truenas/START-HERE-DOCS/TRUENAS_COLLECTION_README.md

TrueNAS Scale Infrastructure Collection System

Version: 1.0.0 TrueNAS Scale API: v2.0 Target Server: 192.168.2.150 Collection Method: Hybrid API + SSH

---

Table of Contents

1. Overview
2. Quick Start
3. Prerequisites
4. Collection Levels
5. Directory Structure
6. API Endpoint Reference
7. SSH Command Reference
8. Usage Examples
9. Security Considerations
10. Troubleshooting
11. Integration with Homelab

---

Overview

This collection system provides comprehensive, READ-ONLY inventory and configuration export capabilities for TrueNAS Scale infrastructure. It follows the same proven pattern as the Proxmox homelab collection script, using a hybrid approach:

• API-First: Leverages TrueNAS Scale REST API v2.0 for structured data
• SSH Fallback: Uses SSH for system-level commands and file access
• Non-Invasive: All operations are read-only - no modifications to your system
• Documented: Generates comprehensive exports with README and summary files

What This System Does:

• Collects pool, dataset, and zvol configurations
• Exports storage metrics and health status
• Documents sharing configurations (NFS, SMB, iSCSI)
• Captures system information and network settings
• Records service states and configurations
• Generates structured JSON and human-readable text outputs

What This System Does NOT Do:

• Modify any TrueNAS configurations
• Export actual data from pools/datasets
• Change permissions or access controls
• Interfere with running services

---

Quick Start

Method 1: Remote Collection via API + SSH (Recommended)

# 1. Set up API authentication
export TRUENAS_HOST="192.168.2.150"
export TRUENAS_API_KEY="your-api-key-here"

# 2. Run collection script (when available)
./collect-truenas-config.sh --level standard --output ./truenas-exports

# 3. Review results
cat ./truenas-exports/truenas-export-$(date +%Y%m%d)/SUMMARY.md

Method 2: Manual Collection via SSH

# SSH to TrueNAS Scale
ssh admin@192.168.2.150

# Run system commands
midclt call system.info
midclt call pool.query
zpool list -v

Method 3: API-Only Collection

# Using curl with API key
curl -X GET "https://192.168.2.150/api/v2.0/system/info" \
  -H "Authorization: Bearer ${TRUENAS_API_KEY}" \
  -H "Content-Type: application/json" \
  --insecure

---

Prerequisites

1. API Access Configuration

Generate API Key:

1. Log in to TrueNAS Scale Web UI (https://192.168.2.150)
2. Navigate to: Account → API Keys
3. Click Add and create new key
4. Copy the key immediately (only shown once)
5. Store securely in password manager

Set Environment Variable:

# Add to ~/.bashrc or ~/.zshrc
export TRUENAS_API_KEY="1-YourActualAPIKeyHere"
export TRUENAS_HOST="192.168.2.150"

Test API Access:

curl -X GET "https://${TRUENAS_HOST}/api/v2.0/system/version" \
  -H "Authorization: Bearer ${TRUENAS_API_KEY}" \
  -H "Content-Type: application/json" \
  --insecure  # Use only if self-signed certificate

Expected response:

{
  "fullversion": "TrueNAS-SCALE-23.10.1",
  "version": "23.10.1"
}

2. SSH Access Configuration

Set Up SSH Key Authentication:

# Generate SSH key (if needed)
ssh-keygen -t ed25519 -C "homelab-collection"

# Copy to TrueNAS
ssh-copy-id admin@192.168.2.150

# Test connection
ssh admin@192.168.2.150 'echo "SSH working"'

3. Required Tools

On Collection Machine (WSL2/Linux):

• curl (7.68.0+)
• jq (1.6+) - JSON processing
• ssh (OpenSSH 8.0+)
• bash (5.0+)

Install on Ubuntu/Debian:

sudo apt update
sudo apt install curl jq openssh-client

4. Disk Space Requirements

Collection Level	Estimated Size	Description
basic	5-10 MB	System info, pool summary
standard	20-50 MB	+ datasets, shares, services
full	50-100 MB	+ detailed metrics, logs
paranoid	100-500 MB	+ debug data, full configs

---

Collection Levels

Basic Level

Use Case: Quick status check, daily monitoring

Collected:

• System version and hardware info
• Pool list and health status
• Dataset tree (top-level only)
• Active services list
• Network interface summary

Execution Time: ~30 seconds

Standard Level (Default)

Use Case: Regular documentation, weekly snapshots

Collected (includes Basic +):

• Complete dataset hierarchy
• ZFS properties for all datasets
• Share configurations (NFS, SMB)
• User and group listings
• Disk information and SMART status
• System logs (last 1000 lines)

Execution Time: ~2-3 minutes

Full Level

Use Case: Pre-maintenance backup, troubleshooting

Collected (includes Standard +):

• Complete service configurations
• Network routing and DNS settings
• Sysctl parameters
• Installed packages list
• Certificate information
• Replication and snapshot tasks
• Cloud sync configurations

Execution Time: ~5-8 minutes

Paranoid Level

Use Case: Complete disaster recovery documentation

Collected (includes Full +):

• System debug data
• Complete middleware configuration
• Historical metrics (if available)
• Boot pool information
• Update history
• Audit logs

Execution Time: ~10-15 minutes

---

Directory Structure

truenas-export-YYYYMMDD-HHMMSS/
├── README.md                          # Export overview and instructions
├── SUMMARY.md                         # Quick reference summary
├── collection.log                     # Detailed collection log
│
├── configs/                           # Configuration files
│   ├── system/
│   │   ├── version.json              # TrueNAS version info
│   │   ├── hardware.json             # CPU, memory, motherboard
│   │   ├── advanced-config.json      # Advanced system settings
│   │   └── general-config.json       # General system configuration
│   │
│   ├── storage/
│   │   ├── pools.json                # Pool configurations
│   │   ├── datasets.json             # Dataset hierarchy
│   │   ├── zvols.json                # ZVol configurations
│   │   ├── disks.json                # Disk inventory
│   │   └── smart-tests.json          # SMART test configurations
│   │
│   ├── sharing/
│   │   ├── nfs-shares.json           # NFS export configurations
│   │   ├── smb-shares.json           # SMB/CIFS shares
│   │   ├── iscsi-config.json         # iSCSI targets and extents
│   │   └── afp-shares.json           # AFP shares (if any)
│   │
│   ├── network/
│   │   ├── interfaces.json           # Network interface configs
│   │   ├── static-routes.json        # Static route table
│   │   ├── dns-config.json           # DNS resolver settings
│   │   └── openvpn-config.json       # VPN configurations
│   │
│   ├── services/
│   │   ├── service-status.json       # All service states
│   │   ├── ssh-config.json           # SSH service configuration
│   │   ├── snmp-config.json          # SNMP settings
│   │   └── ups-config.json           # UPS monitoring config
│   │
│   └── tasks/
│       ├── cron-jobs.json            # Scheduled tasks
│       ├── rsync-tasks.json          # Rsync jobs
│       ├── cloud-sync-tasks.json     # Cloud sync configurations
│       ├── replication-tasks.json    # ZFS replication
│       └── smart-tasks.json          # Scheduled SMART tests
│
├── exports/                           # System state exports
│   ├── storage/
│   │   ├── zpool-list.txt            # Pool status summary
│   │   ├── zpool-status.txt          # Detailed pool health
│   │   ├── zfs-list-all.txt          # All datasets/zvols
│   │   ├── zfs-get-all.txt           # All ZFS properties
│   │   └── disk-list.txt             # Disk inventory
│   │
│   ├── system/
│   │   ├── uname.txt                 # Kernel version
│   │   ├── uptime.txt                # System uptime
│   │   ├── df.txt                    # Filesystem usage
│   │   ├── free.txt                  # Memory usage
│   │   ├── dmesg.txt                 # Kernel messages
│   │   └── sysctl.txt                # Kernel parameters
│   │
│   ├── network/
│   │   ├── ip-addr.txt               # IP address assignments
│   │   ├── ip-route.txt              # Routing table
│   │   ├── ss-listening.txt          # Listening ports
│   │   └── netstat-interfaces.txt    # Network interface stats
│   │
│   └── logs/
│       ├── middleware.log            # TrueNAS middleware log (recent)
│       ├── syslog.txt                # System log (recent)
│       ├── messages.txt              # General messages
│       └── scrub-history.txt         # ZFS scrub history
│
└── metrics/                           # Performance and health metrics
    ├── pool-usage.json               # Pool capacity metrics
    ├── dataset-usage.json            # Per-dataset usage
    ├── disk-temps.json               # Disk temperatures
    ├── smart-status.json             # SMART health status
    └── system-stats.json             # CPU, memory, network stats

---

API Endpoint Reference

System Information Endpoints

Endpoint	Method	Description	Collection Level
/api/v2.0/system/info	GET	System hardware and version	basic
/api/v2.0/system/version	GET	TrueNAS version string	basic
/api/v2.0/system/advanced	GET	Advanced system settings	standard
/api/v2.0/system/general	GET	General configuration	standard
/api/v2.0/system/state	GET	System state and uptime	basic

Storage Endpoints

Endpoint	Method	Description	Collection Level
/api/v2.0/pool	GET	List all pools	basic
/api/v2.0/pool/dataset	GET	List all datasets	standard
/api/v2.0/pool/snapshots	GET	List all snapshots	full
/api/v2.0/pool/scrub	GET	Scrub status and history	standard
/api/v2.0/disk	GET	Disk inventory	standard
/api/v2.0/disk/temperature	GET	Disk temperatures	standard
/api/v2.0/smart/test/results	GET	SMART test results	full

Sharing Endpoints

Endpoint	Method	Description	Collection Level
/api/v2.0/sharing/nfs	GET	NFS share configurations	standard
/api/v2.0/sharing/smb	GET	SMB share configurations	standard
/api/v2.0/sharing/iscsi/target	GET	iSCSI targets	standard
/api/v2.0/sharing/iscsi/extent	GET	iSCSI extents	standard

Network Endpoints

Endpoint	Method	Description	Collection Level
/api/v2.0/interface	GET	Network interfaces	standard
/api/v2.0/network/configuration	GET	Network configuration	standard
/api/v2.0/staticroute	GET	Static routes	standard

Service Endpoints

Endpoint	Method	Description	Collection Level
/api/v2.0/service	GET	All service statuses	basic
/api/v2.0/ssh	GET	SSH service configuration	standard
/api/v2.0/snmp	GET	SNMP configuration	full

Task Endpoints

Endpoint	Method	Description	Collection Level
/api/v2.0/cronjob	GET	Cron jobs	standard
/api/v2.0/rsynctask	GET	Rsync tasks	standard
/api/v2.0/cloudsync	GET	Cloud sync tasks	full
/api/v2.0/replication	GET	Replication tasks	full
/api/v2.0/pool/snapshottask	GET	Snapshot tasks	standard

Full API documentation: See TRUENAS_API_REFERENCE.md for complete details.

---

SSH Command Reference

Storage Commands

Command	Description	Output Format
zpool list -v	Pool list with vdevs	Text table
zpool status -v	Detailed pool health	Text report
zfs list -t all	All datasets/zvols	Text table
zfs get all	All ZFS properties	Text table
midclt call pool.query	Pool info via middleware	JSON

System Commands

Command	Description	Output Format
uname -a	Kernel version	Text
uptime	System uptime	Text
free -h	Memory usage	Text table
df -h	Filesystem usage	Text table
lsblk	Block device tree	Text tree

Network Commands

Command	Description	Output Format
ip addr show	IP addresses	Text
ip route show	Routing table	Text
ss -tuln	Listening sockets	Text table
midclt call interface.query	Interface config	JSON

Disk Health Commands

Command	Description	Output Format
smartctl -a /dev/sda	SMART data for disk	Text report
midclt call disk.query	Disk inventory	JSON
midclt call disk.temperature	Disk temperatures	JSON

---

Usage Examples

Example 1: Basic Daily Check

#!/bin/bash
# quick-truenas-check.sh

TRUENAS_HOST="192.168.2.150"
TRUENAS_API_KEY="${TRUENAS_API_KEY}"

# Get system info
curl -s -X GET "https://${TRUENAS_HOST}/api/v2.0/system/info" \
  -H "Authorization: Bearer ${TRUENAS_API_KEY}" \
  --insecure | jq .

# Get pool status
curl -s -X GET "https://${TRUENAS_HOST}/api/v2.0/pool" \
  -H "Authorization: Bearer ${TRUENAS_API_KEY}" \
  --insecure | jq '.[] | {name: .name, status: .status, healthy: .healthy}'

Example 2: Export All NFS Shares

#!/bin/bash
# export-nfs-config.sh

TRUENAS_HOST="192.168.2.150"
OUTPUT_FILE="nfs-shares-$(date +%Y%m%d).json"

curl -s -X GET "https://${TRUENAS_HOST}/api/v2.0/sharing/nfs" \
  -H "Authorization: Bearer ${TRUENAS_API_KEY}" \
  --insecure | jq . > "${OUTPUT_FILE}"

echo "NFS shares exported to: ${OUTPUT_FILE}"
jq -r '.[] | "\(.path) -> \(.networks[0])"' "${OUTPUT_FILE}"

Example 3: Monitor Pool Health

#!/bin/bash
# monitor-pool-health.sh

ssh admin@192.168.2.150 << 'EOF'
  echo "=== Pool Status ==="
  zpool status

  echo ""
  echo "=== Pool Capacity ==="
  zpool list
EOF

---

Security Considerations

API Key Security

DO:

• Store API keys in environment variables or secure vaults
• Use separate keys for different purposes
• Set appropriate key expiration dates
• Revoke keys when no longer needed
• Use HTTPS for all API calls

DON'T:

• Hardcode API keys in scripts
• Commit keys to version control
• Share keys between users/systems
• Use keys in URLs (query parameters)

SSH Security

Best Practices:

• Use SSH key authentication (not passwords)
• Use Ed25519 keys (more secure than RSA)
• Restrict SSH access by IP if possible
• Use dedicated service accounts for automation
• Regularly rotate SSH keys

Data Sanitization

Before sharing exports:

# Remove sensitive data from export
find truenas-export-*/ -type f -name "*.json" -exec sed -i \
  's/"api_key": "[^"]*"/"api_key": "<REDACTED>"/g' {} \;

# Remove passwords
find truenas-export-*/ -type f -exec sed -i \
  's/"password": "[^"]*"/"password": "<REDACTED>"/g' {} \;

---

Troubleshooting

"API authentication failed"

Symptoms:

HTTP/1.1 401 Unauthorized
{"error": "Invalid API key"}

Solutions:

1. Verify API key is correct: echo $TRUENAS_API_KEY
2. Check key hasn't expired in TrueNAS UI
3. Ensure Authorization header format: Bearer <key>
4. Verify user account has API access permissions

"SSL certificate verify failed"

Symptoms:

curl: (60) SSL certificate problem: self signed certificate

Solutions:

# Option 1: Use --insecure flag (development only)
curl --insecure https://192.168.2.150/api/v2.0/...

# Option 2: Install CA certificate (recommended)
sudo cp /path/to/truenas-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

"Permission denied" on SSH commands

Symptoms:

sudo: a password is required

Solutions:

1. Configure passwordless sudo for specific commands
2. Use middleware API instead: midclt call ...
3. SSH as root user (if allowed)
4. Add user to appropriate groups

"Rate limit exceeded"

Symptoms:

HTTP/1.1 429 Too Many Requests

Solutions:

1. Add delays between API calls: sleep 1
2. Use batch endpoints when available
3. Cache results instead of repeated queries

---

Integration with Homelab

Current Homelab Context

TrueNAS Server:

• IP Address: 192.168.2.150
• Primary Pool: Vauly (3.0T capacity)
• NFS Export: /mnt/Vauly/iso-vault → mounted on Proxmox

Proxmox Integration:

# On serviceslab (192.168.2.200)
nfs: iso-share
	export /mnt/Vauly/iso-vault
	path /mnt/pve/iso-share
	server 192.168.2.150
	content iso

Coordinated Collection Strategy

Recommended workflow:

1. Weekly: Run Proxmox collection (existing script)
2. Weekly: Run TrueNAS collection (this system)
3. Archive: Both exports to same timestamp directory
4. Document: Cross-reference in CLAUDE_STATUS.md

Directory structure:

homelab-exports/
├── export-20251214-030000/
│   ├── proxmox-export/          # From collect-homelab-config.sh
│   └── truenas-export/          # From collect-truenas-config.sh
└── export-20251207-030000/
    ├── proxmox-export/
    └── truenas-export/

---

Next Steps

Phase 1: Foundation (Current)

• Document API endpoints and SSH commands
• Create reference documentation
• Generate API key and test access
• Test individual API endpoints
• Test SSH command execution

Phase 2: Script Development

• Create collect-truenas-config.sh script
• Implement collection levels
• Add error handling and logging
• Create directory structure generation
• Add sanitization functions

Phase 3: Integration

• Integrate with homelab collection workflow
• Add to cron for automated collection
• Create unified export archive structure
• Update CLAUDE_STATUS.md automatically

Phase 4: Monitoring

• Set up Prometheus exporters
• Create Grafana dashboards
• Configure alerting rules

---

Reference Links

Official Documentation:

• TrueNAS Scale Documentation: https://www.truenas.com/docs/scale/
• TrueNAS API Reference: https://www.truenas.com/docs/api/

Related Homelab Documentation:

• Proxmox Collection: /home/jramos/homelab/scripts/crawlers-exporters/collect-homelab-config.sh
• Homelab Index: /home/jramos/homelab/INDEX.md
• Status File: /home/jramos/homelab/CLAUDE_STATUS.md

Community Resources:

• TrueNAS Forums: https://forums.truenas.com/
• r/truenas: https://reddit.com/r/truenas

---

Document Version: 1.0.0 Last Updated: 2025-12-14 Maintained By: Scribe Agent Homelab Repository: /home/jramos/homelab