jramos/truenas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(infrastructure): initialize TrueNAS Scale infrastructure collection system

Browse Source 
Initial repository setup for TrueNAS Scale configuration management and
disaster recovery. This system provides automated collection, versioning,
and documentation of TrueNAS configuration state.

Key components:
- Configuration collection scripts with API integration
- Disaster recovery exports (configs, storage, system state)
- Comprehensive documentation and API reference
- Sub-agent architecture for specialized operations

Infrastructure protected:
- Storage pools and datasets configuration
- Network configuration and routing
- Sharing services (NFS, SMB, iSCSI)
- System tasks (snapshots, replication, cloud sync)
- User and group management

Security measures:
- API keys managed via environment variables
- Sensitive data excluded via .gitignore
- No credentials committed to repository

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Jordan Ramos
2025-12-16 08:03:33 -07:00
commit 52e1822de8
37 changed files with 40881 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

657
START-HERE-DOCS/TRUENAS_COLLECTION_README.md Normal file
View File

@@ -0,0 +1,657 @@
# TrueNAS Scale Infrastructure Collection System
**Version:** 1.0.0
**TrueNAS Scale API:** v2.0
**Target Server:** 192.168.2.150
**Collection Method:** Hybrid API + SSH
---
## Table of Contents
1. [Overview](#overview)
2. [Quick Start](#quick-start)
3. [Prerequisites](#prerequisites)
4. [Collection Levels](#collection-levels)
5. [Directory Structure](#directory-structure)
6. [API Endpoint Reference](#api-endpoint-reference)
7. [SSH Command Reference](#ssh-command-reference)
8. [Usage Examples](#usage-examples)
9. [Security Considerations](#security-considerations)
10. [Troubleshooting](#troubleshooting)
11. [Integration with Homelab](#integration-with-homelab)
---
## Overview
This collection system provides comprehensive, READ-ONLY inventory and configuration export capabilities for TrueNAS Scale infrastructure. It follows the same proven pattern as the Proxmox homelab collection script, using a hybrid approach:
- **API-First**: Leverages TrueNAS Scale REST API v2.0 for structured data
- **SSH Fallback**: Uses SSH for system-level commands and file access
- **Non-Invasive**: All operations are read-only - no modifications to your system
- **Documented**: Generates comprehensive exports with README and summary files
**What This System Does:**
- Collects pool, dataset, and zvol configurations
- Exports storage metrics and health status
- Documents sharing configurations (NFS, SMB, iSCSI)
- Captures system information and network settings
- Records service states and configurations
- Generates structured JSON and human-readable text outputs
**What This System Does NOT Do:**
- Modify any TrueNAS configurations
- Export actual data from pools/datasets
- Change permissions or access controls
- Interfere with running services
---
## Quick Start
### Method 1: Remote Collection via API + SSH (Recommended)
```bash
# 1. Set up API authentication
export TRUENAS_HOST="192.168.2.150"
export TRUENAS_API_KEY="your-api-key-here"
# 2. Run collection script (when available)
./collect-truenas-config.sh --level standard --output ./truenas-exports
# 3. Review results
cat ./truenas-exports/truenas-export-$(date +%Y%m%d)/SUMMARY.md
```
### Method 2: Manual Collection via SSH
```bash
# SSH to TrueNAS Scale
ssh admin@192.168.2.150
# Run system commands
midclt call system.info
midclt call pool.query
zpool list -v
```
### Method 3: API-Only Collection
```bash
# Using curl with API key
curl -X GET "https://192.168.2.150/api/v2.0/system/info" \
-H "Authorization: Bearer ${TRUENAS_API_KEY}" \
-H "Content-Type: application/json" \
--insecure
```
---
## Prerequisites
### 1. API Access Configuration
**Generate API Key:**
1. Log in to TrueNAS Scale Web UI (https://192.168.2.150)
2. Navigate to: **Account** → **API Keys**
3. Click **Add** and create new key
4. Copy the key immediately (only shown once)
5. Store securely in password manager
**Set Environment Variable:**
```bash
# Add to ~/.bashrc or ~/.zshrc
export TRUENAS_API_KEY="1-YourActualAPIKeyHere"
export TRUENAS_HOST="192.168.2.150"
```
**Test API Access:**
```bash
curl -X GET "https://${TRUENAS_HOST}/api/v2.0/system/version" \
-H "Authorization: Bearer ${TRUENAS_API_KEY}" \
-H "Content-Type: application/json" \
--insecure # Use only if self-signed certificate
```
Expected response:
```json
{
"fullversion": "TrueNAS-SCALE-23.10.1",
"version": "23.10.1"
}
```
### 2. SSH Access Configuration
**Set Up SSH Key Authentication:**
```bash
# Generate SSH key (if needed)
ssh-keygen -t ed25519 -C "homelab-collection"
# Copy to TrueNAS
ssh-copy-id admin@192.168.2.150
# Test connection
ssh admin@192.168.2.150 'echo "SSH working"'
```
### 3. Required Tools
**On Collection Machine (WSL2/Linux):**
- `curl` (7.68.0+)
- `jq` (1.6+) - JSON processing
- `ssh` (OpenSSH 8.0+)
- `bash` (5.0+)
**Install on Ubuntu/Debian:**
```bash
sudo apt update
sudo apt install curl jq openssh-client
```
### 4. Disk Space Requirements
| Collection Level | Estimated Size | Description |
|-----------------|----------------|-------------|
| **basic** | 5-10 MB | System info, pool summary |
| **standard** | 20-50 MB | + datasets, shares, services |
| **full** | 50-100 MB | + detailed metrics, logs |
| **paranoid** | 100-500 MB | + debug data, full configs |
---
## Collection Levels
### Basic Level
**Use Case:** Quick status check, daily monitoring
**Collected:**
- System version and hardware info
- Pool list and health status
- Dataset tree (top-level only)
- Active services list
- Network interface summary
**Execution Time:** ~30 seconds
### Standard Level (Default)
**Use Case:** Regular documentation, weekly snapshots
**Collected (includes Basic +):**
- Complete dataset hierarchy
- ZFS properties for all datasets
- Share configurations (NFS, SMB)
- User and group listings
- Disk information and SMART status
- System logs (last 1000 lines)
**Execution Time:** ~2-3 minutes
### Full Level
**Use Case:** Pre-maintenance backup, troubleshooting
**Collected (includes Standard +):**
- Complete service configurations
- Network routing and DNS settings
- Sysctl parameters
- Installed packages list
- Certificate information
- Replication and snapshot tasks
- Cloud sync configurations
**Execution Time:** ~5-8 minutes
### Paranoid Level
**Use Case:** Complete disaster recovery documentation
**Collected (includes Full +):**
- System debug data
- Complete middleware configuration
- Historical metrics (if available)
- Boot pool information
- Update history
- Audit logs
**Execution Time:** ~10-15 minutes
---
## Directory Structure
```
truenas-export-YYYYMMDD-HHMMSS/
├── README.md # Export overview and instructions
├── SUMMARY.md # Quick reference summary
├── collection.log # Detailed collection log
│
├── configs/ # Configuration files
│ ├── system/
│ │ ├── version.json # TrueNAS version info
│ │ ├── hardware.json # CPU, memory, motherboard
│ │ ├── advanced-config.json # Advanced system settings
│ │ └── general-config.json # General system configuration
│ │
│ ├── storage/
│ │ ├── pools.json # Pool configurations
│ │ ├── datasets.json # Dataset hierarchy
│ │ ├── zvols.json # ZVol configurations
│ │ ├── disks.json # Disk inventory
│ │ └── smart-tests.json # SMART test configurations
│ │
│ ├── sharing/
│ │ ├── nfs-shares.json # NFS export configurations
│ │ ├── smb-shares.json # SMB/CIFS shares
│ │ ├── iscsi-config.json # iSCSI targets and extents
│ │ └── afp-shares.json # AFP shares (if any)
│ │
│ ├── network/
│ │ ├── interfaces.json # Network interface configs
│ │ ├── static-routes.json # Static route table
│ │ ├── dns-config.json # DNS resolver settings
│ │ └── openvpn-config.json # VPN configurations
│ │
│ ├── services/
│ │ ├── service-status.json # All service states
│ │ ├── ssh-config.json # SSH service configuration
│ │ ├── snmp-config.json # SNMP settings
│ │ └── ups-config.json # UPS monitoring config
│ │
│ └── tasks/
│ ├── cron-jobs.json # Scheduled tasks
│ ├── rsync-tasks.json # Rsync jobs
│ ├── cloud-sync-tasks.json # Cloud sync configurations
│ ├── replication-tasks.json # ZFS replication
│ └── smart-tasks.json # Scheduled SMART tests
│
├── exports/ # System state exports
│ ├── storage/
│ │ ├── zpool-list.txt # Pool status summary
│ │ ├── zpool-status.txt # Detailed pool health
│ │ ├── zfs-list-all.txt # All datasets/zvols
│ │ ├── zfs-get-all.txt # All ZFS properties
│ │ └── disk-list.txt # Disk inventory
│ │
│ ├── system/
│ │ ├── uname.txt # Kernel version
│ │ ├── uptime.txt # System uptime
│ │ ├── df.txt # Filesystem usage
│ │ ├── free.txt # Memory usage
│ │ ├── dmesg.txt # Kernel messages
│ │ └── sysctl.txt # Kernel parameters
│ │
│ ├── network/
│ │ ├── ip-addr.txt # IP address assignments
│ │ ├── ip-route.txt # Routing table
│ │ ├── ss-listening.txt # Listening ports
│ │ └── netstat-interfaces.txt # Network interface stats
│ │
│ └── logs/
│ ├── middleware.log # TrueNAS middleware log (recent)
│ ├── syslog.txt # System log (recent)
│ ├── messages.txt # General messages
│ └── scrub-history.txt # ZFS scrub history
│
└── metrics/ # Performance and health metrics
├── pool-usage.json # Pool capacity metrics
├── dataset-usage.json # Per-dataset usage
├── disk-temps.json # Disk temperatures
├── smart-status.json # SMART health status
└── system-stats.json # CPU, memory, network stats
```
---
## API Endpoint Reference
### System Information Endpoints
| Endpoint | Method | Description | Collection Level |
|----------|--------|-------------|------------------|
| `/api/v2.0/system/info` | GET | System hardware and version | basic |
| `/api/v2.0/system/version` | GET | TrueNAS version string | basic |
| `/api/v2.0/system/advanced` | GET | Advanced system settings | standard |
| `/api/v2.0/system/general` | GET | General configuration | standard |
| `/api/v2.0/system/state` | GET | System state and uptime | basic |
### Storage Endpoints
| Endpoint | Method | Description | Collection Level |
|----------|--------|-------------|------------------|
| `/api/v2.0/pool` | GET | List all pools | basic |
| `/api/v2.0/pool/dataset` | GET | List all datasets | standard |
| `/api/v2.0/pool/snapshots` | GET | List all snapshots | full |
| `/api/v2.0/pool/scrub` | GET | Scrub status and history | standard |
| `/api/v2.0/disk` | GET | Disk inventory | standard |
| `/api/v2.0/disk/temperature` | GET | Disk temperatures | standard |
| `/api/v2.0/smart/test/results` | GET | SMART test results | full |
### Sharing Endpoints
| Endpoint | Method | Description | Collection Level |
|----------|--------|-------------|------------------|
| `/api/v2.0/sharing/nfs` | GET | NFS share configurations | standard |
| `/api/v2.0/sharing/smb` | GET | SMB share configurations | standard |
| `/api/v2.0/sharing/iscsi/target` | GET | iSCSI targets | standard |
| `/api/v2.0/sharing/iscsi/extent` | GET | iSCSI extents | standard |
### Network Endpoints
| Endpoint | Method | Description | Collection Level |
|----------|--------|-------------|------------------|
| `/api/v2.0/interface` | GET | Network interfaces | standard |
| `/api/v2.0/network/configuration` | GET | Network configuration | standard |
| `/api/v2.0/staticroute` | GET | Static routes | standard |
### Service Endpoints
| Endpoint | Method | Description | Collection Level |
|----------|--------|-------------|------------------|
| `/api/v2.0/service` | GET | All service statuses | basic |
| `/api/v2.0/ssh` | GET | SSH service configuration | standard |
| `/api/v2.0/snmp` | GET | SNMP configuration | full |
### Task Endpoints
| Endpoint | Method | Description | Collection Level |
|----------|--------|-------------|------------------|
| `/api/v2.0/cronjob` | GET | Cron jobs | standard |
| `/api/v2.0/rsynctask` | GET | Rsync tasks | standard |
| `/api/v2.0/cloudsync` | GET | Cloud sync tasks | full |
| `/api/v2.0/replication` | GET | Replication tasks | full |
| `/api/v2.0/pool/snapshottask` | GET | Snapshot tasks | standard |
**Full API documentation:** See `TRUENAS_API_REFERENCE.md` for complete details.
---
## SSH Command Reference
### Storage Commands
| Command | Description | Output Format |
|---------|-------------|---------------|
| `zpool list -v` | Pool list with vdevs | Text table |
| `zpool status -v` | Detailed pool health | Text report |
| `zfs list -t all` | All datasets/zvols | Text table |
| `zfs get all` | All ZFS properties | Text table |
| `midclt call pool.query` | Pool info via middleware | JSON |
### System Commands
| Command | Description | Output Format |
|---------|-------------|---------------|
| `uname -a` | Kernel version | Text |
| `uptime` | System uptime | Text |
| `free -h` | Memory usage | Text table |
| `df -h` | Filesystem usage | Text table |
| `lsblk` | Block device tree | Text tree |
### Network Commands
| Command | Description | Output Format |
|---------|-------------|---------------|
| `ip addr show` | IP addresses | Text |
| `ip route show` | Routing table | Text |
| `ss -tuln` | Listening sockets | Text table |
| `midclt call interface.query` | Interface config | JSON |
### Disk Health Commands
| Command | Description | Output Format |
|---------|-------------|---------------|
| `smartctl -a /dev/sda` | SMART data for disk | Text report |
| `midclt call disk.query` | Disk inventory | JSON |
| `midclt call disk.temperature` | Disk temperatures | JSON |
---
## Usage Examples
### Example 1: Basic Daily Check
```bash
#!/bin/bash
# quick-truenas-check.sh
TRUENAS_HOST="192.168.2.150"
TRUENAS_API_KEY="${TRUENAS_API_KEY}"
# Get system info
curl -s -X GET "https://${TRUENAS_HOST}/api/v2.0/system/info" \
-H "Authorization: Bearer ${TRUENAS_API_KEY}" \
--insecure | jq .
# Get pool status
curl -s -X GET "https://${TRUENAS_HOST}/api/v2.0/pool" \
-H "Authorization: Bearer ${TRUENAS_API_KEY}" \
--insecure | jq '.[] | {name: .name, status: .status, healthy: .healthy}'
```
### Example 2: Export All NFS Shares
```bash
#!/bin/bash
# export-nfs-config.sh
TRUENAS_HOST="192.168.2.150"
OUTPUT_FILE="nfs-shares-$(date +%Y%m%d).json"
curl -s -X GET "https://${TRUENAS_HOST}/api/v2.0/sharing/nfs" \
-H "Authorization: Bearer ${TRUENAS_API_KEY}" \
--insecure | jq . > "${OUTPUT_FILE}"
echo "NFS shares exported to: ${OUTPUT_FILE}"
jq -r '.[] | "\(.path) -> \(.networks[0])"' "${OUTPUT_FILE}"
```
### Example 3: Monitor Pool Health
```bash
#!/bin/bash
# monitor-pool-health.sh
ssh admin@192.168.2.150 << 'EOF'
echo "=== Pool Status ==="
zpool status
echo ""
echo "=== Pool Capacity ==="
zpool list
EOF
```
---
## Security Considerations
### API Key Security
**DO:**
- Store API keys in environment variables or secure vaults
- Use separate keys for different purposes
- Set appropriate key expiration dates
- Revoke keys when no longer needed
- Use HTTPS for all API calls
**DON'T:**
- Hardcode API keys in scripts
- Commit keys to version control
- Share keys between users/systems
- Use keys in URLs (query parameters)
### SSH Security
**Best Practices:**
- Use SSH key authentication (not passwords)
- Use Ed25519 keys (more secure than RSA)
- Restrict SSH access by IP if possible
- Use dedicated service accounts for automation
- Regularly rotate SSH keys
### Data Sanitization
**Before sharing exports:**
```bash
# Remove sensitive data from export
find truenas-export-*/ -type f -name "*.json" -exec sed -i \
's/"api_key": "[^"]*"/"api_key": "<REDACTED>"/g' {} \;
# Remove passwords
find truenas-export-*/ -type f -exec sed -i \
's/"password": "[^"]*"/"password": "<REDACTED>"/g' {} \;
```
---
## Troubleshooting
### "API authentication failed"
**Symptoms:**
```
HTTP/1.1 401 Unauthorized
{"error": "Invalid API key"}
```
**Solutions:**
1. Verify API key is correct: `echo $TRUENAS_API_KEY`
2. Check key hasn't expired in TrueNAS UI
3. Ensure Authorization header format: `Bearer <key>`
4. Verify user account has API access permissions
### "SSL certificate verify failed"
**Symptoms:**
```
curl: (60) SSL certificate problem: self signed certificate
```
**Solutions:**
```bash
# Option 1: Use --insecure flag (development only)
curl --insecure https://192.168.2.150/api/v2.0/...
# Option 2: Install CA certificate (recommended)
sudo cp /path/to/truenas-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
```
### "Permission denied" on SSH commands
**Symptoms:**
```
sudo: a password is required
```
**Solutions:**
1. Configure passwordless sudo for specific commands
2. Use middleware API instead: `midclt call ...`
3. SSH as root user (if allowed)
4. Add user to appropriate groups
### "Rate limit exceeded"
**Symptoms:**
```
HTTP/1.1 429 Too Many Requests
```
**Solutions:**
1. Add delays between API calls: `sleep 1`
2. Use batch endpoints when available
3. Cache results instead of repeated queries
---
## Integration with Homelab
### Current Homelab Context
**TrueNAS Server:**
- **IP Address:** 192.168.2.150
- **Primary Pool:** Vauly (3.0T capacity)
- **NFS Export:** `/mnt/Vauly/iso-vault` → mounted on Proxmox
**Proxmox Integration:**
```
# On serviceslab (192.168.2.200)
nfs: iso-share
export /mnt/Vauly/iso-vault
path /mnt/pve/iso-share
server 192.168.2.150
content iso
```
### Coordinated Collection Strategy
**Recommended workflow:**
1. **Weekly:** Run Proxmox collection (existing script)
2. **Weekly:** Run TrueNAS collection (this system)
3. **Archive:** Both exports to same timestamp directory
4. **Document:** Cross-reference in CLAUDE_STATUS.md
**Directory structure:**
```
homelab-exports/
├── export-20251214-030000/
│ ├── proxmox-export/ # From collect-homelab-config.sh
│ └── truenas-export/ # From collect-truenas-config.sh
└── export-20251207-030000/
├── proxmox-export/
└── truenas-export/
```
---
## Next Steps
### Phase 1: Foundation (Current)
- [x] Document API endpoints and SSH commands
- [x] Create reference documentation
- [ ] Generate API key and test access
- [ ] Test individual API endpoints
- [ ] Test SSH command execution
### Phase 2: Script Development
- [ ] Create `collect-truenas-config.sh` script
- [ ] Implement collection levels
- [ ] Add error handling and logging
- [ ] Create directory structure generation
- [ ] Add sanitization functions
### Phase 3: Integration
- [ ] Integrate with homelab collection workflow
- [ ] Add to cron for automated collection
- [ ] Create unified export archive structure
- [ ] Update CLAUDE_STATUS.md automatically
### Phase 4: Monitoring
- [ ] Set up Prometheus exporters
- [ ] Create Grafana dashboards
- [ ] Configure alerting rules
---
## Reference Links
**Official Documentation:**
- TrueNAS Scale Documentation: https://www.truenas.com/docs/scale/
- TrueNAS API Reference: https://www.truenas.com/docs/api/
**Related Homelab Documentation:**
- Proxmox Collection: `/home/jramos/homelab/scripts/crawlers-exporters/collect-homelab-config.sh`
- Homelab Index: `/home/jramos/homelab/INDEX.md`
- Status File: `/home/jramos/homelab/CLAUDE_STATUS.md`
**Community Resources:**
- TrueNAS Forums: https://forums.truenas.com/
- r/truenas: https://reddit.com/r/truenas
---
**Document Version:** 1.0.0
**Last Updated:** 2025-12-14
**Maintained By:** Scribe Agent
**Homelab Repository:** /home/jramos/homelab