truenas/SCRIPT_UPDATES.md

TrueNAS Collection Scripts - Updates Summary

Date: 2024-12-16 Updated Scripts:

• /home/jramos/truenas/scripts/collect-truenas-config.sh
• /home/jramos/truenas/scripts/collect-truenas-apps.sh

Backups Created:

• collect-truenas-config.sh.backup
• collect-truenas-apps.sh.backup

---

Overview

Enhanced both TrueNAS collection scripts to properly capture Docker containers and TrueNAS SCALE apps. The scripts now collect comprehensive app/container information that was previously missing from disaster recovery exports.

---

Changes to collect-truenas-config.sh

1. Dynamic SSH User Detection (Lines 294-301)

Before: Hardcoded truenas_admin@ username After: Auto-detects SSH user by trying root, admin, and truenas_admin

Benefit: Works on more TrueNAS configurations without modification

2. JSON Format Correction (All Docker API calls)

Before: --format json (produces invalid newline-delimited JSON) After: --format '{{json .}}' | jq -s '.' with fallback to sed conversion

Benefit: Produces valid JSON arrays parseable by jq and other tools

3. File Size Validation

Before: Logs success even if output files are empty After: Checks [[ -s "$file" ]] before logging success

Benefit: Accurate reporting of what was actually collected

4. Enhanced Data Collection

Added:

• Docker containers (JSON + text formats)
• Docker images
• Docker networks
• Docker volumes
• Docker storage sizes

Benefit: Complete Docker infrastructure snapshot

---

Changes to collect-truenas-apps.sh

1. JSON Format Correction (Lines 30-73)

Before: --format json (invalid newline-delimited JSON) After: --format '{{json .}}' | jq -s '.' with sed fallback

Applies to:

• Docker containers
• Docker images
• Docker networks
• Docker volumes
• Docker Compose projects

2. Container Logs Collection (Lines 128-145) - NEW

Collects: Last 500 lines of logs from each container Output: exports/apps/logs/{container-name}.log

Benefit: Troubleshooting information without needing SSH access later

3. Docker Compose Files Collection (Lines 147-171) - NEW

Collects: Compose files from containers with com.docker.compose.project.config_files label Output: configs/apps/compose/{project-name}.yml

Benefit: Reconstruct compose stacks from running containers

4. Individual App Configs Collection (Lines 173-198) - NEW

Collects: YAML, JSON, ENV, CONF files from /mnt/.ix-apps/app_configs/ Output: configs/apps/app_configs/{app-name}/

Benefit: Full TrueNAS app configuration backup

---

New Directory Structure

After running the updated scripts, exports will include:

truenas-export-YYYYMMDD-HHMMSS/
├── configs/
│   └── apps/
│       ├── metadata.yaml          # TrueNAS app metadata
│       ├── user_config.yaml       # TrueNAS user config
│       ├── compose/               # Docker Compose files
│       │   ├── arr-stack.yml
│       │   └── *.yml
│       └── app_configs/           # Individual app configs
│           ├── plex/
│           ├── dockge/
│           └── */
└── exports/
    └── apps/
        ├── docker-containers.json     # JSON array of all containers
        ├── docker-containers.txt      # Human-readable container list
        ├── docker-images.json         # JSON array of images
        ├── docker-networks.json       # JSON array of networks
        ├── docker-volumes.json        # JSON array of volumes
        ├── docker_sizes.txt           # Storage usage
        ├── app_configs_list.txt       # Directory listing
        ├── containers/                # Individual container details
        │   ├── plex.json
        │   ├── dockge.json
        │   └── *.json
        └── logs/                      # Container logs (last 500 lines)
            ├── plex.log
            ├── dockge.log
            └── *.log

---

Testing Instructions

On TrueNAS Server (Direct Collection)

# Copy the script to TrueNAS
scp /home/jramos/truenas/scripts/collect-truenas-apps.sh truenas_admin@192.168.2.150:/tmp/

# SSH to TrueNAS
ssh truenas_admin@192.168.2.150

# Run the collection script
bash /tmp/collect-truenas-apps.sh

# Verify JSON validity (if jq installed)
jq empty /tmp/truenas-apps-export-*/exports/apps/docker-containers.json
jq empty /tmp/truenas-apps-export-*/exports/apps/docker-images.json

# Copy back to workstation
# On your workstation:
scp truenas_admin@192.168.2.150:/tmp/truenas-apps-export-*.tar.gz /home/jramos/truenas/disaster-recovery/

From WSL (Via SSH)

# Set up SSH key authentication first (if not already done)
ssh-copy-id truenas_admin@192.168.2.150

# Run the collection script
cd /home/jramos/truenas
bash scripts/collect-truenas-config.sh

# Check for apps data
ls -la disaster-recovery/truenas-exports/exports/apps/
ls -la disaster-recovery/truenas-exports/configs/apps/

---

Critical Fixes Applied

1. Invalid JSON Bug (High Priority)

• Issue: Docker commands with --format json produce newline-delimited JSON, not arrays
• Impact: Files couldn't be parsed by jq or JSON parsers
• Fix: Pipe through jq -s '.' or use sed to convert to proper JSON arrays

2. Hardcoded SSH User (Medium Priority)

• Issue: Script assumed truenas_admin@ user exists
• Impact: Failed on systems with different admin usernames
• Fix: Auto-detect by trying multiple common usernames

3. Empty File False Positives (Medium Priority)

• Issue: Scripts logged "success" even when output files were empty
• Impact: Misleading collection statistics
• Fix: Added [[ -s "$file" ]] size checks

4. Missing Critical Data (High Priority)

• Issue: No logs, compose files, or individual app configs collected
• Impact: Incomplete disaster recovery information
• Fix: Added three new collection sections

---

Known Limitations

1. SSH Access Required: For remote collection from WSL, SSH key auth must be configured
2. Large Logs: Container logs are limited to last 500 lines to prevent huge files
3. Compose File Detection: Only works for containers with proper Docker Compose labels
4. Sudo Required: All Docker commands require sudo access on TrueNAS

---

Next Steps

1. Test the scripts on your TrueNAS server
2. Verify JSON files are valid with jq
3. Review collected data for completeness
4. Document the apps inventory (similar to Proxmox VMs/CTs)
5. Update TrueNAS README with new collection capabilities

---

Your TrueNAS Environment

Running Containers (from your sudo docker ps output):

• TrueNAS Apps: Plex, Dockge
• Compose Stack (arr-stack): Radarr, Sonarr, Bazarr
• VPN/Networking: Gluetun, NordLynx (2x)
• Download/Media: Deluge, Prowlarr, FlareSolverr
• Monitoring: Beszel-agent

The updated scripts will now capture all of these properly!

---

Review completed by: backend-builder agent Applied by: Main agent Status: ✓ All fixes applied successfully