homelab/start-here-docs/SCRIPT-USAGE.md

Homelab Collection Script - Quick Reference

Overview

The collect-homelab-config.sh script performs comprehensive, read-only collection of your Proxmox VE infrastructure configuration and exports it in an organized, documented format.

Status: ✅ Fully Operational (bugs fixed as of 2025-11-29)

---

Quick Start

On Proxmox Host (Direct Execution)

# Copy script to Proxmox host
scp collect-homelab-config.sh root@192.168.2.100:/tmp/

# SSH to Proxmox host
ssh root@192.168.2.100

# Run with default settings
cd /tmp
bash collect-homelab-config.sh

# Or with verbose output
bash collect-homelab-config.sh --verbose

From Your Workstation (Remote Execution)

# Upload and execute in one command
scp collect-homelab-config.sh root@192.168.2.100:/tmp/ && \
ssh root@192.168.2.100 'cd /tmp && bash /tmp/collect-homelab-config.sh'

# Download the results
scp root@192.168.2.100:/tmp/homelab-export-*.tar.gz ./

---

Command-Line Options

Collection Levels

# Basic: System info + Proxmox/VM/CT configs only
./collect-homelab-config.sh --level basic

# Standard: Basic + storage, network, backup, cluster info (DEFAULT)
./collect-homelab-config.sh --level standard

# Full: Standard + service configs, detailed system state
./collect-homelab-config.sh --level full

# Paranoid: Everything possible (experimental)
./collect-homelab-config.sh --level paranoid

Sanitization Options

# Sanitize everything (IPs, passwords, tokens)
./collect-homelab-config.sh --sanitize all

# Sanitize only IP addresses
./collect-homelab-config.sh --sanitize ips

# No sanitization (CAUTION: sensitive data will be in clear text)
./collect-homelab-config.sh --sanitize none

# Default: Passwords and tokens sanitized, IPs preserved

Output Control

# Custom output directory
./collect-homelab-config.sh --output /backup/my-homelab-export

# Disable compression
./collect-homelab-config.sh --no-compress

# Enable compression (default)
./collect-homelab-config.sh --compress

Debugging

# Verbose output (shows DEBUG messages)
./collect-homelab-config.sh --verbose

# Standard output (INFO/SUCCESS/WARN/ERROR only)
./collect-homelab-config.sh

Combined Examples

# Full collection with complete sanitization and verbose output
./collect-homelab-config.sh --level full --sanitize all --verbose

# Basic collection to specific directory, no compression
./collect-homelab-config.sh --level basic --output /root/configs --no-compress

# Standard collection with IP sanitization
./collect-homelab-config.sh --sanitize all --verbose

---

What Gets Collected

✅ Always Collected (All Levels)

System Information:

• Proxmox VE version
• Hostname, kernel, uptime
• CPU, memory, disk information
• LVM configuration (PV/VG/LV)
• Network interfaces and routing
• Listening services
• Installed packages

Proxmox Configurations:

• Datacenter settings (datacenter.cfg)
• Storage pools (storage.cfg)
• Users and permissions (user.cfg)
• Authentication keys (authkey.pub)
• Firewall rules (if configured)
• Corosync cluster config (if in cluster)
• HA configuration (if configured)

VM Configurations:

• All QEMU/KVM VM configs (/etc/pve/nodes/*/qemu-server/*.conf)
• VM firewall rules
• Format: {VMID}-{VM-name}.conf

LXC Container Configurations:

• All LXC container configs (/etc/pve/nodes/*/lxc/*.conf)
• Container firewall rules
• Format: {CTID}-{container-name}.conf

Network Configurations:

• Interface definitions (/etc/network/interfaces)
• Additional interface configs (/etc/network/interfaces.d/)
• SDN configuration (if configured)
• Hosts file
• DNS resolver config

Storage Information:

• Storage status (pvesm status)
• ZFS pools and datasets (if ZFS is used)
• NFS exports (if configured)
• Samba configuration (if configured)
• iSCSI initiator config (if configured)

➕ Standard Level and Above

Backup Configurations:

• Vzdump configuration
• Scheduled backup jobs
• Proxmox Backup Server integration

Cluster Information:

• Cluster status and membership
• Resource allocation
• Recent tasks history

Guest Information:

• VM and container lists
• Resource usage statistics
• JSON exports for programmatic access

➕ Full Level and Above

Service Configurations:

• Systemd service status
• Proxmox-specific service states
  • pve-cluster
  • pvedaemon
  • pveproxy
  • pvestatd
  • pve-firewall
  • pvescheduler

---

Output Structure

homelab-export-20251129-141328/
├── README.md                    # Comprehensive documentation
├── SUMMARY.md                   # Collection statistics and overview
├── collection.log               # Detailed execution log
├── configs/                     # Configuration files
│   ├── proxmox/                # Proxmox VE configurations
│   │   ├── datacenter.cfg
│   │   ├── storage.cfg
│   │   ├── user.cfg
│   │   └── authkey.pub
│   ├── vms/                    # Virtual machine configs
│   │   ├── 100-docker-hub.conf
│   │   ├── 101-gitlab.conf
│   │   └── ...
│   ├── lxc/                    # LXC container configs
│   │   ├── 102-nginx.conf
│   │   ├── 103-netbox.conf
│   │   └── 112-Anytype.conf
│   ├── storage/                # Storage configurations
│   ├── network/                # Network configurations
│   ├── backup/                 # Backup job configurations
│   └── services/               # System service configs (full level)
├── exports/                     # System state exports
│   ├── system/                 # System information
│   │   ├── pve-version.txt
│   │   ├── cpuinfo.txt
│   │   ├── meminfo.txt
│   │   └── ...
│   ├── cluster/                # Cluster status and resources
│   └── guests/                 # Guest VM/CT information
├── docs/                        # For manual documentation additions
├── scripts/                     # For automation scripts
└── diagrams/                    # For network diagrams

---

Sanitization Behavior

Default Sanitization (Passwords + Tokens)

Patterns Removed:

• password=* → password=<REDACTED>
• passwd=* → passwd=<REDACTED>
• "password": "*" → "password": "<REDACTED>"
• token=* → token=<REDACTED>
• api_key=* → api_key=<REDACTED>
• secret=* → secret=<REDACTED>

IP Addresses: Preserved (not sanitized)

All Sanitization (--sanitize all)

Everything above PLUS:

• All IPv4 addresses → 10.X.X.X

No Sanitization (--sanitize none)

⚠️ DANGER: All sensitive data remains in clear text. Use only for:

• Air-gapped environments
• Encrypted storage destinations
• Troubleshooting purposes

---

Expected Output

Success Indicators

================================================================================
  Collection Complete
================================================================================

[✓] Total items collected: 50
[INFO] Total items skipped: 1
[WARN] Total errors: 5

Export Location: ./homelab-export-20251129-141328
Summary Report: ./homelab-export-20251129-141328/SUMMARY.md
Collection Log: ./homelab-export-20251129-141328/collection.log

Typical Statistics

• Items Collected: 45-60 (depending on your infrastructure)
• Items Skipped: 0-5 (files that don't exist, like domains.cfg)
• Errors: 0-10 (warnings for optional configs that don't exist)
• Archive Size: 30-100KB (compressed)
• Execution Time: 5-15 seconds

Common Warnings (Non-Critical)

[WARN] Failed to copy directory HA configuration from /etc/pve/ha
[WARN] Failed to copy directory Additional interface configs from /etc/network/interfaces.d
[WARN] Failed to copy directory SDN configuration from /etc/pve/sdn
[WARN] Failed to execute: pvecm status (Cluster status)
[WARN] Failed to execute: pvecm nodes (Cluster nodes)

These are expected if you don't have:

• High Availability (HA) configured
• Additional network interface configs
• Software Defined Networking (SDN)
• Multi-node cluster

---

Post-Collection Workflow

1. Review the Collection

# Extract the archive
tar -xzf homelab-export-20251129-141328.tar.gz

# Review the summary
cat homelab-export-20251129-141328/SUMMARY.md

# Check for errors
grep -E "\[ERROR\]|\[WARN\]" homelab-export-20251129-141328/collection.log

2. Verify Sensitive Data

# Verify passwords are sanitized
grep -r "password=" homelab-export-20251129-141328/configs/

# Should show: password=<REDACTED>

3. Version Control

# Initialize git (if not already done)
cd /mnt/c/Users/fam1n/Documents/homelab
git init

# Add the export
git add homelab-export-20251129-141328/
git commit -m "Infrastructure snapshot: 2025-11-29"

# Or just commit the archive
git add homelab-export-20251129-141328.tar.gz
git commit -m "Homelab export archive: 2025-11-29"

4. Documentation Enhancement

Add custom documentation to the collected export:

cd homelab-export-20251129-141328

# Add network diagrams
cp ~/network-diagram.png diagrams/

# Add runbooks
cat > docs/disaster-recovery.md <<EOF
# Disaster Recovery Procedures
...
EOF

# Add automation scripts
cp ~/ansible-playbooks/*.yml scripts/

5. Scheduled Collection

Create a cron job on your Proxmox host:

# Run collection monthly and keep archives
cat > /etc/cron.d/homelab-export <<'EOF'
# Monthly infrastructure export
0 2 1 * * root cd /backup && /opt/scripts/collect-homelab-config.sh --level standard --output /backup/homelab-export-$(date +\%Y\%m) --compress
EOF

---

Troubleshooting

Script Stops Prematurely

✅ FIXED as of 2025-11-29. If you're still experiencing issues:

1. Ensure you're using the latest version of the script
2. Run with --verbose to see detailed progress
3. Check collection.log for specific errors
4. Verify you have root permissions: whoami should return root

Permission Denied Errors

# Ensure script is executable
chmod +x collect-homelab-config.sh

# Run as root
sudo bash collect-homelab-config.sh
# or
ssh root@proxmox-host 'bash /tmp/collect-homelab-config.sh'

Missing Commands

The script requires standard Proxmox utilities:

• pveversion, pvesh, pvesm, pvecm
• qm, pct
• lscpu, df, lsblk, ip, ss

These are all included in a standard Proxmox VE installation.

Archive Too Large

# Exclude large log files (manual step after collection)
cd homelab-export-20251129-141328
find . -name "*.log" -size +10M -delete

# Re-compress
cd ..
tar -czf homelab-export-20251129-141328-slim.tar.gz homelab-export-20251129-141328/

---

Security Considerations

⚠️ Warning: Sensitive Data

Even with sanitization enabled, the export contains:

• ✅ Network topology and IP addressing
• ✅ Usernames (passwords redacted)
• ✅ Storage paths and mount points
• ✅ VM/CT configurations
• ✅ Installed package lists

Best Practices

1. Never commit to public repositories without --sanitize all
2. Encrypt archives before cloud storage:

   gpg -c homelab-export-20251129-141328.tar.gz
   

3. Use private Git repos (GitLab private, GitHub private)
4. Restrict file permissions:

   chmod 600 homelab-export-20251129-141328.tar.gz
   

5. Audit before sharing: Always review contents before sharing with others

---

Integration Examples

Ansible Playbook

---
- name: Collect Proxmox infrastructure state
  hosts: proxmox
  tasks:
    - name: Upload collection script
      copy:
        src: collect-homelab-config.sh
        dest: /tmp/collect-homelab-config.sh
        mode: '0755'

    - name: Run collection
      shell: cd /tmp && bash collect-homelab-config.sh --level full

    - name: Download export
      fetch:
        src: /tmp/homelab-export-{{ ansible_date_time.date }}.tar.gz
        dest: ./exports/
        flat: yes

Backup Script

#!/bin/bash
# backup-homelab-config.sh

BACKUP_DIR="/mnt/backup/homelab-configs"
PROXMOX_HOST="192.168.2.100"

# Run collection on Proxmox
ssh root@${PROXMOX_HOST} 'cd /tmp && bash /tmp/collect-homelab-config.sh --level full --sanitize all'

# Download archive
scp root@${PROXMOX_HOST}:/tmp/homelab-export-*.tar.gz ${BACKUP_DIR}/

# Clean up old exports (keep last 12)
cd ${BACKUP_DIR}
ls -t homelab-export-*.tar.gz | tail -n +13 | xargs rm -f

# Cleanup remote
ssh root@${PROXMOX_HOST} 'rm -rf /tmp/homelab-export-*'

---

Files in This Directory

• collect-homelab-config.sh: The main collection script (FIXED)
• BUGFIX-SUMMARY.md: Detailed technical analysis of bugs and fixes
• SCRIPT-USAGE.md: This file - quick reference guide
• homelab-export-*.tar.gz: Collected infrastructure exports
• CLAUDE.md: Repository context and infrastructure overview

---

Support

For issues, improvements, or questions:

1. Review BUGFIX-SUMMARY.md for technical details
2. Check collection.log in the export for error details
3. Run with --verbose for debugging output
4. Verify you're running the latest fixed version

---

Script version: 1.0.0 (Bugs fixed: 2025-11-29) Documentation maintained in: /mnt/c/Users/fam1n/Documents/homelab/