Initial commit: Homelab infrastructure repository with automated collection system

- Added Proxmox VE configuration collection scripts - Included documentation and quick-start guides - First infrastructure snapshot from serviceslab (2025-11-29) - All VM configs (10 VMs) and LXC configs (3 containers) - Git setup complete with .gitignore protecting sensitive data
2025-11-29 15:55:56 -07:00
commit 4b62fb0a27
75 changed files with 9497 additions and 0 deletions
--- a/README-COLLECTION.md
+++ b/README-COLLECTION.md
@@ -0,0 +1,341 @@
+# Homelab Infrastructure Collection Scripts
+
+Automated collection tools for documenting and backing up your Proxmox VE homelab infrastructure.
+
+## Quick Start
+
+### From WSL2 (Your Current Environment)
+
+The easiest way to collect your Proxmox configuration from WSL2:
+
+```bash
+# 1. Make scripts executable
+chmod +x collect-homelab-config.sh collect-remote.sh
+
+# 2. Set your Proxmox IP address
+PROXMOX_IP="192.168.1.100"  # Replace with your actual IP
+
+# 3. Run the remote collection
+./collect-remote.sh $PROXMOX_IP
+
+# 4. Review results
+ls -la homelab-export-*/
+cat homelab-export-*/SUMMARY.md
+```
+
+That's it! The script will SSH to your Proxmox host, collect all configurations, and download them to your WSL environment.
+
+### Directly on Proxmox Host
+
+If you prefer to run directly on the Proxmox server:
+
+```bash
+# 1. Copy script to Proxmox
+scp collect-homelab-config.sh root@<proxmox-ip>:/root/
+
+# 2. SSH to Proxmox
+ssh root@<proxmox-ip>
+
+# 3. Run collection
+chmod +x /root/collect-homelab-config.sh
+./collect-homelab-config.sh
+
+# 4. Copy results back to WSL
+# From WSL:
+scp -r root@<proxmox-ip>:/root/homelab-export-* ./
+```
+
+## What You Get
+
+After running the collection script, you'll have:
+
+```
+homelab-export-20241128-143022/
+├── README.md              # Detailed documentation
+├── SUMMARY.md            # Collection statistics
+├── collection.log        # Detailed log
+├── configs/
+│   ├── proxmox/         # Core Proxmox settings
+│   ├── vms/             # All VM configurations
+│   ├── lxc/             # All container configurations
+│   ├── storage/         # Storage pool configs
+│   ├── network/         # Network interface configs
+│   └── backup/          # Backup job definitions
+└── exports/
+    ├── system/          # System information
+    ├── cluster/         # Cluster status
+    └── guests/          # VM/container lists
+```
+
+## Files in This Directory
+
+| File | Purpose |
+|------|---------|
+| `collect-homelab-config.sh` | Main collection script (runs on Proxmox host) |
+| `collect-remote.sh` | Wrapper for remote execution from WSL/Linux |
+| `COLLECTION-GUIDE.md` | Comprehensive usage guide and reference |
+| `README-COLLECTION.md` | This file - quick start guide |
+
+## Common Use Cases
+
+### Daily Documentation Snapshot
+
+```bash
+# Run from WSL - executes on Proxmox and brings results back
+./collect-remote.sh 192.168.1.100 --level standard
+```
+
+### Pre-Maintenance Full Backup
+
+```bash
+# Full collection before making changes
+./collect-remote.sh 192.168.1.100 --level full --keep-remote
+```
+
+### Sanitized Export for Sharing
+
+```bash
+# Remove all sensitive data (IPs, passwords, tokens)
+./collect-remote.sh 192.168.1.100 --sanitize all
+```
+
+### Weekly Automated Collection
+
+On your Proxmox host, add to crontab:
+
+```bash
+# Edit crontab
+sudo crontab -e
+
+# Add weekly Sunday 3 AM collection
+0 3 * * 0 /root/collect-homelab-config.sh -l standard -o /backup/homelab/weekly-$(date +\%Y\%W)
+```
+
+## Collection Levels
+
+| Level | What It Includes | Best For |
+|-------|------------------|----------|
+| **basic** | System info, Proxmox configs, VM/LXC configs, network | Quick snapshots |
+| **standard** | Basic + storage, backups, cluster info | Regular documentation |
+| **full** | Standard + service configs, detailed state | Pre-maintenance |
+| **paranoid** | Everything possible | Maximum detail |
+
+## Prerequisites
+
+### SSH Access (Required for Remote Collection)
+
+Set up passwordless SSH authentication:
+
+```bash
+# Generate SSH key (if you don't have one)
+ssh-keygen -t ed25519
+
+# Copy to Proxmox host
+ssh-copy-id root@<proxmox-ip>
+
+# Test connection
+ssh root@<proxmox-ip> 'hostname'
+```
+
+### Disk Space
+
+Ensure adequate space:
+- On Proxmox: ~100-500 MB (depending on collection level)
+- On WSL: ~50-250 MB for compressed archives
+
+### Permissions
+
+The collection script needs root access on the Proxmox host to read all configurations.
+
+## Security Considerations
+
+### Default Sanitization
+
+By default, the script sanitizes:
+- ✓ Passwords
+- ✓ API tokens
+- ✗ IP addresses (preserved for documentation)
+
+### Full Sanitization
+
+For exports that leave your network:
+
+```bash
+./collect-remote.sh <proxmox-ip> --sanitize all
+```
+
+This will redact:
+- Passwords
+- API tokens
+- IP addresses (replaced with 10.x.x.x)
+
+### Storage Recommendations
+
+1. **Keep unsanitized exports secure** - store in encrypted locations
+2. **Use private Git repositories** - never commit to public repos without sanitization
+3. **Encrypt sensitive exports**:
+   ```bash
+   gpg --symmetric --cipher-algo AES256 homelab-export-*.tar.gz
+   ```
+
+## Next Steps
+
+1. **Run your first collection** using the Quick Start steps above
+
+2. **Review the comprehensive guide**:
+   ```bash
+   cat COLLECTION-GUIDE.md
+   ```
+
+3. **Examine your infrastructure**:
+   ```bash
+   # View VM configurations
+   cat homelab-export-*/configs/vms/*.conf
+
+   # Check storage setup
+   cat homelab-export-*/configs/proxmox/storage.cfg
+
+   # Review network configuration
+   cat homelab-export-*/configs/network/interfaces
+   ```
+
+4. **Set up version control**:
+   ```bash
+   cd homelab-export-<timestamp>/
+   git init
+   git add .
+   git commit -m "Initial infrastructure snapshot"
+   git remote add origin <your-private-repo>
+   git push -u origin main
+   ```
+
+5. **Schedule regular collections** (see COLLECTION-GUIDE.md for automation)
+
+6. **Use exports to build Infrastructure as Code** (Terraform, Ansible)
+
+## Troubleshooting
+
+### "Cannot connect to Proxmox host"
+
+Check:
+- Is the IP address correct?
+- Is SSH running on Proxmox? `ssh root@<proxmox-ip>`
+- Can you reach the host? `ping <proxmox-ip>`
+- Is firewall blocking port 22?
+
+### "Permission denied"
+
+Solutions:
+- Ensure scripts are executable: `chmod +x *.sh`
+- Use root user: `./collect-remote.sh 192.168.1.100 --user root`
+- Set up SSH keys: `ssh-copy-id root@<proxmox-ip>`
+
+### "Not a Proxmox VE host"
+
+This script requires a Proxmox VE installation. Verify:
+```bash
+ssh root@<proxmox-ip> 'cat /etc/pve/.version'
+```
+
+### "Some items skipped"
+
+This is normal! Review `SUMMARY.md` to see what was skipped. Common skipped items:
+- ZFS tools (if not using ZFS)
+- Cluster configs (on single-node setups)
+- HA configs (if High Availability not configured)
+
+### Need more help?
+
+1. Run with verbose flag: `./collect-remote.sh <proxmox-ip> -v`
+2. Check the log: `cat homelab-export-*/collection.log`
+3. Review the full guide: `cat COLLECTION-GUIDE.md`
+
+## Examples
+
+### Standard Collection from WSL
+
+```bash
+./collect-remote.sh 192.168.1.100
+```
+
+### Full Collection with Verbose Output
+
+```bash
+./collect-remote.sh 192.168.1.100 --level full --verbose
+```
+
+### Custom SSH Port and User
+
+```bash
+./collect-remote.sh proxmox.local --port 2222 --user admin
+```
+
+### Keep Remote Copy
+
+```bash
+./collect-remote.sh 192.168.1.100 --keep-remote
+```
+
+### Sanitized Export to Custom Location
+
+```bash
+./collect-remote.sh 192.168.1.100 --sanitize all --output ~/Documents/homelab-backups
+```
+
+## File Sizes
+
+Approximate sizes for reference:
+
+| Collection Level | Uncompressed | Compressed |
+|------------------|--------------|------------|
+| Basic | 5-15 MB | 1-3 MB |
+| Standard | 10-30 MB | 2-6 MB |
+| Full | 20-100 MB | 5-20 MB |
+| Paranoid | 50-500 MB | 10-50 MB |
+
+*Actual sizes vary based on number of VMs/containers and configurations*
+
+## Support
+
+For detailed information on any topic:
+
+- **Complete Usage Guide**: See `COLLECTION-GUIDE.md`
+- **Script Help**: Run `./collect-homelab-config.sh --help`
+- **Remote Wrapper Help**: Run `./collect-remote.sh --help`
+
+## What's Collected vs. What's NOT
+
+### ✓ Collected (READ-ONLY)
+- Configuration files
+- System information
+- Resource lists
+- Status information
+- Network settings
+- Storage definitions
+
+### ✗ NOT Collected
+- Actual VM/container disk images
+- ISO files
+- Backup archives
+- User data within VMs/containers
+- Authentication credentials (sanitized)
+
+**Note:** This script collects *configuration* and *metadata*, not actual data. For complete backups, use Proxmox Backup Server or vzdump.
+
+## Integration Ideas
+
+Once you have collected your infrastructure:
+
+1. **Documentation**: Use as source of truth for runbooks
+2. **Change Tracking**: Diff exports to see what changed over time
+3. **Disaster Recovery**: Reference during rebuilds
+4. **IaC Development**: Convert to Terraform/Ansible
+5. **Compliance**: Evidence of configuration state
+6. **Training**: Sandbox for learning without risk
+7. **Migration Planning**: Understand dependencies before moving
+
+---
+
+**Version:** 1.0.0
+**Last Updated:** 2024-11-28
+**Maintained By:** Your homelab automation assistant, Steve