Files
homelab/start-here-docs/QUICK-START.md
Jordan Ramos 4f69420aaa refactor(repo): reorganize repository structure for improved navigation and maintainability
Implement comprehensive directory reorganization to improve discoverability,
logical grouping, and separation of concerns across documentation, scripts,
and infrastructure snapshots.

Major Changes:

1. Documentation Reorganization:
   - Created start-here-docs/ for onboarding documentation
     * Moved QUICK-START.md, START-HERE.md, GIT-SETUP-GUIDE.md
     * Moved GIT-QUICK-REFERENCE.md, SCRIPT-USAGE.md, SETUP-COMPLETE.md
   - Created troubleshooting/ directory
     * Moved BUGFIX-SUMMARY.md for centralized issue resolution
   - Created mcp/ directory for Model Context Protocol configurations
     * Moved OBSIDIAN-MCP-SETUP.md to mcp/obsidian/

2. Scripts Reorganization:
   - Created scripts/crawlers-exporters/ for infrastructure collection
     * Moved collect*.sh scripts and collection documentation
     * Consolidates Proxmox homelab export tooling
   - Created scripts/fixers/ for operational repair scripts
     * Moved fix_n8n_db_*.sh scripts
     * Isolated scripts with embedded credentials (templates tracked)
   - Created scripts/qol/ for quality-of-life utilities
     * Moved git-aliases.sh and git-first-commit.sh

3. Infrastructure Snapshots:
   - Created disaster-recovery/ for active infrastructure state
     * Moved latest homelab-export-20251202-204939/ snapshot
     * Contains current VM/CT configurations and system state
   - Created archive-homelab/ for historical snapshots
     * Moved homelab-export-*.tar.gz archives
     * Preserves point-in-time backups for reference

4. Agent Definitions:
   - Created sub-agents/ directory
     * Added backend-builder.md (development agent)
     * Added lab-operator.md (infrastructure operations agent)
     * Added librarian.md (git/version control agent)
     * Added scribe.md (documentation agent)

5. Updated INDEX.md:
   - Reflects new directory structure throughout
   - Updated all file path references
   - Enhanced navigation with new sections
   - Added agent roles documentation
   - Updated quick reference commands

6. Security Improvements:
   - Updated .gitignore to match reorganized file locations
   - Corrected path for scripts/fixers/fix_n8n_db_c_locale.sh exclusion
   - Maintained template-based credential management pattern

Infrastructure State Update:
   - Latest snapshot: 2025-12-02 20:49:54
   - Removed: VM 101 (gitlab), CT 112 (Anytype)
   - Added: CT 113 (n8n)
   - Total: 9 VMs, 3 Containers

Impact:
   - Improved repository navigation and discoverability
   - Logical separation of documentation, scripts, and snapshots
   - Clearer onboarding path for new users
   - Enhanced maintainability through organized structure
   - Foundation for multi-agent workflow support

Files changed: 90 files (+935/-349)
   - 3 modified, 14 new files, 73 renames/moves

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:39:33 -07:00

9.5 KiB

Homelab Collection - Quick Start Guide

Welcome! This guide will get you up and running with automated homelab infrastructure collection in under 5 minutes.

What This Does

Automatically collects your entire Proxmox homelab configuration including:

  • ✓ All VM configurations
  • ✓ All LXC container configurations
  • ✓ Storage pool definitions
  • ✓ Network configurations
  • ✓ Backup job schedules
  • ✓ System information
  • ✓ Cluster resources

Completely safe - read-only operations, no modifications to your system.

Prerequisites Checklist

  • Proxmox VE server running (yours: serviceslab)
  • SSH access to Proxmox from WSL
  • Proxmox IP address or hostname

5-Minute Setup

Step 1: Test SSH Access

# Replace with your Proxmox IP
ssh root@192.168.1.100

If you get a password prompt, set up SSH keys (recommended):

ssh-keygen -t ed25519 -f ~/.ssh/homelab
ssh-copy-id -i ~/.ssh/homelab root@192.168.1.100

Step 2: Configure Your Environment

cd /mnt/c/Users/fam1n/Documents/homelab

# Copy the example configuration
cp .env.example .env

# Edit with your Proxmox details
nano .env  # or use 'code .env' for VS Code

Minimum required change in .env:

PROXMOX_HOST="192.168.1.100"  # Change to your Proxmox IP

Step 3: Run Your First Collection

# Simple! Just run:
bash collect.sh

That's it! The script will:

  1. SSH to your Proxmox server
  2. Collect all configurations
  3. Download results to ./exports/
  4. Show you a summary

Step 4: Review Your Export

# View the summary
cat exports/homelab-export-*/SUMMARY.md

# Browse collected configs
ls -la exports/homelab-export-*/configs/

What Just Happened?

Your infrastructure is now documented! Here's what you have:

exports/homelab-export-20241128-143000/
├── configs/
│   ├── proxmox/          ← Core Proxmox settings
│   ├── vms/              ← All your VMs (100-docker-hub, 101-gitlab, etc.)
│   ├── lxc/              ← All containers (102-nginx, 103-netbox, etc.)
│   ├── storage/          ← Storage pools (Vault, PBS-Backups, etc.)
│   ├── network/          ← Network configuration
│   └── backup/           ← Backup jobs
├── exports/
│   ├── system/           ← System info (CPU, RAM, disk, etc.)
│   └── cluster/          ← Resource usage
├── README.md             ← Complete documentation
└── SUMMARY.md            ← What was collected

Next Steps

Immediate Actions

  1. Review your infrastructure

    # See your VM configs
    cat exports/homelab-export-*/configs/vms/*.conf
    
    # Check storage setup
    cat exports/homelab-export-*/configs/proxmox/storage.cfg
    
  2. Store safely

    # Initialize git repository
    cd exports/homelab-export-*/
    git init
    git add .
    git commit -m "Initial homelab snapshot"
    
  3. Schedule automatic collections (see Automation section below)

Common Commands

# Standard collection (default)
bash collect.sh

# Full collection with everything
bash collect.sh --level full --verbose

# Quick basic snapshot
bash collect.sh --level basic

# Sanitized export (safe for sharing)
bash collect.sh --sanitize all

# Override Proxmox host
bash collect.sh --host 192.168.1.200

Automation

Weekly Snapshots

On your Proxmox server, run:

# SSH to Proxmox
ssh root@your-proxmox-ip

# Copy the collection script
cat > /root/collect-homelab-config.sh
# Paste the script content, then Ctrl+D

# Make executable
chmod +x /root/collect-homelab-config.sh

# Add to crontab for weekly Sunday 3 AM runs
crontab -e

Add this line:

0 3 * * 0 /root/collect-homelab-config.sh -l standard -o /backup/homelab/weekly-$(date +\%Y\%U) 2>&1 | logger -t homelab-collection

Understanding Your Infrastructure

Now that you have the export, here's what you can learn:

Your Virtual Machines

Based on your environment, you should see configs for:

  • 100-docker-hub.conf - Your container registry
  • 101-gitlab.conf - GitLab CE/EE instance
  • 105-dev.conf - Development environment
  • 106-Ansible-Control.conf - Automation control node
  • 108-CML.conf - Cisco Modeling Labs
  • 109-web-server-01.conf - Web server (clustered)
  • 110-web-server-02.conf - Web server (clustered)
  • 111-db-server-01.conf - Database server

Your Containers

  • 102-nginx.conf - Reverse proxy/load balancer
  • 103-netbox.conf - Network documentation/IPAM
  • 112-Anytype.conf - Knowledge management

Your Storage

Check configs/proxmox/storage.cfg to see:

  • local (Directory) - System storage
  • local-lvm (LVM-Thin) - VM disk images
  • Vault (NFS/Directory) - Secure storage
  • PBS-Backups (Proxmox Backup Server) - Backup repository
  • iso-share (NFS/CIFS) - Installation media

Files in This Directory

File Purpose
collect.sh Start here - Easy wrapper that uses .env
collect-remote.sh Advanced - Direct remote execution
collect-homelab-config.sh Core script (runs on Proxmox)
.env.example Configuration template
.env Your configuration (create from .env.example)
QUICK-START.md This file
README-COLLECTION.md Detailed overview
COLLECTION-GUIDE.md Complete reference manual

Troubleshooting

"Cannot connect to Proxmox"

# Test connection
ping 192.168.1.100
ssh root@192.168.1.100

# If password prompt, set up keys:
ssh-keygen -t ed25519
ssh-copy-id root@192.168.1.100

"PROXMOX_HOST not set"

Edit your .env file:

nano .env
# Set PROXMOX_HOST="your-ip"

"Permission denied"

The scripts might not be executable on Windows filesystems. That's OK! When you run:

bash collect.sh

This explicitly uses bash to execute the script.

Need More Help?

  1. Quick reference: cat README-COLLECTION.md
  2. Complete guide: cat COLLECTION-GUIDE.md
  3. Script help: bash collect-remote.sh --help

Security Notes

What's Sanitized by Default

  • ✓ Passwords (replaced with <REDACTED>)
  • ✓ API tokens (replaced with <REDACTED>)
  • ✗ IP addresses (kept for documentation)

Full Sanitization

For exports leaving your network:

bash collect.sh --sanitize all

This will also replace IP addresses with 10.X.X.X.

Keep It Secure

# Store exports securely
chmod 700 exports/

# Use private Git repositories
# Never commit unsanitized exports to public repos

# Encrypt sensitive exports
gpg --symmetric exports/homelab-export-*.tar.gz

Use Cases

1. Documentation

"What's running on VM 109?" → Check configs/vms/109-web-server-01.conf

2. Disaster Recovery

If Proxmox crashes, you have all configurations to rebuild

3. Change Tracking

# Compare this month vs last month
diff -u exports/january/ exports/february/

4. Infrastructure as Code

Use the collected configs as templates for Terraform/Ansible

5. Compliance/Audit

"What was the configuration on date X?" → Check that export

Advanced Usage

Different Collection Levels

# Minimal - just configs
bash collect.sh --level basic

# Standard - configs + status (default)
bash collect.sh --level standard

# Everything - including service configs
bash collect.sh --level full

# Maximum detail
bash collect.sh --level paranoid

Custom Output Location

# Save to specific directory
bash collect.sh --output ~/backups/homelab

Keep Remote Copy

# Don't delete from Proxmox after download
bash collect.sh --keep-remote

Verbose Output

# See detailed progress
bash collect.sh --verbose

Success Indicators

After running, you should see:

[✓] SSH connection successful
[✓] Confirmed Proxmox VE installation
[✓] Script uploaded successfully
[✓] Collection completed successfully
[✓] Archive downloaded successfully
[✓] Archive extracted

And in your summary:

Total items collected: 45+
Total items skipped: 0-5  (normal)
Total errors: 0

What's Next?

  1. Explore your export - Browse the configs directory
  2. Set up Git - Version control your infrastructure
  3. Schedule automation - Weekly/monthly snapshots
  4. Build IaC - Convert to Terraform/Ansible
  5. Document - Add notes to the docs/ folder in each export
  6. Share - Use sanitized exports to get help or show off your setup

Complete Example Workflow

# 1. Setup (one-time)
cd /mnt/c/Users/fam1n/Documents/homelab
cp .env.example .env
nano .env  # Set PROXMOX_HOST

# 2. Collect
bash collect.sh

# 3. Review
cat exports/homelab-export-*/SUMMARY.md
ls -R exports/homelab-export-*/

# 4. Store
cd exports/homelab-export-*/
git init
git add .
git commit -m "Homelab snapshot $(date +%Y-%m-%d)"

# 5. Automate (on Proxmox)
ssh root@your-proxmox-ip
# Set up cron job (see Automation section)

# 6. Repeat monthly
bash collect.sh --level full

You're All Set!

You now have:

  • ✓ Complete infrastructure documentation
  • ✓ Automated collection capability
  • ✓ Disaster recovery reference
  • ✓ Foundation for Infrastructure as Code
  • ✓ Change tracking capability

Run bash collect.sh anytime to refresh your documentation.


Quick Reference Card:

# Basic collection with defaults
bash collect.sh

# Full collection, verbose, sanitized
bash collect.sh --level full --sanitize all --verbose

# Quick snapshot
bash collect.sh --level basic

# Help
bash collect.sh --help
bash collect-remote.sh --help

Need more details? See COLLECTION-GUIDE.md for comprehensive documentation.

Ready to proceed? Just run: bash collect.sh