Files
homelab/scripts
Jordan Ramos 4f69420aaa refactor(repo): reorganize repository structure for improved navigation and maintainability
Implement comprehensive directory reorganization to improve discoverability,
logical grouping, and separation of concerns across documentation, scripts,
and infrastructure snapshots.

Major Changes:

1. Documentation Reorganization:
   - Created start-here-docs/ for onboarding documentation
     * Moved QUICK-START.md, START-HERE.md, GIT-SETUP-GUIDE.md
     * Moved GIT-QUICK-REFERENCE.md, SCRIPT-USAGE.md, SETUP-COMPLETE.md
   - Created troubleshooting/ directory
     * Moved BUGFIX-SUMMARY.md for centralized issue resolution
   - Created mcp/ directory for Model Context Protocol configurations
     * Moved OBSIDIAN-MCP-SETUP.md to mcp/obsidian/

2. Scripts Reorganization:
   - Created scripts/crawlers-exporters/ for infrastructure collection
     * Moved collect*.sh scripts and collection documentation
     * Consolidates Proxmox homelab export tooling
   - Created scripts/fixers/ for operational repair scripts
     * Moved fix_n8n_db_*.sh scripts
     * Isolated scripts with embedded credentials (templates tracked)
   - Created scripts/qol/ for quality-of-life utilities
     * Moved git-aliases.sh and git-first-commit.sh

3. Infrastructure Snapshots:
   - Created disaster-recovery/ for active infrastructure state
     * Moved latest homelab-export-20251202-204939/ snapshot
     * Contains current VM/CT configurations and system state
   - Created archive-homelab/ for historical snapshots
     * Moved homelab-export-*.tar.gz archives
     * Preserves point-in-time backups for reference

4. Agent Definitions:
   - Created sub-agents/ directory
     * Added backend-builder.md (development agent)
     * Added lab-operator.md (infrastructure operations agent)
     * Added librarian.md (git/version control agent)
     * Added scribe.md (documentation agent)

5. Updated INDEX.md:
   - Reflects new directory structure throughout
   - Updated all file path references
   - Enhanced navigation with new sections
   - Added agent roles documentation
   - Updated quick reference commands

6. Security Improvements:
   - Updated .gitignore to match reorganized file locations
   - Corrected path for scripts/fixers/fix_n8n_db_c_locale.sh exclusion
   - Maintained template-based credential management pattern

Infrastructure State Update:
   - Latest snapshot: 2025-12-02 20:49:54
   - Removed: VM 101 (gitlab), CT 112 (Anytype)
   - Added: CT 113 (n8n)
   - Total: 9 VMs, 3 Containers

Impact:
   - Improved repository navigation and discoverability
   - Logical separation of documentation, scripts, and snapshots
   - Clearer onboarding path for new users
   - Enhanced maintainability through organized structure
   - Foundation for multi-agent workflow support

Files changed: 90 files (+935/-349)
   - 3 modified, 14 new files, 73 renames/moves

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 21:39:33 -07:00
..

Homelab Infrastructure Scripts

This directory contains operational scripts for maintaining and troubleshooting homelab infrastructure services.

Directory Structure

scripts/
├── README.md                          # This file
├── fix_n8n_db_c_locale.sh.template    # Template for PostgreSQL fix (C.utf8 locale)
├── fix_n8n_db_c_locale.sh             # Active script (excluded from git)
├── fix_n8n_db_permissions.sh          # Legacy PostgreSQL permission fix for n8n
└── crawlers-exporters/                # Data export and migration tools
    ├── export_cf_dns.py               # Cloudflare DNS configuration export
    ├── cloudflare_dns_export.json     # Example DNS records export
    └── cloudflare_full_config.json    # Example full config export

Scripts

fix_n8n_db_permissions.sh

Purpose: Fix PostgreSQL 15+ permission issues for n8n database

Background: PostgreSQL 15+ removed default CREATE permission from the PUBLIC role on the 'public' schema. This breaking change causes n8n database migrations to fail with "permission denied for schema public" errors.

What it does:

  1. Creates timestamped backup of existing n8n database
  2. Drops and recreates database with proper ownership (OWNER n8n_user)
  3. Grants explicit schema permissions for PostgreSQL 15+ compatibility
  4. Tests permissions by creating and dropping a test table
  5. Restarts n8n service and verifies successful startup

Usage:

# Method 1: Set password via environment variable (recommended)
export N8N_DB_PASSWORD='your_password_here'
bash fix_n8n_db_permissions.sh

# Method 2: Edit DB_PASSWORD in script directly
# Edit line 28 to replace YOUR_DB_PASSWORD_HERE with actual password
bash fix_n8n_db_permissions.sh

Requirements:

  • Must run as root
  • PostgreSQL service must be running
  • n8n service must be installed

Output:

  • Database backup: /var/backups/n8n/n8n_db_backup_YYYYMMDD_HHMMSS.sql
  • Log file: /var/log/n8n_db_fix_YYYYMMDD_HHMMSS.log

Expected Runtime: 15-30 seconds

See Also:

  • Complete troubleshooting documentation: /home/jramos/homelab/CLAUDE_STATUS.md (section: "Post-Deployment Troubleshooting")
  • n8n setup documentation: /home/jramos/homelab/n8n/N8N-SETUP-PLAN.md

fix_n8n_db_c_locale.sh (Template-Based)

Purpose: Fix PostgreSQL 15+ permission issues for n8n database with correct Debian 12 locale support

Background:

  • PostgreSQL 15+ removed default CREATE permission from the PUBLIC role
  • Debian 12 minimal LXC containers only have C.utf8 locale available (lowercase)
  • Previous scripts used incorrect locale names (en_US.UTF-8 or C.UTF-8) causing database creation failures

Template-Based Security: This script uses a template approach to avoid committing credentials to git:

  • fix_n8n_db_c_locale.sh.template - Tracked in git (no credentials)
  • fix_n8n_db_c_locale.sh - Active script (excluded from git via .gitignore)

Setup Instructions:

# 1. Copy the template to create your working script
cp fix_n8n_db_c_locale.sh.template fix_n8n_db_c_locale.sh

# 2. Set the database password via environment variable (recommended)
export N8N_DB_PASSWORD='your_secure_password_here'

# 3. Run the script
bash fix_n8n_db_c_locale.sh

# 4. Securely delete the script after use (contains credentials in SQL)
shred -u fix_n8n_db_c_locale.sh

What it does:

  1. Validates that N8N_DB_PASSWORD environment variable is set
  2. Creates timestamped backup of existing n8n database (if exists)
  3. Drops and recreates database with:
    • Locale: C.utf8 (matches Debian 12 minimal system)
    • Owner: n8n_user
    • Encoding: UTF8
  4. Grants PostgreSQL 15+ required permissions:
    • GRANT ALL ON SCHEMA public TO n8n_user
    • GRANT CREATE ON SCHEMA public TO n8n_user
  5. Tests permissions by creating/dropping a test table
  6. Restarts n8n service and verifies successful startup

Requirements:

  • Must run as root (or with sudo)
  • PostgreSQL service must be running
  • n8n service must be installed
  • N8N_DB_PASSWORD environment variable must be set

Output:

  • Log file: /var/log/n8n_db_fix_YYYYMMDD_HHMMSS.log
  • Database settings verification in log output

Expected Runtime: 20-40 seconds

Security Notes:

  • Always use environment variables for credentials (never hardcode)
  • The script contains the password in embedded SQL - delete after use
  • Use shred -u instead of rm to securely delete the file
  • The template file is safe to commit (contains no credentials)

Differences from fix_n8n_db_permissions.sh:

  • Uses C.utf8 locale instead of en_US.UTF-8
  • Validates environment variable is set before running
  • Designed for Debian 12 minimal LXC containers
  • Includes reminder to delete script after use

See Also:

  • Complete troubleshooting documentation: /home/jramos/homelab/CLAUDE_STATUS.md
  • n8n setup documentation: /home/jramos/homelab/n8n/N8N-SETUP-PLAN.md
  • PostgreSQL 15 breaking changes: Release Notes

export_cf_dns.py

Purpose: Export Cloudflare DNS configuration and zone settings for backup or migration

What it does:

  1. Fetches all DNS records from specified Cloudflare zone (with pagination support)
  2. Retrieves key zone settings (SSL mode, TLS version, websockets, etc.)
  3. Exports combined configuration to JSON file
  4. Provides clean, structured output for infrastructure-as-code workflows

Usage:

# Method 1: Set credentials via environment variables (recommended)
export CF_ZONE_ID='your_zone_id_here'
export CF_API_TOKEN='your_api_token_here'
python3 export_cf_dns.py

# Method 2: Edit credentials in script directly
# Edit lines 7-8 to replace placeholders with actual credentials
python3 export_cf_dns.py

Requirements:

  • Python 3.6+
  • requests library: pip install requests
  • Cloudflare API token with Zone:Read permissions
  • Cloudflare Zone ID for the target domain

Output:

  • cloudflare_full_config.json - Combined DNS records and zone settings

Example Output Structure:

{
  "metadata": {
    "zone_id": "abc123...",
    "export_date": "Now"
  },
  "zone_settings": {
    "ssl": "strict",
    "always_use_https": "on",
    "min_tls_version": "1.2",
    "websockets": "on"
  },
  "dns_records": [
    {
      "name": "example.com",
      "type": "A",
      "content": "192.168.1.1",
      "proxied": true,
      "ttl": 1
    }
  ]
}

Use Cases:

  • Backup DNS configuration before major changes
  • Document current DNS state for disaster recovery
  • Export for migration to another Cloudflare account
  • Generate infrastructure-as-code templates

Security Notes

Template-Based Script Pattern

This repository uses a template-based approach for scripts containing sensitive data:

  1. Template files (.template extension): Tracked in git, contain placeholder variables
  2. Active scripts: Excluded from git via .gitignore, contain actual credentials

Workflow:

# Copy template to create working script
cp script_name.sh.template script_name.sh

# Set credentials via environment variables
export VARIABLE_NAME='actual_value'

# Run the script
bash script_name.sh

# Securely delete after use
shred -u script_name.sh  # Overwrites and deletes

Benefits:

  • Repository stays credential-free
  • Templates serve as documentation
  • Easy to recreate scripts when needed
  • Supports version control of script logic without exposing secrets

General Security Guidelines

  • Scripts in this directory may require credentials to be set via environment variables
  • Never commit scripts containing plaintext passwords to version control
  • Use .gitignore to exclude credential-containing variants
  • Delete or shred scripts with embedded credentials after use
  • Always use shred -u instead of rm for files containing credentials
  • Prefer environment variables over hardcoded credentials
  • Use dedicated service accounts with minimal permissions

Contributing

When adding new scripts:

  1. Include comprehensive header comments explaining purpose and usage
  2. Parameterize credentials (use environment variables or prompts)
  3. Add error handling and logging
  4. Document in this README
  5. Follow bash best practices (set -euo pipefail, quote variables, etc.)