refactor(repo): reorganize repository structure for improved navigation and maintainability
Implement comprehensive directory reorganization to improve discoverability,
logical grouping, and separation of concerns across documentation, scripts,
and infrastructure snapshots.
Major Changes:
1. Documentation Reorganization:
- Created start-here-docs/ for onboarding documentation
* Moved QUICK-START.md, START-HERE.md, GIT-SETUP-GUIDE.md
* Moved GIT-QUICK-REFERENCE.md, SCRIPT-USAGE.md, SETUP-COMPLETE.md
- Created troubleshooting/ directory
* Moved BUGFIX-SUMMARY.md for centralized issue resolution
- Created mcp/ directory for Model Context Protocol configurations
* Moved OBSIDIAN-MCP-SETUP.md to mcp/obsidian/
2. Scripts Reorganization:
- Created scripts/crawlers-exporters/ for infrastructure collection
* Moved collect*.sh scripts and collection documentation
* Consolidates Proxmox homelab export tooling
- Created scripts/fixers/ for operational repair scripts
* Moved fix_n8n_db_*.sh scripts
* Isolated scripts with embedded credentials (templates tracked)
- Created scripts/qol/ for quality-of-life utilities
* Moved git-aliases.sh and git-first-commit.sh
3. Infrastructure Snapshots:
- Created disaster-recovery/ for active infrastructure state
* Moved latest homelab-export-20251202-204939/ snapshot
* Contains current VM/CT configurations and system state
- Created archive-homelab/ for historical snapshots
* Moved homelab-export-*.tar.gz archives
* Preserves point-in-time backups for reference
4. Agent Definitions:
- Created sub-agents/ directory
* Added backend-builder.md (development agent)
* Added lab-operator.md (infrastructure operations agent)
* Added librarian.md (git/version control agent)
* Added scribe.md (documentation agent)
5. Updated INDEX.md:
- Reflects new directory structure throughout
- Updated all file path references
- Enhanced navigation with new sections
- Added agent roles documentation
- Updated quick reference commands
6. Security Improvements:
- Updated .gitignore to match reorganized file locations
- Corrected path for scripts/fixers/fix_n8n_db_c_locale.sh exclusion
- Maintained template-based credential management pattern
Infrastructure State Update:
- Latest snapshot: 2025-12-02 20:49:54
- Removed: VM 101 (gitlab), CT 112 (Anytype)
- Added: CT 113 (n8n)
- Total: 9 VMs, 3 Containers
Impact:
- Improved repository navigation and discoverability
- Logical separation of documentation, scripts, and snapshots
- Clearer onboarding path for new users
- Enhanced maintainability through organized structure
- Foundation for multi-agent workflow support
Files changed: 90 files (+935/-349)
- 3 modified, 14 new files, 73 renames/moves
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
158
scripts/crawlers-exporters/collect.sh
Normal file
158
scripts/crawlers-exporters/collect.sh
Normal file
@@ -0,0 +1,158 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
################################################################################
|
||||
# Homelab Collection Convenience Wrapper
|
||||
# Purpose: Easy-to-use wrapper that loads settings from .env file
|
||||
################################################################################
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Color codes
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
RED='\033[0;31m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
# Script directory
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# Load .env file if it exists
|
||||
if [[ -f "${SCRIPT_DIR}/.env" ]]; then
|
||||
echo -e "${BLUE}[INFO]${NC} Loading configuration from .env file..."
|
||||
# shellcheck disable=SC1091
|
||||
source "${SCRIPT_DIR}/.env"
|
||||
else
|
||||
echo -e "${YELLOW}[WARN]${NC} No .env file found. Using defaults and command-line arguments."
|
||||
echo -e "${YELLOW}[WARN]${NC} Copy .env.example to .env and customize for your environment."
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Set defaults if not in .env
|
||||
PROXMOX_HOST="${PROXMOX_HOST:-}"
|
||||
PROXMOX_SSH_USER="${PROXMOX_SSH_USER:-root}"
|
||||
PROXMOX_SSH_PORT="${PROXMOX_SSH_PORT:-22}"
|
||||
COLLECTION_LEVEL="${COLLECTION_LEVEL:-standard}"
|
||||
OUTPUT_DIR="${OUTPUT_DIR:-./exports}"
|
||||
KEEP_REMOTE="${KEEP_REMOTE:-false}"
|
||||
VERBOSE="${VERBOSE:-false}"
|
||||
|
||||
# Allow command-line override
|
||||
SANITIZE_ARG=""
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Homelab Collection Convenience Wrapper${NC}
|
||||
|
||||
${BOLD}USAGE:${NC}
|
||||
$0 [OPTIONS]
|
||||
|
||||
${BOLD}DESCRIPTION:${NC}
|
||||
Convenience wrapper that loads settings from .env file and executes
|
||||
the remote collection script.
|
||||
|
||||
${BOLD}OPTIONS:${NC}
|
||||
-h, --host HOST Override Proxmox host from .env
|
||||
-l, --level LEVEL Override collection level (basic, standard, full, paranoid)
|
||||
-s, --sanitize OPT Sanitization option (all, ips, none)
|
||||
-v, --verbose Enable verbose output
|
||||
-k, --keep-remote Keep export on remote host
|
||||
--help Show this help
|
||||
|
||||
${BOLD}CONFIGURATION:${NC}
|
||||
Settings are loaded from .env file in the same directory.
|
||||
Create .env from .env.example and customize for your environment.
|
||||
|
||||
${BOLD}EXAMPLES:${NC}
|
||||
# Use settings from .env file
|
||||
$0
|
||||
|
||||
# Override specific settings
|
||||
$0 --level full --verbose
|
||||
|
||||
# Override host
|
||||
$0 --host 192.168.1.200
|
||||
|
||||
# Full sanitization
|
||||
$0 --sanitize all
|
||||
|
||||
${BOLD}SETUP:${NC}
|
||||
1. Copy .env.example to .env
|
||||
2. Edit .env with your Proxmox details
|
||||
3. Run this script: $0
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
# Parse command-line arguments (they override .env)
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-h|--host)
|
||||
PROXMOX_HOST="$2"
|
||||
shift 2
|
||||
;;
|
||||
-l|--level)
|
||||
COLLECTION_LEVEL="$2"
|
||||
shift 2
|
||||
;;
|
||||
-s|--sanitize)
|
||||
SANITIZE_ARG="--sanitize $2"
|
||||
shift 2
|
||||
;;
|
||||
-v|--verbose)
|
||||
VERBOSE="true"
|
||||
shift
|
||||
;;
|
||||
-k|--keep-remote)
|
||||
KEEP_REMOTE="true"
|
||||
shift
|
||||
;;
|
||||
--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}[ERROR]${NC} Unknown option: $1"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Validate required settings
|
||||
if [[ -z "${PROXMOX_HOST}" ]]; then
|
||||
echo -e "${RED}[ERROR]${NC} PROXMOX_HOST not set"
|
||||
echo -e "${RED}[ERROR]${NC} Either set it in .env or use --host option"
|
||||
echo ""
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build command
|
||||
CMD=("${SCRIPT_DIR}/collect-remote.sh" "${PROXMOX_HOST}")
|
||||
CMD+=("--user" "${PROXMOX_SSH_USER}")
|
||||
CMD+=("--port" "${PROXMOX_SSH_PORT}")
|
||||
CMD+=("--level" "${COLLECTION_LEVEL}")
|
||||
CMD+=("--output" "${OUTPUT_DIR}")
|
||||
|
||||
[[ -n "${SANITIZE_ARG}" ]] && CMD+=("${SANITIZE_ARG}")
|
||||
[[ "${VERBOSE}" == "true" ]] && CMD+=("--verbose")
|
||||
[[ "${KEEP_REMOTE}" == "true" ]] && CMD+=("--keep-remote")
|
||||
|
||||
# Display configuration
|
||||
echo -e "${BOLD}${BLUE}Configuration:${NC}"
|
||||
echo -e " Host: ${PROXMOX_HOST}"
|
||||
echo -e " SSH User: ${PROXMOX_SSH_USER}"
|
||||
echo -e " SSH Port: ${PROXMOX_SSH_PORT}"
|
||||
echo -e " Level: ${COLLECTION_LEVEL}"
|
||||
echo -e " Output: ${OUTPUT_DIR}"
|
||||
echo -e " Verbose: ${VERBOSE}"
|
||||
echo -e " Keep Remote: ${KEEP_REMOTE}"
|
||||
echo ""
|
||||
|
||||
# Execute
|
||||
echo -e "${GREEN}[INFO]${NC} Executing: ${CMD[*]}"
|
||||
echo ""
|
||||
|
||||
exec "${CMD[@]}"
|
||||
Reference in New Issue
Block a user