homelab/services/README.md

Docker Compose Services

This directory contains Docker Compose configurations for various services deployed in the homelab environment.

Migration Information

Migration Date: 2025-12-02 Source: GitLab instance at https://vulcan.apophisnetworking.net/jramos/homelab Target: Gitea instance at http://192.168.2.102:3060/jramos/homelab Migration Tool: Claude Code automated migration

All service configurations have been migrated from the legacy GitLab instance to this repository as part of the infrastructure consolidation effort.

Services Overview

ByteStash

Directory: bytestash/ Port: 5000 Description: Code snippet and text snippet management system with JWT-based authentication Image: ghcr.io/jordan-dalby/bytestash:latest Key Features:

• Snippet storage and organization
• User account management
• OIDC/SSO support (configurable)
• Debug mode available

Deployment:

cd bytestash
docker compose up -d

FileBrowser

Directory: filebrowser/ Port: 8095 Description: Web-based file browser providing file management through a web interface Image: filebrowser/filebrowser:latest Key Features:

• Full filesystem access (mounted at root /)
• User and group ID configuration
• SQLite database for settings
• Customizable via settings.json

Deployment:

cd filebrowser
docker compose up -d

Note: Review volume mounts before deployment - currently configured to mount entire filesystem.

GitLab Utilities

Directory: gitlab/ Description: Quality of Life (QoL) scripts and systemd configurations for GitLab management Contents:

• QoL Scripts/sync-npm-certs.sh: Script to sync Nginx Proxy Manager certificates
• QoL Config Files/sync-npm-certs.service: Systemd service unit
• QoL Config Files/sync-npm-certs.timer: Systemd timer for automated certificate sync

Purpose: Automates certificate synchronization between Nginx Proxy Manager and GitLab instance.

Paperless-ngx

Directory: paperless-ngx/ Port: 8000 URL: https://atlas.apophisnetworking.net Description: Document management system with OCR, full-text search, and automated organization Images:

• ghcr.io/paperless-ngx/paperless-ngx:latest (webserver)
• postgres:17 (database)
• redis:8 (message broker)
• gotenberg:8.20 (document conversion)
• apache/tika:latest (text extraction)

Key Features:

• OCR for scanned documents
• Automated document processing
• Tag and organization system
• PostgreSQL backend
• Redis task queue
• Tika integration for file parsing
• Gotenberg for document conversion

Deployment:

cd paperless-ngx
docker compose up -d

Environment Configuration: Check .env file or Portainer environment variables for production deployment.

Portainer

Directory: portainer/ Ports:

• 8000 (Edge agent)
• 9443 (Web UI - HTTPS)

Description: Docker container management platform with web UI Image: portainer/portainer-ce:latest Key Features:

• Docker container management
• Stack deployment
• Image registry management
• User access control
• Remote agent support

Deployment:

cd portainer
docker compose up -d

Note: Uses external volume portainer_data - ensure volume exists before deployment.

Speedtest Tracker

Directory: speedtest-tracker/ Ports:

• 8180 (HTTP)
• 8143 (HTTPS)

Description: Automated internet speed test tracker with historical data and public dashboard Image: lscr.io/linuxserver/speedtest-tracker:latest Key Features:

• Scheduled speed tests (cron: daily at midnight)
• SQLite database
• Public dashboard view
• Historical speed test data
• LinuxServer.io image with PUID/PGID support

Deployment:

cd speedtest-tracker
docker compose up -d

Monitoring Stack (VM-based)

Deployment: VM 101 (monitoring-docker) at 192.168.2.114 Technology: Docker Compose Components: Grafana, Prometheus, PVE Exporter

Overview

Comprehensive monitoring and observability stack for the Proxmox homelab environment providing real-time metrics, visualization, and alerting capabilities.

Components

Grafana (Port 3000):

• Visualization and dashboards
• Pre-configured Proxmox VE dashboards
• User authentication and RBAC
• Alerting capabilities
• Access: http://192.168.2.114:3000

Prometheus (Port 9090):

• Metrics collection and time-series database
• PromQL query language
• 15-day retention (configurable)
• Service discovery
• Access: http://192.168.2.114:9090

PVE Exporter (Port 9221):

• Proxmox VE metrics exporter
• Connects to Proxmox API
• Exports node, VM, CT, and storage metrics
• Access: http://192.168.2.114:9221

Key Features

• Real-time Proxmox infrastructure monitoring
• VM and container resource utilization tracking
• Storage pool capacity planning
• Network traffic analysis
• Backup job status monitoring
• Custom alerting rules

Deployment

# Navigate to monitoring directory
cd /home/jramos/homelab/monitoring

# Deploy PVE Exporter
cd pve-exporter
docker compose up -d

# Deploy Prometheus
cd ../prometheus
docker compose up -d

# Deploy Grafana
cd ../grafana
docker compose up -d

# Verify all services
docker ps | grep -E 'grafana|prometheus|pve-exporter'

Configuration

PVE Exporter:

• Environment file: monitoring/pve-exporter/.env
• Configuration: monitoring/pve-exporter/pve.yml
• Requires Proxmox API user with PVEAuditor role

Prometheus:

• Configuration: monitoring/prometheus/prometheus.yml
• Scrapes PVE Exporter every 30 seconds
• Targets: localhost:9090, pve-exporter:9221

Grafana:

• Default credentials: admin/admin (change on first login)
• Data source: Prometheus at http://prometheus:9090
• Recommended dashboard: Grafana ID 10347 (Proxmox VE)

Maintenance

# Update images
cd /home/jramos/homelab/monitoring/<component>
docker compose pull
docker compose up -d

# View logs
docker compose logs -f

# Restart services
docker compose restart

Troubleshooting

PVE Exporter connection issues:

1. Verify Proxmox API is accessible: curl -k https://192.168.2.200:8006
2. Check credentials in .env file
3. Verify user has PVEAuditor role: pveum user list (on Proxmox)

Grafana shows no data:

1. Verify Prometheus data source configuration
2. Check Prometheus targets: http://192.168.2.114:9090/targets
3. Test queries in Prometheus UI before using in Grafana

High memory usage:

1. Reduce Prometheus retention period
2. Limit Grafana concurrent queries
3. Increase VM 101 memory allocation

Complete Documentation: See /home/jramos/homelab/monitoring/README.md

---

Twingate Connector

Deployment: CT 112 (twingate-connector) Technology: LXC Container Purpose: Zero-trust network access

Overview

Lightweight connector providing secure remote access to homelab resources without traditional VPN complexity. Part of Twingate's zero-trust network access (ZTNA) solution.

Features

• Zero-Trust Architecture: Grant access to specific resources, not entire networks
• No VPN Required: Simplified connection without VPN client configuration
• Identity-Based Access: User and device authentication
• Automatic Updates: Connector auto-updates for security patches
• Low Resource Overhead: Minimal CPU and memory footprint

Architecture

External User → Twingate Cloud → Twingate Connector (CT 112) → Homelab Resources

Deployment Considerations

LXC vs Docker:

• LXC chosen for lightweight, always-on service
• Minimal resource consumption
• System-level integration
• Quick restart and recovery

Network Placement:

• Deployed on homelab management network (192.168.2.0/24)
• Access to all internal resources
• No inbound port forwarding required

Configuration

The Twingate connector is configured via the Twingate Admin Console:

1. Create Connector in Twingate Admin Console
2. Generate Token for connector authentication
3. Deploy Container with provided token
4. Configure Resources to route through connector
5. Assign Users to resources

Maintenance

Health Monitoring:

• Check connector status in Twingate Admin Console
• Monitor CPU/memory usage on CT 112
• Review connection logs

Updates:

• Connector auto-updates by default
• Manual updates: Restart container or redeploy

Troubleshooting:

• Verify network connectivity to Twingate cloud
• Check connector token validity
• Review resource routing configuration
• Ensure firewall allows outbound HTTPS

Security Best Practices

1. Least Privilege: Grant access only to required resources
2. MFA Enforcement: Require multi-factor authentication for users
3. Device Trust: Enable device posture checks
4. Audit Logs: Regularly review access logs in Twingate Console
5. Connector Isolation: Consider dedicated network segment for connector

Integration with Homelab

Protected Resources:

• Proxmox Web UI (192.168.2.200:8006)
• Grafana Monitoring (192.168.2.114:3000)
• Nginx Proxy Manager (192.168.2.101:81)
• n8n Workflows (192.168.2.107:5678)
• Development VMs and services

Access Policies:

• Admin users: Full access to all resources
• Monitoring users: Read-only Grafana access
• Developers: Access to dev VMs and services

---

General Deployment Instructions

Prerequisites

• Docker Engine 20.10+
• Docker Compose v2.0+
• Sufficient disk space for volumes
• Network ports available (check port conflicts)

Standard Deployment Workflow

1. Review Configuration

   cd services/<service-name>
   cat docker-compose.yaml
   

2. Configure Environment Variables (if applicable)

   # Copy example env file if available
   cp .env.example .env
   # Edit with actual values
   nano .env
   

3. Create Required Directories

   # Ensure volume mount points exist
   # Example for bytestash:
   mkdir -p /home/jramos/docker/bytestash/data
   

4. Deploy Stack

   docker compose up -d
   

5. Verify Deployment

   docker compose ps
   docker compose logs -f
   

6. Configure Reverse Proxy (if using NPM)

   • Access Nginx Proxy Manager at http://192.168.2.101:81
   • Create proxy host pointing to service IP:PORT
   • Configure SSL certificate via Let's Encrypt
   • Set appropriate forwarding scheme (http/https)

Maintenance Commands

View Logs:

cd services/<service-name>
docker compose logs -f

Restart Service:

docker compose restart

Update Service:

docker compose pull
docker compose up -d

Stop Service:

docker compose down

Remove Service and Volumes (DESTRUCTIVE):

docker compose down -v

Directory Structure

services/
├── README.md                    # This file
├── bytestash/
│   ├── docker-compose.yaml
│   └── .gitkeep
├── filebrowser/
│   ├── docker-compose.yaml
│   └── .gitkeep
├── gitlab/
│   ├── QoL Config Files/
│   │   ├── sync-npm-certs.service
│   │   └── sync-npm-certs.timer
│   └── QoL Scripts/
│       └── sync-npm-certs.sh
├── paperless-ngx/
│   ├── docker-compose.yaml
│   └── .env
├── portainer/
│   ├── docker-compose.yaml
│   └── .gitkeep
└── speedtest-tracker/
    ├── docker-compose.yaml
    └── .gitkeep

Volume Mounts and Data Locations

Services use the following host paths for persistent data:

Service	Host Path	Purpose
ByteStash	/home/jramos/docker/bytestash/data	Snippet storage
FileBrowser	/home/docker/filebrowser/	Database and settings
Paperless-ngx	/home/jramos/paperless-ngx/consume	Document intake directory
Speedtest Tracker	/home/jramos/docker/speedtest-tracker/config	Configuration and database
Portainer	portainer_data (Docker volume)	Application data

Important: Ensure these directories exist with appropriate permissions before deploying services.

Network Configuration

All services are configured to use host networking or specific port mappings. If deploying behind Nginx Proxy Manager (CT 102 at 192.168.2.101):

1. Services should be accessible via internal IPs and ports
2. NPM handles external HTTPS access and SSL termination
3. Use http scheme in NPM when forwarding to backend services
4. Enable "Force SSL" in NPM for external HTTPS access

Security Considerations

Environment Files

• .env files are excluded from git via .gitignore
• Never commit credentials or API keys
• Use strong, unique passwords for database services
• Rotate JWT secrets and app keys regularly

Secrets in Docker Compose Files

Several services have embedded secrets in their docker-compose.yaml files:

• ByteStash: JWT_SECRET: your-secret (CHANGE THIS)
• Paperless-ngx: Database password paperless (CHANGE THIS)
• Speedtest Tracker: APP_KEY (already generated, but sensitive)

Action Required: Create .env files and move secrets out of docker-compose.yaml files.

Network Exposure

• Review port mappings before deployment
• Consider using Docker networks instead of host port binding
• Use NPM for external access with SSL
• Implement authentication on all services

Troubleshooting

Service Won't Start

1. Check logs: docker compose logs -f
2. Verify port availability: netstat -tulpn | grep <port>
3. Check volume permissions: ls -la /path/to/volume
4. Validate docker-compose.yaml syntax: docker compose config

Cannot Access Service Externally

1. Verify service is running: docker compose ps
2. Test local access: curl http://localhost:<port>
3. Check NPM proxy host configuration
4. Verify DNS resolution
5. Check firewall rules: iptables -L -n -v

Database Connection Errors (Paperless-ngx)

1. Verify PostgreSQL container is running
2. Check database credentials in environment variables
3. Ensure database initialization completed: docker compose logs db
4. Verify network connectivity between containers

Permission Denied Errors

1. Check PUID/PGID settings in docker-compose.yaml
2. Verify host directory ownership: chown -R <user>:<group> /path/to/volume
3. Check SELinux context (if applicable): ls -Z /path/to/volume

Monitoring Stack Issues

Metrics Not Appearing:

1. Verify PVE Exporter can reach Proxmox API
2. Check Prometheus scrape targets status
3. Ensure Grafana data source is configured correctly
4. Review retention policies (data may be expired)

Authentication Failures (PVE Exporter):

1. Verify Proxmox user credentials in .env file
2. Check user has PVEAuditor role
3. Test API access: curl -k https://192.168.2.200:8006/api2/json/version

High Resource Usage:

1. Adjust Prometheus retention: --storage.tsdb.retention.time=7d
2. Reduce scrape frequency in prometheus.yml
3. Limit Grafana query concurrency
4. Increase VM 101 resources if needed

Twingate Connector Issues

Connector Offline:

1. Check CT 112 is running: pct status 112
2. Verify network connectivity from container
3. Check connector token validity in Twingate Console
4. Review container logs for error messages

Cannot Access Resources:

1. Verify resource is configured in Twingate Console
2. Check user has permission to access resource
3. Ensure connector is online and healthy
4. Verify network routes on CT 112

Migration Notes

Post-Migration Tasks

• Review all .env files and update with production values
• Change default passwords and secrets in docker-compose files
• Verify volume mount paths exist on target system
• Test each service deployment individually
• Configure NPM proxy hosts for external access
• Update DNS records if service URLs changed
• Backup existing service data before redeployment
• Document any service-specific configuration changes

Known Issues

• FileBrowser: Mounts entire filesystem root - review and restrict as needed
• Paperless-ngx: Contains .env file with secrets - ensure it's excluded from git
• GitLab Utilities: May require path adjustments depending on GitLab installation location

Contributing

When adding new services to this directory:

1. Create a new subdirectory with service name (lowercase, hyphenated)
2. Include docker-compose.yaml (or docker-compose.yml)
3. Add .env.example if service requires environment variables
4. Document service in this README under "Services Overview"
5. Update directory structure diagram
6. Test deployment from scratch before committing
7. Ensure .gitignore excludes sensitive files

Additional Resources

• Docker Compose Documentation
• Nginx Proxy Manager Docs
• Proxmox Homelab Documentation
• n8n Setup Guide

Support

For homelab-specific questions or issues:

• Check existing documentation in /home/jramos/homelab/
• Review CLAUDE_STATUS.md for current infrastructure state
• Consult service-specific documentation linked in each service section

---

Last Updated: 2025-12-07 Maintainer: jramos Repository: http://192.168.2.102:3060/jramos/homelab Infrastructure: 10 VMs, 4 LXC Containers