feat(services): migrate Docker Compose configurations from GitLab
Migrate all docker-compose service configurations from legacy GitLab instance to homelab repository for consolidation and version control. Services migrated (6): - bytestash: Code snippet management - filebrowser: Web-based file browser - gitlab: QoL scripts for NPM cert sync - paperless-ngx: Document management with OCR - portainer: Docker management UI - speedtest-tracker: Internet speed test tracker Changes: - Add services/ directory with complete configurations - Update .gitignore with Docker Compose exclusions - Create comprehensive services/README.md documentation - Document migration process in CLAUDE_STATUS.md Migration details: - Source: https://vulcan.apophisnetworking.net/jramos/homelab.git - Files migrated: 10 files (6 compose + 3 utilities + 1 README) - Total size: 84 KB - Lines added: 836 Security notes: - .env files excluded from git - Hardcoded secrets identified in documentation - Review and update secrets before deployment Related: GitLab VM 101 decommissioning preparation 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
358
services/README.md
Normal file
358
services/README.md
Normal file
@@ -0,0 +1,358 @@
|
||||
# Docker Compose Services
|
||||
|
||||
This directory contains Docker Compose configurations for various services deployed in the homelab environment.
|
||||
|
||||
## Migration Information
|
||||
|
||||
**Migration Date**: 2025-12-02
|
||||
**Source**: GitLab instance at https://vulcan.apophisnetworking.net/jramos/homelab
|
||||
**Target**: Gitea instance at http://192.168.2.102:3060/jramos/homelab
|
||||
**Migration Tool**: Claude Code automated migration
|
||||
|
||||
All service configurations have been migrated from the legacy GitLab instance to this repository as part of the infrastructure consolidation effort.
|
||||
|
||||
## Services Overview
|
||||
|
||||
### ByteStash
|
||||
**Directory**: `bytestash/`
|
||||
**Port**: 5000
|
||||
**Description**: Code snippet and text snippet management system with JWT-based authentication
|
||||
**Image**: ghcr.io/jordan-dalby/bytestash:latest
|
||||
**Key Features**:
|
||||
- Snippet storage and organization
|
||||
- User account management
|
||||
- OIDC/SSO support (configurable)
|
||||
- Debug mode available
|
||||
|
||||
**Deployment**:
|
||||
```bash
|
||||
cd bytestash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### FileBrowser
|
||||
**Directory**: `filebrowser/`
|
||||
**Port**: 8095
|
||||
**Description**: Web-based file browser providing file management through a web interface
|
||||
**Image**: filebrowser/filebrowser:latest
|
||||
**Key Features**:
|
||||
- Full filesystem access (mounted at root `/`)
|
||||
- User and group ID configuration
|
||||
- SQLite database for settings
|
||||
- Customizable via settings.json
|
||||
|
||||
**Deployment**:
|
||||
```bash
|
||||
cd filebrowser
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**Note**: Review volume mounts before deployment - currently configured to mount entire filesystem.
|
||||
|
||||
### GitLab Utilities
|
||||
**Directory**: `gitlab/`
|
||||
**Description**: Quality of Life (QoL) scripts and systemd configurations for GitLab management
|
||||
**Contents**:
|
||||
- `QoL Scripts/sync-npm-certs.sh`: Script to sync Nginx Proxy Manager certificates
|
||||
- `QoL Config Files/sync-npm-certs.service`: Systemd service unit
|
||||
- `QoL Config Files/sync-npm-certs.timer`: Systemd timer for automated certificate sync
|
||||
|
||||
**Purpose**: Automates certificate synchronization between Nginx Proxy Manager and GitLab instance.
|
||||
|
||||
### Paperless-ngx
|
||||
**Directory**: `paperless-ngx/`
|
||||
**Port**: 8000
|
||||
**URL**: https://atlas.apophisnetworking.net
|
||||
**Description**: Document management system with OCR, full-text search, and automated organization
|
||||
**Images**:
|
||||
- ghcr.io/paperless-ngx/paperless-ngx:latest (webserver)
|
||||
- postgres:17 (database)
|
||||
- redis:8 (message broker)
|
||||
- gotenberg:8.20 (document conversion)
|
||||
- apache/tika:latest (text extraction)
|
||||
|
||||
**Key Features**:
|
||||
- OCR for scanned documents
|
||||
- Automated document processing
|
||||
- Tag and organization system
|
||||
- PostgreSQL backend
|
||||
- Redis task queue
|
||||
- Tika integration for file parsing
|
||||
- Gotenberg for document conversion
|
||||
|
||||
**Deployment**:
|
||||
```bash
|
||||
cd paperless-ngx
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**Environment Configuration**: Check `.env` file or Portainer environment variables for production deployment.
|
||||
|
||||
### Portainer
|
||||
**Directory**: `portainer/`
|
||||
**Ports**:
|
||||
- 8000 (Edge agent)
|
||||
- 9443 (Web UI - HTTPS)
|
||||
|
||||
**Description**: Docker container management platform with web UI
|
||||
**Image**: portainer/portainer-ce:latest
|
||||
**Key Features**:
|
||||
- Docker container management
|
||||
- Stack deployment
|
||||
- Image registry management
|
||||
- User access control
|
||||
- Remote agent support
|
||||
|
||||
**Deployment**:
|
||||
```bash
|
||||
cd portainer
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**Note**: Uses external volume `portainer_data` - ensure volume exists before deployment.
|
||||
|
||||
### Speedtest Tracker
|
||||
**Directory**: `speedtest-tracker/`
|
||||
**Ports**:
|
||||
- 8180 (HTTP)
|
||||
- 8143 (HTTPS)
|
||||
|
||||
**Description**: Automated internet speed test tracker with historical data and public dashboard
|
||||
**Image**: lscr.io/linuxserver/speedtest-tracker:latest
|
||||
**Key Features**:
|
||||
- Scheduled speed tests (cron: daily at midnight)
|
||||
- SQLite database
|
||||
- Public dashboard view
|
||||
- Historical speed test data
|
||||
- LinuxServer.io image with PUID/PGID support
|
||||
|
||||
**Deployment**:
|
||||
```bash
|
||||
cd speedtest-tracker
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## General Deployment Instructions
|
||||
|
||||
### Prerequisites
|
||||
- Docker Engine 20.10+
|
||||
- Docker Compose v2.0+
|
||||
- Sufficient disk space for volumes
|
||||
- Network ports available (check port conflicts)
|
||||
|
||||
### Standard Deployment Workflow
|
||||
|
||||
1. **Review Configuration**
|
||||
```bash
|
||||
cd services/<service-name>
|
||||
cat docker-compose.yaml
|
||||
```
|
||||
|
||||
2. **Configure Environment Variables** (if applicable)
|
||||
```bash
|
||||
# Copy example env file if available
|
||||
cp .env.example .env
|
||||
# Edit with actual values
|
||||
nano .env
|
||||
```
|
||||
|
||||
3. **Create Required Directories**
|
||||
```bash
|
||||
# Ensure volume mount points exist
|
||||
# Example for bytestash:
|
||||
mkdir -p /home/jramos/docker/bytestash/data
|
||||
```
|
||||
|
||||
4. **Deploy Stack**
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
5. **Verify Deployment**
|
||||
```bash
|
||||
docker compose ps
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
6. **Configure Reverse Proxy** (if using NPM)
|
||||
- Access Nginx Proxy Manager at http://192.168.2.101:81
|
||||
- Create proxy host pointing to service IP:PORT
|
||||
- Configure SSL certificate via Let's Encrypt
|
||||
- Set appropriate forwarding scheme (http/https)
|
||||
|
||||
### Maintenance Commands
|
||||
|
||||
**View Logs**:
|
||||
```bash
|
||||
cd services/<service-name>
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
**Restart Service**:
|
||||
```bash
|
||||
docker compose restart
|
||||
```
|
||||
|
||||
**Update Service**:
|
||||
```bash
|
||||
docker compose pull
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**Stop Service**:
|
||||
```bash
|
||||
docker compose down
|
||||
```
|
||||
|
||||
**Remove Service and Volumes** (DESTRUCTIVE):
|
||||
```bash
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
services/
|
||||
├── README.md # This file
|
||||
├── bytestash/
|
||||
│ ├── docker-compose.yaml
|
||||
│ └── .gitkeep
|
||||
├── filebrowser/
|
||||
│ ├── docker-compose.yaml
|
||||
│ └── .gitkeep
|
||||
├── gitlab/
|
||||
│ ├── QoL Config Files/
|
||||
│ │ ├── sync-npm-certs.service
|
||||
│ │ └── sync-npm-certs.timer
|
||||
│ └── QoL Scripts/
|
||||
│ └── sync-npm-certs.sh
|
||||
├── paperless-ngx/
|
||||
│ ├── docker-compose.yaml
|
||||
│ └── .env
|
||||
├── portainer/
|
||||
│ ├── docker-compose.yaml
|
||||
│ └── .gitkeep
|
||||
└── speedtest-tracker/
|
||||
├── docker-compose.yaml
|
||||
└── .gitkeep
|
||||
```
|
||||
|
||||
## Volume Mounts and Data Locations
|
||||
|
||||
Services use the following host paths for persistent data:
|
||||
|
||||
| Service | Host Path | Purpose |
|
||||
|---------|-----------|---------|
|
||||
| ByteStash | `/home/jramos/docker/bytestash/data` | Snippet storage |
|
||||
| FileBrowser | `/home/docker/filebrowser/` | Database and settings |
|
||||
| Paperless-ngx | `/home/jramos/paperless-ngx/consume` | Document intake directory |
|
||||
| Speedtest Tracker | `/home/jramos/docker/speedtest-tracker/config` | Configuration and database |
|
||||
| Portainer | `portainer_data` (Docker volume) | Application data |
|
||||
|
||||
**Important**: Ensure these directories exist with appropriate permissions before deploying services.
|
||||
|
||||
## Network Configuration
|
||||
|
||||
All services are configured to use host networking or specific port mappings. If deploying behind Nginx Proxy Manager (CT 102 at 192.168.2.101):
|
||||
|
||||
1. Services should be accessible via internal IPs and ports
|
||||
2. NPM handles external HTTPS access and SSL termination
|
||||
3. Use `http` scheme in NPM when forwarding to backend services
|
||||
4. Enable "Force SSL" in NPM for external HTTPS access
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Environment Files
|
||||
- `.env` files are excluded from git via `.gitignore`
|
||||
- Never commit credentials or API keys
|
||||
- Use strong, unique passwords for database services
|
||||
- Rotate JWT secrets and app keys regularly
|
||||
|
||||
### Secrets in Docker Compose Files
|
||||
Several services have embedded secrets in their docker-compose.yaml files:
|
||||
- **ByteStash**: `JWT_SECRET: your-secret` (CHANGE THIS)
|
||||
- **Paperless-ngx**: Database password `paperless` (CHANGE THIS)
|
||||
- **Speedtest Tracker**: `APP_KEY` (already generated, but sensitive)
|
||||
|
||||
**Action Required**: Create `.env` files and move secrets out of docker-compose.yaml files.
|
||||
|
||||
### Network Exposure
|
||||
- Review port mappings before deployment
|
||||
- Consider using Docker networks instead of host port binding
|
||||
- Use NPM for external access with SSL
|
||||
- Implement authentication on all services
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service Won't Start
|
||||
1. Check logs: `docker compose logs -f`
|
||||
2. Verify port availability: `netstat -tulpn | grep <port>`
|
||||
3. Check volume permissions: `ls -la /path/to/volume`
|
||||
4. Validate docker-compose.yaml syntax: `docker compose config`
|
||||
|
||||
### Cannot Access Service Externally
|
||||
1. Verify service is running: `docker compose ps`
|
||||
2. Test local access: `curl http://localhost:<port>`
|
||||
3. Check NPM proxy host configuration
|
||||
4. Verify DNS resolution
|
||||
5. Check firewall rules: `iptables -L -n -v`
|
||||
|
||||
### Database Connection Errors (Paperless-ngx)
|
||||
1. Verify PostgreSQL container is running
|
||||
2. Check database credentials in environment variables
|
||||
3. Ensure database initialization completed: `docker compose logs db`
|
||||
4. Verify network connectivity between containers
|
||||
|
||||
### Permission Denied Errors
|
||||
1. Check PUID/PGID settings in docker-compose.yaml
|
||||
2. Verify host directory ownership: `chown -R <user>:<group> /path/to/volume`
|
||||
3. Check SELinux context (if applicable): `ls -Z /path/to/volume`
|
||||
|
||||
## Migration Notes
|
||||
|
||||
### Post-Migration Tasks
|
||||
- [ ] Review all `.env` files and update with production values
|
||||
- [ ] Change default passwords and secrets in docker-compose files
|
||||
- [ ] Verify volume mount paths exist on target system
|
||||
- [ ] Test each service deployment individually
|
||||
- [ ] Configure NPM proxy hosts for external access
|
||||
- [ ] Update DNS records if service URLs changed
|
||||
- [ ] Backup existing service data before redeployment
|
||||
- [ ] Document any service-specific configuration changes
|
||||
|
||||
### Known Issues
|
||||
- **FileBrowser**: Mounts entire filesystem root - review and restrict as needed
|
||||
- **Paperless-ngx**: Contains `.env` file with secrets - ensure it's excluded from git
|
||||
- **GitLab Utilities**: May require path adjustments depending on GitLab installation location
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new services to this directory:
|
||||
|
||||
1. Create a new subdirectory with service name (lowercase, hyphenated)
|
||||
2. Include `docker-compose.yaml` (or `docker-compose.yml`)
|
||||
3. Add `.env.example` if service requires environment variables
|
||||
4. Document service in this README under "Services Overview"
|
||||
5. Update directory structure diagram
|
||||
6. Test deployment from scratch before committing
|
||||
7. Ensure `.gitignore` excludes sensitive files
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Docker Compose Documentation](https://docs.docker.com/compose/)
|
||||
- [Nginx Proxy Manager Docs](https://nginxproxymanager.com/guide/)
|
||||
- [Proxmox Homelab Documentation](../CLAUDE.md)
|
||||
- [n8n Setup Guide](../n8n/N8N-SETUP-PLAN.md)
|
||||
|
||||
## Support
|
||||
|
||||
For homelab-specific questions or issues:
|
||||
- Check existing documentation in `/home/jramos/homelab/`
|
||||
- Review `CLAUDE_STATUS.md` for current infrastructure state
|
||||
- Consult service-specific documentation linked in each service section
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-02
|
||||
**Maintainer**: jramos
|
||||
**Repository**: http://192.168.2.102:3060/jramos/homelab
|
||||
Reference in New Issue
Block a user