feat(monitoring): resolve Loki-stack syslog ingestion with rsyslog filter fix

Fixed critical issue preventing UniFi router logs from reaching Loki/Promtail/Grafana.

Root Cause:
- rsyslog filter in /etc/rsyslog.d/unifi-router.conf filtered for 192.168.1.1
- VM 101 on VLAN 2, actual source IP is 192.168.2.1 (VLAN 2 gateway)
- Filter silently rejected all incoming syslog traffic

Solution:
- Updated rsyslog filter from 192.168.1.1 to 192.168.2.1
- Logs now flow: UniFi → rsyslog → Promtail → Loki → Grafana

Changes:
- Add services/loki-stack/* - Complete Loki/Promtail/Grafana stack configs
- Add services/logward/* - Logward service configuration
- Update troubleshooting/loki-stack-bugfix.md - Complete 5-phase resolution
- Update CLAUDE_STATUS.md - Document 2025-12-11 resolution
- Update sub-agents/scribe.md - Agent improvements
- Remove services/promtail-config.yml - Duplicate file cleanup

Status:  Monitoring stack fully operational, syslog ingestion active

Technical Details: See troubleshooting/loki-stack-bugfix.md for complete analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
2025-12-11 13:56:27 -07:00
parent 698a5b531a
commit 892684c46e
8 changed files with 526 additions and 3 deletions

View File

@@ -196,9 +196,30 @@ Hybrid approach balancing performance and resource efficiency:
---
## Recent Infrastructure Changes (2025-12-07)
## Recent Infrastructure Changes
### Additions
### 2025-12-11: Loki-Stack Monitoring Fully Operational
**Issue Resolved:** Centralized logging pipeline now receiving syslog from UniFi router
**Root Cause:** rsyslog filter in `/etc/rsyslog.d/unifi-router.conf` was configured for wrong source IP (192.168.1.1 instead of 192.168.2.1)
**Fix Applied:** Updated rsyslog filter to match VLAN 2 gateway IP (192.168.2.1)
**Status:** ✅ Complete - Logs flowing UniFi → rsyslog → Promtail → Loki → Grafana
**Services Affected:**
- VM 101 (monitoring-docker): rsyslog configuration updated
- Loki-stack: All components operational
- Grafana: Dashboards receiving real-time syslog data
**Technical Details:** See `troubleshooting/loki-stack-bugfix.md` for complete 5-phase troubleshooting history
---
### 2025-12-07: Infrastructure Documentation & Monitoring Stack
#### Additions
1. **VM 101 (monitoring-docker)**: New dedicated monitoring infrastructure
- Grafana for visualization
- Prometheus for metrics collection