Add Atlas metrics reporting, security audit tracker, and spec documents

This commit is contained in:
root
2026-04-24 17:30:06 +00:00
parent 8bf8dc55dd
commit 5ffedad02f
11 changed files with 2101 additions and 22 deletions

View File

@@ -21,8 +21,7 @@ A self-hosted vulnerability management dashboard for the NTS-AEO-STEAM and NTS-A
- [Knowledge Base](#knowledge-base)
- [Exports](#exports)
- [Archer Risk Acceptance Tickets](#archer-risk-acceptance-tickets)
- [User Management (Admin)](#user-management-admin)
- [Audit Log (Admin)](#audit-log-admin)
- [Admin Panel](#admin-panel)
- [Scripts](#scripts)
- [API Reference](#api-reference)
- [Architecture](#architecture)
@@ -411,11 +410,18 @@ The Compliance page tracks NTS-AEO team posture against the AEO compliance frame
Admin and Standard_User groups can upload a new compliance report via the **Upload Report** button:
1. Drop or browse for the `NTS_AEO_YYYY_MM_DD.xlsx` file
2. The report is parsed server-side and a **diff preview** is shown — new violations, resolved items, and recurring items since the last upload
3. Click **Confirm Upload** to commit. The upload is recorded and the device table updates immediately.
2. The backend extracts the xlsx schema and runs a **drift check** against the parser configuration (`compliance_config.json`). If structural drift is detected, a drift review phase is shown before the diff preview:
- **Breaking** findings (red) — missing core columns or detail sheets — block the upload until the config is updated
- **Silent-miss** findings (amber) — unknown metrics or sheets that will be miscategorised — warn but allow proceeding
- **Cosmetic** findings (muted) — new columns or stale config entries — informational only
- Admins can click **Reconcile Config** to auto-patch the parser configuration and re-run the check
3. If no breaking drift exists, the **diff preview** is shown — new violations, resolved items, and recurring items since the last upload
4. Click **Confirm Upload** to commit. The upload is recorded and the device table updates immediately.
The report date is extracted automatically from the filename.
**Upload rollback:** Admins can roll back the most recent upload via `POST /api/compliance/rollback/:uploadId`. Rolling back deletes new items introduced by that upload, re-activates items it resolved, and decrements seen counts on recurring items.
#### Metric Health Cards
Each AEO metric (e.g., `2.3.4i`, `5.2.4`) is shown as a health card displaying:
@@ -435,7 +441,7 @@ A slide-out panel for a selected device showing:
- For **2.3.x vulnerability metrics**: the `Ivanti_Vulnerability_ID` is displayed with a **View in Reporting →** button that navigates directly to the Reporting page
- **Resolved Metrics** — previously failing metrics now back in compliance
- **History** — how many times the device has appeared on the report and since when
- **Notes** — timestamped notes per metric with a multi-metric selector if multiple metrics are failing. Requires Admin or Standard_User group.
- **Notes** — timestamped notes per metric with a multi-metric selector if multiple metrics are failing. Notes can be deleted by the author or an Admin — deleting a multi-metric note removes it from all linked metrics. Requires Admin or Standard_User group.
Notes persist across uploads and are keyed to the device hostname and metric ID.
@@ -479,20 +485,17 @@ Track Archer exception tickets (EXC numbers) linked to specific CVE/vendor pairs
---
### User Management (Admin)
### Admin Panel
- Create users with a group assignment (Admin, Standard_User, Leadership, Read_Only)
- Change username, email, password, group, or active status
- Group changes require confirmation; downgrading an Admin shows an additional warning
- Deactivating a user immediately invalidates all their active sessions
- Admins cannot demote themselves or deactivate their own account
- All group changes are audit-logged with previous and new group values
The Admin Panel is a full-page, tabbed interface accessible only to Admin-group users. It replaces the previous inline modal rendering and follows the dashboard's dark tactical intelligence theme. Three tabs provide consolidated access to administrative functions:
---
**User Management** — the default tab. Displays a themed user table with group badges (Admin in red, Standard_User in accent blue, Leadership in amber, Read_Only in muted grey). Admins can create, edit, and delete users, change group assignments, and toggle active status — all through inline forms styled to match the dashboard. Admins cannot demote themselves or deactivate their own account. Deactivating a user immediately invalidates all their active sessions. All group changes are audit-logged with previous and new group values.
### Audit Log (Admin)
**Audit Log** — a paginated, filterable log table showing every state-changing action with timestamp, username, action type, entity type, entity ID, details, and IP address. Action types are colour-coded: login in green, delete in red, create in accent blue, update in amber. Filter by username, action type, entity type, and date range. Results are paginated at 25 per page.
Every state-changing action is recorded with the user identity, IP address, action type, target entity, and a before/after payload. Admins can view the log filtered by user, action type, entity type, and date range. Results are paginated (25 per page).
**System Info** — stat cards showing total user count, active user count, total audit log entries, and users who logged in within the last 7 days. A "Recent Activity" section lists the 10 most recent audit log entries.
The `UserMenu` quick-access links ("Manage Users", "Audit Log") continue to open the existing modal components for fast access without navigating to the admin page.
---
@@ -500,9 +503,9 @@ Every state-changing action is recorded with the user identity, IP address, acti
### `backend/scripts/parse_compliance_xlsx.py`
Called automatically by the compliance upload flow. Parses the NTS_AEO xlsx report and outputs structured JSON to stdout for consumption by the Node compliance route.
Called automatically by the compliance upload flow. Parses the NTS_AEO xlsx report and outputs structured JSON to stdout for consumption by the Node compliance route. Reads metric categories, core columns, and skip sheets from `compliance_config.json` (shared with the drift checker).
- Reads all detail sheets; skips `Summary` and `CMDB_9box`
- Reads all detail sheets; skips sheets listed in `skip_sheets`
- Filters to rows where `Compliant == False`
- Extracts hostname, IP, device type, team, and metric ID per row
- Captures all non-core columns in `extra_json` (CVEs, SLA status, OS, EoL, Splunk, MFA, Ivanti_Vulnerability_ID, etc.)
@@ -511,6 +514,16 @@ Called automatically by the compliance upload flow. Parses the NTS_AEO xlsx repo
**Dependencies:** `pandas>=2.0.0`, `openpyxl>=3.0.0`
### `backend/scripts/extract_xlsx_schema.py`
Called by the preview endpoint before parsing. Extracts the structural schema of an xlsx file as JSON — sheet names, first-row column headers per sheet, and unique metric values from the Summary sheet. The Node.js drift checker compares this schema against `compliance_config.json` to detect breaking, silent-miss, and cosmetic drift.
**Dependencies:** `openpyxl>=3.0.0`
### `backend/scripts/compliance_config.json`
Shared parser configuration file — the single source of truth for `metric_categories` (metric ID → category mapping), `core_cols` (columns that become main item fields), and `skip_sheets` (sheets excluded from parsing). Read by both `parse_compliance_xlsx.py` and the Node.js `driftChecker.js` module. Admins can auto-patch this file via the **Reconcile Config** button in the upload modal.
---
## API Reference
@@ -618,14 +631,17 @@ All endpoints are prefixed with `/api`. All endpoints except `/api/auth/login` a
| Method | Path | Group | Description |
|---|---|---|---|
| POST | `/api/compliance/preview` | Admin, Standard_User | Parse an xlsx upload and return diff + temp file path |
| POST | `/api/compliance/preview` | Admin, Standard_User | Parse an xlsx upload, run drift check, and return drift report + diff + temp file path |
| POST | `/api/compliance/commit` | Admin, Standard_User | Commit a previewed upload to the database |
| POST | `/api/compliance/reconcile-config` | Admin | Auto-patch `compliance_config.json` to resolve breaking and silent-miss drift findings |
| POST | `/api/compliance/rollback/:uploadId` | Admin | Roll back the most recent upload (deletes new items, re-activates resolved items) |
| GET | `/api/compliance/uploads` | Any | List all compliance upload records |
| GET | `/api/compliance/summary` | Any | Metric health summary; `?team=STEAM` |
| GET | `/api/compliance/items` | Any | Device list; `?team=STEAM&status=active` |
| GET | `/api/compliance/items/:hostname` | Any | Full detail for a device (metrics + notes) |
| GET | `/api/compliance/notes/:hostname/:metricId` | Any | Notes for a specific hostname/metric |
| POST | `/api/compliance/notes` | Admin, Standard_User | Add a note for a hostname/metric; accepts `metric_ids` array for multi-metric notes |
| DELETE | `/api/compliance/notes/:id` | Admin, Standard_User | Delete a note by ID; `?group=true` deletes all notes sharing the same `group_id`. Author or Admin only. |
### Knowledge Base
@@ -705,9 +721,13 @@ cve-dashboard/
│ ├── middleware/
│ │ └── auth.js # requireAuth and requireGroup middleware
│ ├── helpers/
│ │ ── auditLog.js # logAudit helper (fire-and-forget)
│ │ ── auditLog.js # logAudit helper (fire-and-forget)
│ │ ├── driftChecker.js # Schema drift detection: compareSchemaToDrift(), loadConfig(), reconcileConfig()
│ │ └── ivantiApi.js # Ivanti API HTTP helpers (multipart, JSON, form POST)
│ ├── migrations/ # Sequential migration scripts (run manually with node)
│ └── scripts/
│ ├── compliance_config.json # Shared parser config (metric_categories, core_cols, skip_sheets)
│ ├── extract_xlsx_schema.py # Extracts xlsx structure as JSON for drift checking
│ ├── parse_compliance_xlsx.py # Parses NTS_AEO xlsx compliance reports
│ ├── import_notes_from_csv.py # Bulk-import finding notes from CSV
│ └── requirements.txt # pandas, openpyxl
@@ -723,14 +743,16 @@ cve-dashboard/
├── NavDrawer.js # Side navigation drawer (Admin Panel link for Admin group)
├── UserMenu.js # User dropdown in header (shows group badge)
├── CalendarWidget.js # Due-date calendar with Ivanti finding indicators
├── UserManagement.js # Admin user management panel (group assignment)
├── AuditLog.js # Admin audit log viewer
├── UserManagement.js # Admin user management modal (quick-access from UserMenu)
├── AuditLog.js # Admin audit log modal (quick-access from UserMenu)
├── NvdSyncModal.js # Bulk NVD sync dialog
├── KnowledgeBaseModal.js # Knowledge base upload/list modal
├── KnowledgeBaseViewer.js # Inline document viewer (sandboxed iframe, sanitized markdown)
├── ConfirmModal.js # Themed confirmation dialog (replaces window.confirm)
├── CveTooltip.js # Hover tooltip for CVE badges (portal-rendered, cached)
├── RedirectModal.js # Queue item redirect modal (workflow type + vendor selection)
└── pages/
├── AdminPage.js # Admin panel: user management, audit log, system info
├── ReportingPage.js # Host findings: charts, table, queue, export
├── CompliancePage.js # AEO compliance: metric cards, device table
├── ComplianceUploadModal.js # xlsx upload with diff preview