Commit Graph

333 Commits

Author SHA1 Message Date
Jordan Ramos
3814de5845 Fix duplicate chart entries on compliance page when multiple verticals share a report_date
Aggregate /trends, /top-recurring, /category-trend by report_date instead of
per-upload row. Add sibling-upload disclosure to /summary. Filter persistUpload
snapshot query by the upload's vertical to prevent cross-vertical contamination.

Fixes GitLab #12 (reported by nkapur — STEAM active findings chart showed 3
entries for 5/11 after uploading three vertical data sets for that date).

Includes 30 property-based tests covering bug condition and preservation.
2026-05-18 15:00:53 -06:00
Jordan Ramos
487489e26c Add unified setup script (configure.js) merging deploy + config wizard
Single-file Node.js CLI that orchestrates the full setup lifecycle:
- Interactive env var configuration with validation and smart defaults
- Postgres provisioning via Docker Compose with readiness polling
- Schema initialization (psql with docker exec fallback)
- npm dependency installation with 120s timeout
- Optional SQLite-to-Postgres data migration with retry logic
- Frontend build with smart skip on reconfiguration

Includes 84 tests: 50 property-based (fast-check) covering 19 correctness
properties, and 34 integration tests for filesystem and parsing flows.
2026-05-18 11:58:21 -06:00
Jordan Ramos
3643c123b4 Fix requeue inserting Postgres array literal instead of JSON into cves_json 2026-05-15 17:41:38 -06:00
Jordan Ramos
be1d357692 Fix todo queue crash on malformed cves_json data 2026-05-15 17:31:19 -06:00
Jordan Ramos
492780fd90 Add aggregated burndown forecast to CCP Metrics overview page 2026-05-15 17:08:55 -06:00
Jordan Ramos
4d255209fd Group history entries together, remove (optional) from change reason
1. History entries saved at the same time by the same user now display
   as a single grouped entry (resolution date + remediation plan together)
2. Removed '(optional)' from the change reason placeholder — engineers
   should treat it as expected, even though the backend allows empty
3. Save button now saves both resolution date AND remediation plan in one
   call (removed the onBlur auto-save on the date field) so they share
   a timestamp and group correctly in history
2026-05-15 15:31:56 -06:00
Jordan Ramos
1fe6c1f84c Add remediation plan and resolution date history tracking
New table compliance_item_history stores an append-only audit trail of
changes to resolution_date and remediation_plan. The current values remain
on compliance_items for fast VCL reporting queries (no double-counting).

Backend:
- Migration: creates compliance_item_history with indexes
- PATCH /items/:hostname/metadata: records old→new in history before updating,
  accepts optional change_reason field (max 500 chars)
- GET /items/:hostname: returns history array (last 10 entries, newest first)
- POST /vcl/bulk-commit: records history for each changed field per hostname

Frontend:
- ComplianceDetailPanel: added change reason input below Save button
- Added Change History section showing field changes with timestamps,
  usernames, old→new values, and reasons
- Re-fetches detail after save to show updated history immediately

Tests updated to match new transaction-based PATCH flow.
2026-05-15 10:53:14 -06:00
Jordan Ramos
97e5d68d8e Fix AEO compliance page not showing metric health cards on dev
The /summary endpoint was fetching the most recent upload regardless of
vertical, which on dev was a PRDCT_VSO multi-vertical upload. Now it
looks for AEO uploads (vertical IS NULL) first, then falls back to the
NTS_AEO multi-vertical upload.

The /items endpoint now includes items from both vertical IS NULL and
vertical = 'NTS_AEO' so the AEO compliance page shows devices uploaded
through either flow.
2026-05-14 15:39:25 -06:00
Jordan Ramos
b808d0e38e Color metric card percentage green/yellow/red based on target, keep NC count always red 2026-05-14 15:30:43 -06:00
Jordan Ramos
a72300475b Clean up metric breakdown panel — compact grid, top 8 with show-all toggle
Replaced the large flex-wrap button cards with a tight CSS grid of compact
cells (130px min). Each cell shows metric ID, current %, and NC count only.
Category text and target removed to reduce noise.

Capped to top 8 metrics by default with a 'Show all N' toggle for the rest.
Removes visual clutter while keeping the data accessible.
2026-05-14 15:29:20 -06:00
Jordan Ramos
7577ab1219 Make Non-Compliant stat clickable — reveals metric breakdown buttons
Clicking the Non-Compliant card on the CCP Metrics overview now toggles a
panel of metric buttons below it, each showing the metric ID, category,
non-compliant count, and compliance % vs target. Styled like the compliance
page's MetricHealthCard pattern.

Backend: added metric_breakdown to the /stats response — aggregated
cross-vertical metric totals (ALL: rows only, grouped by metric_id).

Also updated tech steering file to document the single-port Express
architecture and the requirement to run npm run build after frontend changes.
2026-05-14 15:24:10 -06:00
Jordan Ramos
a2bc1ff564 Add metric sub-team intermediate drill-down view
Clicking a metric now shows a sub-team breakdown page with totals per team
(compliant, non-compliant, total, %) instead of jumping directly to a flat
device list. Clicking a sub-team then shows the device list filtered to
that team only.

Navigation flow: Overview → Vertical → Metric (sub-team totals) → Team (devices)

Backend: added optional ?team= query param to the device list endpoint for
filtered queries.

Frontend: added MetricSubTeamView component with metric-level stats bar and
clickable sub-team table. Updated navigation state to include selectedTeam.

Also updated design brief to reflect the new drill-down hierarchy.
2026-05-14 14:53:41 -06:00
Jordan Ramos
682ee9417f Add metrics calculation explainer and sub-team drill-down docs to design brief 2026-05-14 13:00:09 -06:00
Jordan Ramos
61d7e00d4f Add sub-team level display to CCP Metrics vertical drill-down
Backend: restructured /vertical/:code/metrics endpoint to return metrics
with nested sub_teams arrays. Each metric now has the ALL: rollup as the
primary row and individual team breakdowns (ACCESS-OPS, STEAM, etc.) as
sub_teams. Also returns a teams array for the filter UI.

Frontend: VerticalDetailView now supports two interaction modes:
- Expand/collapse: click the arrow on any metric row to reveal sub-team
  breakdown inline (teal-highlighted rows beneath the parent)
- Team filter: click a team button to filter the entire table to show
  only that team's numbers per metric

Both modes avoid double-counting by using the ALL: rollup for totals
and only showing sub-team data as supplementary detail.
2026-05-14 12:27:46 -06:00
Jordan Ramos
ebaf4cd18c Fix double-counting in VCL multi-vertical stats — use only ALL: rollup rows
The Summary sheet in each vertical spreadsheet contains both sub-team rows
(ACCESS-OPS, STEAM, INTELDEV, etc.) AND a rollup row (ALL: NTS-AEO) per
metric. The rollup row already includes all sub-team totals, so summing
all rows was double-counting every device.

Fixed in three places:
- GET /stats endpoint: added AND team LIKE 'ALL:%' filter
- persistMultiVerticalUpload snapshot creation: only sum ALL: entries
- GET /vertical/:code/metrics category aggregation: only use ALL: rows

Also ran a one-time data fix to correct existing compliance_snapshots.
2026-05-14 12:09:44 -06:00
Jordan Ramos
55238ec71e Fix compliance stats to use Summary sheet data instead of item counts
The compliance_items table only contains non-compliant devices (detail
sheet rows). Compliant devices are never inserted — they only exist in
the Summary sheet totals. This caused Compliant to show 0 and
Compliance % to show 0% for all verticals.

Fix: stats endpoint now reads from vcl_multi_vertical_summary (parsed
Summary sheet data) for total/compliant/non-compliant counts. Snapshot
creation also uses summary data for accurate trend charting.

The compliance_items table is still used for:
- Donut chart (blocked vs in-progress based on resolution_date)
- Burndown forecast (devices with/without resolution dates)
- Device drill-down (actual non-compliant device list)
2026-05-14 12:01:19 -06:00
Jordan Ramos
408aaa7012 Add data management panel with delete vertical, rollback upload, and reset all
Backend:
- DELETE /api/compliance/vcl-multi/vertical/:code — wipe a single vertical
- DELETE /api/compliance/vcl-multi/upload/:uploadId — rollback most recent upload
- DELETE /api/compliance/vcl-multi/all — nuclear reset of all multi-vertical data
- All delete operations are Admin-only and audit-logged

Frontend:
- Manage button (red, Admin-only) in CCP Metrics header
- DataManagementPanel modal showing upload history grouped by vertical
- Per-vertical delete button
- Per-upload rollback button (most recent only)
- Reset All button with confirmation dialog
- Success/error messaging
2026-05-14 11:54:58 -06:00
Jordan Ramos
1eb8eab76f Fix route mount order: vcl-multi must precede general compliance router 2026-05-14 10:15:15 -06:00
Jordan Ramos
232eedce70 Remove unused icon imports to fix ESLint warning count 2026-05-14 10:00:51 -06:00
Jordan Ramos
0ca2fe99e9 Remove unused imports to satisfy ESLint max-warnings threshold 2026-05-14 10:00:00 -06:00
Jordan Ramos
04360cc4bc Add CCP Metrics page with multi-vertical VCL upload and cross-org reporting
New feature: multi-file per-vertical compliance xlsx upload with scoped
resolution logic, executive-level aggregated reporting, and drill-down
by vertical and metric. Supports daily upload cadence and batch commit.

Backend:
- Migration: add vertical column to compliance_items/uploads, create
  vcl_multi_vertical_summary table
- New route module: routes/vclMultiVertical.js with preview, commit,
  stats, trend, metric drill-down, device list, and burndown endpoints
- New helpers: parseVerticalFilename(), computeVerticalBurndown()
- Vertical-scoped resolution: uploading one vertical never resolves
  items from other verticals

Frontend:
- CCPMetricsPage with stats bar, trend chart, donut, vertical table
- Drill-down: vertical -> metrics by category -> device list
- Per-vertical burndown forecast chart
- MultiVerticalUploadModal: multi-file drag-drop, batch preview, commit
- Nav entry: CCP Metrics (Building2 icon)

Docs:
- Design brief for stakeholder meeting (docs/vcl-multi-vertical-design-brief.md)
2026-05-14 09:49:59 -06:00
Jordan Ramos
d61383ac7b Add VCL reporting guide, update reference manual and config wizard; untrack .kiro/steering/workflow.md 2026-05-14 08:15:42 -06:00
Jordan Ramos
808625dab4 Fix requeue: fallback to finding_ids_json when queue items are deleted or absent
The requeue endpoint now handles three scenarios:
1. Original queue items still exist — uses their finding data (ideal case)
2. Queue items deleted (Clear Completed) — looks up findings from
   ivanti_findings table using finding_ids_json
3. FP created outside dashboard (no queue_item_ids) — same fallback
   to finding_ids_json and ivanti_findings lookup
4. Last resort — creates queue items with just finding IDs if the
   findings aren't in ivanti_findings either
2026-05-13 16:57:57 -06:00
Jordan Ramos
0fefd2a707 Add re-queue findings from rejected FP submissions
New feature: users can re-queue findings from a rejected FP submission
back into the Ivanti todo queue under a different workflow type (FP,
Archer, CARD, GRANITE, or DECOM). Primary use case is when an FP is
rejected with a recommendation to submit an Archer risk acceptance.

Backend:
- New migration: add requeued_at column to ivanti_fp_submissions
- New endpoint: POST /api/ivanti/fp-workflow/submissions/:id/requeue
  - Validates workflow_type and vendor (required for FP/Archer/DECOM)
  - Creates new pending queue items from original finding data
  - Marks submission as requeued (prevents double re-queue)
  - Audit logs the action

Frontend (ReportingPage.js):
- RequeueConfirmDialog component with workflow type selector and vendor input
- Re-queue Findings button in Edit FP Modal header (rejected submissions only)
- Already re-queued label when submission.requeued_at is set
- Success notification on completion
2026-05-13 16:46:49 -06:00
Jordan Ramos
828e7cc45d Sync FP submission lifecycle_status from Ivanti currentState on fetch
When GET /submissions enriches submissions with Ivanti API data, it now
checks if batch.currentState (APPROVED, REJECTED, REWORK) differs from
the local lifecycle_status and updates the DB accordingly. This ensures
approved submissions get filtered out of the queue panel as intended.

Also changed safeText() to return null for non-string Ivanti note values
(arrays/objects) instead of JSON-stringifying them. The notes array
filters nulls via .filter(Boolean) so non-string data is simply hidden.
2026-05-13 14:36:05 -06:00
Jordan Ramos
5126ccc6ae Fix History tab crash: coerce Ivanti note fields to strings before rendering
PostgreSQL + Ivanti API enrichment can return non-string values
(objects/arrays) for currentStateUserNotes and similar fields.
React crashes silently (blank page, no console error) when trying
to render non-string values as children. Same root cause pattern
as Bug 3 in ivanti-panel-bugs-2026-05-12.

Added safeText() wrapper that coerces any non-string truthy value
to a JSON string before rendering in the History tab notes section.

Also fixed flaky property test: fc.date() could generate invalid
dates causing RangeError on .toISOString(). Added .filter() guard
and explicit UTC date bounds.
2026-05-13 14:24:16 -06:00
Jordan Ramos
870c0e247a Fix History tab crash: coerce Ivanti note fields to strings before rendering
PostgreSQL + Ivanti API enrichment can return non-string values
(objects/arrays) for currentStateUserNotes and similar fields.
React crashes silently (blank page, no console error) when trying
to render non-string values as children. Same root cause pattern
as Bug 3 in ivanti-panel-bugs-2026-05-12.

Added safeText() wrapper that coerces any non-string truthy value
to a JSON string before rendering in the History tab notes section.
2026-05-13 12:01:52 -06:00
Jordan Ramos
671894ff5f Fix vcl-compliance-reporting test: stats.total → stats.total_devices 2026-05-13 09:56:30 -06:00
Jordan Ramos
0c6830fc6c Add interactive configuration wizard for deployment setup 2026-05-13 09:40:45 -06:00
Jordan Ramos
9eec63ea42 Add VCL vertical metadata: inline-editable team fields, JSDoc on compliance routes, stats query rewrite 2026-05-13 07:57:41 -06:00
Jordan Ramos
0d29a1b84e Document IVANTI_MANAGED_BUS env var in .env.example, reference manual, and API docs 2026-05-13 07:56:03 -06:00
Jordan Ramos
4416f6a25d Make EXPECTED_BUS configurable via IVANTI_MANAGED_BUS env var for multi-tenant drift classification 2026-05-12 15:27:58 -06:00
Jordan Ramos
97d378033b Revert EXPECTED_BUS to STEAM+ACCESS-ENG: findings leaving managed teams are BU reassignments 2026-05-12 15:22:52 -06:00
Jordan Ramos
537cf96a0a Fix BU drift checker: derive EXPECTED_BUS from IVANTI_BU_FILTER env var instead of hardcoded 2 BUs 2026-05-12 15:18:44 -06:00
Jordan Ramos
f3d7f2ac1d Fix archive bar chart: fmtDate now handles ISO datetime strings from PostgreSQL date columns 2026-05-12 14:57:15 -06:00
Jordan Ramos
8c93e86fe0 Fix Ivanti panel bugs: Invalid Date, wrong workflow count, crash on archive click, BU scope filtering 2026-05-12 14:21:46 -06:00
Jordan Ramos
d093a3d113 Add VCL compliance reporting: exec report page, device metadata fields, bulk upload 2026-05-11 15:48:10 -06:00
Jordan Ramos
955036145d Fix property test CI failure: mock db module before importing route 2026-05-11 14:51:16 -06:00
Jordan Ramos
7245352496 Add FP submissions cleanup: auto-clear approved, dismiss rejected, collapsible section 2026-05-11 14:29:50 -06:00
Jordan Ramos
cda1eaadc9 Add DECOM workflow type, auto-note/hide on decom, show CVEs on CARD queue items, auto-run migrations in pipeline
- Add DECOM to queue workflow types (red badge, inventory-style display)
- When findings are added as DECOM, auto-set note to 'DECOM' and hide row
- Hidden rows are excluded from donut charts (removes from pending count)
- Show CVEs on CARD/GRANITE/DECOM queue items (was previously omitted)
- Add backend/migrations/run-all.js for CI/CD auto-migration execution
- Pipeline now runs migrations before service restart on both staging and prod
- Add add_decom_workflow_type.js migration (updates CHECK constraint)
2026-05-08 14:51:05 -06:00
Jordan Ramos
3cf0d6be3d Fix CI: install root deps in frontend test job for cross-directory backend imports 2026-05-08 13:55:15 -06:00
Jordan Ramos
cc652ba964 Fix CI: add npm ci to each job since runner cache is unreliable, use local jest binary 2026-05-08 13:35:50 -06:00
Jordan Ramos
f76996a161 Fix CI: add express/pg devDeps for atlas test, allow lint warnings, drop forceExit 2026-05-08 13:25:58 -06:00
Jordan Ramos
b870f47e67 Fix CI: allow 10 lint warnings for unused vars, drop --forceExit from frontend tests 2026-05-08 13:18:17 -06:00
Jordan Ramos
890d7b82dc Fix CI: exclude test files from lint, mock db.js in jira test for runner env 2026-05-08 13:11:06 -06:00
Jordan Ramos
1b0fc072cc Track package-lock.json files for deterministic CI installs 2026-05-08 13:05:20 -06:00
Jordan Ramos
3f00f4c941 Fix pipeline: remove verify-staging from deploy-production needs (manual gate is sufficient) 2026-05-08 13:02:12 -06:00
Jordan Ramos
eef324936d Fix pipeline: mark verify-staging as optional dependency for deploy-production 2026-05-08 12:57:39 -06:00
Jordan Ramos
de2c5f245e Add CI/CD pipeline, feedback modal, Atlas qualys_id fallback, and health endpoint
- Rewrite .gitlab-ci.yml with proper stages, blocking tests, staging
  environment on dev box, and SSH-based production deploy to 71.85.90.6
- Add POST /api/health endpoint for pipeline verification
- Add POST /atlas/hosts/:hostId/refresh-cache for Atlas cache staleness
- AtlasSlideOutPanel: auto-resolve qualys_id from Atlas vulnerabilities,
  prefer qualys_id over active_host_findings_id, retry on failure
- Add FeedbackModal component with bug report button in header and
  feature request in UserMenu, creates GitLab issues via /api/feedback
- Fix all frontend test failures (ESM transforms, TextDecoder polyfill,
  fast-check resolution, App.test.js boilerplate replacement)
- Fix root package.json test script to run jest
- Add deploy/ directory with staging systemd service and setup script
2026-05-08 12:50:11 -06:00
Jordan Ramos
86fdd084ac docs: update README and reference manual for PostgreSQL migration and systemd scripts 2026-05-08 09:17:38 -06:00