The DELETE /completed endpoint failed with a FK violation when completed
queue items had associated rows in jira_ticket_queue_items. Replaced the
bare DELETE query with a transaction that removes junction table references
before deleting the queue items themselves.
Transaction sequence: BEGIN → SELECT completed IDs → DELETE junction rows →
DELETE queue items → COMMIT, with ROLLBACK on error and client release in
finally block.
Per-metric remediation plan scoping (GitLab issue #19):
- Add metric_id column to compliance_item_history table (migration)
- Extend PATCH /items/:hostname/metadata to accept metric_id/metric_ids
for targeting specific metrics instead of all active items
- Add MetricChipSelector UI in detail panel for choosing which metrics
to apply resolution_date and remediation_plan changes to
- Display per-metric labels (MetricChip or 'All metrics') on history entries
- Backward compatible: omitting metric_ids preserves hostname-level behavior
CI/CD pipeline improvements:
- Add migration idempotency integration test (runs against real Postgres)
- Add post-deploy smoke tests for compliance and VCL endpoints
- Bump lint --max-warnings from 10 to 25
- Configure varsIgnorePattern for _ prefix convention on unused vars
Closes#19
- Drop CHECK constraint on jira_tickets.status to allow any status string
- Store raw Jira status directly in status column during sync (remove mapJiraStatusToLocal)
- Remove VALID_TICKET_STATUSES validation on create/update endpoints
- Remove separate Jira Status column from table (status IS the Jira status now)
- Update frontend status badges to color-code dynamically based on status category
- Update Open Tickets widget and CVE detail view to use isClosedStatus() helper
- Make filter dropdown dynamic based on actual ticket statuses
- Add migration script for dropping the constraint on other deployments
PostgreSQL DATE columns return JS Date objects which serialize to ISO
timestamps (e.g. 2025-05-22T00:00:00.000Z). The CalendarWidget expects
plain YYYY-MM-DD strings for its date key lookup. Added formatDate()
helper to normalize due_date and last_found_on before sending the
API response.
Library documents from the knowledge base were not checked against
the allowed file extensions before being sent to Ivanti. If a doc
had an unsupported type (e.g. .msg, .eml), Ivanti would reject the
entire workflow with a 400. Now validates library docs the same way
as local uploads and returns a clear error naming the offending file.
Allowed: .pdf, .png, .jpg, .jpeg, .gif, .doc, .docx, .xlsx, .csv, .txt, .zip
When the Ivanti API returns a non-success status, the error message
now includes the actual response body from Ivanti instead of just
the HTTP status code. This makes troubleshooting much easier since
you can see what Ivanti rejected (e.g. invalid field, too many
attachments, malformed request).
Select multiple queue items and create a single consolidated Jira ticket
with aggregated summary and description. Adds multi-select mode with
checkboxes, floating action bar, consolidation modal, and junction table
to track which queue items contributed to each ticket.
- Migration: jira_ticket_queue_items junction table
- POST /api/jira-tickets/:id/queue-items endpoint
- GET /api/ivanti/todo-queue/ticket-links endpoint
- ConsolidationModal component with aggregation logic
- IvantiTodoQueuePage with selection mode and ticket link badges
- Pure utility functions for summary/description generation
- 34 tests passing (backend + frontend)
- Add add_jira_sync_columns_pg.js migration (jira_id, jira_status, last_synced_at, created_by)
- Register in run-all.js before the flexible creation migration
- Replace all generic 'Internal server error' with actual err.message in jiraTickets routes
- Users and admins can now see the real failure reason instead of a useless generic message
- Pass through actual Jira error details instead of generic 'Jira API error'
- Parse errorMessages and errors from Jira response for human-readable display
- Make cve_id and vendor optional on local POST /api/jira-tickets (for Save to Dashboard)
- Update getIssue comment for clarity (logic unchanged — JQL search per compliance spec)
- Replace issue type text input with dropdown of STEAM project types (Story default)
- Add Save to Dashboard button on lookup results to link existing Jira tickets locally
- Make cve_id and vendor optional on local POST /api/jira-tickets endpoint
- Fix: use normalized values in local ticket INSERT query
Make CVE ID and Vendor optional when creating Jira tickets. Add source_context
field to track ticket origin (cve, archer, ivanti_queue, email, manual).
- Migration: drop NOT NULL on cve_id/vendor, add source_context column with CHECK
- Backend: update create/update/get endpoints for optional fields and source_context
- Frontend: update creation modal with optional labels and source context dropdown
- Add Create Jira Ticket action from Ivanti queue (pre-populates from finding)
- Add Create Jira Ticket action from Archer detail view (pre-populates from ticket)
- Add source context badge column, filter dropdown, and search to ticket list
Flip stacked bar chart so non-compliant (orange) renders on top and
compliant (blue) on bottom for better visual emphasis.
Use the file's report_date for compliance_snapshots month instead of
the current date, so historical uploads land in the correct monthly
bucket. Also fix rollback to delete the correct month's snapshot.
Remove cve-frontend systemd service ( Express serves theredundant
built frontend on port 3001).
Devices appearing in multiple verticals were counted multiple times,
causing non_compliant > totalAssets and negative compliance percentages.
Deduplicate by hostname before passing to the forecast helper.
- Fix Date object handling for resolution_date from PostgreSQL
- Fix totalAssets using per-metric summary (vcl_multi_vertical_summary)
instead of vertical-level compliance_snapshots total_devices
- Fix duplicate current month in chart (forecast starts from next month)
- Fix multi-vertical metrics summing across all relevant verticals
- Fix bar stacking: orange (non-compliant) on bottom, blue (compliant)
on top, both sharing same baseline (stacked to total)
- Add fill props to Bar components for correct legend colors
- Backfill historical snapshots with per-metric totalAssets
New feature: combined historical + forecast burndown chart with metric
selector on the CCP Metrics page. Shows stacked bars (total assets vs
non-compliant) with a compliance percentage trend line. A bold divider
separates actual historical data from projected future remediation.
Forecast assumes constant asset count and on-schedule remediation plans.
Backend:
- computeMetricForecastBurndown helper in vclHelpers.js (pure function)
- GET /api/compliance/vcl-multi/metrics-list endpoint
- GET /api/compliance/vcl-multi/metric/:metricId/forecast-burndown endpoint
Frontend:
- MetricSelector dropdown with device counts per metric
- ForecastBurndownChart using recharts ComposedChart (Bar + Line + ReferenceLine)
- Forecast bars render at 50% opacity to distinguish from actuals
- Race condition handling for rapid metric switching
- Queue panel width increased from 420px to 600px
Closes#18
- Feedback modal now supports up to 3 image attachments (PNG/JPG/GIF/WebP, 5MB
each) with thumbnail previews. Images are uploaded to GitLab project uploads
and embedded as markdown in the issue description.
- New webhook endpoint (POST /api/webhooks/gitlab) receives issue close events,
parses the submitter from the description, looks up their email, and sends a
Webex DM via the Patches O'Houlihan bot.
- New helper: backend/helpers/webexBot.js (fire-and-forget DM sender).
- Requires WEBEX_BOT_TOKEN and GITLAB_WEBHOOK_SECRET in backend/.env.
Deduplicate (hostname, metric_id) rows across verticals using DISTINCT ON in
GET /items, GET /items/:hostname, GET /vcl/stats (heavy-hitters + forecast),
GET /mttr, and persistUpload() snapshot block. Add defensive groupByHostname
Set and hostname_status CTE for snapshot classification.
Includes 38 property-based tests (11 exploration + 27 preservation) covering
all six affected sites.
Closes#13
Aggregate /trends, /top-recurring, /category-trend by report_date instead of
per-upload row. Add sibling-upload disclosure to /summary. Filter persistUpload
snapshot query by the upload's vertical to prevent cross-vertical contamination.
Fixes GitLab #12 (reported by nkapur — STEAM active findings chart showed 3
entries for 5/11 after uploading three vertical data sets for that date).
Includes 30 property-based tests covering bug condition and preservation.
New table compliance_item_history stores an append-only audit trail of
changes to resolution_date and remediation_plan. The current values remain
on compliance_items for fast VCL reporting queries (no double-counting).
Backend:
- Migration: creates compliance_item_history with indexes
- PATCH /items/:hostname/metadata: records old→new in history before updating,
accepts optional change_reason field (max 500 chars)
- GET /items/:hostname: returns history array (last 10 entries, newest first)
- POST /vcl/bulk-commit: records history for each changed field per hostname
Frontend:
- ComplianceDetailPanel: added change reason input below Save button
- Added Change History section showing field changes with timestamps,
usernames, old→new values, and reasons
- Re-fetches detail after save to show updated history immediately
Tests updated to match new transaction-based PATCH flow.
The /summary endpoint was fetching the most recent upload regardless of
vertical, which on dev was a PRDCT_VSO multi-vertical upload. Now it
looks for AEO uploads (vertical IS NULL) first, then falls back to the
NTS_AEO multi-vertical upload.
The /items endpoint now includes items from both vertical IS NULL and
vertical = 'NTS_AEO' so the AEO compliance page shows devices uploaded
through either flow.
Clicking the Non-Compliant card on the CCP Metrics overview now toggles a
panel of metric buttons below it, each showing the metric ID, category,
non-compliant count, and compliance % vs target. Styled like the compliance
page's MetricHealthCard pattern.
Backend: added metric_breakdown to the /stats response — aggregated
cross-vertical metric totals (ALL: rows only, grouped by metric_id).
Also updated tech steering file to document the single-port Express
architecture and the requirement to run npm run build after frontend changes.
Clicking a metric now shows a sub-team breakdown page with totals per team
(compliant, non-compliant, total, %) instead of jumping directly to a flat
device list. Clicking a sub-team then shows the device list filtered to
that team only.
Navigation flow: Overview → Vertical → Metric (sub-team totals) → Team (devices)
Backend: added optional ?team= query param to the device list endpoint for
filtered queries.
Frontend: added MetricSubTeamView component with metric-level stats bar and
clickable sub-team table. Updated navigation state to include selectedTeam.
Also updated design brief to reflect the new drill-down hierarchy.
Backend: restructured /vertical/:code/metrics endpoint to return metrics
with nested sub_teams arrays. Each metric now has the ALL: rollup as the
primary row and individual team breakdowns (ACCESS-OPS, STEAM, etc.) as
sub_teams. Also returns a teams array for the filter UI.
Frontend: VerticalDetailView now supports two interaction modes:
- Expand/collapse: click the arrow on any metric row to reveal sub-team
breakdown inline (teal-highlighted rows beneath the parent)
- Team filter: click a team button to filter the entire table to show
only that team's numbers per metric
Both modes avoid double-counting by using the ALL: rollup for totals
and only showing sub-team data as supplementary detail.
The Summary sheet in each vertical spreadsheet contains both sub-team rows
(ACCESS-OPS, STEAM, INTELDEV, etc.) AND a rollup row (ALL: NTS-AEO) per
metric. The rollup row already includes all sub-team totals, so summing
all rows was double-counting every device.
Fixed in three places:
- GET /stats endpoint: added AND team LIKE 'ALL:%' filter
- persistMultiVerticalUpload snapshot creation: only sum ALL: entries
- GET /vertical/:code/metrics category aggregation: only use ALL: rows
Also ran a one-time data fix to correct existing compliance_snapshots.
The compliance_items table only contains non-compliant devices (detail
sheet rows). Compliant devices are never inserted — they only exist in
the Summary sheet totals. This caused Compliant to show 0 and
Compliance % to show 0% for all verticals.
Fix: stats endpoint now reads from vcl_multi_vertical_summary (parsed
Summary sheet data) for total/compliant/non-compliant counts. Snapshot
creation also uses summary data for accurate trend charting.
The compliance_items table is still used for:
- Donut chart (blocked vs in-progress based on resolution_date)
- Burndown forecast (devices with/without resolution dates)
- Device drill-down (actual non-compliant device list)
The requeue endpoint now handles three scenarios:
1. Original queue items still exist — uses their finding data (ideal case)
2. Queue items deleted (Clear Completed) — looks up findings from
ivanti_findings table using finding_ids_json
3. FP created outside dashboard (no queue_item_ids) — same fallback
to finding_ids_json and ivanti_findings lookup
4. Last resort — creates queue items with just finding IDs if the
findings aren't in ivanti_findings either
New feature: users can re-queue findings from a rejected FP submission
back into the Ivanti todo queue under a different workflow type (FP,
Archer, CARD, GRANITE, or DECOM). Primary use case is when an FP is
rejected with a recommendation to submit an Archer risk acceptance.
Backend:
- New migration: add requeued_at column to ivanti_fp_submissions
- New endpoint: POST /api/ivanti/fp-workflow/submissions/:id/requeue
- Validates workflow_type and vendor (required for FP/Archer/DECOM)
- Creates new pending queue items from original finding data
- Marks submission as requeued (prevents double re-queue)
- Audit logs the action
Frontend (ReportingPage.js):
- RequeueConfirmDialog component with workflow type selector and vendor input
- Re-queue Findings button in Edit FP Modal header (rejected submissions only)
- Already re-queued label when submission.requeued_at is set
- Success notification on completion
When GET /submissions enriches submissions with Ivanti API data, it now
checks if batch.currentState (APPROVED, REJECTED, REWORK) differs from
the local lifecycle_status and updates the DB accordingly. This ensures
approved submissions get filtered out of the queue panel as intended.
Also changed safeText() to return null for non-string Ivanti note values
(arrays/objects) instead of JSON-stringifying them. The notes array
filters nulls via .filter(Boolean) so non-string data is simply hidden.
- Add DECOM to queue workflow types (red badge, inventory-style display)
- When findings are added as DECOM, auto-set note to 'DECOM' and hide row
- Hidden rows are excluded from donut charts (removes from pending count)
- Show CVEs on CARD/GRANITE/DECOM queue items (was previously omitted)
- Add backend/migrations/run-all.js for CI/CD auto-migration execution
- Pipeline now runs migrations before service restart on both staging and prod
- Add add_decom_workflow_type.js migration (updates CHECK constraint)
- Rewrite .gitlab-ci.yml with proper stages, blocking tests, staging
environment on dev box, and SSH-based production deploy to 71.85.90.6
- Add POST /api/health endpoint for pipeline verification
- Add POST /atlas/hosts/:hostId/refresh-cache for Atlas cache staleness
- AtlasSlideOutPanel: auto-resolve qualys_id from Atlas vulnerabilities,
prefer qualys_id over active_host_findings_id, retry on failure
- Add FeedbackModal component with bug report button in header and
feature request in UserMenu, creates GitLab issues via /api/feedback
- Fix all frontend test failures (ESM transforms, TextDecoder polyfill,
fast-check resolution, App.test.js boilerplate replacement)
- Fix root package.json test script to run jest
- Add deploy/ directory with staging systemd service and setup script
- Rewrite /fp-workflow-counts endpoint to query ivanti_findings table
directly with optional teams ILIKE filter (replaces pre-computed JSON blob)
- Frontend passes getActiveTeamsParam() to FP counts fetch
- FP counts refresh on scope toggle change alongside open/closed counts
- Both FP Finding Status and FP Workflow Status donuts now respect BU scope
- Create ivanti_counts_history_by_bu table (bu_ownership, state, count per sync)
- Sync writes per-BU snapshot alongside global history on each sync
- Seed table with current counts for immediate first data point
- GET /counts/history accepts ?teams param — queries per-BU table when filtered
- IvantiCountsChart accepts teamsParam prop, re-fetches on scope change
- ReportingPage passes getActiveTeamsParam() to the chart
- Historical per-BU data accumulates from this point forward
- Global history (no filter) still uses the original aggregate table
- Replace 2.6MB JSON blob with individual rows in ivanti_findings table
- Batch upsert via INSERT ... ON CONFLICT in chunks of 100
- Sync stores both open AND closed findings as rows with state column
- Per-BU closed counts now possible via SQL GROUP BY
- GET /findings queries indexed table with optional ILIKE BU filter
- GET /counts returns per-BU open+closed via GROUP BY state
- Notes and overrides are columns on ivanti_findings (no separate tables)
- Removed: readState, readStateWithNotes, _findingsCache, initTables
- Preserved: extractFinding, archive detection, FP workflow counts, anomaly log
- Response shape unchanged — frontend works without modification
- All 16 route files now import pool from ../db directly
- Removed db parameter from all factory functions
- All callbacks replaced with async/await pool.query()
- All ? placeholders converted to $1, $2... numbered params
- datetime('now') → NOW(), INSERT OR IGNORE → ON CONFLICT DO NOTHING
- LIKE → ILIKE for case-insensitive searches
- Error detection: err.code === '23505' for unique violations
- server.js no longer passes pool/db/requireAuth to route factories
- Only ivantiFindings.js still receives pool (pending task 8 rewrite)
- docs/guides/postgres-migration-plan.md: full migration manual with
phases, port allocation, rollback plan, and timeline
- .kiro/specs/postgres-migration/: requirements, design, and tasks
- Replaces findings_json blob with individual indexed rows
- Enables per-BU closed counts via SQL queries
- Uses existing Postgres instance (port 5432), new cve_dashboard DB
- Testing on port 3003, cutover to 3001 with 30s downtime
- Add bu_teams column to users table (migration + fresh schema)
- Create shared KNOWN_TEAMS constant and validateTeams helper
- Expose user teams in auth middleware, login, and /me responses
- Add bu_teams CRUD to user management routes with audit logging
- Make Ivanti FINDINGS_FILTERS configurable via IVANTI_BU_FILTER env var
- Add query-time team filtering to GET /findings and /findings/counts
- Update AuthContext with teams helpers and admin scope toggle
- Create AdminScopeToggle component (My Teams / All BUs)
- Scope ReportingPage findings fetch by user teams
- Scope CompliancePage team selector by user teams
- Scope ExportsPage findings exports by user teams
- Add BU teams multi-select to UserManagement create/edit forms
- Display team badges in user list table
