cve-dashboard/.kiro/specs/sync-anomaly-detection/tasks.md

Implementation Plan: Sync Anomaly Detection and BU Drift Monitoring

Overview

This plan implements the sync anomaly detection feature in incremental steps: database migration first, then core classification and summary logic, BU tracking in the sync pipeline, new API endpoints, archive transition enhancement, and finally the React anomaly banner. Each task builds on the previous, with property-based tests validating correctness properties from the design document.

Tasks

• 1. Create database migration script

  • 1.1 Create backend/migrations/add_sync_anomaly_tables.js

    • Create ivanti_sync_anomaly_log table with columns: id (autoincrement PK), sync_timestamp (datetime NOT NULL DEFAULT CURRENT_TIMESTAMP), open_count_delta (integer NOT NULL DEFAULT 0), closed_count_delta (integer NOT NULL DEFAULT 0), newly_archived_count (integer NOT NULL DEFAULT 0), returned_count (integer NOT NULL DEFAULT 0), classification_json (text NOT NULL DEFAULT '{}'), is_significant (integer NOT NULL DEFAULT 0), created_at (datetime DEFAULT CURRENT_TIMESTAMP)
    • Create ivanti_finding_bu_history table with columns: id (autoincrement PK), finding_id (text NOT NULL), finding_title (text NOT NULL DEFAULT ''), host_name (text NOT NULL DEFAULT ''), previous_bu (text NOT NULL), new_bu (text NOT NULL), detected_at (datetime NOT NULL DEFAULT CURRENT_TIMESTAMP), created_at (datetime DEFAULT CURRENT_TIMESTAMP)
    • Create index idx_anomaly_sync_timestamp on ivanti_sync_anomaly_log(sync_timestamp)
    • Create index idx_bu_history_finding_id on ivanti_finding_bu_history(finding_id)
    • Create index idx_bu_history_detected_at on ivanti_finding_bu_history(detected_at)
    • Use CREATE TABLE IF NOT EXISTS and CREATE INDEX IF NOT EXISTS for idempotency
    • Follow the standalone Node script pattern from add_closed_gone_state.js (open DB directly, promise-based helpers, run/all wrappers)
    • Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6

  • 1.2 Run migration and verify tables exist

    • Execute node backend/migrations/add_sync_anomaly_tables.js
    • Verify both tables and all three indexes were created
    • Run migration a second time to confirm idempotency (no errors on re-run)
    • Requirements: 4.6

• 2. Implement BU drift classifier and batch logic

  • 2.1 Implement runBUDriftChecker function in backend/routes/ivantiFindings.js

    • Add runBUDriftChecker(db, newlyArchivedIds, apiKey, clientId, skipTls) async function
    • Chunk newlyArchivedIds into batches of 50
    • For each batch, call ivantiPost() with a filter on id field only (no BU, severity, or state filters) using the unfiltered query pattern from bu-reassignment-check.js
    • Classify each finding: BU not in {NTS-AEO-ACCESS-ENG, NTS-AEO-STEAM} → bu_reassignment; BU matches + severity < 8.5 → severity_drift; BU matches + state Closed → closed_on_platform; not found → decommissioned
    • Update the corresponding ivanti_archive_transitions row's reason field with the formatted classification
    • Return classification summary object: { bu_reassignment: N, severity_drift: N, closed_on_platform: N, decommissioned: N }
    • Wrap each batch in try/catch — log errors, skip failed batches, never throw
    • Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7

  • * 2.2 Write property test for classification correctness

    • Property 1: Classification correctness
    • Generate random {bu, severity, state, found} tuples, verify classifier produces the correct classification based on BU value, severity threshold (8.5), and state
    • Validates: Requirements 1.2, 1.3, 1.4, 1.5

  • * 2.3 Write property test for archive transition reason formatting

    • Property 2: Archive transition reason formatting
    • Generate random classification results with BU values and severity scores, verify reason string format: bu_reassignment:B, severity_drift:S, closed_on_platform, decommissioned
    • Validates: Requirements 6.1, 6.2, 6.3, 6.4

  • * 2.4 Write property test for batch size constraint

    • Property 3: Batch size constraint
    • Generate random-length ID arrays (0 to 500), verify chunking produces ceil(N/50) batches, each batch has at most 50 IDs, and the union of all batches equals the original list
    • Validates: Requirements 1.7

• 3. Implement anomaly summary computation

  • 3.1 Implement computeAnomalySummary function in backend/routes/ivantiFindings.js

    • Add computeAnomalySummary(db, openCountDelta, closedCountDelta, newlyArchivedCount, returnedCount, classificationBreakdown) async function
    • Compute is_significant: true if newlyArchivedCount > 5
    • Insert a row into ivanti_sync_anomaly_log with all fields
    • Log the summary to console
    • Wrap in try/catch — log errors, never throw (anomaly summary is informational)
    • Requirements: 2.1, 2.4, 2.7

  • * 3.2 Write property test for significance threshold

    • Property 4: Significance threshold
    • Generate random non-negative integers for newly_archived_count, verify is_significant is true if and only if newly_archived_count > 5
    • Validates: Requirements 2.7

  • * 3.3 Write property test for count delta computation

    • Property 5: Count delta computation
    • Generate random pairs of non-negative integers (previous_count, current_count), verify delta equals current_count - previous_count for both open and closed counts
    • Validates: Requirements 2.1

• 4. Checkpoint — Verify core logic

  • Ensure all tests pass, ask the user if questions arise.

• 5. Integrate BU comparison into syncFindings

  • 5.1 Add per-finding BU comparison logic to syncFindings() in backend/routes/ivantiFindings.js

    • After reading previousFindings and before writing the new cache, compare each finding's buOwnership against the previous finding's buOwnership
    • When both values are non-empty and differ, insert a row into ivanti_finding_bu_history with finding_id, finding_title, host_name, previous_bu, new_bu, and detected_at
    • When a finding appears for the first time (no previous entry), store the BU without recording a change event
    • Wrap in try/catch per finding — log errors, continue processing remaining findings
    • Requirements: 3.1, 3.2, 3.3, 3.6

  • * 5.2 Write property test for BU extraction preservation

    • Property 6: BU extraction preservation
    • Generate random raw Ivanti finding objects with varying assetCustomAttributes['1550_host_1'] arrays, verify extractFinding produces a finding whose buOwnership equals the first element of that array
    • Validates: Requirements 3.1

  • * 5.3 Write property test for BU change detection

    • Property 7: BU change detection and recording
    • Generate random previous/current finding pairs with varying BU values (same, different, empty), verify that a BU history row is inserted only when both values are non-empty and differ
    • Validates: Requirements 3.2, 3.3, 3.6

• 6. Wire drift checker and anomaly summary into sync pipeline

  • 6.1 Integrate runBUDriftChecker and computeAnomalySummary into syncFindings() flow
    • Collect newlyArchivedIds from detectArchiveChanges (modify it to return the list of disappeared IDs)
    • Collect returnedCount from detectArchiveChanges (count of ARCHIVED → RETURNED transitions)
    • After detectClosedGoneFindings, call runBUDriftChecker with the newly archived IDs
    • Compute openCountDelta and closedCountDelta by comparing current counts against previous counts from ivanti_counts_cache
    • Call computeAnomalySummary with all collected metrics
    • Wrap both calls in try/catch — failures are non-fatal and must not block sync completion
    • Export runBUDriftChecker, computeAnomalySummary, and extractFinding from the module for testing
    • Requirements: 1.1, 2.1, 2.2, 2.3, 2.4, 2.7

• 7. Checkpoint — Verify pipeline integration

  • Ensure all tests pass, ask the user if questions arise.

• 8. Add anomaly and BU history API endpoints

  • 8.1 Add GET /anomaly/latest endpoint to createIvantiFindingsRouter

    • Query ivanti_sync_anomaly_log for the row with the maximum sync_timestamp
    • Parse classification_json into an object in the response
    • Return { anomaly: row } or { anomaly: null } if no records exist
    • Requirements: 2.5

  • 8.2 Add GET /anomaly/history endpoint to createIvantiFindingsRouter

    • Accept optional from and to query parameters (ISO date strings)
    • If date params provided, filter by sync_timestamp range (inclusive)
    • If no date params, return last 30 rows ordered by sync_timestamp descending
    • Parse classification_json into an object for each row
    • Return each row with: sync_timestamp, open_count_delta, closed_count_delta, newly_archived_count, returned_count, classification (parsed object), is_significant
    • Requirements: 2.6, 7.1, 7.2, 7.3, 7.4

  • 8.3 Add GET /bu-changes endpoint to createIvantiFindingsRouter

    • Query all rows from ivanti_finding_bu_history ordered by detected_at descending
    • Return { changes: rows }
    • Requirements: 3.4

  • 8.4 Add GET /:findingId/bu-history endpoint to createIvantiFindingsRouter

    • Query ivanti_finding_bu_history where finding_id matches the URL param, ordered by detected_at descending
    • Return { finding_id, history: rows }
    • Place this route definition carefully to avoid conflicts with existing /:findingId/note and /:findingId/override routes
    • Requirements: 3.5

  • * 8.5 Write property tests for API query behaviors

    • Property 8: Latest anomaly returns most recent — Generate random sequences of anomaly rows with distinct timestamps, verify /anomaly/latest returns the row with the maximum timestamp
    • Property 9: Anomaly history ordering and limit — Generate random sets of N anomaly rows, verify /anomaly/history returns min(N, 30) rows ordered by timestamp descending
    • Property 10: Date-range filtering with complete response shape — Generate random date ranges and anomaly rows, verify only rows within range are returned with correct fields
    • Property 11: BU changes endpoint ordering — Generate random BU change rows, verify /bu-changes returns all rows ordered by detected_at descending
    • Property 12: Per-finding BU history filtering — Generate random BU history rows across multiple findings, verify /:findingId/bu-history returns only matching rows ordered by detected_at descending
    • Validates: Requirements 2.5, 2.6, 3.4, 3.5, 7.1, 7.2, 7.4

• 9. Checkpoint — Verify endpoints and properties

  • Ensure all tests pass, ask the user if questions arise.

• 10. Implement Anomaly Banner UI component

  • 10.1 Create frontend/src/components/pages/AnomalyBanner.js

    • Fetch latest anomaly summary from /api/ivanti/findings/anomaly/latest on mount
    • If is_significant is false or no anomaly exists, render nothing
    • If is_significant is true, render a warning banner with:
      • Amber background tint (rgba(245, 158, 11, 0.15)) with amber border (rgba(245, 158, 11, 0.3))
      • AlertTriangle icon from lucide-react
      • Summary text showing total count change and classification breakdown (e.g., "45 findings archived — 38 BU reassignment, 5 severity drift, 2 decommissioned")
      • Expandable detail section (click to toggle) for affected findings grouped by classification
      • Dismiss button (X icon) that hides the banner for the current session via useState
    • Use monospace typography (JetBrains Mono) and dark theme colors per DESIGN_SYSTEM.md
    • Match the inline style pattern used by IvantiCountsChart.js
    • Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7

  • 10.2 Integrate AnomalyBanner into the Vulnerability Triage page

    • Import and render AnomalyBanner above the IvantiCountsChart component on the Vulnerability Triage page
    • Requirements: 5.1

• 11. Enhance archive transition reasons

  • 11.1 Verify archive transition reason updates from runBUDriftChecker
    • Confirm that runBUDriftChecker (task 2.1) correctly updates ivanti_archive_transitions.reason with formatted values: bu_reassignment:<new_bu>, severity_drift:<new_severity>, closed_on_platform, decommissioned
    • Confirm existing rows with severity_score_drift are not modified — enhanced reasons apply only to new transitions
    • Requirements: 6.1, 6.2, 6.3, 6.4, 6.5

• 12. Final checkpoint — Ensure all tests pass

  • Ensure all tests pass, ask the user if questions arise.

Notes

• Tasks marked with * are optional and can be skipped for faster MVP
• Each task references specific requirements for traceability
• Checkpoints ensure incremental validation after each major integration point
• Property tests validate the 12 correctness properties defined in the design document
• The migration must be run before any other tasks (task 1)
• All new functions are added to the existing ivantiFindings.js module — no new route files
• The BU comparison in syncFindings (task 5) must happen before the cache is overwritten, which is the only point where both old and new BU values are in memory