cve-dashboard/.kiro/specs/sync-anomaly-detection/tasks.md

# Implementation Plan: Sync Anomaly Detection and BU Drift Monitoring

## Overview

This plan implements the sync anomaly detection feature in incremental steps: database migration first, then core classification and summary logic, BU tracking in the sync pipeline, new API endpoints, archive transition enhancement, and finally the React anomaly banner. Each task builds on the previous, with property-based tests validating correctness properties from the design document.

## Tasks

- [x] 1. Create database migration script
  - [x] 1.1 Create `backend/migrations/add_sync_anomaly_tables.js`
    - Create `ivanti_sync_anomaly_log` table with columns: id (autoincrement PK), sync_timestamp (datetime NOT NULL DEFAULT CURRENT_TIMESTAMP), open_count_delta (integer NOT NULL DEFAULT 0), closed_count_delta (integer NOT NULL DEFAULT 0), newly_archived_count (integer NOT NULL DEFAULT 0), returned_count (integer NOT NULL DEFAULT 0), classification_json (text NOT NULL DEFAULT '{}'), is_significant (integer NOT NULL DEFAULT 0), created_at (datetime DEFAULT CURRENT_TIMESTAMP)
    - Create `ivanti_finding_bu_history` table with columns: id (autoincrement PK), finding_id (text NOT NULL), finding_title (text NOT NULL DEFAULT ''), host_name (text NOT NULL DEFAULT ''), previous_bu (text NOT NULL), new_bu (text NOT NULL), detected_at (datetime NOT NULL DEFAULT CURRENT_TIMESTAMP), created_at (datetime DEFAULT CURRENT_TIMESTAMP)
    - Create index `idx_anomaly_sync_timestamp` on `ivanti_sync_anomaly_log(sync_timestamp)`
    - Create index `idx_bu_history_finding_id` on `ivanti_finding_bu_history(finding_id)`
    - Create index `idx_bu_history_detected_at` on `ivanti_finding_bu_history(detected_at)`
    - Use `CREATE TABLE IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS` for idempotency
    - Follow the standalone Node script pattern from `add_closed_gone_state.js` (open DB directly, promise-based helpers, run/all wrappers)
    - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6_

  - [x] 1.2 Run migration and verify tables exist
    - Execute `node backend/migrations/add_sync_anomaly_tables.js`
    - Verify both tables and all three indexes were created
    - Run migration a second time to confirm idempotency (no errors on re-run)
    - _Requirements: 4.6_

- [x] 2. Implement BU drift classifier and batch logic
  - [x] 2.1 Implement `runBUDriftChecker` function in `backend/routes/ivantiFindings.js`
    - Add `runBUDriftChecker(db, newlyArchivedIds, apiKey, clientId, skipTls)` async function
    - Chunk `newlyArchivedIds` into batches of 50
    - For each batch, call `ivantiPost()` with a filter on `id` field only (no BU, severity, or state filters) using the unfiltered query pattern from `bu-reassignment-check.js`
    - Classify each finding: BU not in {NTS-AEO-ACCESS-ENG, NTS-AEO-STEAM} → `bu_reassignment`; BU matches + severity < 8.5 → `severity_drift`; BU matches + state Closed → `closed_on_platform`; not found → `decommissioned`
    - Update the corresponding `ivanti_archive_transitions` row's reason field with the formatted classification
    - Return classification summary object: `{ bu_reassignment: N, severity_drift: N, closed_on_platform: N, decommissioned: N }`
    - Wrap each batch in try/catch — log errors, skip failed batches, never throw
    - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7_

  - [ ]* 2.2 Write property test for classification correctness
    - **Property 1: Classification correctness**
    - Generate random {bu, severity, state, found} tuples, verify classifier produces the correct classification based on BU value, severity threshold (8.5), and state
    - **Validates: Requirements 1.2, 1.3, 1.4, 1.5**

  - [ ]* 2.3 Write property test for archive transition reason formatting
    - **Property 2: Archive transition reason formatting**
    - Generate random classification results with BU values and severity scores, verify reason string format: `bu_reassignment:B`, `severity_drift:S`, `closed_on_platform`, `decommissioned`
    - **Validates: Requirements 6.1, 6.2, 6.3, 6.4**

  - [ ]* 2.4 Write property test for batch size constraint
    - **Property 3: Batch size constraint**
    - Generate random-length ID arrays (0 to 500), verify chunking produces ceil(N/50) batches, each batch has at most 50 IDs, and the union of all batches equals the original list
    - **Validates: Requirements 1.7**

- [x] 3. Implement anomaly summary computation
  - [x] 3.1 Implement `computeAnomalySummary` function in `backend/routes/ivantiFindings.js`
    - Add `computeAnomalySummary(db, openCountDelta, closedCountDelta, newlyArchivedCount, returnedCount, classificationBreakdown)` async function
    - Compute `is_significant`: true if `newlyArchivedCount > 5`
    - Insert a row into `ivanti_sync_anomaly_log` with all fields
    - Log the summary to console
    - Wrap in try/catch — log errors, never throw (anomaly summary is informational)
    - _Requirements: 2.1, 2.4, 2.7_

  - [ ]* 3.2 Write property test for significance threshold
    - **Property 4: Significance threshold**
    - Generate random non-negative integers for `newly_archived_count`, verify `is_significant` is true if and only if `newly_archived_count > 5`
    - **Validates: Requirements 2.7**

  - [ ]* 3.3 Write property test for count delta computation
    - **Property 5: Count delta computation**
    - Generate random pairs of non-negative integers (previous_count, current_count), verify delta equals `current_count - previous_count` for both open and closed counts
    - **Validates: Requirements 2.1**

- [x] 4. Checkpoint — Verify core logic
  - Ensure all tests pass, ask the user if questions arise.

- [x] 5. Integrate BU comparison into syncFindings
  - [x] 5.1 Add per-finding BU comparison logic to `syncFindings()` in `backend/routes/ivantiFindings.js`
    - After reading `previousFindings` and before writing the new cache, compare each finding's `buOwnership` against the previous finding's `buOwnership`
    - When both values are non-empty and differ, insert a row into `ivanti_finding_bu_history` with finding_id, finding_title, host_name, previous_bu, new_bu, and detected_at
    - When a finding appears for the first time (no previous entry), store the BU without recording a change event
    - Wrap in try/catch per finding — log errors, continue processing remaining findings
    - _Requirements: 3.1, 3.2, 3.3, 3.6_

  - [ ]* 5.2 Write property test for BU extraction preservation
    - **Property 6: BU extraction preservation**
    - Generate random raw Ivanti finding objects with varying `assetCustomAttributes['1550_host_1']` arrays, verify `extractFinding` produces a finding whose `buOwnership` equals the first element of that array
    - **Validates: Requirements 3.1**

  - [ ]* 5.3 Write property test for BU change detection
    - **Property 7: BU change detection and recording**
    - Generate random previous/current finding pairs with varying BU values (same, different, empty), verify that a BU history row is inserted only when both values are non-empty and differ
    - **Validates: Requirements 3.2, 3.3, 3.6**

- [x] 6. Wire drift checker and anomaly summary into sync pipeline
  - [x] 6.1 Integrate `runBUDriftChecker` and `computeAnomalySummary` into `syncFindings()` flow
    - Collect `newlyArchivedIds` from `detectArchiveChanges` (modify it to return the list of disappeared IDs)
    - Collect `returnedCount` from `detectArchiveChanges` (count of ARCHIVED → RETURNED transitions)
    - After `detectClosedGoneFindings`, call `runBUDriftChecker` with the newly archived IDs
    - Compute `openCountDelta` and `closedCountDelta` by comparing current counts against previous counts from `ivanti_counts_cache`
    - Call `computeAnomalySummary` with all collected metrics
    - Wrap both calls in try/catch — failures are non-fatal and must not block sync completion
    - Export `runBUDriftChecker`, `computeAnomalySummary`, and `extractFinding` from the module for testing
    - _Requirements: 1.1, 2.1, 2.2, 2.3, 2.4, 2.7_

- [x] 7. Checkpoint — Verify pipeline integration
  - Ensure all tests pass, ask the user if questions arise.

- [x] 8. Add anomaly and BU history API endpoints
  - [x] 8.1 Add `GET /anomaly/latest` endpoint to `createIvantiFindingsRouter`
    - Query `ivanti_sync_anomaly_log` for the row with the maximum `sync_timestamp`
    - Parse `classification_json` into an object in the response
    - Return `{ anomaly: row }` or `{ anomaly: null }` if no records exist
    - _Requirements: 2.5_

  - [x] 8.2 Add `GET /anomaly/history` endpoint to `createIvantiFindingsRouter`
    - Accept optional `from` and `to` query parameters (ISO date strings)
    - If date params provided, filter by `sync_timestamp` range (inclusive)
    - If no date params, return last 30 rows ordered by `sync_timestamp` descending
    - Parse `classification_json` into an object for each row
    - Return each row with: sync_timestamp, open_count_delta, closed_count_delta, newly_archived_count, returned_count, classification (parsed object), is_significant
    - _Requirements: 2.6, 7.1, 7.2, 7.3, 7.4_

  - [x] 8.3 Add `GET /bu-changes` endpoint to `createIvantiFindingsRouter`
    - Query all rows from `ivanti_finding_bu_history` ordered by `detected_at` descending
    - Return `{ changes: rows }`
    - _Requirements: 3.4_

  - [x] 8.4 Add `GET /:findingId/bu-history` endpoint to `createIvantiFindingsRouter`
    - Query `ivanti_finding_bu_history` where `finding_id` matches the URL param, ordered by `detected_at` descending
    - Return `{ finding_id, history: rows }`
    - Place this route definition carefully to avoid conflicts with existing `/:findingId/note` and `/:findingId/override` routes
    - _Requirements: 3.5_

  - [ ]* 8.5 Write property tests for API query behaviors
    - **Property 8: Latest anomaly returns most recent** — Generate random sequences of anomaly rows with distinct timestamps, verify `/anomaly/latest` returns the row with the maximum timestamp
    - **Property 9: Anomaly history ordering and limit** — Generate random sets of N anomaly rows, verify `/anomaly/history` returns min(N, 30) rows ordered by timestamp descending
    - **Property 10: Date-range filtering with complete response shape** — Generate random date ranges and anomaly rows, verify only rows within range are returned with correct fields
    - **Property 11: BU changes endpoint ordering** — Generate random BU change rows, verify `/bu-changes` returns all rows ordered by `detected_at` descending
    - **Property 12: Per-finding BU history filtering** — Generate random BU history rows across multiple findings, verify `/:findingId/bu-history` returns only matching rows ordered by `detected_at` descending
    - **Validates: Requirements 2.5, 2.6, 3.4, 3.5, 7.1, 7.2, 7.4**

- [x] 9. Checkpoint — Verify endpoints and properties
  - Ensure all tests pass, ask the user if questions arise.

- [x] 10. Implement Anomaly Banner UI component
  - [x] 10.1 Create `frontend/src/components/pages/AnomalyBanner.js`
    - Fetch latest anomaly summary from `/api/ivanti/findings/anomaly/latest` on mount
    - If `is_significant` is false or no anomaly exists, render nothing
    - If `is_significant` is true, render a warning banner with:
      - Amber background tint (`rgba(245, 158, 11, 0.15)`) with amber border (`rgba(245, 158, 11, 0.3)`)
      - `AlertTriangle` icon from lucide-react
      - Summary text showing total count change and classification breakdown (e.g., "45 findings archived — 38 BU reassignment, 5 severity drift, 2 decommissioned")
      - Expandable detail section (click to toggle) for affected findings grouped by classification
      - Dismiss button (X icon) that hides the banner for the current session via `useState`
    - Use monospace typography (`JetBrains Mono`) and dark theme colors per `DESIGN_SYSTEM.md`
    - Match the inline style pattern used by `IvantiCountsChart.js`
    - _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7_

  - [x] 10.2 Integrate `AnomalyBanner` into the Vulnerability Triage page
    - Import and render `AnomalyBanner` above the `IvantiCountsChart` component on the Vulnerability Triage page
    - _Requirements: 5.1_

- [x] 11. Enhance archive transition reasons
  - [x] 11.1 Verify archive transition reason updates from `runBUDriftChecker`
    - Confirm that `runBUDriftChecker` (task 2.1) correctly updates `ivanti_archive_transitions.reason` with formatted values: `bu_reassignment:<new_bu>`, `severity_drift:<new_severity>`, `closed_on_platform`, `decommissioned`
    - Confirm existing rows with `severity_score_drift` are not modified — enhanced reasons apply only to new transitions
    - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_

- [x] 12. Final checkpoint — Ensure all tests pass
  - Ensure all tests pass, ask the user if questions arise.

## Notes

- Tasks marked with `*` are optional and can be skipped for faster MVP
- Each task references specific requirements for traceability
- Checkpoints ensure incremental validation after each major integration point
- Property tests validate the 12 correctness properties defined in the design document
- The migration must be run before any other tasks (task 1)
- All new functions are added to the existing `ivantiFindings.js` module — no new route files
- The BU comparison in `syncFindings` (task 5) must happen before the cache is overwritten, which is the only point where both old and new BU values are in memory