cve-dashboard/.kiro/specs/compliance-chart-replacements/design.md

Design Document: Compliance Chart Replacements

Overview

This feature replaces two underperforming charts in the ComplianceChartsPanel with more actionable visualizations:

Slot	Old Chart	New Chart
Chart 4	MTTR by Team (horizontal bar)	Aging Findings Distribution — stacked vertical bar showing active findings bucketed by seen_count with per-team color segments
Chart 5	Most Persistent Findings (horizontal bar)	Net Change Waterfall — waterfall bar chart showing per-cycle net movement (start → +new → +recurring → −resolved → end)

Both new charts derive data from columns already present in the SQLite schema (compliance_items.seen_count, compliance_uploads.new_count / recurring_count / resolved_count). No database migrations are required. The four other charts (Active Findings Over Time, Change per Cycle, Team Compliance Health, Archer Exception Pipeline) remain unchanged.

The backend reuses the existing route paths (/mttr → aging, /top-recurring → waterfall) so no new API routes are introduced. The frontend replaces the two component functions and their corresponding state variables while preserving the shared infrastructure (ChartCard, DarkTooltip, NoData, TEAM_COLORS, fmtDate, style tokens).

Architecture

The change is scoped to two files and follows the existing vertical-slice pattern:

graph LR
    subgraph Backend ["Express Backend"]
        R1["/compliance/mttr<br/>(was MTTR → now Aging)"]
        R2["/compliance/top-recurring<br/>(was Persistent → now Waterfall)"]
        DB[(SQLite<br/>compliance_items<br/>compliance_uploads)]
        R1 -->|"SELECT … GROUP BY bucket, team"| DB
        R2 -->|"SELECT … ORDER BY report_date"| DB
    end

    subgraph Frontend ["React Frontend"]
        Panel["ComplianceChartsPanel"]
        AC["AgingChart"]
        WC["WaterfallChart"]
        Panel --> AC
        Panel --> WC
    end

    R1 -->|JSON| AC
    R2 -->|JSON| WC

Key Architectural Decisions

1. Route path reuse — The old /mttr and /top-recurring paths are replaced in-place rather than creating new paths. This avoids route proliferation and means no proxy/CORS changes are needed.

2. No new dependencies — Both charts are built with the existing Recharts BarChart / Bar components. The waterfall effect is achieved using a transparent "base" Bar with fill="transparent" to offset the visible segments, which is a standard Recharts stacking technique.

3. No database migration — The aging chart queries compliance_items.seen_count (already populated by the upload pipeline). The waterfall chart reads new_count, recurring_count, resolved_count from compliance_uploads (already written by persistUpload). Starting/ending counts are computed in the route handler via a running accumulator.

4. Shared component reuse — Both new chart components use the existing ChartCard, DarkTooltip, NoData, TEAM_COLORS, AXIS_STYLE, GRID_STYLE, and LEGEND_STYLE tokens already defined in ComplianceChartsPanel.js.

Components and Interfaces

Backend: Aging Endpoint (replaces /mttr)

Route: GET /compliance/mttr

SQL Query:

SELECT
  CASE
    WHEN seen_count = 1       THEN '1 cycle'
    WHEN seen_count BETWEEN 2 AND 3 THEN '2–3 cycles'
    WHEN seen_count BETWEEN 4 AND 6 THEN '4–6 cycles'
    ELSE '7+ cycles'
  END AS bucket,
  team,
  COUNT(*) AS count
FROM compliance_items
WHERE status = 'active'
GROUP BY bucket, team
ORDER BY
  CASE bucket
    WHEN '1 cycle'     THEN 1
    WHEN '2–3 cycles'  THEN 2
    WHEN '4–6 cycles'  THEN 3
    WHEN '7+ cycles'   THEN 4
  END

Response shape:

{
  "aging": [
    { "bucket": "1 cycle",     "total": 12, "STEAM": 5, "ACCESS-ENG": 3, "ACCESS-OPS": 2, "INTELDEV": 2 },
    { "bucket": "2–3 cycles",  "total": 8,  "STEAM": 2, "ACCESS-ENG": 4, "ACCESS-OPS": 1, "INTELDEV": 1 },
    { "bucket": "4–6 cycles",  "total": 3,  "STEAM": 1, "ACCESS-ENG": 0, "ACCESS-OPS": 2, "INTELDEV": 0 },
    { "bucket": "7+ cycles",   "total": 2,  "STEAM": 0, "ACCESS-ENG": 1, "ACCESS-OPS": 0, "INTELDEV": 1 }
  ]
}

Handler logic:

1. Query active items grouped by bucket CASE expression and team.
2. Pivot the flat rows into one object per bucket with per-team counts and a total.
3. Return the array sorted by bucket order (ascending age).

Backend: Waterfall Endpoint (replaces /top-recurring)

Route: GET /compliance/top-recurring

SQL Query:

SELECT id, report_date,
       COALESCE(new_count, 0)       AS new_count,
       COALESCE(recurring_count, 0) AS recurring_count,
       COALESCE(resolved_count, 0)  AS resolved_count
FROM compliance_uploads
ORDER BY report_date ASC

Response shape:

{
  "waterfall": [
    {
      "date": "2026-04-13",
      "start": 0,
      "new_count": 15,
      "recurring_count": 0,
      "resolved_count": 0,
      "end": 15
    },
    {
      "date": "2026-04-20",
      "start": 15,
      "new_count": 3,
      "recurring_count": 12,
      "resolved_count": 5,
      "end": 25
    }
  ]
}

Handler logic:

1. Fetch all uploads ordered by report_date ASC.
2. Iterate with a running accumulator: start for cycle N = end of cycle N−1 (first cycle starts at 0).
3. Compute end = start + new_count + recurring_count - resolved_count.
4. Return the array.

Frontend: AgingChart Component

Replaces MttrChart. Renders a vertical stacked BarChart:

• X-axis: bucket labels (1 cycle, 2–3 cycles, 4–6 cycles, 7+ cycles)
• Y-axis: finding count
• Stacked bars: one <Bar> per team key from TEAM_COLORS, using stackId="aging"
• Tooltip: DarkTooltip showing bucket label, per-team counts, and total
• Empty state: <NoData /> when aging array is empty
• Wrapper: <ChartCard title="Aging Findings Distribution" subtitle="Active findings by age bucket — stacked by team" />

Frontend: WaterfallChart Component

Replaces RecurringChart. Renders a vertical BarChart with stacked transparent base:

• X-axis: cycle dates formatted via fmtDate
• Y-axis: finding count
• Bars (stacked):
  1. <Bar dataKey="start" stackId="w" fill="transparent" /> — invisible base
  2. <Bar dataKey="new_count" stackId="w" fill="#EF4444" /> — red new findings
  3. <Bar dataKey="recurring_count" stackId="w" fill="#F59E0B" /> — amber recurring
• Separate bar: <Bar dataKey="resolved_count" fill="#10B981" /> rendered as a standalone (not stacked) to visually represent the downward resolution. Since Recharts doesn't natively support negative waterfall segments, the resolved bar is rendered separately alongside the stacked group, matching the existing DeltaChart pattern already used in Chart 2.
• Tooltip: Custom WaterfallTooltip (extends DarkTooltip pattern) showing date, start, new, recurring, resolved, and end values.
• Empty state: <NoData /> when waterfall array is empty
• Wrapper: <ChartCard title="Net Change Waterfall" subtitle="Per-cycle net movement: start → +new → +recurring → −resolved → end" />

Frontend: State & Fetch Changes in ComplianceChartsPanel

Old	New
const [mttr, setMttr] = useState([])	const [aging, setAging] = useState([])
const [recurring, setRecurring] = useState([])	const [waterfall, setWaterfall] = useState([])
fetch(…/compliance/mttr) → setMttr(d.mttr)	fetch(…/compliance/mttr) → setAging(d.aging)
fetch(…/compliance/top-recurring) → setRecurring(d.items)	fetch(…/compliance/top-recurring) → setWaterfall(d.waterfall)

The Promise.all pattern and error handling remain identical. The other two fetches (/trends, /archer-tickets/status-trend) are untouched.

Code Removal

• Delete MttrChart function component
• Delete RecurringChart function component
• Remove mttr and recurring state variables and their setter calls
• Remove old /mttr handler SQL and logic in compliance.js
• Remove old /top-recurring handler SQL and logic in compliance.js

Data Models

No schema changes are required. The feature reads from existing columns:

compliance_items (read by Aging endpoint)

Column	Type	Usage
status	TEXT ('active' / 'resolved')	Filter to active items only
seen_count	INTEGER (default 1)	Bucketed into age groups via CASE expression
team	TEXT	Group-by for per-team stacked segments

compliance_uploads (read by Waterfall endpoint)

Column	Type	Usage
report_date	TEXT (YYYY-MM-DD)	X-axis label, ordering
new_count	INTEGER (default 0)	New findings in this cycle
recurring_count	INTEGER (default 0)	Recurring findings in this cycle
resolved_count	INTEGER (default 0)	Resolved findings in this cycle

Derived Waterfall Fields (computed in handler, not stored)

Field	Computation
start	Previous cycle's end (0 for first cycle)
end	start + new_count + recurring_count - resolved_count

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Aging bucketing correctness and conservation

For any array of active compliance items with arbitrary seen_count values (≥ 1) and arbitrary team assignments, applying the aging bucketing logic SHALL:

• assign every item to exactly one of the four buckets (1 cycle for seen_count = 1, 2–3 cycles for 2 ≤ seen_count ≤ 3, 4–6 cycles for 4 ≤ seen_count ≤ 6, 7+ cycles for seen_count ≥ 7),
• produce per-team counts within each bucket that sum to the bucket's total count, and
• produce bucket totals that sum to the total number of input items.

Validates: Requirements 1.1, 1.2

Property 2: Waterfall chain linkage and arithmetic invariant

For any ordered sequence of upload records with non-negative new_count, recurring_count, and resolved_count values, computing the waterfall SHALL produce a result where:

• the first row has start = 0,
• for every subsequent row N, start[N] = end[N-1] (chain linkage), and
• for every row, end = start + new_count + recurring_count - resolved_count (arithmetic invariant).

Validates: Requirements 3.1, 3.2, 3.3

Error Handling

Backend

Scenario	Behavior
No active findings (aging endpoint)	Return { aging: [] } with HTTP 200
No uploads (waterfall endpoint)	Return { waterfall: [] } with HTTP 200
SQLite query error (either endpoint)	Log error to console, return HTTP 500 with { error: "Database error" }
seen_count is NULL or missing	COALESCE(seen_count, 1) in the CASE expression treats NULL as 1 cycle
new_count / recurring_count / resolved_count is NULL	COALESCE(..., 0) in the SELECT ensures zero-default (already present in the uploads table pattern)

Frontend

Scenario	Behavior
Aging fetch fails (network error or non-200)	aging state remains [], AgingChart renders <NoData />
Waterfall fetch fails (network error or non-200)	waterfall state remains [], WaterfallChart renders <NoData />
One fetch fails, others succeed	Unaffected charts render normally — each fetch result is handled independently in the existing try/catch pattern
API returns unexpected shape	Component guards (data.length === 0) trigger <NoData /> gracefully

Testing Strategy

Property-Based Tests (fast-check)

The project backend already uses Jest. Property-based tests will use fast-check (already available in the Node ecosystem, zero-config with Jest).

Each property test runs a minimum of 100 iterations with randomly generated inputs.

Test 1: Aging bucketing correctness and conservation

• Tag: Feature: compliance-chart-replacements, Property 1: Aging bucketing correctness and conservation
• Generator: Array of objects { seen_count: fc.integer({ min: 1, max: 200 }), team: fc.constantFrom('STEAM', 'ACCESS-ENG', 'ACCESS-OPS', 'INTELDEV') }
• Assertion: Extract the pure bucketing/pivoting function from the route handler. For each generated array, verify:
  1. Every item maps to exactly one of the four bucket labels
  2. Per-team counts in each bucket sum to the bucket total
  3. All bucket totals sum to the input array length

Test 2: Waterfall chain linkage and arithmetic invariant

• Tag: Feature: compliance-chart-replacements, Property 2: Waterfall chain linkage and arithmetic invariant
• Generator: Array of objects { new_count: fc.nat({ max: 50 }), recurring_count: fc.nat({ max: 50 }), resolved_count: fc.nat({ max: 50 }), report_date: fc.date().map(d => d.toISOString().slice(0, 10)) }
• Assertion: Extract the pure waterfall computation function. For each generated sequence, verify:
  1. result[0].start === 0
  2. For all i > 0: result[i].start === result[i-1].end
  3. For all i: result[i].end === result[i].start + result[i].new_count + result[i].recurring_count - result[i].resolved_count

Unit Tests (Jest, example-based)

Test	Validates
Aging endpoint returns correct JSON shape with known seed data	Req 1.3
Aging endpoint returns empty array when no active items exist	Req 1.4
Aging endpoint returns 500 on DB error	Req 1.5
Waterfall endpoint returns empty array when no uploads exist	Req 3.5
Waterfall endpoint returns 500 on DB error	Req 3.6
AgingChart renders <NoData /> for empty data	Req 2.5
AgingChart renders ChartCard with correct title	Req 2.3
WaterfallChart renders <NoData /> for empty data	Req 4.5
WaterfallChart renders ChartCard with correct title	Req 4.6

Integration Tests

Test	Validates
Dashboard fetches /compliance/mttr and /compliance/top-recurring on mount	Req 5.1, 5.2
Dashboard still fetches /compliance/trends and /archer-tickets/status-trend	Req 5.3
All four fetches run in parallel via Promise.all	Req 5.4
Single fetch failure does not break other charts	Req 5.5

Smoke Tests

Test	Validates
/compliance/mttr route responds and does NOT return old { mttr: [...] } shape	Req 1.6
/compliance/top-recurring route responds and does NOT return old { items: [...] } shape	Req 3.7

Testability Design Note

To enable property-based testing, the bucketing logic and waterfall accumulator will be extracted as pure exported functions (e.g., bucketAgingItems(items) and computeWaterfall(uploads)) separate from the Express route handlers. This allows the property tests to call the pure functions directly without needing HTTP or database mocking.