141 lines
9.5 KiB
Markdown
141 lines
9.5 KiB
Markdown
# Implementation Plan: Compliance Chart Replacements
|
||
|
||
## Overview
|
||
|
||
Replace Chart 4 (MTTR by Team) and Chart 5 (Most Persistent Findings) in the compliance dashboard with two new visualizations: Aging Findings Distribution (stacked bar by age bucket) and Net Change Waterfall (per-cycle net movement). The implementation reuses existing route paths, requires no database migrations, and extracts pure functions for testability.
|
||
|
||
## Tasks
|
||
|
||
- [x] 1. Extract pure backend functions and implement aging endpoint
|
||
- [x] 1.1 Create `bucketAgingItems(items)` pure function in `cve-dashboard/backend/routes/compliance.js`
|
||
- Export a pure function that accepts an array of `{ seen_count, team }` objects
|
||
- Bucket each item into one of four age groups: `1 cycle` (seen_count = 1), `2–3 cycles` (2–3), `4–6 cycles` (4–6), `7+ cycles` (≥ 7)
|
||
- Pivot into one object per bucket with per-team counts and a total, sorted by ascending age
|
||
- Return the array in the response shape defined in the design: `[{ bucket, total, STEAM, ACCESS-ENG, ACCESS-OPS, INTELDEV }]`
|
||
- _Requirements: 1.1, 1.2, 1.3_
|
||
|
||
- [x] 1.2 Replace the `/mttr` route handler with the aging endpoint
|
||
- Remove the old MTTR SQL query and handler logic
|
||
- Add a new handler at `GET /mttr` that queries active compliance items grouped by bucket CASE expression and team
|
||
- Use the SQL from the design: `SELECT CASE WHEN seen_count = 1 THEN '1 cycle' ... END AS bucket, team, COUNT(*) AS count FROM compliance_items WHERE status = 'active' GROUP BY bucket, team ORDER BY ...`
|
||
- Call `bucketAgingItems()` to pivot the flat rows, then return `{ aging: [...] }`
|
||
- Return `{ aging: [] }` with HTTP 200 when no active findings exist
|
||
- Return HTTP 500 with `{ error: "Database error" }` on SQLite errors
|
||
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 6.3_
|
||
|
||
- [ ]* 1.3 Write property test for aging bucketing (Property 1)
|
||
- **Property 1: Aging bucketing correctness and conservation**
|
||
- Create `cve-dashboard/backend/__tests__/compliance-aging-bucketing.property.test.js`
|
||
- Generator: array of `{ seen_count: fc.integer({ min: 1, max: 200 }), team: fc.constantFrom('STEAM', 'ACCESS-ENG', 'ACCESS-OPS', 'INTELDEV') }`
|
||
- Assert: every item maps to exactly one bucket, per-team counts sum to bucket total, all bucket totals sum to input length
|
||
- **Validates: Requirements 1.1, 1.2**
|
||
|
||
- [ ]* 1.4 Write unit tests for aging endpoint
|
||
- Create `cve-dashboard/backend/__tests__/compliance-aging-endpoint.test.js`
|
||
- Test correct JSON shape with known seed data (Req 1.3)
|
||
- Test empty array returned when no active items exist (Req 1.4)
|
||
- Test HTTP 500 returned on DB error (Req 1.5)
|
||
- _Requirements: 1.3, 1.4, 1.5_
|
||
|
||
- [x] 2. Implement waterfall endpoint with pure function
|
||
- [x] 2.1 Create `computeWaterfall(uploads)` pure function in `cve-dashboard/backend/routes/compliance.js`
|
||
- Export a pure function that accepts an ordered array of `{ report_date, new_count, recurring_count, resolved_count }`
|
||
- Iterate with a running accumulator: first row starts at 0, each subsequent row's start = previous row's end
|
||
- Compute `end = start + new_count + recurring_count - resolved_count` for each row
|
||
- Return array of `{ date, start, new_count, recurring_count, resolved_count, end }`
|
||
- _Requirements: 3.1, 3.2, 3.3_
|
||
|
||
- [x] 2.2 Replace the `/top-recurring` route handler with the waterfall endpoint
|
||
- Remove the old persistent-findings SQL query and handler logic
|
||
- Add a new handler at `GET /top-recurring` that queries `compliance_uploads` ordered by `report_date ASC`
|
||
- Use `COALESCE` for null safety on count columns
|
||
- Call `computeWaterfall()` to build the chain, then return `{ waterfall: [...] }`
|
||
- Return `{ waterfall: [] }` with HTTP 200 when no uploads exist
|
||
- Return HTTP 500 with `{ error: "Database error" }` on SQLite errors
|
||
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 6.4_
|
||
|
||
- [ ]* 2.3 Write property test for waterfall computation (Property 2)
|
||
- **Property 2: Waterfall chain linkage and arithmetic invariant**
|
||
- Create `cve-dashboard/backend/__tests__/compliance-waterfall-chain.property.test.js`
|
||
- Generator: array of `{ new_count: fc.nat({ max: 50 }), recurring_count: fc.nat({ max: 50 }), resolved_count: fc.nat({ max: 50 }), report_date: fc.date().map(d => d.toISOString().slice(0, 10)) }`
|
||
- Assert: first row start = 0, each row N start = row N-1 end, each row end = start + new_count + recurring_count - resolved_count
|
||
- **Validates: Requirements 3.1, 3.2, 3.3**
|
||
|
||
- [ ]* 2.4 Write unit tests for waterfall endpoint
|
||
- Create `cve-dashboard/backend/__tests__/compliance-waterfall-endpoint.test.js`
|
||
- Test empty array returned when no uploads exist (Req 3.5)
|
||
- Test HTTP 500 returned on DB error (Req 3.6)
|
||
- Test correct chain computation with known seed data (Req 3.1, 3.2, 3.3)
|
||
- _Requirements: 3.1, 3.2, 3.3, 3.5, 3.6_
|
||
|
||
- [x] 3. Checkpoint — Backend complete
|
||
- Ensure all tests pass, ask the user if questions arise.
|
||
|
||
- [x] 4. Implement frontend AgingChart component
|
||
- [x] 4.1 Replace `MttrChart` with `AgingChart` in `cve-dashboard/frontend/src/components/pages/ComplianceChartsPanel.js`
|
||
- Remove the `MttrChart` function component entirely
|
||
- Add `AgingChart({ data })` that renders a vertical stacked `BarChart` using Recharts
|
||
- X-axis: bucket labels (`1 cycle`, `2–3 cycles`, `4–6 cycles`, `7+ cycles`)
|
||
- Y-axis: finding count with monospace axis styling
|
||
- One `<Bar>` per team key from `TEAM_COLORS` using `stackId="aging"`
|
||
- Use `DarkTooltip` for hover showing bucket label, per-team counts, and total
|
||
- Render `<NoData />` when data array is empty
|
||
- Wrap in `<ChartCard title="Aging Findings Distribution" subtitle="Active findings by age bucket — stacked by team" />`
|
||
- Place in the Chart 4 grid slot previously held by MTTR
|
||
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7_
|
||
|
||
- [ ]* 4.2 Write unit tests for AgingChart
|
||
- Test `<NoData />` renders for empty data (Req 2.5)
|
||
- Test ChartCard renders with correct title (Req 2.3)
|
||
- _Requirements: 2.3, 2.5_
|
||
|
||
- [x] 5. Implement frontend WaterfallChart component
|
||
- [x] 5.1 Replace `RecurringChart` with `WaterfallChart` in `cve-dashboard/frontend/src/components/pages/ComplianceChartsPanel.js`
|
||
- Remove the `RecurringChart` function component entirely
|
||
- Add `WaterfallChart({ data })` that renders a vertical `BarChart` using Recharts
|
||
- Stacked bars: transparent base `<Bar dataKey="start" stackId="w" fill="transparent" />`, red new `<Bar dataKey="new_count" fill="#EF4444" />`, amber recurring `<Bar dataKey="recurring_count" fill="#F59E0B" />`
|
||
- Separate bar for resolved: `<Bar dataKey="resolved_count" fill="#10B981" />` (not stacked, matching existing DeltaChart pattern)
|
||
- X-axis: cycle dates formatted via `fmtDate`, Y-axis: finding count with monospace styling
|
||
- Custom `WaterfallTooltip` showing date, start, new, recurring, resolved, and end values
|
||
- Render `<NoData />` when data array is empty
|
||
- Wrap in `<ChartCard title="Net Change Waterfall" subtitle="Per-cycle net movement: start → +new → +recurring → −resolved → end" />`
|
||
- Place in the Chart 5 grid slot previously held by Most Persistent Findings
|
||
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8_
|
||
|
||
- [ ]* 5.2 Write unit tests for WaterfallChart
|
||
- Test `<NoData />` renders for empty data (Req 4.5)
|
||
- Test ChartCard renders with correct title (Req 4.6)
|
||
- _Requirements: 4.5, 4.6_
|
||
|
||
- [x] 6. Update state and fetch logic in ComplianceChartsPanel
|
||
- [x] 6.1 Replace state variables and fetch calls
|
||
- Replace `const [mttr, setMttr] = useState([])` with `const [aging, setAging] = useState([])`
|
||
- Replace `const [recurring, setRecurring] = useState([])` with `const [waterfall, setWaterfall] = useState([])`
|
||
- In the `Promise.all` fetch block, update the `/compliance/mttr` response handler: `setAging(d.aging || [])`
|
||
- Update the `/compliance/top-recurring` response handler: `setWaterfall(d.waterfall || [])`
|
||
- Keep `/compliance/trends` and `/archer-tickets/status-trend` fetches unchanged
|
||
- Pass `aging` to `<AgingChart data={aging} />` and `waterfall` to `<WaterfallChart data={waterfall} />`
|
||
- Ensure individual fetch failures render `<NoData />` without affecting other charts
|
||
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 6.1, 6.2_
|
||
|
||
- [ ]* 6.2 Write integration tests for dashboard fetch logic
|
||
- Test that dashboard fetches `/compliance/mttr` and `/compliance/top-recurring` on mount (Req 5.1, 5.2)
|
||
- Test that `/compliance/trends` and `/archer-tickets/status-trend` are still fetched (Req 5.3)
|
||
- Test that all four fetches run in parallel via `Promise.all` (Req 5.4)
|
||
- Test that a single fetch failure does not break other charts (Req 5.5)
|
||
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5_
|
||
|
||
- [x] 7. Final checkpoint — All tests pass
|
||
- Ensure all tests pass, ask the user if questions arise.
|
||
|
||
## Notes
|
||
|
||
- Tasks marked with `*` are optional and can be skipped for faster MVP
|
||
- Each task references specific requirements for traceability
|
||
- Checkpoints ensure incremental validation after backend and full integration
|
||
- Property tests validate universal correctness properties from the design (Properties 1 and 2)
|
||
- Unit tests validate specific examples and edge cases
|
||
- Pure functions `bucketAgingItems()` and `computeWaterfall()` are extracted first to enable property-based testing
|
||
- No new route paths are introduced — `/mttr` and `/top-recurring` are reused in-place
|
||
- No database migrations are needed — all data comes from existing columns
|