# Design Document: VCL Aggregated Burndown ## Overview This feature adds an aggregated (cross-vertical) burndown forecast to the CCP Metrics overview page. Currently, burndown data is only available per-vertical via `GET /api/compliance/vcl-multi/vertical/:code/burndown`. This feature introduces a new endpoint `GET /api/compliance/vcl-multi/burndown` that rolls up burndown data across all verticals, a new pure helper function `computeAggregatedBurndown` for testable computation logic, and a new `AggregatedBurndownChart` React component displayed on the overview page. The design reuses the existing `computeVerticalBurndown` pattern from `vclHelpers.js` and extends it with hostname deduplication (a device appearing in multiple metrics counts once) and per-vertical contribution breakdown. The frontend component follows the same Recharts `BarChart` pattern used in the per-vertical burndown chart within `VerticalDetailView`. ## Architecture ```mermaid sequenceDiagram participant FE as CCPMetricsPage participant BE as Express Backend participant DB as PostgreSQL Note over FE,DB: Overview Page Load (existing + new) FE->>BE: GET /api/compliance/vcl-multi/stats BE->>DB: Aggregate stats across verticals BE-->>FE: { stats, donut, vertical_breakdown } FE->>BE: GET /api/compliance/vcl-multi/trend BE->>DB: Monthly snapshots BE-->>FE: { months: [...] } FE->>BE: GET /api/compliance/vcl-multi/burndown BE->>DB: SELECT hostname, resolution_date, vertical FROM compliance_items WHERE vertical IS NOT NULL AND status = 'active' BE->>BE: Deduplicate by hostname (earliest resolution_date) BE->>BE: computeAggregatedBurndown(devices) BE-->>FE: { total_non_compliant, blockers, with_dates, monthly_forecast, projected_clear_date, by_vertical } FE->>FE: Render AggregatedBurndownChart ``` ### Data Flow 1. **Query** — Fetch all active non-compliant devices across all verticals from `compliance_items`. 2. **Deduplicate** — Group by hostname, keeping the earliest non-null `resolution_date` across metric entries. A device appearing in 3 metrics counts as 1 device. 3. **Compute** — Pass deduplicated device list to `computeAggregatedBurndown` which produces totals, monthly buckets, cumulative projection, and per-vertical breakdown. 4. **Respond** — Return the computed burndown data to the frontend. 5. **Render** — `AggregatedBurndownChart` displays a bar chart of monthly remediations, summary stats header, and per-vertical contribution table. ## Components and Interfaces ### Backend #### New Endpoint **`GET /api/compliance/vcl-multi/burndown`** Returns aggregated burndown forecast across all verticals. - Auth: `requireAuth()` - Route file: `backend/routes/vclMultiVertical.js` - Response: ```json { "total_non_compliant": 400, "blockers": 120, "with_dates": 280, "monthly_forecast": { "2026-06": 85, "2026-07": 110, "2026-08": 55, "2026-09": 30 }, "projected_clear_date": "2026-09", "by_vertical": [ { "vertical": "NTS_AEO", "total": 180, "blockers": 50, "with_dates": 130 }, { "vertical": "SDIT_CISO", "total": 120, "blockers": 40, "with_dates": 80 }, { "vertical": "TSI", "total": 100, "blockers": 30, "with_dates": 70 } ] } ``` When no active non-compliant devices exist: ```json { "total_non_compliant": 0, "blockers": 0, "with_dates": 0, "monthly_forecast": {}, "projected_clear_date": null, "by_vertical": [] } ``` #### New Pure Helper Function Added to `backend/helpers/vclHelpers.js`: ```javascript /** * Deduplicates devices by hostname, keeping the earliest non-null resolution_date. * A device appearing in multiple metrics counts once. * * @param {Array<{ hostname: string, resolution_date: string|null, vertical: string }>} items * @returns {Array<{ hostname: string, resolution_date: string|null, vertical: string }>} */ function deduplicateByHostname(items) { ... } /** * Computes aggregated burndown from a deduplicated array of device objects. * Each device has { hostname, resolution_date, vertical }. * * @param {Array<{ hostname: string, resolution_date: string|null, vertical: string }>} devices * @returns {{ * total: number, * blockers: number, * with_dates: number, * monthly: Object, * projection: Object, * projected_clear_date: string|null, * by_vertical: Array<{ vertical: string, total: number, blockers: number, with_dates: number }> * }} */ function computeAggregatedBurndown(devices) { ... } ``` **`deduplicateByHostname` logic:** - Groups items by hostname - For each hostname, selects the earliest non-null `resolution_date` across all entries - If all entries for a hostname have null dates, the device is a blocker - Preserves the `vertical` from the first entry (for per-vertical breakdown, the endpoint groups separately) **`computeAggregatedBurndown` logic:** - Counts total devices, blockers (null date), and with_dates (non-null date) - Buckets with_dates devices by YYYY-MM of resolution_date into `monthly` - Sorts monthly keys chronologically - Computes `projection` as cumulative remaining: starts at `total`, subtracts each month's count - Sets `projected_clear_date` to the last month key if blockers = 0, otherwise null - Groups devices by vertical for `by_vertical`, sorted descending by total, omitting verticals with zero devices #### Endpoint Implementation Pattern The endpoint follows the same pattern as the existing per-vertical burndown: ```javascript router.get('/burndown', async (req, res) => { try { const { rows } = await pool.query( `SELECT hostname, resolution_date, vertical FROM compliance_items WHERE vertical IS NOT NULL AND status = 'active'` ); // Deduplicate by hostname (earliest non-null resolution_date) const devices = deduplicateByHostname(rows); const burndown = computeAggregatedBurndown(devices); res.json({ total_non_compliant: burndown.total, blockers: burndown.blockers, with_dates: burndown.with_dates, monthly_forecast: burndown.monthly, projected_clear_date: burndown.projected_clear_date, by_vertical: burndown.by_vertical, }); } catch (err) { console.error('[VCL Multi] GET /burndown error:', err.message); res.status(500).json({ error: 'Database error' }); } }); ``` ### Frontend #### New Component: `AggregatedBurndownChart` Inline component within `CCPMetricsPage.js` (following the existing pattern where `StatsBar`, `DonutChart`, `TrendChart`, and `VerticalTable` are all defined in the same file). **Placement:** Below the existing charts row (TrendChart + DonutChart), above the VerticalTable. **Behavior:** - Fetches `GET /api/compliance/vcl-multi/burndown` on page load alongside existing stats/trend calls - Displays a summary header with total non-compliant, blockers, in-progress, and projected clear date - Renders a Recharts `BarChart` with one bar per monthly bucket (purple fill, matching existing burndown chart style) - Below the chart, renders a compact per-vertical contribution table sorted by total descending - Shows "No non-compliant devices" message when total = 0 - Shows "All X non-compliant devices lack remediation dates" when monthly_forecast is empty but blockers > 0 - Shows loading spinner while fetching - Shows inline error message on API failure **Chart specification:** ```javascript ``` ## Data Models No schema changes required. The feature reads from the existing `compliance_items` table: ```sql -- Existing columns used: -- hostname TEXT -- vertical TEXT (nullable) -- status TEXT ('active'|'resolved') -- resolution_date DATE (nullable) ``` The query for the aggregated burndown: ```sql SELECT hostname, resolution_date, vertical FROM compliance_items WHERE vertical IS NOT NULL AND status = 'active' ``` This is the same data source used by the per-vertical burndown endpoint, just without the `vertical = $1` filter. ## Correctness Properties *A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* ### Property 1: Partition Invariant *For any* array of device objects passed to `computeAggregatedBurndown`, the result must satisfy `blockers + with_dates = total`. Every device is either a blocker (null resolution_date) or in-progress (non-null resolution_date), with no device uncounted or double-counted. **Validates: Requirements 2.2** ### Property 2: Monthly Bucket Conservation *For any* array of device objects passed to `computeAggregatedBurndown`, the sum of all values in the `monthly` object must equal `with_dates`. Every in-progress device appears in exactly one monthly bucket, and no device is lost or duplicated during bucketing. **Validates: Requirements 2.3, 1.5** ### Property 3: Chronological Monthly Ordering *For any* array of device objects passed to `computeAggregatedBurndown`, the keys of the `monthly` object must be in ascending chronological order (lexicographic sort of YYYY-MM strings). **Validates: Requirements 2.4** ### Property 4: Cumulative Projection Consistency *For any* array of device objects passed to `computeAggregatedBurndown`, the `projection` object must satisfy: for each month in chronological order, `projection[month].remaining = total - (cumulative sum of monthly[m] for all m <= month)`. The first month's remaining equals `total - monthly[first_month]`. **Validates: Requirements 2.5** ### Property 5: Projected Clear Date Logic *For any* array of device objects passed to `computeAggregatedBurndown`: if `blockers > 0`, then `projected_clear_date` must be `null`; if `blockers = 0` and `with_dates > 0`, then `projected_clear_date` must equal the last (chronologically greatest) key in `monthly`. **Validates: Requirements 1.7** ### Property 6: Hostname Deduplication with Earliest Date *For any* array of items where the same hostname appears multiple times with different resolution_dates, `deduplicateByHostname` must produce exactly one entry per unique hostname, and that entry's `resolution_date` must be the earliest non-null date among all entries for that hostname (or null if all entries have null dates). **Validates: Requirements 1.6** ### Property 7: Aggregation Consistency with Per-Vertical Computation *For any* array of device objects spanning multiple verticals, the aggregated `total` must equal the sum of per-vertical totals, the aggregated `blockers` must equal the sum of per-vertical blockers, the aggregated `with_dates` must equal the sum of per-vertical with_dates, and for each month key, the aggregated monthly count must equal the sum of that month's count across all per-vertical computations. **Validates: Requirements 4.1, 4.2, 4.3, 4.4** ### Property 8: By-Vertical Sorting and Filtering *For any* array of device objects spanning multiple verticals, the `by_vertical` array must be sorted in descending order by `total`, must not contain any entry where `total = 0`, and the sum of all `by_vertical[i].total` must equal the overall `total`. **Validates: Requirements 5.1, 5.2, 5.4** ## Error Handling | Condition | HTTP Status | Response | Behavior | |---|---|---|---| | Database error | 500 | `{ "error": "Database error" }` | Log error, return 500 | | Unauthenticated request | 401 | `{ "error": "Authentication required" }` | Middleware rejects | | No active non-compliant devices | 200 | Zero/empty response (see above) | Graceful empty state | Frontend error handling: - API failure: inline error message in red monospace text, consistent with existing error patterns on the page - Loading state: `` spinner with "Loading..." text - Empty state (total = 0): informational message instead of empty chart - All blockers (monthly empty, blockers > 0): message indicating all devices lack dates ## Testing Strategy ### Property-Based Testing Use `fast-check` (already used in this project). Each correctness property maps to a single property-based test with minimum 100 iterations. Property tests target the pure helper functions exported from `backend/helpers/vclHelpers.js`: - `deduplicateByHostname` — Property 6 - `computeAggregatedBurndown` — Properties 1, 2, 3, 4, 5, 7, 8 Tag format: **Feature: vcl-aggregated-burndown, Property {number}: {title}** Test file: `backend/__tests__/vcl-aggregated-burndown.property.test.js` ### Unit Testing Unit tests cover specific examples and edge cases: - **Empty input** — verify all-zero response (Requirement 2.6) - **All blockers** — verify with_dates = 0, monthly = {}, projected_clear_date = null (Requirement 2.7) - **Single device, single metric** — verify basic computation - **Duplicate hostnames across metrics** — verify deduplication picks earliest date - **Duplicate hostnames where all dates are null** — verify device is a blocker - **API endpoint integration** — verify response shape with mocked DB data - **Auth middleware** — verify 401 without session Test file: `backend/__tests__/vcl-aggregated-burndown.test.js` ### Frontend Testing - Component renders loading state - Component renders empty state message when total = 0 - Component renders blocker-only message when monthly is empty - Component renders bar chart with correct data - Component renders per-vertical table sorted correctly - Component renders error message on API failure