cve-dashboard/.kiro/specs/vcl-aggregated-burndown/design.md

# Design Document: VCL Aggregated Burndown

## Overview

This feature adds an aggregated (cross-vertical) burndown forecast to the CCP Metrics overview page. Currently, burndown data is only available per-vertical via `GET /api/compliance/vcl-multi/vertical/:code/burndown`. This feature introduces a new endpoint `GET /api/compliance/vcl-multi/burndown` that rolls up burndown data across all verticals, a new pure helper function `computeAggregatedBurndown` for testable computation logic, and a new `AggregatedBurndownChart` React component displayed on the overview page.

The design reuses the existing `computeVerticalBurndown` pattern from `vclHelpers.js` and extends it with hostname deduplication (a device appearing in multiple metrics counts once) and per-vertical contribution breakdown. The frontend component follows the same Recharts `BarChart` pattern used in the per-vertical burndown chart within `VerticalDetailView`.

## Architecture

```mermaid
sequenceDiagram
    participant FE as CCPMetricsPage
    participant BE as Express Backend
    participant DB as PostgreSQL

    Note over FE,DB: Overview Page Load (existing + new)
    FE->>BE: GET /api/compliance/vcl-multi/stats
    BE->>DB: Aggregate stats across verticals
    BE-->>FE: { stats, donut, vertical_breakdown }

    FE->>BE: GET /api/compliance/vcl-multi/trend
    BE->>DB: Monthly snapshots
    BE-->>FE: { months: [...] }

    FE->>BE: GET /api/compliance/vcl-multi/burndown
    BE->>DB: SELECT hostname, resolution_date, vertical FROM compliance_items WHERE vertical IS NOT NULL AND status = 'active'
    BE->>BE: Deduplicate by hostname (earliest resolution_date)
    BE->>BE: computeAggregatedBurndown(devices)
    BE-->>FE: { total_non_compliant, blockers, with_dates, monthly_forecast, projected_clear_date, by_vertical }

    FE->>FE: Render AggregatedBurndownChart
```

### Data Flow

1. **Query** — Fetch all active non-compliant devices across all verticals from `compliance_items`.
2. **Deduplicate** — Group by hostname, keeping the earliest non-null `resolution_date` across metric entries. A device appearing in 3 metrics counts as 1 device.
3. **Compute** — Pass deduplicated device list to `computeAggregatedBurndown` which produces totals, monthly buckets, cumulative projection, and per-vertical breakdown.
4. **Respond** — Return the computed burndown data to the frontend.
5. **Render** — `AggregatedBurndownChart` displays a bar chart of monthly remediations, summary stats header, and per-vertical contribution table.

## Components and Interfaces

### Backend

#### New Endpoint

**`GET /api/compliance/vcl-multi/burndown`**

Returns aggregated burndown forecast across all verticals.

- Auth: `requireAuth()`
- Route file: `backend/routes/vclMultiVertical.js`
- Response:
```json
{
  "total_non_compliant": 400,
  "blockers": 120,
  "with_dates": 280,
  "monthly_forecast": {
    "2026-06": 85,
    "2026-07": 110,
    "2026-08": 55,
    "2026-09": 30
  },
  "projected_clear_date": "2026-09",
  "by_vertical": [
    { "vertical": "NTS_AEO", "total": 180, "blockers": 50, "with_dates": 130 },
    { "vertical": "SDIT_CISO", "total": 120, "blockers": 40, "with_dates": 80 },
    { "vertical": "TSI", "total": 100, "blockers": 30, "with_dates": 70 }
  ]
}
```

When no active non-compliant devices exist:
```json
{
  "total_non_compliant": 0,
  "blockers": 0,
  "with_dates": 0,
  "monthly_forecast": {},
  "projected_clear_date": null,
  "by_vertical": []
}
```

#### New Pure Helper Function

Added to `backend/helpers/vclHelpers.js`:

```javascript
/**
 * Deduplicates devices by hostname, keeping the earliest non-null resolution_date.
 * A device appearing in multiple metrics counts once.
 *
 * @param {Array<{ hostname: string, resolution_date: string|null, vertical: string }>} items
 * @returns {Array<{ hostname: string, resolution_date: string|null, vertical: string }>}
 */
function deduplicateByHostname(items) { ... }

/**
 * Computes aggregated burndown from a deduplicated array of device objects.
 * Each device has { hostname, resolution_date, vertical }.
 *
 * @param {Array<{ hostname: string, resolution_date: string|null, vertical: string }>} devices
 * @returns {{
 *   total: number,
 *   blockers: number,
 *   with_dates: number,
 *   monthly: Object<string, number>,
 *   projection: Object<string, { remediated: number, remaining: number }>,
 *   projected_clear_date: string|null,
 *   by_vertical: Array<{ vertical: string, total: number, blockers: number, with_dates: number }>
 * }}
 */
function computeAggregatedBurndown(devices) { ... }
```

**`deduplicateByHostname` logic:**
- Groups items by hostname
- For each hostname, selects the earliest non-null `resolution_date` across all entries
- If all entries for a hostname have null dates, the device is a blocker
- Preserves the `vertical` from the first entry (for per-vertical breakdown, the endpoint groups separately)

**`computeAggregatedBurndown` logic:**
- Counts total devices, blockers (null date), and with_dates (non-null date)
- Buckets with_dates devices by YYYY-MM of resolution_date into `monthly`
- Sorts monthly keys chronologically
- Computes `projection` as cumulative remaining: starts at `total`, subtracts each month's count
- Sets `projected_clear_date` to the last month key if blockers = 0, otherwise null
- Groups devices by vertical for `by_vertical`, sorted descending by total, omitting verticals with zero devices

#### Endpoint Implementation Pattern

The endpoint follows the same pattern as the existing per-vertical burndown:

```javascript
router.get('/burndown', async (req, res) => {
    try {
        const { rows } = await pool.query(
            `SELECT hostname, resolution_date, vertical
             FROM compliance_items
             WHERE vertical IS NOT NULL AND status = 'active'`
        );

        // Deduplicate by hostname (earliest non-null resolution_date)
        const devices = deduplicateByHostname(rows);
        const burndown = computeAggregatedBurndown(devices);

        res.json({
            total_non_compliant: burndown.total,
            blockers: burndown.blockers,
            with_dates: burndown.with_dates,
            monthly_forecast: burndown.monthly,
            projected_clear_date: burndown.projected_clear_date,
            by_vertical: burndown.by_vertical,
        });
    } catch (err) {
        console.error('[VCL Multi] GET /burndown error:', err.message);
        res.status(500).json({ error: 'Database error' });
    }
});
```

### Frontend

#### New Component: `AggregatedBurndownChart`

Inline component within `CCPMetricsPage.js` (following the existing pattern where `StatsBar`, `DonutChart`, `TrendChart`, and `VerticalTable` are all defined in the same file).

**Placement:** Below the existing charts row (TrendChart + DonutChart), above the VerticalTable.

**Behavior:**
- Fetches `GET /api/compliance/vcl-multi/burndown` on page load alongside existing stats/trend calls
- Displays a summary header with total non-compliant, blockers, in-progress, and projected clear date
- Renders a Recharts `BarChart` with one bar per monthly bucket (purple fill, matching existing burndown chart style)
- Below the chart, renders a compact per-vertical contribution table sorted by total descending
- Shows "No non-compliant devices" message when total = 0
- Shows "All X non-compliant devices lack remediation dates" when monthly_forecast is empty but blockers > 0
- Shows loading spinner while fetching
- Shows inline error message on API failure

**Chart specification:**
```javascript
<ResponsiveContainer width="100%" height={200}>
    <BarChart data={monthlyData}>
        <CartesianGrid stroke="rgba(255,255,255,0.05)" strokeDasharray="3 3" />
        <XAxis dataKey="month" tick={{ fontSize: 10, fill: '#64748B' }} />
        <YAxis tick={{ fontSize: 10, fill: '#64748B' }} />
        <Tooltip contentStyle={{ background: '#1E293B', border: '1px solid #334155', borderRadius: '0.5rem', fontSize: '0.75rem' }} />
        <Bar dataKey="count" fill="#A78BFA" fillOpacity={0.7} name="Projected Remediations" />
    </BarChart>
</ResponsiveContainer>
```

## Data Models

No schema changes required. The feature reads from the existing `compliance_items` table:

```sql
-- Existing columns used:
--   hostname     TEXT
--   vertical     TEXT (nullable)
--   status       TEXT ('active'|'resolved')
--   resolution_date  DATE (nullable)
```

The query for the aggregated burndown:
```sql
SELECT hostname, resolution_date, vertical
FROM compliance_items
WHERE vertical IS NOT NULL AND status = 'active'
```

This is the same data source used by the per-vertical burndown endpoint, just without the `vertical = $1` filter.

## Correctness Properties

*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*

### Property 1: Partition Invariant

*For any* array of device objects passed to `computeAggregatedBurndown`, the result must satisfy `blockers + with_dates = total`. Every device is either a blocker (null resolution_date) or in-progress (non-null resolution_date), with no device uncounted or double-counted.

**Validates: Requirements 2.2**

### Property 2: Monthly Bucket Conservation

*For any* array of device objects passed to `computeAggregatedBurndown`, the sum of all values in the `monthly` object must equal `with_dates`. Every in-progress device appears in exactly one monthly bucket, and no device is lost or duplicated during bucketing.

**Validates: Requirements 2.3, 1.5**

### Property 3: Chronological Monthly Ordering

*For any* array of device objects passed to `computeAggregatedBurndown`, the keys of the `monthly` object must be in ascending chronological order (lexicographic sort of YYYY-MM strings).

**Validates: Requirements 2.4**

### Property 4: Cumulative Projection Consistency

*For any* array of device objects passed to `computeAggregatedBurndown`, the `projection` object must satisfy: for each month in chronological order, `projection[month].remaining = total - (cumulative sum of monthly[m] for all m <= month)`. The first month's remaining equals `total - monthly[first_month]`.

**Validates: Requirements 2.5**

### Property 5: Projected Clear Date Logic

*For any* array of device objects passed to `computeAggregatedBurndown`: if `blockers > 0`, then `projected_clear_date` must be `null`; if `blockers = 0` and `with_dates > 0`, then `projected_clear_date` must equal the last (chronologically greatest) key in `monthly`.

**Validates: Requirements 1.7**

### Property 6: Hostname Deduplication with Earliest Date

*For any* array of items where the same hostname appears multiple times with different resolution_dates, `deduplicateByHostname` must produce exactly one entry per unique hostname, and that entry's `resolution_date` must be the earliest non-null date among all entries for that hostname (or null if all entries have null dates).

**Validates: Requirements 1.6**

### Property 7: Aggregation Consistency with Per-Vertical Computation

*For any* array of device objects spanning multiple verticals, the aggregated `total` must equal the sum of per-vertical totals, the aggregated `blockers` must equal the sum of per-vertical blockers, the aggregated `with_dates` must equal the sum of per-vertical with_dates, and for each month key, the aggregated monthly count must equal the sum of that month's count across all per-vertical computations.

**Validates: Requirements 4.1, 4.2, 4.3, 4.4**

### Property 8: By-Vertical Sorting and Filtering

*For any* array of device objects spanning multiple verticals, the `by_vertical` array must be sorted in descending order by `total`, must not contain any entry where `total = 0`, and the sum of all `by_vertical[i].total` must equal the overall `total`.

**Validates: Requirements 5.1, 5.2, 5.4**

## Error Handling

| Condition | HTTP Status | Response | Behavior |
|---|---|---|---|
| Database error | 500 | `{ "error": "Database error" }` | Log error, return 500 |
| Unauthenticated request | 401 | `{ "error": "Authentication required" }` | Middleware rejects |
| No active non-compliant devices | 200 | Zero/empty response (see above) | Graceful empty state |

Frontend error handling:
- API failure: inline error message in red monospace text, consistent with existing error patterns on the page
- Loading state: `<Loader />` spinner with "Loading..." text
- Empty state (total = 0): informational message instead of empty chart
- All blockers (monthly empty, blockers > 0): message indicating all devices lack dates

## Testing Strategy

### Property-Based Testing

Use `fast-check` (already used in this project). Each correctness property maps to a single property-based test with minimum 100 iterations.

Property tests target the pure helper functions exported from `backend/helpers/vclHelpers.js`:
- `deduplicateByHostname` — Property 6
- `computeAggregatedBurndown` — Properties 1, 2, 3, 4, 5, 7, 8

Tag format: **Feature: vcl-aggregated-burndown, Property {number}: {title}**

Test file: `backend/__tests__/vcl-aggregated-burndown.property.test.js`

### Unit Testing

Unit tests cover specific examples and edge cases:

- **Empty input** — verify all-zero response (Requirement 2.6)
- **All blockers** — verify with_dates = 0, monthly = {}, projected_clear_date = null (Requirement 2.7)
- **Single device, single metric** — verify basic computation
- **Duplicate hostnames across metrics** — verify deduplication picks earliest date
- **Duplicate hostnames where all dates are null** — verify device is a blocker
- **API endpoint integration** — verify response shape with mocked DB data
- **Auth middleware** — verify 401 without session

Test file: `backend/__tests__/vcl-aggregated-burndown.test.js`

### Frontend Testing

- Component renders loading state
- Component renders empty state message when total = 0
- Component renders blocker-only message when monthly is empty
- Component renders bar chart with correct data
- Component renders per-vertical table sorted correctly
- Component renders error message on API failure