cve-dashboard/.kiro/specs/ccp-metrics-view-restructure/design.md

Design Document

Overview

This design restructures the CCP Metrics page from a vertical-first drill-down model to a metric-first model. The overview table changes from one row per vertical to one row per metric (aggregated across all verticals). Two new backend endpoints support the metric-centric view. The "By Vertical" table in the AggregatedBurndownChart is removed. Existing vertical-first endpoints are preserved for backward compatibility.

Architecture

The restructure touches two layers:

1. Backend — Two new Express route handlers added to backend/routes/vclMultiVertical.js, querying vcl_multi_vertical_summary with aggregation across verticals.
2. Frontend — Replace VerticalTable and VerticalDetailView components with MetricTable and MetricDetailView in frontend/src/components/pages/CCPMetricsPage.js. Remove the "By Vertical" table JSX from AggregatedBurndownChart. Adjust drill-down state from (selectedVertical, selectedMetric, selectedTeam) to (selectedMetric, selectedVertical, selectedTeam).

No database schema changes are required. No new tables or columns are needed — the existing vcl_multi_vertical_summary table already contains all data needed for metric-centric aggregation.

Components and Interfaces

Backend: New Endpoints

GET /api/compliance/vcl-multi/metrics

Aggregates all metrics across verticals using only ALL: rollup rows from the latest upload per vertical.

// Query: get latest upload ID per vertical
const { rows: latestUploads } = await pool.query(`
    SELECT DISTINCT ON (vertical) id, vertical
    FROM compliance_uploads
    WHERE vertical IS NOT NULL
    ORDER BY vertical, id DESC
`);

const latestUploadIds = latestUploads.map(u => u.id);

// Aggregate metrics across verticals (ALL: rows only)
const { rows: metrics } = await pool.query(`
    SELECT metric_id,
           MAX(metric_desc) AS metric_desc,
           MAX(category) AS category,
           SUM(non_compliant)::int AS non_compliant,
           SUM(compliant)::int AS compliant,
           SUM(total)::int AS total,
           ROUND(AVG(target::numeric), 4) AS target
    FROM vcl_multi_vertical_summary
    WHERE upload_id = ANY($1) AND team LIKE 'ALL:%'
    GROUP BY metric_id
    ORDER BY non_compliant DESC
`, [latestUploadIds]);

// Compute compliance_pct for each metric
const result = metrics.map(m => ({
    ...m,
    compliance_pct: m.total > 0 ? m.compliant / m.total : 0,
}));

Response shape:

{
  "metrics": [
    {
      "metric_id": "VM-001",
      "metric_desc": "Vulnerability Management - Patching",
      "category": "Vulnerability Management",
      "non_compliant": 450,
      "compliant": 3200,
      "total": 3650,
      "compliance_pct": 0.8767,
      "target": 0.95
    }
  ]
}

GET /api/compliance/vcl-multi/metric/:id/verticals

Returns per-vertical breakdown for a specific metric, including sub-team data within each vertical.

const metricId = req.params.id;
if (!metricId || metricId.length > 50) {
    return res.status(400).json({ error: 'Invalid metric ID' });
}

// Get latest upload per vertical
const { rows: latestUploads } = await pool.query(`
    SELECT DISTINCT ON (vertical) id, vertical
    FROM compliance_uploads
    WHERE vertical IS NOT NULL
    ORDER BY vertical, id DESC
`);
const latestUploadIds = latestUploads.map(u => u.id);

// Get all rows for this metric from latest uploads
const { rows: allRows } = await pool.query(`
    SELECT vertical, metric_desc, category, team,
           non_compliant, compliant, total, compliance_pct, target
    FROM vcl_multi_vertical_summary
    WHERE upload_id = ANY($1) AND metric_id = $2
    ORDER BY vertical, team
`, [latestUploadIds, metricId]);

// Separate rollup rows (ALL:) from sub-team rows
// Build per-vertical entries with nested sub_teams

Response shape:

{
  "metric_id": "VM-001",
  "metric_desc": "Vulnerability Management - Patching",
  "category": "Vulnerability Management",
  "verticals": [
    {
      "vertical": "NTS_AEO",
      "non_compliant": 200,
      "compliant": 1800,
      "total": 2000,
      "compliance_pct": 0.90,
      "target": 0.95,
      "sub_teams": [
        {
          "team": "STEAM",
          "non_compliant": 120,
          "compliant": 1080,
          "total": 1200,
          "compliance_pct": 0.90
        },
        {
          "team": "ACCESS-ENG",
          "non_compliant": 80,
          "compliant": 720,
          "total": 800,
          "compliance_pct": 0.90
        }
      ]
    }
  ]
}

Frontend: Component Changes

AggregatedBurndownChart (Modified)

Remove the "By Vertical" contribution table JSX block. The component continues to receive data.by_vertical in its props (for backward compat with the /burndown API response) but no longer renders it.

Before: Summary header + bar chart + "By Vertical" table After: Summary header + bar chart only

MetricTable (New — replaces VerticalTable)

Renders one row per metric from the /metrics endpoint response. Columns: Metric ID, Description, Category, Compliant, Non-Compliant, Total, Compliance %, Target %. Rows are clickable — clicking triggers onSelectMetric(metricId).

function MetricTable({ metrics, onSelectMetric }) {
    if (!metrics || metrics.length === 0) return null;
    // Render table with columns: metric_id, metric_desc, category,
    // compliant, non_compliant, total, compliance_pct, target
    // onClick row → onSelectMetric(metric.metric_id)
}

MetricDetailView (New — replaces VerticalDetailView)

Fetches data from GET /metric/:id/verticals and displays:

• Header with metric ID, description, category
• Aggregated stats cards (total, compliant, non-compliant, compliance %)
• Table of verticals with per-vertical compliance numbers
• Clicking a vertical row navigates to the sub-team view

function MetricDetailView({ metricId, onBack, onSelectVertical }) {
    const [data, setData] = useState(null);
    const [loading, setLoading] = useState(true);

    useEffect(() => {
        fetch(`${API_BASE}/compliance/vcl-multi/metric/${encodeURIComponent(metricId)}/verticals`,
              { credentials: 'include' })
            .then(r => r.json())
            .then(d => { setData(d); setLoading(false); })
            .catch(() => setLoading(false));
    }, [metricId]);
    // Render: back button, header, stats, verticals table
}

MetricSubTeamView (Reused with minor adjustments)

The existing MetricSubTeamView component is reused. The only change is that it now receives its metricData (including sub_teams) from the MetricDetailView's vertical entry rather than from the VerticalDetailView's metric entry. The props interface remains the same: { vertical, metricId, metricData, onBack, onSelectTeam }.

MetricDeviceList (Reused unchanged)

The existing MetricDeviceList component is reused without modification. It already accepts { vertical, metricId, team, onBack } and calls the existing devices endpoint.

Frontend: Drill-Down State Changes

Current state model:

selectedVertical → selectedMetric → selectedTeam

New state model:

selectedMetric → selectedVertical → selectedTeam

The main component's render logic changes from:

// OLD
if (selectedTeam && selectedMetric && selectedVertical) → MetricDeviceList
if (selectedMetric && selectedVertical) → MetricSubTeamView
if (selectedVertical) → VerticalDetailView
else → Overview (VerticalTable)

To:

// NEW
if (selectedTeam !== null && selectedMetric && selectedVertical) → MetricDeviceList
if (selectedVertical && selectedMetric) → MetricSubTeamView
if (selectedMetric) → MetricDetailView
else → Overview (MetricTable)

State variables:

• selectedMetric — string (metric_id) or null
• selectedMetricData — object with metric context (metric_desc, category) or null
• selectedVertical — string (vertical code) or null
• selectedVerticalData — object with vertical's sub_teams for the selected metric or null
• selectedTeam — string (team name) or null (null with selectedVertical set = "View All Devices")

Data Flow

┌─────────────────────────────────────────────────────────────────┐
│ Overview                                                         │
│  GET /stats → StatsBar, DonutChart, TrendChart                  │
│  GET /burndown → AggregatedBurndownChart (no By Vertical table) │
│  GET /metrics → MetricTable                                      │
└──────────────────────────┬──────────────────────────────────────┘
                           │ click metric row
                           ▼
┌─────────────────────────────────────────────────────────────────┐
│ MetricDetailView                                                 │
│  GET /metric/:id/verticals → header + verticals table            │
└──────────────────────────┬──────────────────────────────────────┘
                           │ click vertical row
                           ▼
┌─────────────────────────────────────────────────────────────────┐
│ MetricSubTeamView                                                │
│  (data passed from parent — no additional fetch)                 │
└──────────────────────────┬──────────────────────────────────────┘
                           │ click team row
                           ▼
┌─────────────────────────────────────────────────────────────────┐
│ MetricDeviceList                                                 │
│  GET /vertical/:code/metric/:metricId/devices?team=X             │
└─────────────────────────────────────────────────────────────────┘

Interfaces

New API Endpoints

Method	Path	Auth	Description
GET	/api/compliance/vcl-multi/metrics	requireAuth()	Metrics aggregated across all verticals
GET	/api/compliance/vcl-multi/metric/:id/verticals	requireAuth()	Per-vertical breakdown for a metric

Preserved API Endpoints (Backward Compatibility)

Method	Path	Notes
GET	/api/compliance/vcl-multi/stats	Unchanged — still returns vertical_breakdown and metric_breakdown
GET	/api/compliance/vcl-multi/vertical/:code/metrics	Unchanged
GET	/api/compliance/vcl-multi/vertical/:code/metric/:metricId/devices	Unchanged
GET	/api/compliance/vcl-multi/vertical/:code/burndown	Unchanged
GET	/api/compliance/vcl-multi/burndown	Unchanged — still returns by_vertical in response

Component Props Interfaces

// MetricTable
MetricTable.propTypes = {
    metrics: Array,       // from GET /metrics response
    onSelectMetric: Function, // (metricId: string) => void
};

// MetricDetailView
MetricDetailView.propTypes = {
    metricId: String,
    onBack: Function,         // () => void — returns to overview
    onSelectVertical: Function, // (vertical: string, verticalData: object) => void
};

// MetricSubTeamView (existing — props unchanged)
MetricSubTeamView.propTypes = {
    vertical: String,
    metricId: String,
    metricData: Object,       // { metric_desc, sub_teams, target, ... }
    onBack: Function,         // () => void — returns to metric-vertical view
    onSelectTeam: Function,   // (team: string|null) => void
};

// MetricDeviceList (existing — props unchanged)
MetricDeviceList.propTypes = {
    vertical: String,
    metricId: String,
    team: String,             // null = all teams
    onBack: Function,         // () => void — returns to sub-team view
};

Data Models

No new database tables or columns. The feature uses existing data:

vcl_multi_vertical_summary (existing)

Column	Type	Notes
id	SERIAL	PK
upload_id	INTEGER	FK → compliance_uploads
vertical	TEXT	Vertical code (e.g., "NTS_AEO")
metric_id	TEXT	Metric identifier (e.g., "VM-001")
metric_desc	TEXT	Human-readable description
category	TEXT	Metric category
team	TEXT	"ALL: NTS-AEO" for rollup, "STEAM" for sub-team
non_compliant	INTEGER	Count
compliant	INTEGER	Count
total	INTEGER	Count
compliance_pct	NUMERIC(5,2)	Stored as decimal (0.95 = 95%)
target	NUMERIC(5,2)	Stored as decimal

Key convention: Rows where team LIKE 'ALL:%' are rollup rows that include all sub-team totals. Sub-team rows are individual team breakdowns. Only rollup rows should be used for aggregation to avoid double-counting.

Error Handling

Backend

Scenario	Response
No auth cookie / expired session	401 { "error": "Authentication required" }
Metric ID > 50 chars	400 { "error": "Invalid metric ID" }
Database query failure	500 { "error": "Database error" }
No data for metric	200 with empty verticals array
No summary data at all	200 with empty metrics array

Frontend

Scenario	Behavior
Fetch failure on overview	Existing error state renders error message
Fetch failure on MetricDetailView	Show error message in component
Empty metrics array	Show "No metrics data" empty state
Empty verticals array for a metric	Show "No verticals found for this metric" message

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Metrics aggregation uses only rollup rows from latest uploads

For any set of vcl_multi_vertical_summary rows across multiple verticals and uploads, the GET /metrics endpoint SHALL return exactly one entry per distinct metric_id, with non_compliant, compliant, and total equal to the sum of only ALL:-prefixed team rows from the single latest upload per vertical.

Validates: Requirements 3.1, 3.3

Property 2: Metrics computed fields are mathematically correct

For any metric entry returned by the GET /metrics endpoint, compliance_pct SHALL equal compliant / total (or 0 when total is 0), and target SHALL equal the arithmetic mean of the target values across all verticals that have that metric in their latest upload.

Validates: Requirements 3.4, 3.5

Property 3: Metrics response is sorted by non-compliant descending

For any response from GET /metrics containing two or more metric entries, each entry's non_compliant value SHALL be greater than or equal to the next entry's non_compliant value.

Validates: Requirements 3.6

Property 4: Metric-verticals breakdown is complete and correct

For any metric ID that exists in the database, the GET /metric/:id/verticals endpoint SHALL return one vertical entry for each vertical that has that metric in its latest upload, with each entry's sub_teams array containing exactly the non-rollup, non-"(Other)" team rows for that metric in that vertical.

Validates: Requirements 4.1, 4.3

Property 5: Metric-verticals response is sorted by non-compliant descending

For any response from GET /metric/:id/verticals containing two or more vertical entries, each entry's non_compliant value SHALL be greater than or equal to the next entry's non_compliant value.

Validates: Requirements 4.4

Property 6: Drill-down state determines rendered view

For any combination of (selectedMetric, selectedVertical, selectedTeam) state values, the CCP_Metrics_Page SHALL render exactly one view: Overview when no metric is selected, MetricDetailView when only metric is selected, MetricSubTeamView when metric and vertical are selected (team is null or absent), and MetricDeviceList when all three are selected.

Validates: Requirements 6.1, 6.5

Testing Strategy

Property-Based Tests (Backend)

Properties 1–5 are testable via property-based tests against the backend endpoints. The test setup generates random vcl_multi_vertical_summary rows with varying numbers of verticals, metrics, uploads, and team configurations, then asserts the invariants hold.

Example-Based Tests (Frontend)

Requirements 1.x, 2.x, 5.x, and 6.2–6.4 are UI rendering and interaction tests best covered by example-based component tests using React Testing Library.

Integration Tests (Backward Compatibility)

Requirements 7.x are integration tests that verify existing endpoints continue to return the same response shape after the new endpoints are added.