cve-dashboard/.kiro/specs/compliance-metric-grouping/design.md

Design Document: Compliance Metric Grouping

Overview

The Compliance Metric Grouping feature consolidates the AEO Compliance page's metric health cards from one-per-summary-entry to one-per-metric-family. A metric family is the set of summary entries that share the same base metric_id (e.g., 5.2.5). The same base ID can appear multiple times in the summary data because the backend parser produces one entry per team/variant row in the xlsx Summary sheet.

The feature adds three new capabilities on top of the grouping:

1. Variant pills inside each grouped card showing per-variant compliance percentages and status indicators
2. Hover tooltip (300ms delay) displaying metric title, business justification, and data sources from a static definitions file
3. Info panel opened via an info icon on each card, showing the full metric definition (scope, filters, exclusions, notes)

A static JSON file (metricDefinitions.json) ships with the frontend containing structured metric definition data for all tracked metrics. No new backend endpoints are needed.

Design Decisions

• Grouping is frontend-only. The backend summary endpoint returns flat entries. The frontend groups them by metric_id at render time. This avoids backend changes and keeps the grouping logic testable as a pure function.
• metricFilter changes from a single string to an array. Currently metricFilter is a single metric_id string or null. With grouping, clicking a card sets the filter to the array of all metric_id values in that family. For single-entry families this is a one-element array. The device table filter checks metricFilter.includes(m.metric_id) instead of m.metric_id === metricFilter.
• Worst-status drives card color. Each grouped card computes the most severe status across its variants using a defined severity ordering. This gives engineers an at-a-glance signal when any variant is failing.
• Definitions file is static, not an API. The metric definitions table has ~130 rows that change infrequently. A static JSON import avoids API round-trips and keeps the tooltip/panel responsive. The file can be regenerated from the source xlsx definitions table when metrics change.
• MetricInfoPanel is a new component. The detail panel is complex enough (12+ fields, dark-themed sections) to warrant its own component file rather than inlining it in CompliancePage.js. The hover tooltip, being lightweight, stays inline.
• Info icon click uses stopPropagation. The info icon sits inside the card button. Clicking it opens the detail panel without triggering the card's metric filter toggle.

Architecture

graph TD
    A[CompliancePage.js] -->|imports| B[metricDefinitions.json]
    A -->|renders| C[MetricHealthCard - grouped]
    A -->|renders| D[MetricInfoPanel]
    A -->|renders| E[HoverTooltip - inline]
    A -->|fetches| F[GET /api/compliance/summary?team=X]
    A -->|fetches| G[GET /api/compliance/items?team=X&status=Y]

    F -->|returns| H[summary.entries array]
    H -->|groupByMetricFamily| I[Map of metricId → entries array]
    I -->|one card per group| C

    C -->|click outside info icon| J[setMetricFilter - array of IDs]
    C -->|click info icon| K[setInfoMetric - opens MetricInfoPanel]
    C -->|hover 300ms| E

    J -->|filters| G2[filteredDevices]
    B -->|lookup by metric_id| E
    B -->|lookup by metric_id| D

Component Hierarchy

CompliancePage
├── PageHeader (unchanged)
├── TeamTabs (unchanged)
├── MetricHealthSection
│   ├── SectionHeader ("Metric Health — click to filter" + clear button)
│   └── MetricHealthCard (one per metric family)
│       ├── CardTitle (base metric_id + category)
│       ├── VariantPill[] (one per summary entry in family)
│       ├── WorstStatusPill (computed from all variants)
│       ├── TargetDisplay (shared target %)
│       ├── InfoIcon (lucide-react Info, top-right)
│       └── HoverTooltip (inline, 300ms delay)
├── MetricInfoPanel (slide-out/overlay, opened by info icon click)
├── ComplianceChartsPanel (unchanged)
├── DeviceTable (unchanged, filter logic updated)
├── ComplianceDetailPanel (unchanged)
├── ComplianceUploadModal (unchanged)
└── RollbackModal (unchanged)

Data Flow

1. CompliancePage fetches summary entries from /api/compliance/summary?team=X on mount and team change.
2. The groupByMetricFamily(entries) helper groups the flat entries array into a Map<string, SummaryEntry[]> keyed by base metric_id.
3. One MetricHealthCard renders per map entry. Each card receives the full array of entries for that family.
4. The card computes worstStatus from the entries' status fields and uses it for border/pill coloring.
5. On card click (outside info icon), metricFilter is set to the array of metric_id values in that family (or cleared if already active).
6. The device table filter changes from d.failing_metrics.some(m => m.metric_id === metricFilter) to d.failing_metrics.some(m => metricFilter.includes(m.metric_id)).
7. On hover (300ms), a tooltip renders using data from the metricDefinitions.json lookup map, falling back to the summary entry description.
8. On info icon click, infoMetric state is set, opening MetricInfoPanel with the full definition.

Components and Interfaces

groupByMetricFamily (pure helper function)

// In CompliancePage.js — replaces the current teamMetrics() helper
// Input: entries (array of SummaryEntry from the summary endpoint), team (string)
// Output: array of { metricId, entries, category, target, worstStatus }

function groupByMetricFamily(allEntries, team) {
  const teamEntries = allEntries.filter(e => e.team === team);
  const familyMap = {};

  for (const entry of teamEntries) {
    const baseId = entry.metric_id; // already the base ID from the parser
    if (!familyMap[baseId]) {
      familyMap[baseId] = [];
    }
    familyMap[baseId].push(entry);
  }

  return Object.entries(familyMap).map(([metricId, entries]) => ({
    metricId,
    entries,
    category: entries[0].category,
    target: entries[0].target,
    worstStatus: computeWorstStatus(entries.map(e => e.status)),
  }));
}

computeWorstStatus (pure helper function)

// Input: array of status strings
// Output: the most severe status string
// Severity order: "Below 15% of Target" > "Within 15% of Target" > "Meets/Exceeds Target"

const STATUS_SEVERITY = {
  'Below 15% of Target':   0,
  'Within 15% of Target':  1,
  'Meets/Exceeds Target':  2,
};

function computeWorstStatus(statuses) {
  let worst = 'Meets/Exceeds Target';
  let worstSev = 2;
  for (const s of statuses) {
    const sev = STATUS_SEVERITY[s] ?? 0;
    if (sev < worstSev) {
      worstSev = sev;
      worst = s;
    }
  }
  return worst;
}

MetricHealthCard (redesigned)

// Props:
//   family: { metricId, entries, category, target, worstStatus }
//   active: boolean (is this family's filter currently active)
//   onClick: () => void (toggle metric filter)
//   onInfoClick: (metricId) => void (open detail panel)
//   definitionLookup: Map<string, MetricDefinition> (for tooltip)
//
// Renders:
//   - Base metric ID as title
//   - Category label
//   - One VariantPill per entry in family.entries
//   - Shared target percentage
//   - Worst-status pill with border color
//   - Info icon (top-right, stopPropagation on click)
//   - HoverTooltip (300ms delay, positioned near card)

VariantPill (new inline sub-component)

// Props:
//   entry: SummaryEntry (single variant)
//
// Renders:
//   - Label: entry.team or entry.description (distinguishing text)
//   - Compliance percentage in monospace
//   - Background tint from entry's status color at ~12% opacity
//   - Glow dot if status !== "Meets/Exceeds Target"
//
// Layout: inline-flex, wraps via parent flexWrap

HoverTooltip (inline in CompliancePage.js)

// State managed in CompliancePage:
//   hoveredMetric: string | null
//   hoverTimeout: ref (setTimeout ID)
//   tooltipPosition: { top, left } (computed from card bounding rect)
//
// On mouseEnter on MetricHealthCard:
//   Set timeout for 300ms → set hoveredMetric to family.metricId
// On mouseLeave:
//   Clear timeout, set hoveredMetric to null
//
// Renders (when hoveredMetric matches):
//   - Fixed-position div near the card
//   - Metric title (from definitionLookup or entry.description)
//   - Business justification (from definitionLookup)
//   - Data sources required (from definitionLookup)
//   - Dark card background, subtle border, shadow per DESIGN_SYSTEM.md
//   - Falls back to summary entry description if no definition found

MetricInfoPanel (new component file)

// frontend/src/components/pages/MetricInfoPanel.js
// Props:
//   metricId: string (base metric ID)
//   definition: MetricDefinition | null (from lookup)
//   summaryEntries: SummaryEntry[] (the family's entries, for fallback)
//   onClose: () => void
//
// Renders:
//   - Overlay/slide-out panel with dark theme
//   - Close button (X icon, top-right)
//   - Metric title (h3, monospace)
//   - Sections with monospace uppercase labels:
//     - Asset Types / Asset Types In Scope
//     - Application Types In Scope
//     - Environment In Scope
//     - Status In Scope
//     - Instance Types In Scope
//     - Criticality Levels In Scope
//     - Exclusions
//     - Special Conditions
//     - Data Sources Required
//     - Business Justification
//     - Notes
//   - If definition is null: "No detailed definition available" + summary description fallback
//   - Click outside or close button → onClose()

Integration with CompliancePage.js

// New imports:
import { Info } from 'lucide-react';
import MetricInfoPanel from './MetricInfoPanel';
import metricDefinitionsRaw from '../../data/metricDefinitions.json';

// Build lookup map once at module level:
const METRIC_DEFINITIONS = {};
for (const def of metricDefinitionsRaw) {
  METRIC_DEFINITIONS[def.metric_id] = def;
}

// State changes in CompliancePage:
// - metricFilter: null → null | string[] (array of metric IDs)
// - New state: infoMetric (string | null) — which metric's info panel is open
// - New state: hoveredMetric (string | null) — which metric is being hovered
// - New ref: hoverTimeoutRef — for 300ms delay

// Filter logic change:
// Old: .filter(d => !metricFilter || d.failing_metrics.some(m => m.metric_id === metricFilter))
// New: .filter(d => !metricFilter || d.failing_metrics.some(m => metricFilter.includes(m.metric_id)))

// Card rendering change:
// Old: metrics.map(entry => <MetricHealthCard entry={entry} ... />)
// New: families.map(family => <MetricHealthCard family={family} ... />)

Data Models

SummaryEntry (from backend, unchanged)

{
  metric_id: string,       // e.g. "5.2.5" — base ID, no suffix
  team: string,            // e.g. "STEAM", "ACCESS-ENG"
  priority: string,
  non_compliant: number,
  compliant: number,
  total: number,
  compliance_pct: number,  // 0.0–1.0
  target: number,          // 0.0–1.0
  status: string,          // "Meets/Exceeds Target" | "Within 15% of Target" | "Below 15% of Target"
  description: string,
  category: string         // from compliance_config.json metric_categories
}

MetricFamily (computed client-side)

{
  metricId: string,              // base metric ID (e.g. "5.2.5")
  entries: SummaryEntry[],       // all summary entries for this base ID
  category: string,              // from first entry
  target: number,                // from first entry (shared across variants)
  worstStatus: string            // computed worst status across all entries
}

MetricDefinition (from metricDefinitions.json)

{
  metric_id: string,                    // e.g. "5.2.5"
  metric_title: string,                 // e.g. "MFA for Privileged Access"
  asset_types: string,                  // e.g. "Servers, Network Devices"
  asset_types_in_scope: string,
  application_types_in_scope: string,
  environment_in_scope: string,
  status_in_scope: string,
  instance_types_in_scope: string,
  criticality_levels_in_scope: string,
  exclusions: string,                   // empty string if none
  special_conditions: string,           // empty string if none
  data_sources_required: string,
  business_justification: string,
  notes: string                         // empty string if none
}

metricDefinitions.json (file structure)

[
  {
    "metric_id": "1.1.1",
    "metric_title": "...",
    "asset_types": "...",
    "asset_types_in_scope": "...",
    "application_types_in_scope": "...",
    "environment_in_scope": "...",
    "status_in_scope": "...",
    "instance_types_in_scope": "...",
    "criticality_levels_in_scope": "...",
    "exclusions": "",
    "special_conditions": "",
    "data_sources_required": "...",
    "business_justification": "...",
    "notes": ""
  }
]

All entries use the same set of keys. Optional fields with no value use empty strings, never null or omitted keys.

Status Severity Map

const STATUS_SEVERITY = {
  'Below 15% of Target':   0,  // worst
  'Within 15% of Target':  1,
  'Meets/Exceeds Target':  2,  // best
};

Updated metricFilter State

// Old: metricFilter: string | null
// New: metricFilter: string[] | null
//
// null = no filter (show all devices)
// string[] = show devices with failing_metrics matching any ID in the array

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Grouping invariant — no entries lost or misplaced

For any array of summary entries and any team string, grouping by metric family SHALL produce groups where (a) every entry appears in exactly one group, (b) all entries within a group share the same metric_id, and (c) the total number of entries across all groups equals the number of entries for that team in the input.

Validates: Requirements 1.1, 1.2

Property 2: Worst-status computation follows severity ordering

For any non-empty array of status strings drawn from {"Below 15% of Target", "Within 15% of Target", "Meets/Exceeds Target"}, the computed worst status SHALL be the status with the lowest severity rank present in the array. If the array contains "Below 15% of Target", the result SHALL be "Below 15% of Target" regardless of other values.

Validates: Requirements 1.6, 3.1

Property 3: Device filtering with metric family includes all matching devices

For any array of device objects (each with a failing_metrics array of {metric_id} objects) and for any non-empty array of filter metric IDs, the filtered result SHALL contain exactly those devices that have at least one failing_metrics entry whose metric_id is included in the filter array. No matching device is excluded and no non-matching device is included.

Validates: Requirements 1.8, 7.1, 7.2

Property 4: Definition lookup returns correct entry or null

For any array of metric definition objects with unique metric_id values, building a lookup map and querying it with a metric_id that exists in the array SHALL return the corresponding definition object. Querying with a metric_id not in the array SHALL return undefined.

Validates: Requirements 4.2, 4.6

Property 5: Detail panel renders all required definition fields

For any valid metric definition object (with all 14 fields present), the set of field keys rendered by the detail panel SHALL include: metric_title, asset_types, asset_types_in_scope, application_types_in_scope, environment_in_scope, status_in_scope, instance_types_in_scope, criticality_levels_in_scope, exclusions, special_conditions, data_sources_required, business_justification, and notes.

Validates: Requirements 5.3

Property 6: Definitions schema validation — all entries have required fields

For any entry in the metric definitions array, the entry SHALL have all 14 required keys present, and the metric_id field SHALL be a non-empty string. Optional fields (exclusions, special_conditions, notes) SHALL be present as strings (empty string if no value), never omitted or null.

Validates: Requirements 6.2, 8.3, 8.4

Property 7: Lookup map construction preserves all definitions

For any array of metric definition objects with unique metric_id values, building a lookup map keyed by metric_id SHALL produce a map with exactly as many entries as the input array, and every input definition SHALL be retrievable by its metric_id.

Validates: Requirements 6.4

Property 8: JSON round-trip preserves metric definition data

For any valid metric definition object, JSON.parse(JSON.stringify(definition)) SHALL produce an object deeply equal to the original.

Validates: Requirements 8.1, 8.2

Error Handling

Metric Definitions File

Error Scenario	Handling
metricDefinitions.json fails to import (malformed JSON)	Build-time error caught by Create React App. The file is validated at development time.
Metric ID not found in definitions lookup	Tooltip falls back to entry.description from summary data. Info panel shows "No detailed definition available" with summary description.
Definition entry has empty optional fields	Rendered sections show "—" placeholder for empty strings. No error thrown.

Grouping Logic

Error Scenario	Handling
Summary entries array is empty	groupByMetricFamily returns empty array. No cards rendered. Existing "No compliance data" empty state shown.
Summary entry has missing or empty metric_id	Entry is skipped during grouping (filtered out).
All entries for a team have the same metric_id	Single family group with multiple variant pills. Works correctly.

Hover Tooltip

Error Scenario	Handling
User moves mouse away before 300ms	Timeout cleared, tooltip never shown. No side effects.
Tooltip would render off-screen	Position clamped to viewport bounds using getBoundingClientRect().
Rapid hover/unhover across multiple cards	Previous timeout cleared on each mouseLeave. Only the currently hovered card's tooltip can appear.

Info Panel

Error Scenario	Handling
Info icon click while tooltip is visible	Tooltip dismissed (mouseLeave fires). Panel opens. No conflict.
Multiple rapid info icon clicks	infoMetric state is set to the latest clicked metric. Only one panel open at a time.
Click outside panel while scrolled	Overlay backdrop captures click, closes panel. Scroll position preserved.

Device Table Filtering

Error Scenario	Handling
metricFilter is set to an array but no devices match	Empty state message shown: "No non-compliant devices".
Device has failing_metrics with IDs not in any family	Device only shown when no filter is active or when its metric IDs match the active filter.

Testing Strategy

Unit Tests (Example-Based)

Unit tests cover specific rendering, interaction, and integration scenarios:

Grouping and display:

• Groups entries with the same metric_id into one card
• Single-entry families render one variant pill
• Multi-entry families render one pill per entry
• Card title shows base metric ID and category from first entry
• Card shows shared target percentage

Worst-status computation:

• All "Meets/Exceeds Target" → card shows "OK" with success color
• Mix of statuses → card uses the worst status color
• Single "Below 15% of Target" among passing variants → card shows danger color

Variant pills:

• Each pill shows the entry's team label and compliance percentage
• Pill background tint matches the entry's individual status color
• Non-passing variants show a glow dot

Hover tooltip:

• Tooltip appears after 300ms hover delay
• Tooltip shows metric title, business justification, data sources from definitions
• Tooltip disappears on mouse leave
• Tooltip falls back to summary description when no definition exists
• Tooltip does not interfere with card click

Info panel:

• Info icon click opens MetricInfoPanel with correct metric definition
• Info icon click does not trigger card's metric filter toggle (stopPropagation)
• Panel displays all 12+ definition fields with section labels
• Panel shows fallback message when no definition exists
• Panel closes on outside click or close button

Device table filtering:

• Clicking a grouped card sets filter to all metric IDs in that family
• Filtered device table shows devices matching any ID in the family
• Clicking the same card again clears the filter
• Clear filter button resets to show all devices
• Active card shows highlighted styling

Definitions file:

• File imports without error
• Lookup map contains all metric IDs from the file
• All entries have the required 14 fields

Property-Based Tests

Property-based tests use fast-check to verify universal properties across generated inputs. Each test runs a minimum of 100 iterations.

Property	Test Description	Tag
Property 1	Generate random arrays of summary entries with varying metric_id values and team strings. Group them and verify: all entries accounted for, entries within each group share the same metric_id, group count equals unique metric_id count.	Feature: compliance-metric-grouping, Property 1: Grouping invariant — no entries lost or misplaced
Property 2	Generate random non-empty arrays of status strings from the valid set. Compute worst status and verify it matches the minimum severity rank in the array.	Feature: compliance-metric-grouping, Property 2: Worst-status computation follows severity ordering
Property 3	Generate random device arrays (each with random failing_metrics) and random filter ID arrays. Filter and verify the result contains exactly the devices with at least one matching metric.	Feature: compliance-metric-grouping, Property 3: Device filtering with metric family includes all matching devices
Property 4	Generate random arrays of metric definitions with unique IDs. Build lookup map, query with IDs from the array (expect hit) and IDs not in the array (expect miss).	Feature: compliance-metric-grouping, Property 4: Definition lookup returns correct entry or null
Property 5	Generate random metric definition objects with all 14 fields. Extract the rendered field keys and verify all required keys are present.	Feature: compliance-metric-grouping, Property 5: Detail panel renders all required definition fields
Property 6	Generate random arrays of metric definition objects. Verify every entry has all 14 keys present, metric_id is a non-empty string, and optional fields are strings (not null/undefined).	Feature: compliance-metric-grouping, Property 6: Definitions schema validation — all entries have required fields
Property 7	Generate random definition arrays with unique IDs. Build lookup map and verify map size equals array length, and every definition is retrievable by its metric_id.	Feature: compliance-metric-grouping, Property 7: Lookup map construction preserves all definitions
Property 8	Generate random metric definition objects with string values. Round-trip through JSON.stringify then JSON.parse and verify deep equality.	Feature: compliance-metric-grouping, Property 8: JSON round-trip preserves metric definition data

Test Configuration

• Library: fast-check (JavaScript property-based testing)
• Runner: Jest (via react-scripts test)
• Iterations: Minimum 100 per property test (fc.assert(property, { numRuns: 100 }))
• Tag format: Comment at top of each property test referencing the design property