Update .kiro: remove SQLite hooks, add PostgreSQL migration hook, add workflow steering, sync specs

2026-05-12 14:45:58 -06:00
parent 3ee8487286
commit 1bb8ec1658
35 changed files with 4645 additions and 48 deletions
--- a/.kiro/specs/vcl-compliance-reporting/.config.kiro
+++ b/.kiro/specs/vcl-compliance-reporting/.config.kiro
@@ -0,0 +1 @@
+{"specId": "a7c3e1d4-9f82-4b6a-a3d1-7e5f2c8b9a04", "workflowType": "requirements-first", "specType": "feature"}
--- a/.kiro/specs/vcl-compliance-reporting/design.md
+++ b/.kiro/specs/vcl-compliance-reporting/design.md
@@ -0,0 +1,505 @@
+# Design Document: VCL Compliance Reporting
+
+## Overview
+
+This feature adds an executive-level VCL (Vulnerability Compliance Level) reporting page to the existing Compliance module, extends device records with remediation tracking fields (resolution date, remediation plan), and introduces a bulk upload mechanism for updating device metadata in batch. The VCL Report Page mirrors the layout of the leadership's existing spreadsheet deck — summary statistics bar, trend chart with forecast, non-compliant asset donut chart, heavy hitters table, and vertical breakdown table with burndown projections.
+
+The implementation builds on the existing `compliance.js` route module, `compliance_items` table, and `CompliancePage.js` frontend component. New backend endpoints compute VCL statistics from existing data plus the new `resolution_date` and `remediation_plan` columns. The frontend adds a new `VCLReportPage.js` component accessible from the Compliance module navigation.
+
+## Architecture
+
+```mermaid
+sequenceDiagram
+    participant U as User
+    participant FE as React Frontend
+    participant BE as Express Backend
+    participant DB as PostgreSQL
+
+    Note over FE,DB: Device Metadata Update (single device)
+    U->>FE: Edit resolution_date / remediation_plan in DetailPanel
+    FE->>BE: PATCH /api/compliance/items/:hostname/metadata
+    BE->>DB: UPDATE compliance_items SET resolution_date, remediation_plan WHERE hostname = $1
+    BE-->>FE: 200 OK { updated: count }
+
+    Note over FE,DB: VCL Report Page Load
+    FE->>BE: GET /api/compliance/vcl/stats
+    BE->>DB: Aggregate compliance_items (counts, percentages, categorization)
+    DB-->>BE: Raw counts
+    BE->>BE: Compute stats, categorization, heavy hitters, vertical breakdown
+    BE-->>FE: JSON { stats, donut, heavyHitters, verticalBreakdown }
+
+    FE->>BE: GET /api/compliance/vcl/trend
+    BE->>DB: Monthly aggregation from compliance_uploads + compliance_items history
+    DB-->>BE: Monthly data points
+    BE->>BE: Compute actuals + forecast
+    BE-->>FE: JSON { months: [...] }
+
+    Note over U,DB: Bulk Upload Flow
+    U->>FE: Select xlsx file in bulk upload control
+    FE->>FE: Parse xlsx with 'xlsx' library (client-side)
+    FE->>FE: Map columns, validate fields, match hostnames
+    FE->>BE: POST /api/compliance/vcl/bulk-preview { rows: [...] }
+    BE->>DB: Match hostnames against compliance_items
+    BE-->>FE: JSON { matched, unmatched, changes, invalid }
+    FE->>FE: Display Diff_Preview
+    U->>FE: Confirm changes
+    FE->>BE: POST /api/compliance/vcl/bulk-commit { changes: [...] }
+    BE->>DB: BEGIN; UPDATE compliance_items ...; COMMIT;
+    BE-->>FE: 200 OK { committed: count }
+```
+
+### Data Flow Summary
+
+1. **Device metadata** — stored directly on `compliance_items` rows. Updated via PATCH endpoint (single) or bulk commit (batch).
+2. **VCL statistics** — computed on-demand from current `compliance_items` state. No separate materialized table needed since the dataset is small (~1000 devices).
+3. **Trend data** — derived from `compliance_uploads` history (existing) plus monthly snapshots of compliance percentages stored in a new `compliance_snapshots` table.
+4. **Burndown projections** — computed from `resolution_date` values on active non-compliant items, bucketed by month.
+
+## Components and Interfaces
+
+### Backend
+
+#### New Endpoints (added to `backend/routes/compliance.js`)
+
+**`PATCH /api/compliance/items/:hostname/metadata`**
+
+Updates resolution_date and/or remediation_plan for all active items matching a hostname.
+
+- Auth: `requireAuth()`, `requireGroup('Admin', 'Standard_User')`
+- Body: `{ resolution_date?: string|null, remediation_plan?: string|null }`
+- Validation: resolution_date must be a valid ISO date or null; remediation_plan must be <= 2000 chars
+- Response: `{ updated: number }`
+
+**`GET /api/compliance/vcl/stats`**
+
+Returns computed VCL executive summary statistics.
+
+- Auth: `requireAuth()`
+- Response:
+```json
+{
+  "stats": {
+    "total_devices": 1200,
+    "in_scope": 1100,
+    "compliant": 950,
+    "non_compliant": 150,
+    "remediations_required": 150,
+    "compliance_pct": 86,
+    "target_pct": 95
+  },
+  "donut": {
+    "blocked": { "count": 45, "pct": 30 },
+    "in_progress": { "count": 105, "pct": 70 }
+  },
+  "heavy_hitters": [
+    { "vertical": "Network Ops", "team": "STEAM", "non_compliant": 42, "compliance_date": "2026-06-30", "notes": "..." }
+  ],
+  "vertical_breakdown": [
+    {
+      "vertical": "Network Ops",
+      "compliance_pct": 82,
+      "team": "STEAM",
+      "non_compliant": 42,
+      "actual_burndown": { "2026-01": 5, "2026-02": 8 },
+      "forecast_burndown": { "2026-03": 10, "2026-04": 12 },
+      "blockers": 8,
+      "risk_acceptances": 3,
+      "notes": ""
+    }
+  ]
+}
+```
+
+**`GET /api/compliance/vcl/trend`**
+
+Returns monthly compliance trend data for the overview chart.
+
+- Auth: `requireAuth()`
+- Query params: none
+- Response:
+```json
+{
+  "months": [
+    {
+      "month": "2026-01",
+      "compliant_count": 900,
+      "compliance_pct": 82,
+      "forecast_pct": null,
+      "target_pct": 95
+    }
+  ]
+}
+```
+
+Forecast is computed using linear regression on the last 3+ months of actual data, projected forward.
+
+**`POST /api/compliance/vcl/bulk-preview`**
+
+Accepts parsed bulk upload rows and returns a diff preview.
+
+- Auth: `requireAuth()`, `requireGroup('Admin', 'Standard_User')`
+- Body: `{ rows: [{ hostname, resolution_date?, remediation_plan?, notes? }] }`
+- Response:
+```json
+{
+  "matched": 850,
+  "unmatched": 12,
+  "changes": 200,
+  "invalid": 5,
+  "details": [
+    {
+      "hostname": "srv-001",
+      "status": "changed",
+      "fields": {
+        "resolution_date": { "old": null, "new": "2026-06-15" },
+        "remediation_plan": { "old": "", "new": "Patch in next window" }
+      }
+    }
+  ],
+  "unmatched_rows": ["unknown-host-1"],
+  "invalid_rows": [{ "hostname": "srv-bad", "errors": ["resolution_date: invalid date format"] }]
+}
+```
+
+**`POST /api/compliance/vcl/bulk-commit`**
+
+Commits validated bulk changes in a single transaction.
+
+- Auth: `requireAuth()`, `requireGroup('Admin', 'Standard_User')`
+- Body: `{ changes: [{ hostname, resolution_date?, remediation_plan?, notes? }] }`
+- Response: `{ committed: number }`
+- Audit: logs `compliance_bulk_update` action
+
+#### Pure Helper Functions (exported for testing)
+
+```javascript
+// Truncates text to maxLen chars with ellipsis
+function truncateText(text, maxLen = 80) { ... }
+
+// Validates remediation_plan length
+function validateRemediationPlan(text) { ... }
+
+// Validates a date string (ISO format)
+function isValidDateString(str) { ... }
+
+// Computes VCL summary stats from device rows
+function computeVCLStats(items, targetPct) { ... }
+
+// Categorizes non-compliant devices into blocked/in-progress
+function categorizeNonCompliant(items) { ... }
+
+// Ranks verticals by non-compliant count descending
+function rankHeavyHitters(verticalData) { ... }
+
+// Computes forecasted burndown from resolution_date values
+function computeForecastBurndown(items) { ... }
+
+// Matches uploaded rows to existing devices by hostname
+function matchByHostname(uploadedRows, existingHostnames) { ... }
+
+// Computes diff between uploaded values and current DB values
+function computeBulkDiff(matchedRows, currentData) { ... }
+
+// Maps column headers to known field names
+function mapColumnHeaders(headers) { ... }
+
+// Formats a decimal as a whole-number percentage string
+function formatPct(decimal) { ... }
+```
+
+### Frontend
+
+#### New Component: `VCLReportPage.js`
+
+Located at `frontend/src/components/pages/VCLReportPage.js`. Accessible via a tab/button on the existing CompliancePage or as a separate nav entry.
+
+**Sub-components:**
+
+| Component | Purpose |
+|-----------|---------|
+| `VCLStatsBar` | Horizontal bar with 7 stat cards (Total, In-Scope, Compliant, Non-Compliant, Remediations, Current %, Target %) |
+| `ComplianceOverviewChart` | Recharts ComposedChart — bars for compliant count, solid line for actual %, dashed line for forecast %, ReferenceLine for target |
+| `NonCompliantDonutChart` | Recharts PieChart (donut) — Blocked vs In-Progress segments |
+| `HeavyHittersTable` | Sorted table of top verticals by non-compliant count |
+| `VerticalBreakdownTable` | Full breakdown table with burndown columns |
+| `BulkUploadModal` | Modal with file picker, column mapping preview, diff display, confirm/cancel |
+
+#### Modified Component: `ComplianceDetailPanel.js`
+
+Add two new fields to the device detail panel:
+- **Resolution Date** — `<input type="date">` with save on blur/enter
+- **Remediation Plan** — `<textarea>` with character counter (max 2000) and save button
+
+#### Modified Component: `CompliancePage.js`
+
+- Add "VCL Report" tab/button in the page header that navigates to VCLReportPage
+- Add `resolution_date` and `remediation_plan` columns to the device table
+
+### Chart Specifications
+
+#### Compliance Overview Chart (Recharts ComposedChart)
+
+```javascript
+<ComposedChart data={months}>
+  <CartesianGrid stroke="rgba(255,255,255,0.05)" strokeDasharray="3 3" />
+  <XAxis dataKey="month" tick={AXIS_STYLE} />
+  <YAxis yAxisId="count" tick={AXIS_STYLE} />
+  <YAxis yAxisId="pct" orientation="right" domain={[0, 100]} unit="%" tick={AXIS_STYLE} />
+  <Bar yAxisId="count" dataKey="compliant_count" fill="#10B981" fillOpacity={0.7} />
+  <Line yAxisId="pct" dataKey="compliance_pct" stroke={TEAL} strokeWidth={2} dot={{ r: 3 }} />
+  <Line yAxisId="pct" dataKey="forecast_pct" stroke={TEAL} strokeWidth={2} strokeDasharray="5 3" dot={false} />
+  <ReferenceLine yAxisId="pct" y={targetPct} stroke="#F59E0B" strokeDasharray="4 4" label="Target" />
+</ComposedChart>
+```
+
+#### Non-Compliant Assets Donut (Recharts PieChart)
+
+```javascript
+<PieChart>
+  <Pie data={donutData} innerRadius={60} outerRadius={90} dataKey="count" nameKey="name">
+    <Cell fill="#EF4444" />  {/* Blocked */}
+    <Cell fill="#F59E0B" />  {/* In-Progress */}
+  </Pie>
+  <Legend />
+</PieChart>
+```
+
+## Data Models
+
+### Schema Changes to `compliance_items`
+
+Two new columns:
+
+```sql
+ALTER TABLE compliance_items ADD COLUMN IF NOT EXISTS resolution_date DATE DEFAULT NULL;
+ALTER TABLE compliance_items ADD COLUMN IF NOT EXISTS remediation_plan TEXT DEFAULT NULL;
+```
+
+- `resolution_date` — target date for remediation completion. NULL means no date set.
+- `remediation_plan` — free-text description of the fix approach. NULL or empty means no plan documented. Max 2000 characters enforced at application layer.
+
+### New Table: `compliance_snapshots`
+
+Stores monthly compliance percentage snapshots for trend charting. One row per vertical per month.
+
+```sql
+CREATE TABLE IF NOT EXISTS compliance_snapshots (
+    id              SERIAL PRIMARY KEY,
+    snapshot_month  TEXT NOT NULL,          -- 'YYYY-MM' format
+    vertical        TEXT NOT NULL,
+    total_devices   INTEGER NOT NULL DEFAULT 0,
+    compliant       INTEGER NOT NULL DEFAULT 0,
+    non_compliant   INTEGER NOT NULL DEFAULT 0,
+    compliance_pct  NUMERIC(5,2) DEFAULT 0,
+    created_at      TIMESTAMPTZ DEFAULT NOW(),
+    UNIQUE(snapshot_month, vertical)
+);
+
+CREATE INDEX IF NOT EXISTS idx_compliance_snapshots_month
+ON compliance_snapshots(snapshot_month);
+```
+
+Snapshots are created automatically when a new compliance upload is committed — the commit logic inserts/updates the snapshot for the current month.
+
+### Migration Script: `backend/migrations/add_vcl_reporting_columns.js`
+
+```javascript
+const pool = require('../db');
+
+async function run() {
+    console.log('Starting VCL reporting migration...');
+    try {
+        await pool.query(`ALTER TABLE compliance_items ADD COLUMN IF NOT EXISTS resolution_date DATE DEFAULT NULL`);
+        console.log('✓ resolution_date column added');
+
+        await pool.query(`ALTER TABLE compliance_items ADD COLUMN IF NOT EXISTS remediation_plan TEXT DEFAULT NULL`);
+        console.log('✓ remediation_plan column added');
+
+        await pool.query(`
+            CREATE TABLE IF NOT EXISTS compliance_snapshots (
+                id              SERIAL PRIMARY KEY,
+                snapshot_month  TEXT NOT NULL,
+                vertical        TEXT NOT NULL,
+                total_devices   INTEGER NOT NULL DEFAULT 0,
+                compliant       INTEGER NOT NULL DEFAULT 0,
+                non_compliant   INTEGER NOT NULL DEFAULT 0,
+                compliance_pct  NUMERIC(5,2) DEFAULT 0,
+                created_at      TIMESTAMPTZ DEFAULT NOW(),
+                UNIQUE(snapshot_month, vertical)
+            )
+        `);
+        console.log('✓ compliance_snapshots table created');
+
+        await pool.query(`CREATE INDEX IF NOT EXISTS idx_compliance_snapshots_month ON compliance_snapshots(snapshot_month)`);
+        console.log('✓ compliance_snapshots index created');
+    } catch (err) {
+        console.error('Migration error:', err.message);
+        process.exit(1);
+    }
+    console.log('Migration complete.');
+    process.exit(0);
+}
+
+run();
+```
+
+## Correctness Properties
+
+*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
+
+### Property 1: Device Metadata Persistence Round-Trip
+
+*For any* valid resolution_date (ISO date string or null) and any valid remediation_plan (string of 0–2000 characters or null), saving the metadata via the update endpoint and then fetching the device should return the same resolution_date and remediation_plan values.
+
+**Validates: Requirements 1.3, 2.3**
+
+### Property 2: Text Truncation
+
+*For any* string, `truncateText(text, 80)` should return the original string if its length is <= 80, or the first 80 characters followed by "…" if its length exceeds 80. The output length should never exceed 81 characters (80 + ellipsis).
+
+**Validates: Requirements 2.4**
+
+### Property 3: Remediation Plan Length Validation
+
+*For any* string, `validateRemediationPlan(text)` should return valid if and only if the string length is <= 2000 characters. Strings exceeding 2000 characters should be flagged as invalid.
+
+**Validates: Requirements 2.5, 9.4**
+
+### Property 4: Summary Statistics Computation Invariants
+
+*For any* set of compliance items with total, compliant, and non-compliant counts where total >= compliant >= 0 and non_compliant = total - compliant, `computeVCLStats(items, target)` should produce: non_compliant + compliant = total, compliance_pct = Math.round((compliant / total) * 100) when total > 0, and compliance_pct = 0 when total = 0.
+
+**Validates: Requirements 3.2, 7.3**
+
+### Property 5: Percentage Formatting
+
+*For any* decimal number between 0 and 1 (inclusive), `formatPct(decimal)` should return `Math.round(decimal * 100) + '%'`. The output should always match the regex pattern `/^\d{1,3}%$/`.
+
+**Validates: Requirements 3.3**
+
+### Property 6: Non-Compliant Device Categorization Partition
+
+*For any* array of non-compliant device objects, `categorizeNonCompliant(items)` should produce two groups (blocked, in_progress) where: every input item appears in exactly one group, blocked.count + in_progress.count = items.length, and each group's percentage equals Math.round((group.count / items.length) * 100) when items.length > 0.
+
+**Validates: Requirements 5.2, 5.3**
+
+### Property 7: Heavy Hitters Descending Sort
+
+*For any* array of vertical objects with non_compliant counts, `rankHeavyHitters(verticals)` should return the array sorted in strictly non-increasing order by non_compliant count. For all consecutive pairs (a, b) in the output, a.non_compliant >= b.non_compliant.
+
+**Validates: Requirements 6.1, 6.3**
+
+### Property 8: Forecasted Burndown Projection
+
+*For any* set of non-compliant devices with resolution_date values (some null, some valid future dates), `computeForecastBurndown(items)` should produce monthly buckets where: the sum of all monthly forecast counts equals the number of items with non-null resolution_dates, and each item with a resolution_date appears in exactly the bucket corresponding to its resolution month.
+
+**Validates: Requirements 7.5**
+
+### Property 9: Hostname Matching with Unmatched Flagging
+
+*For any* array of uploaded rows (each with a hostname) and a set of existing hostnames, `matchByHostname(rows, existing)` should produce: matched rows (hostname exists in the set) + unmatched rows (hostname not in set) = total input rows. Every matched row's hostname must be in the existing set, and every unmatched row's hostname must not be in the existing set.
+
+**Validates: Requirements 8.2, 8.7**
+
+### Property 10: Bulk Diff Change Detection
+
+*For any* array of matched row pairs (uploaded value, current DB value) for fields resolution_date and remediation_plan, `computeBulkDiff(matched, current)` should flag a row as "changed" if and only if at least one field value differs between uploaded and current. Rows where all fields are identical should be flagged as "unchanged".
+
+**Validates: Requirements 8.3, 8.4**
+
+### Property 11: Column Header Mapping
+
+*For any* array of column header strings, `mapColumnHeaders(headers)` should: return a mapping that includes "hostname" if any header case-insensitively matches "Hostname", include "resolution_date" if any header matches "Resolution Date", include "remediation_plan" if any header matches "Remediation Plan", and include "notes" if any header matches "Notes". Headers not matching any known field should be ignored.
+
+**Validates: Requirements 9.2**
+
+### Property 12: Date String Validation
+
+*For any* string, `isValidDateString(str)` should return true if and only if the string can be parsed into a valid Date object representing a real calendar date (e.g., "2026-02-30" is invalid). Null and empty string should return false.
+
+**Validates: Requirements 9.3**
+
+### Property 13: Row Count Arithmetic Invariant
+
+*For any* bulk upload preview result with matched, unmatched, and invalid counts, the sum matched + unmatched must equal the total number of input rows. Additionally, within matched rows, changed + unchanged must equal matched count.
+
+**Validates: Requirements 9.6**
+
+## Error Handling
+
+### Device Metadata Update Errors
+
+| Condition | HTTP Status | Response | Behavior |
+|-----------|-------------|----------|----------|
+| Hostname not found | 404 | `{ "error": "Device not found" }` | No state change |
+| Invalid date format | 400 | `{ "error": "Invalid resolution_date format" }` | No state change |
+| Remediation plan > 2000 chars | 400 | `{ "error": "Remediation plan exceeds 2000 characters" }` | No state change |
+| Database error | 500 | `{ "error": "Failed to update device metadata" }` | No state change |
+
+### VCL Stats Endpoint Errors
+
+| Condition | HTTP Status | Response | Behavior |
+|-----------|-------------|----------|----------|
+| No compliance data | 200 | `{ "stats": { all zeros }, ... }` | Return empty/zero stats gracefully |
+| Database error | 500 | `{ "error": "Database error" }` | Log error |
+
+### Bulk Upload Errors
+
+| Condition | HTTP Status | Response | Behavior |
+|-----------|-------------|----------|----------|
+| No rows in file | 400 | `{ "error": "File contains no data rows" }` | No state change |
+| No Hostname column | 400 | `{ "error": "File must contain a Hostname column" }` | No state change |
+| No updatable columns | 400 | `{ "error": "No updatable fields found (need Resolution Date, Remediation Plan, or Notes)" }` | No state change |
+| File exceeds 2000 rows | 400 | `{ "error": "File exceeds maximum of 2000 rows" }` | No state change |
+| Transaction failure on commit | 500 | `{ "error": "Failed to commit changes" }` | Full rollback, no partial updates |
+
+### Frontend Error Handling
+
+- API failures display inline error messages (red text, monospace, consistent with existing patterns)
+- Bulk upload validation errors are shown per-row in the diff preview with red highlighting
+- Network errors show a retry prompt
+- File parsing errors (corrupt xlsx) show a user-friendly message suggesting re-export from the source
+
+## Testing Strategy
+
+### Property-Based Testing
+
+Use `fast-check` as the property-based testing library (already used in this project). Each correctness property maps to a single property-based test with a minimum of 100 iterations.
+
+Property tests focus on the pure helper functions exported from the compliance route module:
+- `truncateText` — Property 2
+- `validateRemediationPlan` — Property 3
+- `computeVCLStats` — Property 4
+- `formatPct` — Property 5
+- `categorizeNonCompliant` — Property 6
+- `rankHeavyHitters` — Property 7
+- `computeForecastBurndown` — Property 8
+- `matchByHostname` — Property 9
+- `computeBulkDiff` — Property 10
+- `mapColumnHeaders` — Property 11
+- `isValidDateString` — Property 12
+
+Tag format: **Feature: vcl-compliance-reporting, Property {number}: {title}**
+
+Test file: `backend/__tests__/vcl-compliance-reporting.property.test.js`
+
+### Unit Testing
+
+Unit tests cover specific examples, edge cases, and integration points:
+
+- **PATCH metadata endpoint** — happy path, invalid date, plan too long, hostname not found
+- **VCL stats with no data** — verify zero/empty response
+- **Bulk preview with all unmatched** — verify correct counts
+- **Bulk preview with mixed valid/invalid** — verify row classification
+- **Bulk commit transactional** — verify all-or-nothing behavior
+- **Donut chart with single category** — verify full donut rendering
+- **Trend chart with < 2 months** — verify no forecast line
+- **Vertical with zero non-compliant** — verify zero display
+
+Test file: `backend/__tests__/vcl-compliance-reporting.test.js`
+
+### Integration Testing
+
+- Full bulk upload flow: parse → preview → commit → verify DB state
+- Device metadata update → verify VCL stats reflect the change
+- Snapshot creation on upload commit → verify trend data includes new month
--- a/.kiro/specs/vcl-compliance-reporting/requirements.md
+++ b/.kiro/specs/vcl-compliance-reporting/requirements.md
@@ -0,0 +1,135 @@
+# Requirements Document
+
+## Introduction
+
+This feature adds executive-level VCL (Vulnerability Compliance Level) reporting capabilities to the STEAM Security Dashboard's Compliance module. Leadership requires a dedicated reporting page that mirrors the layout of their existing VCL spreadsheet deck used for executive presentations. The feature also extends device records with remediation tracking fields and introduces bulk upload support for updating approximately 1,000 device records at once with resolution dates, remediation plans, and notes.
+
+## Glossary
+
+- **Dashboard**: The STEAM Security Dashboard application
+- **Compliance_Module**: The existing AEO Compliance section of the Dashboard, which handles weekly xlsx upload, diff preview, per-team metric health cards, and device-level violation tracking
+- **VCL_Report_Page**: A new page within the Compliance_Module that displays executive-level compliance summary statistics, trend charts, and vertical breakdown tables
+- **Device_Record**: A compliance_items row representing a non-compliant asset tracked in the system, identified by hostname and metric_id
+- **Resolution_Date**: A date field on a Device_Record indicating the target date by which remediation is expected to be complete
+- **Remediation_Plan**: A free-text field on a Device_Record describing the planned approach to bring the device into compliance
+- **Vertical**: A logical grouping of devices by business function or organizational unit (e.g., a team or department responsible for a set of assets)
+- **Compliance_Percentage**: The ratio of compliant devices to total in-scope devices, expressed as a percentage
+- **Target_Compliance**: The organization-defined compliance percentage threshold that all verticals are expected to meet
+- **Burndown**: A month-over-month projection of how many non-compliant devices will be remediated, shown as actual (historical) and forecasted (future) values
+- **Heavy_Hitter**: A vertical with a disproportionately high count of non-compliant devices relative to other verticals
+- **Bulk_Upload**: The process of uploading an xlsx file to update multiple Device_Records simultaneously with remediation metadata
+- **Diff_Preview**: A summary view showing proposed changes before they are committed to the database, allowing the user to review and confirm or cancel
+
+## Requirements
+
+### Requirement 1: Resolution Date Field on Device Records
+
+**User Story:** As a compliance analyst, I want to set a target resolution date on each non-compliant device, so that leadership can track remediation timelines.
+
+#### Acceptance Criteria
+
+1. THE Dashboard SHALL store a Resolution_Date field on each Device_Record in the compliance_items table
+2. WHEN a user opens the ComplianceDetailPanel for a device, THE Dashboard SHALL display the Resolution_Date field as an editable date input
+3. WHEN the user sets or changes the Resolution_Date value and saves, THE Dashboard SHALL persist the updated value to the database
+4. THE Dashboard SHALL display the Resolution_Date as a column in the compliance device table view
+5. IF the user clears the Resolution_Date field, THEN THE Dashboard SHALL store a null value and display the column cell as empty
+
+### Requirement 2: Remediation Plan Field on Device Records
+
+**User Story:** As a compliance analyst, I want to document a remediation plan for each non-compliant device, so that teams have a clear record of the intended fix.
+
+#### Acceptance Criteria
+
+1. THE Dashboard SHALL store a Remediation_Plan field on each Device_Record in the compliance_items table
+2. WHEN a user opens the ComplianceDetailPanel for a device, THE Dashboard SHALL display the Remediation_Plan field as an editable text area
+3. WHEN the user enters or modifies the Remediation_Plan text and saves, THE Dashboard SHALL persist the updated value to the database
+4. THE Dashboard SHALL display the Remediation_Plan as a column in the compliance device table view, truncated to 80 characters with an ellipsis when the text exceeds that length
+5. THE Dashboard SHALL limit the Remediation_Plan field to 2000 characters
+
+### Requirement 3: VCL Executive Summary Statistics Bar
+
+**User Story:** As an executive, I want to see high-level compliance statistics at a glance, so that I can quickly assess the organization's security posture.
+
+#### Acceptance Criteria
+
+1. THE VCL_Report_Page SHALL display a summary statistics bar containing: Total Devices, In-Scope count, Compliant count, Non-Compliant count, Remediations Required count, Current Compliance_Percentage, and Target_Compliance percentage
+2. WHEN the VCL_Report_Page loads, THE Dashboard SHALL compute the summary statistics from the current compliance data in the database
+3. THE Dashboard SHALL format Compliance_Percentage and Target_Compliance as whole-number percentages with a percent sign
+4. WHEN the underlying compliance data changes due to a new upload, THE Dashboard SHALL reflect updated statistics upon page refresh
+
+### Requirement 4: Compliance Overview Trend Chart
+
+**User Story:** As an executive, I want to see a monthly compliance trend chart, so that I can understand whether compliance is improving over time.
+
+#### Acceptance Criteria
+
+1. THE VCL_Report_Page SHALL display a "Compliance Overview" chart showing monthly compliance data
+2. THE Chart SHALL render compliant asset counts as vertical bars for each month
+3. THE Chart SHALL render the actual Compliance_Percentage as a solid line overlaid on the bar chart
+4. THE Chart SHALL render the forecasted Compliance_Percentage as a dashed line extending beyond the current month
+5. THE Chart SHALL render the Target_Compliance threshold as a horizontal reference line
+6. WHEN fewer than two months of historical data exist, THE Dashboard SHALL display the chart with available data points without forecasting
+
+### Requirement 5: Non-Compliant Assets Status Chart
+
+**User Story:** As an executive, I want to see the breakdown of non-compliant asset statuses, so that I can understand how many are blocked versus in progress.
+
+#### Acceptance Criteria
+
+1. THE VCL_Report_Page SHALL display a "Status of Non-Compliant Assets" donut chart
+2. THE Chart SHALL segment non-compliant devices into "Blocked" and "In-Progress" categories
+3. THE Chart SHALL display the count and percentage for each segment
+4. WHEN all non-compliant devices belong to a single category, THE Chart SHALL render a full donut in that category's color with the count displayed
+
+### Requirement 6: Heavy Hitters Table
+
+**User Story:** As an executive, I want to see which verticals have the most non-compliant devices, so that I can focus leadership attention on the highest-impact areas.
+
+#### Acceptance Criteria
+
+1. THE VCL_Report_Page SHALL display a "Heavy Hitters" table showing verticals ranked by non-compliant device count in descending order
+2. THE Table SHALL include columns: Vertical name with Responsible Team, Non-Compliant count, Compliance Date (target), and Notes
+3. THE Dashboard SHALL derive the Heavy Hitters list from verticals with the highest non-compliant device counts
+4. WHEN a vertical has no target Compliance Date set, THE Table SHALL display the cell as empty
+
+### Requirement 7: Vertical Breakdown Table
+
+**User Story:** As an executive, I want a detailed breakdown of compliance by vertical with burndown projections, so that I can track each team's remediation progress against their timeline.
+
+#### Acceptance Criteria
+
+1. THE VCL_Report_Page SHALL display a detailed vertical breakdown table
+2. THE Table SHALL include columns: Vertical, Current Vertical Compliance_Percentage, Responsible Team, Non-Compliant Devices count, Actual Burndown (by month), Forecasted Burndown (by month from current month through Q4 2026), Count Blockers, Count RAs, and Notes
+3. THE Dashboard SHALL compute Current Vertical Compliance_Percentage as the ratio of compliant to total in-scope devices within each vertical
+4. THE Dashboard SHALL populate Actual Burndown columns with historical monthly remediation counts from upload history
+5. THE Dashboard SHALL populate Forecasted Burndown columns with projected monthly remediation counts based on Resolution_Date values of non-compliant devices in each vertical
+6. WHEN a vertical has no non-compliant devices, THE Table SHALL display zero values for Non-Compliant Devices, Blockers, and RAs columns
+
+### Requirement 8: Bulk Upload for Device Remediation Metadata
+
+**User Story:** As a compliance analyst, I want to upload an xlsx file to update resolution dates, remediation plans, and notes for many devices at once, so that I do not have to edit each device individually.
+
+#### Acceptance Criteria
+
+1. THE Dashboard SHALL provide a bulk upload control on the VCL_Report_Page that accepts an xlsx file
+2. WHEN the user uploads an xlsx file, THE Dashboard SHALL parse the file and match rows to existing Device_Records by hostname
+3. WHEN the file is parsed successfully, THE Dashboard SHALL display a Diff_Preview showing: total rows matched, rows with changes, and a summary of fields to be updated (Resolution_Date, Remediation_Plan, Notes)
+4. THE Diff_Preview SHALL highlight rows where the uploaded value differs from the current database value
+5. WHEN the user confirms the Diff_Preview, THE Dashboard SHALL commit all changes to the database in a single transaction
+6. IF the user cancels the Diff_Preview, THEN THE Dashboard SHALL discard the parsed data without modifying any records
+7. IF a row in the uploaded file references a hostname not found in the database, THEN THE Dashboard SHALL flag that row as unmatched in the Diff_Preview and exclude it from the commit
+8. THE Dashboard SHALL support xlsx files containing up to 2000 rows without timeout or failure
+
+### Requirement 9: Bulk Upload Field Mapping and Validation
+
+**User Story:** As a compliance analyst, I want the bulk upload to validate my data before committing, so that I do not accidentally corrupt device records with malformed data.
+
+#### Acceptance Criteria
+
+1. THE Dashboard SHALL require the uploaded xlsx file to contain a "Hostname" column for row matching
+2. THE Dashboard SHALL recognize and map the following optional columns: "Resolution Date", "Remediation Plan", "Notes"
+3. WHEN a Resolution Date value cannot be parsed as a valid date, THE Dashboard SHALL flag that cell as invalid in the Diff_Preview and exclude the row from the commit
+4. WHEN a Remediation Plan value exceeds 2000 characters, THE Dashboard SHALL flag that cell as invalid in the Diff_Preview and exclude the row from the commit
+5. IF the uploaded file contains no recognizable columns beyond Hostname, THEN THE Dashboard SHALL display an error message indicating no updatable fields were found
+6. THE Dashboard SHALL display the count of valid rows, invalid rows, and unmatched rows in the Diff_Preview summary
+
--- a/.kiro/specs/vcl-compliance-reporting/tasks.md
+++ b/.kiro/specs/vcl-compliance-reporting/tasks.md
@@ -0,0 +1,158 @@
+# Tasks
+
+## Task 1: Database Migration
+
+- [x] 1.1 Create migration file `backend/migrations/add_vcl_reporting_columns.js` that adds `resolution_date DATE DEFAULT NULL` and `remediation_plan TEXT DEFAULT NULL` columns to the `compliance_items` table using `ALTER TABLE ... ADD COLUMN IF NOT EXISTS`
+- [x] 1.2 In the same migration, create the `compliance_snapshots` table with columns: id (SERIAL PK), snapshot_month (TEXT NOT NULL), vertical (TEXT NOT NULL), total_devices (INTEGER), compliant (INTEGER), non_compliant (INTEGER), compliance_pct (NUMERIC(5,2)), created_at (TIMESTAMPTZ), with a UNIQUE constraint on (snapshot_month, vertical)
+- [x] 1.3 Add index `idx_compliance_snapshots_month` on `compliance_snapshots(snapshot_month)`
+- [x] 1.4 Run the migration and verify columns and table exist in the database
+
+## Task 2: Backend — Pure Helper Functions
+
+- [x] 2.1 Implement and export `truncateText(text, maxLen)` — returns original if length <= maxLen, otherwise first maxLen chars + "…"
+- [x] 2.2 Implement and export `validateRemediationPlan(text)` — returns `{ valid: true }` if length <= 2000, otherwise `{ valid: false, error: "..." }`
+- [x] 2.3 Implement and export `isValidDateString(str)` — returns true only for strings parseable as real calendar dates (rejects "2026-02-30", null, empty)
+- [x] 2.4 Implement and export `formatPct(decimal)` — returns `Math.round(decimal * 100) + '%'`
+- [x] 2.5 Implement and export `computeVCLStats(items, targetPct)` — computes total, in_scope, compliant, non_compliant, remediations_required, compliance_pct from an array of device objects
+- [x] 2.6 Implement and export `categorizeNonCompliant(items)` — partitions items into "blocked" (no resolution_date set) and "in_progress" (resolution_date set) with counts and percentages
+- [x] 2.7 Implement and export `rankHeavyHitters(verticalData)` — sorts verticals by non_compliant count descending
+- [x] 2.8 Implement and export `computeForecastBurndown(items)` — buckets items by resolution_date month, returns monthly forecast counts
+- [x] 2.9 Implement and export `matchByHostname(uploadedRows, existingHostnames)` — returns { matched, unmatched } arrays
+- [x] 2.10 Implement and export `computeBulkDiff(matchedRows, currentData)` — compares uploaded vs current values, returns change details
+- [x] 2.11 Implement and export `mapColumnHeaders(headers)` — maps column header strings to known field names (case-insensitive matching)
+
+## Task 3: Backend — Device Metadata Endpoint
+
+- [x] 3.1 Add `PATCH /items/:hostname/metadata` endpoint to `backend/routes/compliance.js` with `requireAuth()` and `requireGroup('Admin', 'Standard_User')` middleware
+- [x] 3.2 Implement request body validation: resolution_date must be a valid ISO date or null, remediation_plan must be <= 2000 chars or null
+- [x] 3.3 Update all active `compliance_items` rows matching the hostname with the provided fields
+- [x] 3.4 Return 404 if no rows match the hostname, 200 with `{ updated: count }` on success
+- [x] 3.5 Log audit entry with action `compliance_metadata_update`
+
+## Task 4: Backend — VCL Stats Endpoint
+
+- [x] 4.1 Add `GET /vcl/stats` endpoint to `backend/routes/compliance.js` with `requireAuth()` middleware
+- [x] 4.2 Query compliance_items to compute summary statistics (total devices, compliant, non-compliant counts) using `computeVCLStats`
+- [x] 4.3 Compute donut chart data using `categorizeNonCompliant` on active non-compliant items
+- [x] 4.4 Compute heavy hitters using `rankHeavyHitters` grouped by team/vertical
+- [x] 4.5 Compute vertical breakdown with actual burndown (from upload history) and forecast burndown (from resolution_dates) using `computeForecastBurndown`
+- [x] 4.6 Return the combined JSON response with stats, donut, heavy_hitters, and vertical_breakdown
+
+## Task 5: Backend — VCL Trend Endpoint
+
+- [x] 5.1 Add `GET /vcl/trend` endpoint to `backend/routes/compliance.js` with `requireAuth()` middleware
+- [x] 5.2 Query `compliance_snapshots` table for monthly compliance data points
+- [x] 5.3 Compute forecast line using linear regression on the last 3+ months of actual data
+- [x] 5.4 Return monthly data array with compliant_count, compliance_pct, forecast_pct, and target_pct
+
+## Task 6: Backend — Bulk Upload Endpoints
+
+- [x] 6.1 Add `POST /vcl/bulk-preview` endpoint with `requireAuth()` and `requireGroup('Admin', 'Standard_User')` middleware
+- [x] 6.2 Validate request body: require rows array, enforce 2000 row limit, require hostname field on each row
+- [x] 6.3 Use `mapColumnHeaders` to identify updatable fields; return 400 if no updatable fields found
+- [x] 6.4 Use `matchByHostname` to separate matched/unmatched rows
+- [x] 6.5 Validate each row's fields using `isValidDateString` and `validateRemediationPlan`; flag invalid rows
+- [x] 6.6 Use `computeBulkDiff` to identify changed rows and return the full diff preview response
+- [x] 6.7 Add `POST /vcl/bulk-commit` endpoint with `requireAuth()` and `requireGroup('Admin', 'Standard_User')` middleware
+- [x] 6.8 Accept validated changes array, execute all updates in a single PostgreSQL transaction
+- [x] 6.9 Log audit entry with action `compliance_bulk_update` including count of rows updated
+
+## Task 7: Backend — Snapshot Creation on Upload
+
+- [x] 7.1 Modify the existing `persistUpload` function in `compliance.js` to insert/update a `compliance_snapshots` row for the current month after each upload commit
+- [x] 7.2 Compute per-vertical compliance percentages at snapshot time from the current state of compliance_items
+
+## Task 8: Frontend — Device Detail Panel Fields
+
+- [x] 8.1 Add a "Resolution Date" section to `ComplianceDetailPanel.js` with an `<input type="date">` field
+- [x] 8.2 Add a "Remediation Plan" section with a `<textarea>` field and character counter showing current/max (2000)
+- [x] 8.3 Implement save logic that calls `PATCH /api/compliance/items/:hostname/metadata` on blur or explicit save button click
+- [x] 8.4 Display loading state during save and error message on failure
+- [x] 8.5 Fetch and display existing resolution_date and remediation_plan values when the panel opens (extend the existing GET /items/:hostname response)
+
+## Task 9: Frontend — Device Table Columns
+
+- [x] 9.1 Add "Resolution Date" column to the device table in `CompliancePage.js`, displaying the date or empty cell
+- [x] 9.2 Add "Remediation Plan" column to the device table, truncated to 80 characters with ellipsis using `truncateText` logic
+
+## Task 10: Frontend — VCL Report Page Shell
+
+- [x] 10.1 Create `frontend/src/components/pages/VCLReportPage.js` with the page layout structure (header, stats bar area, charts area, tables area)
+- [x] 10.2 Add navigation to VCLReportPage from CompliancePage (tab or button in the page header)
+- [x] 10.3 Implement data fetching on mount: call `/api/compliance/vcl/stats` and `/api/compliance/vcl/trend`
+- [x] 10.4 Add loading and error states consistent with existing page patterns
+
+## Task 11: Frontend — VCL Stats Bar
+
+- [x] 11.1 Implement `VCLStatsBar` component displaying 7 stat cards in a horizontal row: Total Devices, In-Scope, Compliant, Non-Compliant, Remediations Required, Current %, Target %
+- [x] 11.2 Style stat cards with the dark tactical theme (monospace font, teal accents, gradient backgrounds)
+- [x] 11.3 Format percentage values as whole numbers with percent sign
+
+## Task 12: Frontend — Compliance Overview Trend Chart
+
+- [x] 12.1 Implement `ComplianceOverviewChart` using Recharts ComposedChart with dual Y-axes (count left, percentage right)
+- [x] 12.2 Render compliant asset counts as green bars on the count axis
+- [x] 12.3 Render actual compliance percentage as a solid teal line on the percentage axis
+- [x] 12.4 Render forecasted compliance percentage as a dashed teal line
+- [x] 12.5 Render target compliance as a horizontal amber ReferenceLine
+- [x] 12.6 Handle edge case: when < 2 months of data, show available points without forecast line
+
+## Task 13: Frontend — Non-Compliant Assets Donut Chart
+
+- [x] 13.1 Implement `NonCompliantDonutChart` using Recharts PieChart with innerRadius/outerRadius for donut shape
+- [x] 13.2 Render "Blocked" segment in red (#EF4444) and "In-Progress" segment in amber (#F59E0B)
+- [x] 13.3 Display count and percentage labels for each segment
+- [x] 13.4 Handle edge case: single category renders full donut in that color
+
+## Task 14: Frontend — Heavy Hitters Table
+
+- [x] 14.1 Implement `HeavyHittersTable` component with columns: Vertical (with team), Non-Compliant count, Compliance Date, Notes
+- [x] 14.2 Display rows sorted by non-compliant count descending (pre-sorted from API)
+- [x] 14.3 Display empty cell when compliance date is null
+- [x] 14.4 Style table with dark theme (border colors, monospace text, teal header accents)
+
+## Task 15: Frontend — Vertical Breakdown Table
+
+- [x] 15.1 Implement `VerticalBreakdownTable` component with all specified columns: Vertical, Compliance %, Team, Non-Compliant, Actual Burndown months, Forecast Burndown months, Blockers, RAs, Notes
+- [x] 15.2 Render actual burndown columns with historical monthly counts
+- [x] 15.3 Render forecast burndown columns from current month through Q4 2026
+- [x] 15.4 Display zero values for verticals with no non-compliant devices
+
+## Task 16: Frontend — Bulk Upload Modal
+
+- [x] 16.1 Create `BulkUploadModal` component with file picker accepting .xlsx files
+- [x] 16.2 Parse uploaded xlsx file client-side using the `xlsx` library to extract rows and headers
+- [x] 16.3 Use column mapping logic to identify Hostname, Resolution Date, Remediation Plan, Notes columns
+- [x] 16.4 Validate fields client-side (date format, plan length) before sending to backend
+- [x] 16.5 Call `POST /api/compliance/vcl/bulk-preview` with parsed rows and display the diff preview
+- [x] 16.6 Render diff preview showing: matched count, unmatched count, changes count, invalid count
+- [x] 16.7 Highlight changed rows with field-level old/new value comparison
+- [x] 16.8 Display unmatched hostnames and invalid rows with error details
+- [x] 16.9 Implement confirm button that calls `POST /api/compliance/vcl/bulk-commit` and closes modal on success
+- [x] 16.10 Implement cancel button that discards data and closes modal without changes
+- [x] 16.11 Show error message if file has no Hostname column or no updatable fields
+
+## Task 17: Property-Based Tests
+
+- [x] 17.1 Create `backend/__tests__/vcl-compliance-reporting.property.test.js` with fast-check
+- [x] 17.2 Implement Property 2 test: truncateText returns original for short strings, truncated + ellipsis for long strings (min 100 iterations)
+- [x] 17.3 Implement Property 3 test: validateRemediationPlan accepts strings <= 2000 chars, rejects longer (min 100 iterations)
+- [x] 17.4 Implement Property 4 test: computeVCLStats produces correct arithmetic relationships (non_compliant + compliant = total, correct percentage) (min 100 iterations)
+- [x] 17.5 Implement Property 5 test: formatPct produces correct percentage string matching /^\d{1,3}%$/ (min 100 iterations)
+- [x] 17.6 Implement Property 6 test: categorizeNonCompliant partitions all items into exactly two groups summing to total (min 100 iterations)
+- [x] 17.7 Implement Property 7 test: rankHeavyHitters output is sorted in non-increasing order by non_compliant (min 100 iterations)
+- [x] 17.8 Implement Property 8 test: computeForecastBurndown monthly bucket sum equals count of items with non-null resolution_dates (min 100 iterations)
+- [x] 17.9 Implement Property 9 test: matchByHostname matched + unmatched = total input, matched hostnames all exist in set (min 100 iterations)
+- [x] 17.10 Implement Property 10 test: computeBulkDiff flags row as changed iff at least one field differs (min 100 iterations)
+- [x] 17.11 Implement Property 11 test: mapColumnHeaders correctly identifies known columns case-insensitively (min 100 iterations)
+- [x] 17.12 Implement Property 12 test: isValidDateString rejects invalid calendar dates and non-date strings (min 100 iterations)
+- [x] 17.13 Implement Property 13 test: bulk preview row counts (matched + unmatched = total) invariant holds (min 100 iterations)
+
+## Task 18: Unit and Integration Tests
+
+- [x] 18.1 Write unit tests for PATCH /items/:hostname/metadata endpoint (happy path, invalid date, plan too long, not found)
+- [x] 18.2 Write unit tests for GET /vcl/stats with no data (verify zero/empty response)
+- [x] 18.3 Write unit tests for bulk preview with all unmatched hostnames
+- [x] 18.4 Write unit tests for bulk preview with mixed valid/invalid rows
+- [x] 18.5 Write integration test for full bulk upload flow: preview → commit → verify DB state
+- [x] 18.6 Write unit test for trend endpoint with < 2 months of data (no forecast)
				`@@ -0,0 +1 @@`
				`{"specId": "a7c3e1d4-9f82-4b6a-a3d1-7e5f2c8b9a04", "workflowType": "requirements-first", "specType": "feature"}`