diff --git a/.kiro/hooks/migration-registration-check.kiro.hook b/.kiro/hooks/migration-registration-check.kiro.hook index 90a7a79..6f3afef 100644 --- a/.kiro/hooks/migration-registration-check.kiro.hook +++ b/.kiro/hooks/migration-registration-check.kiro.hook @@ -1,16 +1,14 @@ { "enabled": true, - "name": "Migration Registration Reminder", - "description": "When a new migration file is created in backend/migrations/, reminds the agent to add it to the POSTGRES_MIGRATIONS array in run-all.js so the pipeline will execute it during deploy.", - "version": "1", + "name": "Migration Registration Check", + "description": "After any write to backend/migrations/, verify the file is registered in POSTGRES_MIGRATIONS array in run-all.js. Blocks until confirmed.", + "version": "2", "when": { - "type": "fileCreated", - "patterns": [ - "backend/migrations/*.js" - ] + "type": "postToolUse", + "toolTypes": ["write"] }, "then": { "type": "askAgent", - "prompt": "A new migration file was created. If this migration uses the Postgres pool (require('../db')), add its filename to the POSTGRES_MIGRATIONS array in backend/migrations/run-all.js so the CI/CD pipeline will run it automatically during deploy. Skip this if the file is run-all.js itself." + "prompt": "STOP. If the tool just wrote or created a file matching backend/migrations/*.js (but NOT run-all.js itself), you MUST immediately:\n1. Read backend/migrations/run-all.js\n2. Check if the migration filename exists in the POSTGRES_MIGRATIONS array\n3. If NOT present, add it to the end of the array RIGHT NOW before doing anything else\n4. Do NOT proceed with other work until this is done\n\nThis is a hard requirement — migrations not in run-all.js will not run in CI/CD and will break production deploys. If the written file is not a migration file, ignore this message." } -} \ No newline at end of file +} diff --git a/.kiro/specs/archer-template-library/.config.kiro b/.kiro/specs/archer-template-library/.config.kiro new file mode 100644 index 0000000..cd313f8 --- /dev/null +++ b/.kiro/specs/archer-template-library/.config.kiro @@ -0,0 +1 @@ +{"specId": "69cc3734-7a9a-4d50-be5e-6e47a2806651", "workflowType": "requirements-first", "specType": "feature"} diff --git a/.kiro/specs/archer-template-library/design.md b/.kiro/specs/archer-template-library/design.md new file mode 100644 index 0000000..dfbf74e --- /dev/null +++ b/.kiro/specs/archer-template-library/design.md @@ -0,0 +1,518 @@ +# Design Document: Archer Template Library + +## Overview + +The Archer Template Library adds a template management system to the STEAM Security Dashboard, allowing editors to store, browse, and reuse pre-filled content for Archer Risk Acceptance forms. Templates are organized by a Vendor > Platform > Model hierarchy and contain the static and semi-static content sections that map directly to the external Archer eGRC application's form fields. + +The feature integrates into the existing Ivanti Todo Queue page where Archer workflow items are worked. When an editor selects an Archer queue item, they can pick a template, view its sections, and copy content to their clipboard for pasting into the external Archer application. + +### Design Decisions + +1. **Separate route module** (`routes/archerTemplates.js`) following the project's factory pattern — keeps template CRUD isolated from existing `archerTickets.js` which handles EXC number tracking. +2. **PostgreSQL table** with a composite unique index on `(LOWER(TRIM(vendor)), LOWER(TRIM(platform)), LOWER(TRIM(model)))` to enforce case-insensitive uniqueness at the database level. +3. **Single page component** (`ArcherTemplatePage.js`) for the Template Manager, plus a **reusable TemplateSelector component** embedded in the existing `IvantiTodoQueuePage.js` for the queue workflow integration. +4. **No separate search endpoint** — the list endpoint handles search/filter via query parameters, keeping the API surface small. +5. **Clipboard API** (`navigator.clipboard.writeText`) for copy operations — the dashboard already runs on HTTPS in production. + +--- + +## Architecture + +```mermaid +graph TD + subgraph Frontend + TM[ArcherTemplatePage.js
Template Manager] + TS[TemplateSelector.js
Queue Integration] + IQ[IvantiTodoQueuePage.js] + end + + subgraph Backend + R[routes/archerTemplates.js
Express Router] + A[helpers/auditLog.js] + DB[(PostgreSQL
archer_templates)] + end + + TM -->|fetch /api/archer-templates| R + TS -->|fetch /api/archer-templates| R + IQ -->|embeds| TS + R -->|pool.query| DB + R -->|logAudit| A + A -->|fire-and-forget INSERT| DB +``` + +### Request Flow + +1. Frontend component calls `/api/archer-templates/*` with session cookie +2. `requireAuth()` middleware validates session, attaches `req.user` +3. Write operations additionally pass through `requireGroup('Admin', 'Standard_User')` +4. Route handler validates input, executes PostgreSQL query via `pool` +5. On success of write operations, `logAudit()` is called fire-and-forget +6. Response returned to frontend + +--- + +## Components and Interfaces + +### Backend: `routes/archerTemplates.js` + +Factory function signature: + +```javascript +function createArcherTemplatesRouter() { + // Returns Express Router + // Imports pool from '../db', auth from '../middleware/auth', logAudit from '../helpers/auditLog' +} +``` + +#### API Endpoints + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| `GET` | `/api/archer-templates` | requireAuth | List/search/filter templates | +| `GET` | `/api/archer-templates/:id` | requireAuth | Get single template by ID | +| `POST` | `/api/archer-templates` | requireAuth + requireGroup | Create template | +| `PUT` | `/api/archer-templates/:id` | requireAuth + requireGroup | Update template | +| `DELETE` | `/api/archer-templates/:id` | requireAuth + requireGroup | Delete template | +| `POST` | `/api/archer-templates/:id/clone` | requireAuth + requireGroup | Clone template | +| `GET` | `/api/archer-templates/hierarchy/vendors` | requireAuth | Distinct vendors list | +| `GET` | `/api/archer-templates/hierarchy/platforms` | requireAuth | Distinct platforms for vendor | +| `GET` | `/api/archer-templates/hierarchy/models` | requireAuth | Distinct models for vendor+platform | + +#### `GET /api/archer-templates` + +**Query Parameters:** + +| Param | Type | Description | +|-------|------|-------------| +| `search` | string | Substring match across vendor, platform, model (case-insensitive). Ignored if empty/whitespace-only. | +| `vendor` | string | Exact match filter on vendor (case-insensitive) | +| `platform` | string | Exact match filter on platform (case-insensitive) | +| `model` | string | Exact match filter on model (case-insensitive) | + +**Response:** `200 OK` +```json +[ + { + "id": 1, + "vendor": "Harmonic", + "platform": "vCMTS", + "model": "3.29.1", + "environment_overview": "...", + "segmentation": "...", + "mitigating_controls": "...", + "additional_info": "...", + "charter_network_banner": "...", + "data_classification": "...", + "charter_network": "...", + "additional_access_list": "...", + "created_by": 3, + "created_at": "2025-01-15T10:30:00Z", + "updated_at": "2025-01-15T10:30:00Z" + } +] +``` + +**Filtering logic:** +- `search` applies as `ILIKE '%value%'` across vendor OR platform OR model +- Field-specific filters apply as `LOWER(TRIM(field)) = LOWER(TRIM(value))` +- When both search and filters are present, they combine with AND logic +- Results always sorted by `vendor ASC, platform ASC, model ASC` + +#### `POST /api/archer-templates` + +**Request Body:** +```json +{ + "vendor": "Harmonic", + "platform": "vCMTS", + "model": "3.29.1", + "environment_overview": "content...", + "segmentation": "content...", + "mitigating_controls": "", + "additional_info": "", + "charter_network_banner": "", + "data_classification": "", + "charter_network": "", + "additional_access_list": "" +} +``` + +**Validation:** +- `vendor`, `platform`, `model`: required, 1-100 chars after trim, non-empty after trim +- Section fields: optional, max 10,000 chars each, default to empty string +- Uniqueness: `LOWER(TRIM(vendor)) + LOWER(TRIM(platform)) + LOWER(TRIM(model))` must be unique + +**Response:** `201 Created` +```json +{ + "id": 1, + "vendor": "Harmonic", + "platform": "vCMTS", + "model": "3.29.1", + "environment_overview": "content...", + "...": "...", + "created_by": 3, + "created_at": "2025-01-15T10:30:00Z", + "updated_at": "2025-01-15T10:30:00Z" +} +``` + +**Error Responses:** +- `400` — missing or invalid fields (includes field names in error message) +- `409` — duplicate vendor/platform/model combination + +#### `PUT /api/archer-templates/:id` + +**Request Body:** Partial — only provided fields are updated. + +```json +{ + "vendor": "Harmonic", + "mitigating_controls": "updated content..." +} +``` + +**Validation:** Same rules as create for any provided field. If vendor/platform/model change, uniqueness is re-checked against other templates (excluding self). + +**Response:** `200 OK` — returns full updated template object. + +**Error Responses:** +- `400` — invalid field values +- `404` — template ID not found +- `409` — new vendor/platform/model combination conflicts with another template + +#### `DELETE /api/archer-templates/:id` + +**Response:** `200 OK` +```json +{ "message": "Template deleted successfully" } +``` + +**Error Responses:** +- `404` — template ID not found + +#### `POST /api/archer-templates/:id/clone` + +**Request Body:** +```json +{ + "vendor": "Harmonic", + "platform": "vCMTS", + "model": "3.30.0" +} +``` + +**Validation:** +- All three hierarchy fields required +- At least one must differ from source (enforced by uniqueness constraint) +- Same length/format validation as create + +**Response:** `201 Created` — returns full new template object with copied section content. + +**Error Responses:** +- `400` — missing/invalid hierarchy fields +- `404` — source template not found +- `409` — target combination already exists + +#### `GET /api/archer-templates/hierarchy/vendors` + +**Response:** `200 OK` +```json +["Adtran", "Harmonic", "Vecima"] +``` + +#### `GET /api/archer-templates/hierarchy/platforms?vendor=Harmonic` + +**Query Parameters:** `vendor` (required) + +**Response:** `200 OK` +```json +["RPD", "vCMTS"] +``` + +**Error:** `400` if `vendor` param missing. + +#### `GET /api/archer-templates/hierarchy/models?vendor=Harmonic&platform=vCMTS` + +**Query Parameters:** `vendor` (required), `platform` (required) + +**Response:** `200 OK` +```json +["3.29.1", "3.30.0"] +``` + +**Error:** `400` if either param missing. + +--- + +### Frontend: `ArcherTemplatePage.js` + +Full-page Template Manager component at `frontend/src/components/pages/ArcherTemplatePage.js`. + +**Component Structure:** + +``` +ArcherTemplatePage +├── Header (title + create button) +├── TemplateList (grouped by vendor > platform) +│ ├── VendorGroup (collapsible) +│ │ ├── PlatformSubgroup +│ │ │ └── TemplateRow (model, edit/clone/delete buttons) +│ │ └── ... +│ └── ... +├── TemplateFormModal (create/edit/clone) +│ ├── Hierarchy fields (vendor, platform, model) +│ └── Section textareas (8 sections with labels) +└── DeleteConfirmModal +``` + +**State Management:** Local `useState` — no global state needed. Template list fetched on mount and after mutations. + +**Role-based rendering:** Uses `useAuth().canWrite()` to conditionally show action buttons. Viewers see the list but no create/edit/delete/clone controls. + +--- + +### Frontend: `TemplateSelector.js` + +Reusable component embedded in `IvantiTodoQueuePage.js` when viewing an Archer workflow item. + +**Component Structure:** + +``` +TemplateSelector +├── SearchableDropdown +│ ├── Search input +│ └── Filtered template list +├── SectionPanel (shown after selection) +│ ├── StaticSections (Environment Overview, Segmentation, Mitigating Controls) +│ ├── SemiStaticSections (Additional Info, Charter Network Banner, etc.) +│ └── CopyAllButton +└── SectionBlock (repeated for each section) + ├── Label + ├── Content (or "No content stored" placeholder) + └── CopyButton (with confirmation state) +``` + +**Props:** +```javascript +// No props needed — fetches templates from API independently +// Appears as an expandable panel within the Archer queue item view +``` + +--- + +## Data Models + +### Database Table: `archer_templates` + +```sql +CREATE TABLE IF NOT EXISTS archer_templates ( + id SERIAL PRIMARY KEY, + vendor VARCHAR(100) NOT NULL, + platform VARCHAR(100) NOT NULL, + model VARCHAR(100) NOT NULL, + environment_overview TEXT NOT NULL DEFAULT '', + segmentation TEXT NOT NULL DEFAULT '', + mitigating_controls TEXT NOT NULL DEFAULT '', + additional_info TEXT NOT NULL DEFAULT '', + charter_network_banner TEXT NOT NULL DEFAULT '', + data_classification TEXT NOT NULL DEFAULT '', + charter_network TEXT NOT NULL DEFAULT '', + additional_access_list TEXT NOT NULL DEFAULT '', + created_by INTEGER REFERENCES users(id) ON DELETE SET NULL, + created_at TIMESTAMPTZ DEFAULT NOW(), + updated_at TIMESTAMPTZ DEFAULT NOW() +); + +-- Case-insensitive uniqueness on trimmed vendor/platform/model +CREATE UNIQUE INDEX IF NOT EXISTS idx_archer_templates_unique_combo + ON archer_templates (LOWER(TRIM(vendor)), LOWER(TRIM(platform)), LOWER(TRIM(model))); + +-- Index for list ordering and search +CREATE INDEX IF NOT EXISTS idx_archer_templates_vendor ON archer_templates(vendor); +CREATE INDEX IF NOT EXISTS idx_archer_templates_platform ON archer_templates(platform); +``` + +### Migration File: `backend/migrations/add_archer_templates_table.js` + +Follows the project's migration pattern — idempotent, uses `IF NOT EXISTS`. + +### Section Field Mapping + +| DB Column | Archer Form Label | Type | +|-----------|------------------|------| +| `environment_overview` | Environment Overview | Static | +| `segmentation` | Segmentation | Static | +| `mitigating_controls` | Mitigating Controls | Static | +| `additional_info` | Additional Info/Background | Semi-Static | +| `charter_network_banner` | Charter Network Banner | Semi-Static | +| `data_classification` | Data Classification | Semi-Static | +| `charter_network` | Charter Network | Semi-Static | +| `additional_access_list` | Additional Access List | Semi-Static | + +--- + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +### Property 1: Template data round-trip preservation + +*For any* valid template with vendor (1-100 chars), platform (1-100 chars), model (1-100 chars), and section content (0-10,000 chars each), creating the template via the API and then retrieving it by ID SHALL return the exact same field values (after trimming vendor/platform/model whitespace). + +**Validates: Requirements 1.1, 1.2, 1.5, 2.1** + +### Property 2: Uniqueness enforcement across all write operations + +*For any* two templates where `LOWER(TRIM(vendor))`, `LOWER(TRIM(platform))`, and `LOWER(TRIM(model))` are identical, the system SHALL reject the second operation (create, update, or clone) with a 409 status code, regardless of case or leading/trailing whitespace differences. + +**Validates: Requirements 1.3, 2.2, 2.8, 3.2, 3.5** + +### Property 3: Input validation rejects invalid hierarchy fields + +*For any* template create or update request where vendor, platform, or model is empty after trimming, consists only of whitespace, or exceeds 100 characters, the API SHALL reject the request with a 400 status code and an error message identifying which field(s) failed validation. + +**Validates: Requirements 1.6, 2.3** + +### Property 4: List and search results are sorted + +*For any* set of templates in the database and any combination of search/filter parameters, the API response SHALL return results ordered by vendor ascending, then platform ascending, then model ascending (lexicographic, case-insensitive). + +**Validates: Requirements 2.4, 6.4** + +### Property 5: Search and filter semantics with AND logic + +*For any* search query (substring match on vendor/platform/model) combined with any field-specific exact-match filters (vendor, platform, model), the API SHALL return only templates that satisfy ALL conditions simultaneously: the search substring appears in at least one hierarchy field AND each provided filter exactly matches its respective field (case-insensitive). + +**Validates: Requirements 2.5, 2.6, 6.1, 6.2, 6.3** + +### Property 6: Partial update preserves unspecified fields + +*For any* existing template and any subset of updatable fields provided in a PUT request, the API SHALL modify only the specified fields and leave all other fields unchanged, while setting `updated_at` to the current timestamp. + +**Validates: Requirements 2.7** + +### Property 7: Delete removes template permanently + +*For any* existing template, after a successful DELETE request, a subsequent GET request for that template's ID SHALL return 404. + +**Validates: Requirements 2.9** + +### Property 8: Write operations require editor or admin role + +*For any* user with viewer role (or no authenticated session), all write operations (POST create, PUT update, DELETE, POST clone) SHALL be rejected with 401 (unauthenticated) or 403 (insufficient permissions). + +**Validates: Requirements 2.10, 4.13** + +### Property 9: Non-existent template ID returns 404 + +*For any* template ID that does not exist in the database, GET, PUT, DELETE, and clone requests targeting that ID SHALL return a 404 status code. + +**Validates: Requirements 2.12, 3.4** + +### Property 10: Clone preserves all section content + +*For any* existing template, cloning it with new vendor/platform/model values SHALL produce a new template whose 8 section content fields are byte-for-byte identical to the source template, with a different ID, new `created_at` timestamp, and `created_by` set to the requesting user. + +**Validates: Requirements 3.1, 3.3** + +### Property 11: Hierarchy endpoints return distinct sorted values + +*For any* set of templates, the vendors endpoint SHALL return a deduplicated, alphabetically sorted array of vendor names; the platforms endpoint (given a vendor) SHALL return only distinct platforms from templates matching that vendor; and the models endpoint (given vendor+platform) SHALL return only distinct models from matching templates — all sorted alphabetically ascending. + +**Validates: Requirements 7.1, 7.2, 7.3** + +### Property 12: Copy All concatenation format + +*For any* template with a mix of populated and empty sections, the "Copy All" operation SHALL produce a string that concatenates only non-empty sections, each preceded by its human-readable section header, in the order: static sections first (Environment Overview, Segmentation, Mitigating Controls) then semi-static sections (Additional Info/Background, Charter Network Banner, Data Classification, Charter Network, Additional Access List). + +**Validates: Requirements 5.9** + +--- + +## Error Handling + +### Backend Error Strategy + +| Scenario | HTTP Status | Error Response Format | +|----------|-------------|----------------------| +| Missing required field | 400 | `{ "error": "Vendor is required" }` or `{ "error": "Missing fields: vendor, model" }` | +| Field too long | 400 | `{ "error": "Vendor must be 100 characters or fewer" }` | +| Section too long | 400 | `{ "error": "environment_overview must be 10,000 characters or fewer" }` | +| Duplicate combination | 409 | `{ "error": "A template with this vendor/platform/model combination already exists" }` | +| Template not found | 404 | `{ "error": "Template not found" }` | +| Not authenticated | 401 | `{ "error": "Authentication required" }` | +| Insufficient permissions | 403 | `{ "error": "Insufficient permissions", "required": [...], "current": "..." }` | +| Database error | 500 | `{ "error": "Internal server error" }` | + +### Frontend Error Handling + +- **Network errors:** Caught in try/catch around `fetch()`, displayed as a banner message within the component. +- **Validation errors (400):** Displayed inline next to the relevant form field or as a general form error. +- **Conflict errors (409):** Displayed as a warning banner at the top of the form indicating the duplicate. +- **Auth errors (401/403):** Handled by AuthContext — 401 triggers redirect to login, 403 shows permission denied message. +- **Clipboard failures:** If `navigator.clipboard.writeText` rejects (e.g., permissions denied), display a fallback message suggesting manual copy. + +### Audit Log Resilience + +Per requirement 8.5, `logAudit()` is called fire-and-forget. If the audit insert fails, the error is logged to `console.error` but the main operation's response is unaffected. This matches the existing pattern in `helpers/auditLog.js`. + +--- + +## Testing Strategy + +### Property-Based Tests (fast-check) + +The project uses `fast-check` for property-based testing (visible in existing `__tests__/*.property.test.js` files). Each correctness property above maps to one property-based test with a minimum of 100 iterations. + +**Test file:** `backend/__tests__/archer-template-library.property.test.js` + +**Properties to implement:** + +| Property | Test Description | +|----------|-----------------| +| 1 | Template data round-trip — create + GET preserves all fields | +| 2 | Uniqueness enforcement — duplicate combinations rejected across create/update/clone | +| 3 | Input validation — invalid hierarchy fields rejected with 400 | +| 4 | Sorted results — list/search always returns sorted by vendor/platform/model | +| 5 | Search + filter AND logic — combined criteria narrow results correctly | +| 6 | Partial update semantics — unspecified fields preserved | +| 7 | Delete permanence — deleted templates return 404 | +| 8 | Access control — viewer role cannot perform write operations | +| 9 | Non-existent ID — 404 for all operations on missing templates | +| 10 | Clone content preservation — cloned sections match source | +| 11 | Hierarchy distinct values — deduplicated and sorted | +| 12 | Copy All format — concatenation includes only non-empty sections in order | + +**Tag format:** Each test tagged with `// Feature: archer-template-library, Property N: ` + +**Configuration:** 100 iterations minimum per property. + +### Unit / Example-Based Tests + +**Test file:** `backend/__tests__/archer-template-library.test.js` + +- Template creation with all sections populated +- Template creation with no sections (defaults to empty strings) +- Timestamp metadata correctness (created_at, updated_at, created_by) +- Clone metadata (new created_at, new created_by) +- Hierarchy endpoint without required params returns 400 +- Empty search results return 200 with empty array +- Whitespace-only search param is ignored +- Authentication required for read endpoints (401 without session) + +### Integration Tests + +**Test file:** `backend/__tests__/archer-template-library.integration.test.js` + +- Audit log entries created for create/update/delete/clone +- Audit log failure does not block template operation +- Failed operations do not produce audit entries +- Full workflow: create → list → update → clone → delete + +### Frontend Tests + +- TemplateSelector copy-to-clipboard behavior +- TemplateFormModal validation prevents submission with empty required fields +- Viewer role sees no action buttons +- Template list grouped by vendor/platform renders correctly diff --git a/.kiro/specs/archer-template-library/requirements.md b/.kiro/specs/archer-template-library/requirements.md new file mode 100644 index 0000000..08fbadb --- /dev/null +++ b/.kiro/specs/archer-template-library/requirements.md @@ -0,0 +1,143 @@ +# Requirements Document + +## Introduction + +The Archer Template Library adds a template management system to the Ivanti Queue's Archer Risk Acceptance workflow. Templates store the static and semi-static content sections of Archer Risk Acceptance forms, keyed by Vendor > Platform > Model hierarchy. When working an Archer queue item, users select a template and copy pre-filled section content into the external Archer application (https://egrcprod.corp.chartercom.com), eliminating repetitive manual entry of identical content across submissions for the same vendor/platform/model combination. + +## Glossary + +- **Dashboard**: The STEAM Security Dashboard application +- **Template_Library**: The collection of stored Archer Risk Acceptance templates, organized by Vendor > Platform > Model hierarchy +- **Template**: A single template record containing static and semi-static section content for a specific Vendor/Platform/Model combination +- **Template_Manager**: The UI section where users create, edit, clone, and delete templates +- **Template_Selector**: The UI component within the Archer queue workflow that allows users to pick a template and copy section content +- **Template_API**: The backend REST endpoints for CRUD operations on templates +- **Section**: A named content block within a template corresponding to an Archer form section (e.g., Environment Overview, Segmentation, Mitigating Controls) +- **Static_Section**: A template section whose content is identical across all submissions for a given Vendor/Platform/Model (Environment Overview, Segmentation, Mitigating Controls) +- **Semi_Static_Section**: A template section with default content that may need per-request editing (Additional Info/Background, Charter Network Banner, Data Classification, Charter Network, Additional Access List) +- **Vendor**: The equipment manufacturer (e.g., Harmonic, Vecima, Adtran) +- **Platform**: The product category under a vendor (e.g., vCMTS, RPD, OLT) +- **Model**: The specific hardware or software version under a platform (e.g., 3.29.1, ERM3-2-2, 9504N) +- **Archer_Form**: The external Archer Risk Acceptance form in the eGRC platform with sections for General Information, Issues, Justification, Remediation Plans, Related Items, Mitigating Controls, and Attachments + +## Requirements + +### Requirement 1: Template Data Model + +**User Story:** As an editor, I want templates to store content organized by Vendor/Platform/Model and form section, so that I can maintain reusable content for each device type I submit Archer exceptions for. + +#### Acceptance Criteria + +1. THE Template_API SHALL store each Template with a vendor field (1 to 100 characters), a platform field (1 to 100 characters), and a model field (1 to 100 characters) +2. THE Template_API SHALL store each Template with named Section content blocks for: environment_overview, segmentation, mitigating_controls, additional_info, charter_network_banner, data_classification, charter_network, and additional_access_list, each with a maximum length of 10,000 characters +3. THE Template_API SHALL enforce uniqueness on the combination of vendor, platform, and model after trimming leading and trailing whitespace (case-insensitive) +4. THE Template_API SHALL store a created_at timestamp, updated_at timestamp, and created_by user reference for each Template +5. WHEN a Template is created without content for any section, THE Template_API SHALL store empty strings for those sections +6. IF a Template creation or update request provides a vendor, platform, or model value that is empty or exceeds 100 characters, THEN THE Template_API SHALL reject the request with a 400 status code and an error message indicating which field failed validation + +### Requirement 2: Template CRUD API + +**User Story:** As an editor, I want to create, read, update, and delete templates through the API, so that I can manage my template library programmatically and through the UI. + +#### Acceptance Criteria + +1. WHEN a valid Template creation request is received with vendor, platform, model, and at least one section content field, THE Template_API SHALL create the Template and return the created record including its generated ID with a 201 status code +2. WHEN a Template creation request contains a vendor/platform/model combination that already exists (case-insensitive after trimming), THE Template_API SHALL return a 409 status code with an error message identifying the conflicting combination +3. WHEN a Template creation request is missing vendor, platform, or model, THE Template_API SHALL return a 400 status code with an error message identifying the missing fields +4. WHEN a GET request is made to the templates list endpoint, THE Template_API SHALL return all templates ordered by vendor, platform, and model ascending +5. WHEN a GET request includes a vendor query parameter, THE Template_API SHALL filter results to templates matching that vendor (case-insensitive) +6. WHEN a GET request includes a platform query parameter, THE Template_API SHALL filter results to templates matching that platform (case-insensitive) +7. WHEN a valid Template update request is received, THE Template_API SHALL update only the specified fields and set the updated_at timestamp to the current time +8. WHEN a Template update request changes vendor, platform, or model to a combination that already exists in another Template, THE Template_API SHALL return a 409 status code +9. WHEN a Template delete request is received for an existing Template, THE Template_API SHALL remove the Template record and return a 200 status code +10. THE Template_API SHALL require an authenticated session with editor or admin role for all write operations (create, update, delete, clone) +11. THE Template_API SHALL require an authenticated session for all read operations +12. IF a Template ID referenced in a GET, PUT, or DELETE request does not exist, THEN THE Template_API SHALL return a 404 status code with an error message indicating the Template was not found + +### Requirement 3: Template Cloning + +**User Story:** As an editor, I want to clone an existing template to create a new one for a similar device, so that I can reuse shared content between models on the same platform without retyping it. + +#### Acceptance Criteria + +1. WHEN a clone request is received for an existing Template, THE Template_API SHALL create a new Template that copies all section content from the source Template and return the newly created Template including its generated ID +2. WHEN cloning a Template, THE Template_API SHALL require the caller to supply vendor, platform, and model values where at least one value differs from the source Template, such that the resulting combination is unique across all Templates in the system +3. WHEN cloning a Template, THE Template_API SHALL set created_at to the current time and created_by to the requesting user +4. IF the source Template ID does not exist, THEN THE Template_API SHALL return a 404 status code with an error message indicating the source Template was not found +5. IF the supplied vendor, platform, and model combination already exists in another Template, THEN THE Template_API SHALL reject the clone request with a 409 status code and an error message indicating the combination is not unique + +### Requirement 4: Template Manager UI + +**User Story:** As an editor, I want a dedicated Template Manager section in the dashboard, so that I can browse, create, edit, and delete templates with a clear view of what is available. + +#### Acceptance Criteria + +1. THE Template_Manager SHALL display a list of all templates grouped by vendor, then by platform +2. THE Template_Manager SHALL display the vendor, platform, and model for each template in the list +3. WHEN the user clicks a create button, THE Template_Manager SHALL display a form with fields for vendor (max 100 characters), platform (max 100 characters), model (max 100 characters), and all eight section content areas defined in the Template data model +4. WHEN the user clicks an edit button on a template, THE Template_Manager SHALL display the template content pre-populated in an editable form with the same fields as the create form +5. WHEN the user clicks a delete button on a template, THE Template_Manager SHALL display a confirmation dialog identifying the template by vendor, platform, and model before deleting +6. WHEN the user confirms the delete action in the confirmation dialog, THE Template_Manager SHALL delete the template via the API and remove it from the displayed list +7. IF the user cancels the delete confirmation dialog, THEN THE Template_Manager SHALL dismiss the dialog and leave the template unchanged +8. THE Template_Manager SHALL display section content fields as multi-line text areas with labels matching the Archer form section names +9. WHEN the user submits the create or edit form with vendor, platform, or model fields empty, THE Template_Manager SHALL prevent submission and indicate which required fields are missing +10. WHEN the user submits the create or edit form with a duplicate vendor/platform/model combination, THE Template_Manager SHALL display the conflict error returned by the API +11. WHEN a create, edit, or delete operation succeeds, THE Template_Manager SHALL refresh the template list to reflect the change +12. THE Template_Manager SHALL provide a clone button for each template that opens the create form pre-filled with the source template section content and empty vendor/platform/model fields +13. WHILE the user has viewer role only, THE Template_Manager SHALL hide create, edit, delete, and clone buttons + +### Requirement 5: Template Selection in Archer Queue Workflow + +**User Story:** As an editor, I want to select a template when working an Archer queue item, so that I can quickly access pre-filled content to paste into the Archer application. + +#### Acceptance Criteria + +1. WHEN the user is viewing an Archer workflow queue item, THE Template_Selector SHALL display a searchable dropdown that lists all available templates by their vendor/platform/model label +2. WHEN the user types in the template picker search field, THE Template_Selector SHALL filter the template list to show only templates whose vendor, platform, or model contains the typed text as a case-insensitive substring +3. WHEN the user selects a template from the dropdown, THE Template_Selector SHALL display all populated sections of the selected template in a panel, with Static_Sections (Environment Overview, Segmentation, Mitigating Controls) displayed first, followed by Semi_Static_Sections +4. THE Template_Selector SHALL display each section with a human-readable label matching the Archer form section name (e.g., "Environment Overview" not "environment_overview") +5. THE Template_Selector SHALL display a copy-to-clipboard button adjacent to each section content block +6. WHEN the user clicks a copy button, THE Template_Selector SHALL copy that section's full text content to the system clipboard using the Clipboard API +7. WHEN a section is copied successfully, THE Template_Selector SHALL display a brief visual confirmation (checkmark icon or "Copied!" text) for 2 seconds before reverting to the copy button state +8. WHEN a template has an empty section, THE Template_Selector SHALL display that section with a muted italic placeholder text "No content stored" and disable the copy button for that section +9. THE Template_Selector SHALL provide a "Copy All" button that concatenates all non-empty sections with section headers and copies the combined text to the clipboard + +### Requirement 6: Template Search and Filtering API + +**User Story:** As an editor, I want to search templates by vendor, platform, or model, so that I can quickly find the right template when working a queue item. + +#### Acceptance Criteria + +1. WHEN a search query parameter of at least 1 non-whitespace character is provided, THE Template_API SHALL return templates where the vendor, platform, or model contains the search string as a substring (case-insensitive partial match) +2. WHEN vendor, platform, or model filter parameters are provided, THE Template_API SHALL return only templates whose corresponding field exactly matches the filter value (case-insensitive exact match), supporting any combination of these filters simultaneously +3. WHEN both a search query parameter and field-specific filter parameters are provided, THE Template_API SHALL return only templates that satisfy both the search substring match AND all field-specific exact match filters (AND logic) +4. THE Template_API SHALL return search and filter results ordered by vendor, platform, and model ascending +5. WHEN no templates match the search or filter criteria, THE Template_API SHALL return an empty array with a 200 status code +6. IF a search query parameter is provided but contains only whitespace or is empty, THEN THE Template_API SHALL ignore the search parameter and return results as if no search was specified + +### Requirement 7: Distinct Values API for Hierarchy Navigation + +**User Story:** As an editor, I want to browse the template hierarchy by vendor and platform, so that I can navigate to templates without knowing the exact model identifier. + +#### Acceptance Criteria + +1. WHEN a request is made to the vendors endpoint, THE Template_API SHALL return an array of distinct vendor names from all templates sorted alphabetically ascending +2. WHEN a request is made to the platforms endpoint with a vendor parameter, THE Template_API SHALL return an array of distinct platform names for templates matching that vendor (case-insensitive) sorted alphabetically ascending +3. WHEN a request is made to the models endpoint with vendor and platform parameters, THE Template_API SHALL return an array of distinct model names for templates matching that vendor and platform combination (case-insensitive) sorted alphabetically ascending +4. IF the platforms endpoint is called without a vendor parameter, THEN THE Template_API SHALL return a 400 status code with an error message indicating vendor is required +5. IF the models endpoint is called without both vendor and platform parameters, THEN THE Template_API SHALL return a 400 status code with an error message indicating which parameters are missing +6. WHEN no templates match the provided parameters, THE Template_API SHALL return an empty array with a 200 status code +7. THE Template_API SHALL require an authenticated session for all hierarchy endpoints + +### Requirement 8: Audit Logging for Template Operations + +**User Story:** As an admin, I want all template create, update, delete, and clone operations logged, so that I can track who modified template content and when. + +#### Acceptance Criteria + +1. WHEN a Template is created successfully, THE Template_API SHALL log an audit entry with action "template_created", the template ID, the requesting user's ID and username, and the requester's IP address +2. WHEN a Template is updated successfully, THE Template_API SHALL log an audit entry with action "template_updated", the template ID, the requesting user's ID and username, the requester's IP address, and the list of field names that changed +3. WHEN a Template is deleted successfully, THE Template_API SHALL log an audit entry with action "template_deleted", the template ID, the requesting user's ID and username, the requester's IP address, and the vendor/platform/model of the deleted template +4. WHEN a Template is cloned successfully, THE Template_API SHALL log an audit entry with action "template_cloned", both the source and new template IDs, the requesting user's ID and username, and the requester's IP address +5. IF an audit log entry fails to persist, THEN THE Template_API SHALL log the failure to the application error log and SHALL NOT block or fail the original template operation +6. IF a template create, update, delete, or clone operation fails, THEN THE Template_API SHALL NOT log an audit entry for that operation diff --git a/.kiro/specs/archer-template-library/tasks.md b/.kiro/specs/archer-template-library/tasks.md new file mode 100644 index 0000000..927c6e5 --- /dev/null +++ b/.kiro/specs/archer-template-library/tasks.md @@ -0,0 +1,242 @@ +# Implementation Plan: Archer Template Library + +## Overview + +This plan implements the Archer Template Library feature — a template management system for storing and reusing Archer Risk Acceptance form content organized by Vendor > Platform > Model hierarchy. The implementation progresses from database schema, through backend CRUD/search API, to frontend Template Manager page and TemplateSelector integration in the Ivanti Todo Queue workflow. + +The backend uses Node.js/Express 5 with PostgreSQL (pool from `backend/db.js`), following the existing factory-pattern route module convention. The frontend uses React 19 (plain JavaScript) with the project's dark theme aesthetic. Property-based tests use `fast-check` in `backend/__tests__/`. + +## Tasks + +- [x] 1. Database migration and schema setup + - [x] 1.1 Create the `archer_templates` table migration + - Create `backend/migrations/add_archer_templates_table.js` using the idempotent migration pattern (IF NOT EXISTS) + - Define the `archer_templates` table with columns: `id SERIAL PRIMARY KEY`, `vendor VARCHAR(100) NOT NULL`, `platform VARCHAR(100) NOT NULL`, `model VARCHAR(100) NOT NULL`, 8 TEXT section columns (all `NOT NULL DEFAULT ''`), `created_by INTEGER REFERENCES users(id) ON DELETE SET NULL`, `created_at TIMESTAMPTZ DEFAULT NOW()`, `updated_at TIMESTAMPTZ DEFAULT NOW()` + - Create unique index `idx_archer_templates_unique_combo` on `(LOWER(TRIM(vendor)), LOWER(TRIM(platform)), LOWER(TRIM(model)))` + - Create index `idx_archer_templates_vendor` on `(vendor)` and `idx_archer_templates_platform` on `(platform)` for list query performance + - Register the migration in `backend/migrations/run-all.js` by appending `'add_archer_templates_table.js'` to the `POSTGRES_MIGRATIONS` array + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_ + +- [x] 2. Backend route module — core CRUD + - [x] 2.1 Create `backend/routes/archerTemplates.js` with factory function and create endpoint + - Create the file following the factory pattern: `function createArcherTemplatesRouter()` returning an Express Router + - Import `pool` from `'../db'`, `{ requireAuth, requireGroup }` from `'../middleware/auth'`, and `logAudit` from `'../helpers/auditLog'` + - Implement `POST /` — validate vendor/platform/model (required, 1-100 chars after trim, non-empty after trim), validate section fields (max 10,000 chars each, default to empty string), INSERT with RETURNING *, handle unique constraint violation (23505) as 409, call `logAudit` fire-and-forget on success with action `template_created` + - Implement `GET /` — query all templates with optional search (ILIKE across vendor/platform/model), optional exact-match filters on vendor/platform/model (case-insensitive via LOWER/TRIM), combine with AND logic, ORDER BY vendor, platform, model ASC + - Implement `GET /:id` — fetch single template by ID, return 404 if not found + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.10, 2.11, 2.12, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 8.1_ + + - [x] 2.2 Implement update, delete, and clone endpoints + - Implement `PUT /:id` — validate provided fields (same rules as create), build dynamic UPDATE SET clause for only provided fields, always set `updated_at = NOW()`, check uniqueness if vendor/platform/model changed (excluding self), return 404/409/400 as appropriate, call `logAudit` with action `template_updated` and changed field names + - Implement `DELETE /:id` — verify template exists (404 if not), DELETE, call `logAudit` with action `template_deleted` and vendor/platform/model of the deleted template, return 200 + - Implement `POST /:id/clone` — verify source exists (404 if not), validate new vendor/platform/model (required, same rules as create), INSERT copying all 8 section fields from source with new hierarchy values and `created_by = req.user.id`, handle uniqueness violation as 409, call `logAudit` with action `template_cloned` and both source/new IDs + - _Requirements: 2.7, 2.8, 2.9, 2.10, 2.12, 3.1, 3.2, 3.3, 3.4, 3.5, 8.2, 8.3, 8.4, 8.5, 8.6_ + + - [x] 2.3 Implement hierarchy endpoints (vendors, platforms, models) + - Implement `GET /hierarchy/vendors` — SELECT DISTINCT vendor ORDER BY vendor ASC + - Implement `GET /hierarchy/platforms` — require `vendor` query param (400 if missing), SELECT DISTINCT platform WHERE LOWER(TRIM(vendor)) matches, ORDER BY platform ASC + - Implement `GET /hierarchy/models` — require `vendor` and `platform` query params (400 if either missing), SELECT DISTINCT model WHERE both match, ORDER BY model ASC + - All hierarchy endpoints require `requireAuth()` middleware + - _Requirements: 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7_ + + - [x] 2.4 Wire the router into `backend/server.js` + - Add `const createArcherTemplatesRouter = require('./routes/archerTemplates');` to the imports section + - Mount with `app.use('/api/archer-templates', createArcherTemplatesRouter());` after the existing archer-tickets route + - _Requirements: 2.10, 2.11_ + +- [x] 3. Checkpoint - Backend API verification + - Ensure all tests pass, ask the user if questions arise. + +- [ ] 4. Backend property-based tests + - [ ]* 4.1 Write property test for template data round-trip preservation + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 1: Template data round-trip preservation** + - Generate valid templates with random vendor (1-100 chars), platform (1-100 chars), model (1-100 chars), and section content (0-10,000 chars each); create via POST then GET by ID; assert all fields match (vendor/platform/model trimmed) + - Minimum 100 iterations + - **Validates: Requirements 1.1, 1.2, 1.5, 2.1** + + - [ ]* 4.2 Write property test for uniqueness enforcement + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 2: Uniqueness enforcement across all write operations** + - Generate template pairs with matching LOWER(TRIM(vendor/platform/model)) but varying case/whitespace; assert second create returns 409, update to matching combo returns 409, clone to matching combo returns 409 + - Minimum 100 iterations + - **Validates: Requirements 1.3, 2.2, 2.8, 3.2, 3.5** + + - [ ]* 4.3 Write property test for input validation + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 3: Input validation rejects invalid hierarchy fields** + - Generate invalid inputs: empty strings, whitespace-only, strings > 100 chars for vendor/platform/model; assert POST and PUT return 400 with error identifying which field(s) failed + - Minimum 100 iterations + - **Validates: Requirements 1.6, 2.3** + + - [ ]* 4.4 Write property test for sorted results + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 4: List and search results are sorted** + - Generate multiple templates with random hierarchy values, insert all, then GET with various search/filter combos; assert response is sorted by vendor ASC, platform ASC, model ASC (case-insensitive) + - Minimum 100 iterations + - **Validates: Requirements 2.4, 6.4** + + - [ ]* 4.5 Write property test for search and filter AND logic + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 5: Search and filter semantics with AND logic** + - Generate template sets with overlapping vendor/platform/model values; apply combined search substring + field exact-match filters; assert every result satisfies ALL conditions and no matching template is missing + - Minimum 100 iterations + - **Validates: Requirements 2.5, 2.6, 6.1, 6.2, 6.3** + + - [ ]* 4.6 Write property test for partial update preservation + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 6: Partial update preserves unspecified fields** + - Generate a template, then PUT with a random subset of fields; assert unmodified fields remain unchanged and `updated_at` is updated + - Minimum 100 iterations + - **Validates: Requirements 2.7** + + - [ ]* 4.7 Write property test for delete permanence + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 7: Delete removes template permanently** + - Generate a template, DELETE it, then GET by ID; assert 404 response + - Minimum 100 iterations + - **Validates: Requirements 2.9** + + - [ ]* 4.8 Write property test for access control + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 8: Write operations require editor or admin role** + - Generate write operation requests (POST/PUT/DELETE/clone) with viewer-role or unauthenticated sessions; assert all are rejected with 401 or 403 + - Minimum 100 iterations + - **Validates: Requirements 2.10, 4.13** + + - [ ]* 4.9 Write property test for non-existent ID handling + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 9: Non-existent template ID returns 404** + - Generate random IDs that don't exist; assert GET, PUT, DELETE, and clone all return 404 + - Minimum 100 iterations + - **Validates: Requirements 2.12, 3.4** + + - [ ]* 4.10 Write property test for clone content preservation + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 10: Clone preserves all section content** + - Generate a template with random section content, clone with new hierarchy values; assert all 8 section fields are byte-for-byte identical, ID differs, `created_at` is new, `created_by` matches requesting user + - Minimum 100 iterations + - **Validates: Requirements 3.1, 3.3** + + - [ ]* 4.11 Write property test for hierarchy distinct values + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 11: Hierarchy endpoints return distinct sorted values** + - Generate templates with overlapping vendors/platforms/models; assert vendors endpoint returns deduplicated sorted array, platforms for a vendor returns only that vendor's platforms sorted, models for vendor+platform returns only matching models sorted + - Minimum 100 iterations + - **Validates: Requirements 7.1, 7.2, 7.3** + + - [ ]* 4.12 Write property test for Copy All concatenation format + - File: `backend/__tests__/archer-template-library.property.test.js` + - **Property 12: Copy All concatenation format** + - Generate templates with random mix of populated and empty sections; construct the expected "Copy All" output (only non-empty sections, with headers, static first then semi-static); assert the concatenation matches expected format and order + - Minimum 100 iterations + - **Validates: Requirements 5.9** + +- [x] 5. Checkpoint - Backend tests pass + - Ensure all tests pass, ask the user if questions arise. + +- [x] 6. Frontend Template Manager page + - [x] 6.1 Create `frontend/src/components/pages/ArcherTemplatePage.js` with list view + - Create the page component following the project's dark-theme inline-styles pattern + - Implement template list fetch on mount from `GET /api/archer-templates` + - Display templates grouped by vendor, then platform, with collapsible vendor groups + - Show vendor, platform, and model for each template row + - Conditionally render create/edit/clone/delete buttons using `useAuth()` role check (hide for viewer role) + - _Requirements: 4.1, 4.2, 4.13_ + + - [x] 6.2 Implement TemplateFormModal (create/edit/clone) + - Create a modal component (inline in ArcherTemplatePage.js or as a separate subcomponent) + - Include fields for vendor (max 100 chars), platform (max 100 chars), model (max 100 chars), and all 8 section content textareas with human-readable labels + - For create mode: all fields empty; for edit mode: pre-populate from template; for clone mode: pre-populate sections from source, leave hierarchy fields empty + - Client-side validation: prevent submission with empty vendor/platform/model, show which required fields are missing + - On submit: POST for create/clone, PUT for edit; handle 409 conflict errors by displaying the conflict message from API + - On success: close modal and refresh template list + - _Requirements: 4.3, 4.4, 4.8, 4.9, 4.10, 4.11, 4.12_ + + - [x] 6.3 Implement DeleteConfirmModal and delete flow + - Create a confirmation dialog that identifies the template by vendor/platform/model + - On confirm: call DELETE API, remove template from displayed list, close dialog + - On cancel: dismiss dialog, leave template unchanged + - _Requirements: 4.5, 4.6, 4.7_ + + - [x] 6.4 Wire ArcherTemplatePage into navigation + - Add a navigation entry in `NavDrawer.js` for the Template Manager page + - Add the page routing in `App.js` (following the existing page-switching pattern) + - _Requirements: 4.1_ + +- [x] 7. Frontend TemplateSelector component + - [x] 7.1 Create `frontend/src/components/TemplateSelector.js` with searchable dropdown + - Create the reusable component that fetches templates from `GET /api/archer-templates` + - Implement search input that filters the template list (case-insensitive substring match on vendor/platform/model — client-side filter for responsiveness) + - Display templates in dropdown as `Vendor / Platform / Model` label + - _Requirements: 5.1, 5.2_ + + - [x] 7.2 Implement section display panel and copy functionality + - When a template is selected, display all populated sections in a panel + - Order: static sections first (Environment Overview, Segmentation, Mitigating Controls), then semi-static sections (Additional Info/Background, Charter Network Banner, Data Classification, Charter Network, Additional Access List) + - Display each section with human-readable label (not DB column name) + - Show copy-to-clipboard button adjacent to each section; disable for empty sections (show muted "No content stored" placeholder) + - On copy: use `navigator.clipboard.writeText()`, show checkmark/"Copied!" for 2 seconds then revert + - Implement "Copy All" button: concatenate all non-empty sections with section headers, copy combined text + - _Requirements: 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9_ + + - [x] 7.3 Integrate TemplateSelector into IvantiTodoQueuePage + - Embed the TemplateSelector component in `IvantiTodoQueuePage.js` when viewing an Archer workflow queue item + - Display as an expandable panel within the Archer queue item view + - _Requirements: 5.1_ + +- [x] 8. Checkpoint - Frontend build and integration verification + - Run `cd frontend && npm run build` to verify no ESLint or build errors + - Ensure all tests pass, ask the user if questions arise. + +- [ ] 9. Backend unit and integration tests + - [ ]* 9.1 Write unit tests for template CRUD operations + - File: `backend/__tests__/archer-template-library.test.js` + - Test template creation with all sections populated returns 201 with full record + - Test template creation with no sections defaults to empty strings + - Test timestamp metadata correctness (created_at, updated_at, created_by set properly) + - Test clone metadata (new created_at, new created_by, different ID) + - Test hierarchy endpoint without required params returns 400 + - Test empty search results return 200 with empty array + - Test whitespace-only search param is ignored (returns all templates) + - Test authentication required for read endpoints (401 without session) + - _Requirements: 1.1, 1.2, 1.4, 1.5, 2.1, 2.3, 2.4, 2.11, 2.12, 3.1, 3.3, 6.5, 6.6, 7.4, 7.5, 7.6_ + + - [ ]* 9.2 Write integration tests for audit logging and full workflow + - File: `backend/__tests__/archer-template-library.integration.test.js` + - Test audit log entries created for create/update/delete/clone operations + - Test audit log failure does not block template operation (mock logAudit to throw) + - Test failed operations (400, 409, 404) do not produce audit entries + - Test full workflow: create → list → update → clone → delete + - _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6_ + +- [x] 10. Final checkpoint - Full test suite passes + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional test sub-tasks and can be skipped for a faster MVP. +- Each task references specific requirements for traceability. +- Property tests use `fast-check` with a minimum of 100 iterations and are tagged per the design's Testing Strategy. +- The migration must be registered in `run-all.js` to be picked up by CI/CD deploys. +- The route module follows the existing `createArcherTicketsRouter()` pattern in `routes/archerTickets.js`. +- Frontend styling follows the dark-theme tactical intelligence aesthetic defined in `DESIGN_SYSTEM.md`. +- Checkpoints (tasks 3, 5, 8, 10) ensure incremental validation at each layer. + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1"] }, + { "id": 1, "tasks": ["2.1"] }, + { "id": 2, "tasks": ["2.2", "2.3"] }, + { "id": 3, "tasks": ["2.4"] }, + { "id": 4, "tasks": ["4.1", "4.2", "4.3", "4.4", "4.5", "4.6", "4.7", "4.8", "4.9", "4.10", "4.11", "4.12"] }, + { "id": 5, "tasks": ["6.1", "7.1"] }, + { "id": 6, "tasks": ["6.2", "6.3", "7.2"] }, + { "id": 7, "tasks": ["6.4", "7.3"] }, + { "id": 8, "tasks": ["9.1", "9.2"] } + ] +} +``` diff --git a/.kiro/specs/ccp-metrics-view-restructure/.config.kiro b/.kiro/specs/ccp-metrics-view-restructure/.config.kiro new file mode 100644 index 0000000..5da88cd --- /dev/null +++ b/.kiro/specs/ccp-metrics-view-restructure/.config.kiro @@ -0,0 +1 @@ +{"specId": "54d073e2-9532-42f1-9315-f2129a2158af", "workflowType": "fast-task", "specType": "feature"} diff --git a/.kiro/specs/ccp-metrics-view-restructure/design.md b/.kiro/specs/ccp-metrics-view-restructure/design.md new file mode 100644 index 0000000..aa2f8a3 --- /dev/null +++ b/.kiro/specs/ccp-metrics-view-restructure/design.md @@ -0,0 +1,413 @@ +# Design Document + +## Overview + +This design restructures the CCP Metrics page from a vertical-first drill-down model to a metric-first model. The overview table changes from one row per vertical to one row per metric (aggregated across all verticals). Two new backend endpoints support the metric-centric view. The "By Vertical" table in the AggregatedBurndownChart is removed. Existing vertical-first endpoints are preserved for backward compatibility. + +## Architecture + +The restructure touches two layers: + +1. **Backend** — Two new Express route handlers added to `backend/routes/vclMultiVertical.js`, querying `vcl_multi_vertical_summary` with aggregation across verticals. +2. **Frontend** — Replace `VerticalTable` and `VerticalDetailView` components with `MetricTable` and `MetricDetailView` in `frontend/src/components/pages/CCPMetricsPage.js`. Remove the "By Vertical" table JSX from `AggregatedBurndownChart`. Adjust drill-down state from `(selectedVertical, selectedMetric, selectedTeam)` to `(selectedMetric, selectedVertical, selectedTeam)`. + +No database schema changes are required. No new tables or columns are needed — the existing `vcl_multi_vertical_summary` table already contains all data needed for metric-centric aggregation. + +## Components and Interfaces + +### Backend: New Endpoints + +#### GET /api/compliance/vcl-multi/metrics + +Aggregates all metrics across verticals using only `ALL:` rollup rows from the latest upload per vertical. + +```javascript +// Query: get latest upload ID per vertical +const { rows: latestUploads } = await pool.query(` + SELECT DISTINCT ON (vertical) id, vertical + FROM compliance_uploads + WHERE vertical IS NOT NULL + ORDER BY vertical, id DESC +`); + +const latestUploadIds = latestUploads.map(u => u.id); + +// Aggregate metrics across verticals (ALL: rows only) +const { rows: metrics } = await pool.query(` + SELECT metric_id, + MAX(metric_desc) AS metric_desc, + MAX(category) AS category, + SUM(non_compliant)::int AS non_compliant, + SUM(compliant)::int AS compliant, + SUM(total)::int AS total, + ROUND(AVG(target::numeric), 4) AS target + FROM vcl_multi_vertical_summary + WHERE upload_id = ANY($1) AND team LIKE 'ALL:%' + GROUP BY metric_id + ORDER BY non_compliant DESC +`, [latestUploadIds]); + +// Compute compliance_pct for each metric +const result = metrics.map(m => ({ + ...m, + compliance_pct: m.total > 0 ? m.compliant / m.total : 0, +})); +``` + +**Response shape:** +```json +{ + "metrics": [ + { + "metric_id": "VM-001", + "metric_desc": "Vulnerability Management - Patching", + "category": "Vulnerability Management", + "non_compliant": 450, + "compliant": 3200, + "total": 3650, + "compliance_pct": 0.8767, + "target": 0.95 + } + ] +} +``` + +#### GET /api/compliance/vcl-multi/metric/:id/verticals + +Returns per-vertical breakdown for a specific metric, including sub-team data within each vertical. + +```javascript +const metricId = req.params.id; +if (!metricId || metricId.length > 50) { + return res.status(400).json({ error: 'Invalid metric ID' }); +} + +// Get latest upload per vertical +const { rows: latestUploads } = await pool.query(` + SELECT DISTINCT ON (vertical) id, vertical + FROM compliance_uploads + WHERE vertical IS NOT NULL + ORDER BY vertical, id DESC +`); +const latestUploadIds = latestUploads.map(u => u.id); + +// Get all rows for this metric from latest uploads +const { rows: allRows } = await pool.query(` + SELECT vertical, metric_desc, category, team, + non_compliant, compliant, total, compliance_pct, target + FROM vcl_multi_vertical_summary + WHERE upload_id = ANY($1) AND metric_id = $2 + ORDER BY vertical, team +`, [latestUploadIds, metricId]); + +// Separate rollup rows (ALL:) from sub-team rows +// Build per-vertical entries with nested sub_teams +``` + +**Response shape:** +```json +{ + "metric_id": "VM-001", + "metric_desc": "Vulnerability Management - Patching", + "category": "Vulnerability Management", + "verticals": [ + { + "vertical": "NTS_AEO", + "non_compliant": 200, + "compliant": 1800, + "total": 2000, + "compliance_pct": 0.90, + "target": 0.95, + "sub_teams": [ + { + "team": "STEAM", + "non_compliant": 120, + "compliant": 1080, + "total": 1200, + "compliance_pct": 0.90 + }, + { + "team": "ACCESS-ENG", + "non_compliant": 80, + "compliant": 720, + "total": 800, + "compliance_pct": 0.90 + } + ] + } + ] +} +``` + +### Frontend: Component Changes + +#### AggregatedBurndownChart (Modified) + +Remove the "By Vertical" contribution table JSX block. The component continues to receive `data.by_vertical` in its props (for backward compat with the `/burndown` API response) but no longer renders it. + +**Before:** Summary header + bar chart + "By Vertical" table +**After:** Summary header + bar chart only + +#### MetricTable (New — replaces VerticalTable) + +Renders one row per metric from the `/metrics` endpoint response. Columns: Metric ID, Description, Category, Compliant, Non-Compliant, Total, Compliance %, Target %. Rows are clickable — clicking triggers `onSelectMetric(metricId)`. + +```javascript +function MetricTable({ metrics, onSelectMetric }) { + if (!metrics || metrics.length === 0) return null; + // Render table with columns: metric_id, metric_desc, category, + // compliant, non_compliant, total, compliance_pct, target + // onClick row → onSelectMetric(metric.metric_id) +} +``` + +#### MetricDetailView (New — replaces VerticalDetailView) + +Fetches data from `GET /metric/:id/verticals` and displays: +- Header with metric ID, description, category +- Aggregated stats cards (total, compliant, non-compliant, compliance %) +- Table of verticals with per-vertical compliance numbers +- Clicking a vertical row navigates to the sub-team view + +```javascript +function MetricDetailView({ metricId, onBack, onSelectVertical }) { + const [data, setData] = useState(null); + const [loading, setLoading] = useState(true); + + useEffect(() => { + fetch(`${API_BASE}/compliance/vcl-multi/metric/${encodeURIComponent(metricId)}/verticals`, + { credentials: 'include' }) + .then(r => r.json()) + .then(d => { setData(d); setLoading(false); }) + .catch(() => setLoading(false)); + }, [metricId]); + // Render: back button, header, stats, verticals table +} +``` + +#### MetricSubTeamView (Reused with minor adjustments) + +The existing `MetricSubTeamView` component is reused. The only change is that it now receives its `metricData` (including `sub_teams`) from the `MetricDetailView`'s vertical entry rather than from the `VerticalDetailView`'s metric entry. The props interface remains the same: `{ vertical, metricId, metricData, onBack, onSelectTeam }`. + +#### MetricDeviceList (Reused unchanged) + +The existing `MetricDeviceList` component is reused without modification. It already accepts `{ vertical, metricId, team, onBack }` and calls the existing devices endpoint. + +### Frontend: Drill-Down State Changes + +**Current state model:** +``` +selectedVertical → selectedMetric → selectedTeam +``` + +**New state model:** +``` +selectedMetric → selectedVertical → selectedTeam +``` + +The main component's render logic changes from: + +```javascript +// OLD +if (selectedTeam && selectedMetric && selectedVertical) → MetricDeviceList +if (selectedMetric && selectedVertical) → MetricSubTeamView +if (selectedVertical) → VerticalDetailView +else → Overview (VerticalTable) +``` + +To: + +```javascript +// NEW +if (selectedTeam !== null && selectedMetric && selectedVertical) → MetricDeviceList +if (selectedVertical && selectedMetric) → MetricSubTeamView +if (selectedMetric) → MetricDetailView +else → Overview (MetricTable) +``` + +**State variables:** +- `selectedMetric` — string (metric_id) or null +- `selectedMetricData` — object with metric context (metric_desc, category) or null +- `selectedVertical` — string (vertical code) or null +- `selectedVerticalData` — object with vertical's sub_teams for the selected metric or null +- `selectedTeam` — string (team name) or null (null with selectedVertical set = "View All Devices") + +## Data Flow + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Overview │ +│ GET /stats → StatsBar, DonutChart, TrendChart │ +│ GET /burndown → AggregatedBurndownChart (no By Vertical table) │ +│ GET /metrics → MetricTable │ +└──────────────────────────┬──────────────────────────────────────┘ + │ click metric row + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ MetricDetailView │ +│ GET /metric/:id/verticals → header + verticals table │ +└──────────────────────────┬──────────────────────────────────────┘ + │ click vertical row + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ MetricSubTeamView │ +│ (data passed from parent — no additional fetch) │ +└──────────────────────────┬──────────────────────────────────────┘ + │ click team row + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ MetricDeviceList │ +│ GET /vertical/:code/metric/:metricId/devices?team=X │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## Interfaces + +### New API Endpoints + +| Method | Path | Auth | Description | +|---|---|---|---| +| GET | `/api/compliance/vcl-multi/metrics` | requireAuth() | Metrics aggregated across all verticals | +| GET | `/api/compliance/vcl-multi/metric/:id/verticals` | requireAuth() | Per-vertical breakdown for a metric | + +### Preserved API Endpoints (Backward Compatibility) + +| Method | Path | Notes | +|---|---|---| +| GET | `/api/compliance/vcl-multi/stats` | Unchanged — still returns `vertical_breakdown` and `metric_breakdown` | +| GET | `/api/compliance/vcl-multi/vertical/:code/metrics` | Unchanged | +| GET | `/api/compliance/vcl-multi/vertical/:code/metric/:metricId/devices` | Unchanged | +| GET | `/api/compliance/vcl-multi/vertical/:code/burndown` | Unchanged | +| GET | `/api/compliance/vcl-multi/burndown` | Unchanged — still returns `by_vertical` in response | + +### Component Props Interfaces + +```javascript +// MetricTable +MetricTable.propTypes = { + metrics: Array, // from GET /metrics response + onSelectMetric: Function, // (metricId: string) => void +}; + +// MetricDetailView +MetricDetailView.propTypes = { + metricId: String, + onBack: Function, // () => void — returns to overview + onSelectVertical: Function, // (vertical: string, verticalData: object) => void +}; + +// MetricSubTeamView (existing — props unchanged) +MetricSubTeamView.propTypes = { + vertical: String, + metricId: String, + metricData: Object, // { metric_desc, sub_teams, target, ... } + onBack: Function, // () => void — returns to metric-vertical view + onSelectTeam: Function, // (team: string|null) => void +}; + +// MetricDeviceList (existing — props unchanged) +MetricDeviceList.propTypes = { + vertical: String, + metricId: String, + team: String, // null = all teams + onBack: Function, // () => void — returns to sub-team view +}; +``` + +## Data Models + +No new database tables or columns. The feature uses existing data: + +### vcl_multi_vertical_summary (existing) + +| Column | Type | Notes | +|---|---|---| +| id | SERIAL | PK | +| upload_id | INTEGER | FK → compliance_uploads | +| vertical | TEXT | Vertical code (e.g., "NTS_AEO") | +| metric_id | TEXT | Metric identifier (e.g., "VM-001") | +| metric_desc | TEXT | Human-readable description | +| category | TEXT | Metric category | +| team | TEXT | "ALL: NTS-AEO" for rollup, "STEAM" for sub-team | +| non_compliant | INTEGER | Count | +| compliant | INTEGER | Count | +| total | INTEGER | Count | +| compliance_pct | NUMERIC(5,2) | Stored as decimal (0.95 = 95%) | +| target | NUMERIC(5,2) | Stored as decimal | + +**Key convention:** Rows where `team LIKE 'ALL:%'` are rollup rows that include all sub-team totals. Sub-team rows are individual team breakdowns. Only rollup rows should be used for aggregation to avoid double-counting. + +## Error Handling + +### Backend + +| Scenario | Response | +|---|---| +| No auth cookie / expired session | 401 `{ "error": "Authentication required" }` | +| Metric ID > 50 chars | 400 `{ "error": "Invalid metric ID" }` | +| Database query failure | 500 `{ "error": "Database error" }` | +| No data for metric | 200 with empty `verticals` array | +| No summary data at all | 200 with empty `metrics` array | + +### Frontend + +| Scenario | Behavior | +|---|---| +| Fetch failure on overview | Existing error state renders error message | +| Fetch failure on MetricDetailView | Show error message in component | +| Empty metrics array | Show "No metrics data" empty state | +| Empty verticals array for a metric | Show "No verticals found for this metric" message | + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +### Property 1: Metrics aggregation uses only rollup rows from latest uploads + +*For any* set of `vcl_multi_vertical_summary` rows across multiple verticals and uploads, the `GET /metrics` endpoint SHALL return exactly one entry per distinct `metric_id`, with `non_compliant`, `compliant`, and `total` equal to the sum of only `ALL:`-prefixed team rows from the single latest upload per vertical. + +**Validates: Requirements 3.1, 3.3** + +### Property 2: Metrics computed fields are mathematically correct + +*For any* metric entry returned by the `GET /metrics` endpoint, `compliance_pct` SHALL equal `compliant / total` (or 0 when total is 0), and `target` SHALL equal the arithmetic mean of the `target` values across all verticals that have that metric in their latest upload. + +**Validates: Requirements 3.4, 3.5** + +### Property 3: Metrics response is sorted by non-compliant descending + +*For any* response from `GET /metrics` containing two or more metric entries, each entry's `non_compliant` value SHALL be greater than or equal to the next entry's `non_compliant` value. + +**Validates: Requirements 3.6** + +### Property 4: Metric-verticals breakdown is complete and correct + +*For any* metric ID that exists in the database, the `GET /metric/:id/verticals` endpoint SHALL return one vertical entry for each vertical that has that metric in its latest upload, with each entry's `sub_teams` array containing exactly the non-rollup, non-"(Other)" team rows for that metric in that vertical. + +**Validates: Requirements 4.1, 4.3** + +### Property 5: Metric-verticals response is sorted by non-compliant descending + +*For any* response from `GET /metric/:id/verticals` containing two or more vertical entries, each entry's `non_compliant` value SHALL be greater than or equal to the next entry's `non_compliant` value. + +**Validates: Requirements 4.4** + +### Property 6: Drill-down state determines rendered view + +*For any* combination of `(selectedMetric, selectedVertical, selectedTeam)` state values, the CCP_Metrics_Page SHALL render exactly one view: Overview when no metric is selected, MetricDetailView when only metric is selected, MetricSubTeamView when metric and vertical are selected (team is null or absent), and MetricDeviceList when all three are selected. + +**Validates: Requirements 6.1, 6.5** + +## Testing Strategy + +### Property-Based Tests (Backend) + +Properties 1–5 are testable via property-based tests against the backend endpoints. The test setup generates random `vcl_multi_vertical_summary` rows with varying numbers of verticals, metrics, uploads, and team configurations, then asserts the invariants hold. + +### Example-Based Tests (Frontend) + +Requirements 1.x, 2.x, 5.x, and 6.2–6.4 are UI rendering and interaction tests best covered by example-based component tests using React Testing Library. + +### Integration Tests (Backward Compatibility) + +Requirements 7.x are integration tests that verify existing endpoints continue to return the same response shape after the new endpoints are added. diff --git a/.kiro/specs/ccp-metrics-view-restructure/requirements.md b/.kiro/specs/ccp-metrics-view-restructure/requirements.md new file mode 100644 index 0000000..1d06cbe --- /dev/null +++ b/.kiro/specs/ccp-metrics-view-restructure/requirements.md @@ -0,0 +1,112 @@ +# Requirements Document + +## Introduction + +This feature restructures the CCP Metrics page drill-down hierarchy from a vertical-first model (Vertical → Metric → Subteam → Devices) to a metric-first model (Metric → Vertical → Subteam → Devices). The overview table changes from showing one row per vertical to one row per metric (aggregated across all verticals), with each metric appearing once with totals summed across verticals. Clicking a metric drills into which verticals have that metric. Additionally, the "By Vertical" contribution table currently rendered below the AggregatedBurndownChart is removed — the burndown chart and summary stats remain. + +Two new backend endpoints support the metric-centric view alongside existing vertical-first endpoints, which are preserved for backward compatibility. + +## Glossary + +- **CCP_Metrics_Page**: The React page component (`CCPMetricsPage.js`) that displays cross-organizational compliance posture and provides drill-down navigation. +- **Overview_Table**: The primary data table on the CCP_Metrics_Page overview that users interact with to begin drill-down navigation. +- **Metrics_API**: The new backend endpoint `GET /api/compliance/vcl-multi/metrics` that returns all metrics aggregated across verticals. +- **Metric_Verticals_API**: The new backend endpoint `GET /api/compliance/vcl-multi/metric/:id/verticals` that returns per-vertical breakdown for a specific metric. +- **Devices_API**: The existing backend endpoint `GET /api/compliance/vcl-multi/vertical/:code/metric/:metricId/devices` that returns non-compliant devices filtered by vertical, metric, and optionally team. +- **Drill_Down_State**: The set of React state variables that track the user's current navigation depth within the metric-centric hierarchy. +- **AggregatedBurndownChart**: The existing React component that displays the cross-vertical burndown forecast bar chart and summary statistics on the overview page. +- **Metric_Row**: A single row in the Overview_Table representing one metric with compliance totals summed across all verticals. + +## Requirements + +### Requirement 1: Remove "By Vertical" Table from AggregatedBurndownChart + +**User Story:** As a compliance analyst, I want the burndown chart section to show only the forecast chart and summary stats without the per-vertical contribution table, so that the overview page is less cluttered and the vertical breakdown is accessible through the drill-down instead. + +#### Acceptance Criteria + +1. THE AggregatedBurndownChart component SHALL render the summary header (total non-compliant, blockers, in-progress, projected clear date) and the bar chart. +2. THE AggregatedBurndownChart component SHALL omit the "By Vertical" contribution table that previously appeared below the bar chart. +3. THE AggregatedBurndownChart component SHALL preserve all existing loading, error, empty-state, and all-blockers display behaviors. + +### Requirement 2: Metric-Aggregated Overview Table + +**User Story:** As a senior leader, I want the overview table to show one row per metric with totals summed across all verticals, so that I can immediately see which metrics have the most non-compliance across the organization. + +#### Acceptance Criteria + +1. THE Overview_Table SHALL display one Metric_Row per distinct metric, with compliance totals (compliant, non_compliant, total, compliance_pct, target) aggregated across all verticals. +2. THE Overview_Table SHALL display columns for metric ID, description, category, compliant count, non-compliant count, total count, compliance percentage, and target percentage. +3. THE Overview_Table SHALL sort Metric_Rows by non-compliant count in descending order by default. +4. WHEN a user clicks a Metric_Row, THE CCP_Metrics_Page SHALL navigate to the metric-vertical drill-down view for that metric. +5. THE Overview_Table SHALL replace the existing VerticalTable component on the overview page. + +### Requirement 3: Metrics Aggregated API Endpoint + +**User Story:** As a frontend developer, I want a single API endpoint that returns all metrics aggregated across verticals, so that the overview table can display metric-centric data without client-side aggregation. + +#### Acceptance Criteria + +1. WHEN a GET request is made to `/api/compliance/vcl-multi/metrics`, THE Metrics_API SHALL return a JSON response containing a `metrics` array where each entry represents one distinct metric aggregated across all verticals. +2. THE Metrics_API SHALL compute each metric entry with fields: `metric_id`, `metric_desc`, `category`, `non_compliant`, `compliant`, `total`, `compliance_pct`, and `target`. +3. THE Metrics_API SHALL aggregate totals by summing `non_compliant`, `compliant`, and `total` across all verticals for each metric, using only rollup rows (team starting with "ALL:") from the latest upload per vertical. +4. THE Metrics_API SHALL compute `compliance_pct` as `compliant / total` for each aggregated metric. +5. THE Metrics_API SHALL compute `target` as the average target across verticals for each metric. +6. THE Metrics_API SHALL sort the returned metrics array by `non_compliant` in descending order. +7. WHEN no summary data exists, THE Metrics_API SHALL return an empty `metrics` array. +8. THE Metrics_API SHALL require authentication via `requireAuth()` middleware. +9. IF a database error occurs, THEN THE Metrics_API SHALL return HTTP 500 with `{ "error": "Database error" }`. + +### Requirement 4: Metric-Verticals Drill-Down API Endpoint + +**User Story:** As a frontend developer, I want an API endpoint that returns which verticals have a specific metric and their per-vertical compliance numbers, so that the drill-down view can show vertical breakdown for a selected metric. + +#### Acceptance Criteria + +1. WHEN a GET request is made to `/api/compliance/vcl-multi/metric/:id/verticals`, THE Metric_Verticals_API SHALL return a JSON response containing a `verticals` array with per-vertical breakdown for the specified metric. +2. THE Metric_Verticals_API SHALL return each vertical entry with fields: `vertical`, `non_compliant`, `compliant`, `total`, `compliance_pct`, `target`, and `sub_teams`. +3. THE Metric_Verticals_API SHALL include a `sub_teams` array within each vertical entry containing per-team breakdown with fields: `team`, `non_compliant`, `compliant`, `total`, `compliance_pct`. +4. THE Metric_Verticals_API SHALL sort the `verticals` array by `non_compliant` in descending order. +5. THE Metric_Verticals_API SHALL also return `metric_id`, `metric_desc`, and `category` fields at the top level for display context. +6. WHEN the specified metric ID does not exist in any vertical, THE Metric_Verticals_API SHALL return an empty `verticals` array with the metric_id echoed back. +7. IF the metric ID parameter exceeds 50 characters, THEN THE Metric_Verticals_API SHALL return HTTP 400 with `{ "error": "Invalid metric ID" }`. +8. THE Metric_Verticals_API SHALL require authentication via `requireAuth()` middleware. +9. IF a database error occurs, THEN THE Metric_Verticals_API SHALL return HTTP 500 with `{ "error": "Database error" }`. + +### Requirement 5: Metric-Centric Drill-Down Hierarchy + +**User Story:** As a compliance analyst, I want to drill down from a metric to see which verticals are affected, then into a vertical's sub-teams, and finally to the device list, so that I can trace non-compliance from the metric level down to individual devices. + +#### Acceptance Criteria + +1. WHEN a user clicks a Metric_Row in the Overview_Table, THE CCP_Metrics_Page SHALL display a metric-vertical view showing per-vertical breakdown for that metric. +2. THE metric-vertical view SHALL display the metric ID, description, and category as a header, along with aggregated compliance stats for that metric. +3. THE metric-vertical view SHALL display a table of verticals with columns: vertical name, compliant, non-compliant, total, and compliance percentage. +4. WHEN a user clicks a vertical row in the metric-vertical view, THE CCP_Metrics_Page SHALL display a sub-team breakdown view for that vertical and metric combination. +5. THE sub-team breakdown view SHALL display per-team compliance numbers with columns: team name, compliant, non-compliant, total, and compliance percentage. +6. WHEN a user clicks a sub-team row in the sub-team view, THE CCP_Metrics_Page SHALL display the device list filtered by the selected vertical, metric, and team. +7. THE device list view SHALL reuse the existing Devices_API endpoint (`GET /api/compliance/vcl-multi/vertical/:code/metric/:metricId/devices`) with the vertical and metric parameters derived from the drill-down context. +8. WHEN a user clicks a "View All Devices" option in the sub-team view, THE CCP_Metrics_Page SHALL display the device list for all teams within that vertical and metric. + +### Requirement 6: Drill-Down Navigation State + +**User Story:** As a user navigating the drill-down hierarchy, I want clear back-navigation at each level so that I can return to the previous view without losing context. + +#### Acceptance Criteria + +1. THE Drill_Down_State SHALL track the selected metric, selected vertical, and selected team to determine which view to render. +2. WHEN the metric-vertical view is displayed, THE CCP_Metrics_Page SHALL provide a "Back to Overview" navigation element that clears the selected metric and returns to the overview. +3. WHEN the sub-team view is displayed, THE CCP_Metrics_Page SHALL provide a "Back to Verticals" navigation element that clears the selected vertical and returns to the metric-vertical view. +4. WHEN the device list view is displayed, THE CCP_Metrics_Page SHALL provide a "Back to Sub-Teams" navigation element that clears the selected team and returns to the sub-team view. +5. THE CCP_Metrics_Page SHALL render the overview (with Overview_Table) when no metric is selected in the Drill_Down_State. + +### Requirement 7: Backward Compatibility + +**User Story:** As a developer maintaining the system, I want existing vertical-first endpoints to remain functional so that any external consumers or future features relying on them continue to work. + +#### Acceptance Criteria + +1. THE backend SHALL preserve the existing `GET /api/compliance/vcl-multi/vertical/:code/metrics` endpoint with unchanged behavior and response shape. +2. THE backend SHALL preserve the existing `GET /api/compliance/vcl-multi/vertical/:code/metric/:metricId/devices` endpoint with unchanged behavior and response shape. +3. THE backend SHALL preserve the existing `GET /api/compliance/vcl-multi/vertical/:code/burndown` endpoint with unchanged behavior and response shape. +4. THE backend SHALL preserve the existing `GET /api/compliance/vcl-multi/stats` endpoint with unchanged behavior and response shape, including the `vertical_breakdown` and `metric_breakdown` fields. diff --git a/.kiro/specs/ccp-metrics-view-restructure/tasks.md b/.kiro/specs/ccp-metrics-view-restructure/tasks.md new file mode 100644 index 0000000..de6a726 --- /dev/null +++ b/.kiro/specs/ccp-metrics-view-restructure/tasks.md @@ -0,0 +1,151 @@ +# Implementation Plan: CCP Metrics View Restructure + +## Overview + +Restructure the CCP Metrics page from a vertical-first drill-down model to a metric-first model. Two new backend endpoints aggregate metrics across verticals. The frontend replaces VerticalTable/VerticalDetailView with MetricTable/MetricDetailView, removes the "By Vertical" table from AggregatedBurndownChart, and inverts the drill-down state hierarchy. + +## Tasks + +- [x] 1. Add backend endpoints for metric-centric aggregation + - [x] 1.1 Implement GET /metrics endpoint in backend/routes/vclMultiVertical.js + - Add route handler that queries `vcl_multi_vertical_summary` using only `ALL:` rollup rows from the latest upload per vertical + - Aggregate by `metric_id`: SUM non_compliant, compliant, total; MAX metric_desc, category; AVG target + - Compute `compliance_pct` as `compliant / total` (0 when total is 0) + - Sort by `non_compliant` DESC + - Return `{ metrics: [...] }` with empty array when no data exists + - Require `requireAuth()` middleware + - Return HTTP 500 with `{ "error": "Database error" }` on query failure + - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9_ + + - [x] 1.2 Implement GET /metric/:id/verticals endpoint in backend/routes/vclMultiVertical.js + - Validate metric ID parameter (reject if > 50 chars with HTTP 400) + - Query all rows for the metric from latest uploads + - Separate `ALL:` rollup rows from sub-team rows + - Build per-vertical entries with nested `sub_teams` arrays + - Sort verticals by `non_compliant` DESC + - Return `{ metric_id, metric_desc, category, verticals: [...] }` with empty verticals array when metric not found + - Require `requireAuth()` middleware + - Return HTTP 500 with `{ "error": "Database error" }` on query failure + - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9_ + + - [ ]* 1.3 Write property test: Metrics aggregation uses only rollup rows from latest uploads + - **Property 1: Metrics aggregation uses only rollup rows from latest uploads** + - Generate random `vcl_multi_vertical_summary` rows with multiple verticals, metrics, uploads, and team configurations + - Assert the endpoint returns exactly one entry per distinct `metric_id` with totals matching only `ALL:`-prefixed rows from the latest upload per vertical + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - **Validates: Requirements 3.1, 3.3** + + - [ ]* 1.4 Write property test: Metrics computed fields are mathematically correct + - **Property 2: Metrics computed fields are mathematically correct** + - Assert `compliance_pct` equals `compliant / total` (or 0 when total is 0) for each metric + - Assert `target` equals the arithmetic mean of target values across verticals for each metric + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - **Validates: Requirements 3.4, 3.5** + + - [ ]* 1.5 Write property test: Metrics response is sorted by non-compliant descending + - **Property 3: Metrics response is sorted by non-compliant descending** + - Assert each entry's `non_compliant` is >= the next entry's `non_compliant` + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - **Validates: Requirements 3.6** + + - [ ]* 1.6 Write property test: Metric-verticals breakdown is complete and correct + - **Property 4: Metric-verticals breakdown is complete and correct** + - Assert one vertical entry per vertical that has the metric in its latest upload + - Assert each entry's `sub_teams` contains exactly the non-rollup team rows for that metric/vertical + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - **Validates: Requirements 4.1, 4.3** + + - [ ]* 1.7 Write property test: Metric-verticals response is sorted by non-compliant descending + - **Property 5: Metric-verticals response is sorted by non-compliant descending** + - Assert each vertical entry's `non_compliant` is >= the next entry's `non_compliant` + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - **Validates: Requirements 4.4** + +- [x] 2. Checkpoint - Backend endpoints verified + - Ensure all tests pass, ask the user if questions arise. + +- [x] 3. Remove "By Vertical" table from AggregatedBurndownChart + - [x] 3.1 Remove the "By Vertical" contribution table JSX from AggregatedBurndownChart in frontend/src/components/pages/CCPMetricsPage.js + - Remove the entire `{data.by_vertical && data.by_vertical.length > 0 && (...)}` block that renders the per-vertical table below the bar chart + - Preserve the summary header (total non-compliant, blockers, in-progress, projected clear date) and bar chart + - Preserve all loading, error, empty-state, and all-blockers display behaviors + - _Requirements: 1.1, 1.2, 1.3_ + +- [x] 4. Implement MetricTable component and replace VerticalTable + - [x] 4.1 Create MetricTable component in frontend/src/components/pages/CCPMetricsPage.js + - Render one row per metric from the `/metrics` endpoint response + - Columns: Metric ID, Description, Category, Compliant, Non-Compliant, Total, Compliance %, Target % + - Sort rows by non-compliant descending (server-side, already sorted) + - Rows are clickable — clicking triggers `onSelectMetric(metricId)` + - Handle empty state (no metrics data) + - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_ + + - [x] 4.2 Replace VerticalTable usage with MetricTable in the overview render logic + - Add fetch call to `GET /api/compliance/vcl-multi/metrics` in the overview data loading + - Pass fetched metrics data to MetricTable + - Wire `onSelectMetric` to set `selectedMetric` state + - _Requirements: 2.4, 2.5_ + +- [x] 5. Implement MetricDetailView and update drill-down state + - [x] 5.1 Create MetricDetailView component in frontend/src/components/pages/CCPMetricsPage.js + - Fetch data from `GET /metric/:id/verticals` on mount + - Display header with metric ID, description, category + - Display aggregated stats cards (total, compliant, non-compliant, compliance %) + - Display table of verticals with columns: vertical name, compliant, non-compliant, total, compliance % + - Clicking a vertical row calls `onSelectVertical(vertical, verticalData)` + - Include "Back to Overview" button that calls `onBack` + - Handle loading, error, and empty states + - _Requirements: 5.1, 5.2, 5.3, 5.4, 6.2_ + + - [x] 5.2 Restructure drill-down state in CCPMetricsPage main component + - Change state model from `selectedVertical → selectedMetric → selectedTeam` to `selectedMetric → selectedVertical → selectedTeam` + - Add state variables: `selectedMetric`, `selectedMetricData`, `selectedVertical`, `selectedVerticalData`, `selectedTeam` + - Update render logic: no metric = Overview (MetricTable), metric only = MetricDetailView, metric+vertical = MetricSubTeamView, all three = MetricDeviceList + - Wire back-navigation handlers at each level (clear appropriate state to go up one level) + - Pass `selectedVerticalData.sub_teams` to MetricSubTeamView as `metricData` + - _Requirements: 5.4, 5.5, 5.6, 5.7, 5.8, 6.1, 6.2, 6.3, 6.4, 6.5_ + + - [ ]* 5.3 Write property test: Drill-down state determines rendered view + - **Property 6: Drill-down state determines rendered view** + - Generate random combinations of `(selectedMetric, selectedVertical, selectedTeam)` state values + - Assert exactly one view is rendered for each combination following the state hierarchy rules + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - **Validates: Requirements 6.1, 6.5** + +- [ ] 6. Verify backward compatibility of existing endpoints + - [ ]* 6.1 Write integration tests verifying existing endpoints are unchanged + - Verify `GET /vertical/:code/metrics` returns same response shape + - Verify `GET /vertical/:code/metric/:metricId/devices` returns same response shape + - Verify `GET /vertical/:code/burndown` returns same response shape + - Verify `GET /stats` still returns `vertical_breakdown` and `metric_breakdown` fields + - Test file: `backend/__tests__/ccp-metrics-view-restructure.property.test.js` + - _Requirements: 7.1, 7.2, 7.3, 7.4_ + +- [x] 7. Final checkpoint - Ensure all tests pass + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional and can be skipped for faster MVP +- Each task references specific requirements for traceability +- Checkpoints ensure incremental validation +- Property tests validate universal correctness properties from the design document +- The existing `MetricSubTeamView` and `MetricDeviceList` components are reused with minor prop adjustments — no new component creation needed for those +- All backend endpoints are added to the existing `backend/routes/vclMultiVertical.js` file +- All frontend changes are within `frontend/src/components/pages/CCPMetricsPage.js` +- Property-based tests use `fast-check` (already in project dependencies) + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1", "1.2"] }, + { "id": 1, "tasks": ["1.3", "1.4", "1.5", "1.6", "1.7", "3.1"] }, + { "id": 2, "tasks": ["4.1"] }, + { "id": 3, "tasks": ["4.2", "5.1"] }, + { "id": 4, "tasks": ["5.2"] }, + { "id": 5, "tasks": ["5.3", "6.1"] } + ] +} +``` diff --git a/.kiro/specs/compliance-list-stale-after-sidebar-edit/.config.kiro b/.kiro/specs/compliance-list-stale-after-sidebar-edit/.config.kiro new file mode 100644 index 0000000..37166b4 --- /dev/null +++ b/.kiro/specs/compliance-list-stale-after-sidebar-edit/.config.kiro @@ -0,0 +1 @@ +{"specId": "7b867406-4d3d-4728-9b3c-94ea7c6f51ef", "workflowType": "requirements-first", "specType": "bugfix"} diff --git a/.kiro/specs/compliance-list-stale-after-sidebar-edit/bugfix.md b/.kiro/specs/compliance-list-stale-after-sidebar-edit/bugfix.md new file mode 100644 index 0000000..f1c1d63 --- /dev/null +++ b/.kiro/specs/compliance-list-stale-after-sidebar-edit/bugfix.md @@ -0,0 +1,44 @@ +# Bugfix Requirements Document + +## Introduction + +GitLab issue [#23](http://steam-gitlab.charterlab.com/steam/cve-dashboard/-/issues/23) — "[Bug] Update Res Date/Remed Plan in list after updating in sidebar" (reported by nkapur on the AEO Compliance page). + +After a user sets a resolution date and/or remediation plan in the host detail sidebar (`ComplianceDetailPanel`) and closes it, the new value does not appear in the host list on `CompliancePage` until the list is refreshed by some other action (changing filters, switching teams, switching tabs, completing an upload, adding a note, or clicking the manual refresh button). + +This is purely a frontend state-refresh issue. The predecessor spec `compliance-remediation-display-fix` already fixed the list endpoint (`GET /api/compliance/items?team=X&status=Y`) to include `resolution_date` and `remediation_plan` in the SELECT and propagate them through `groupByHostname()`, so the backend list endpoint already returns the correct data. The defect is that `CompliancePage` never re-fetches the device list after a sidebar metadata save, so the in-memory `devices` state (populated by `fetchDevices`) keeps the stale value until one of the existing refresh triggers fires. The backend is out of scope for this fix. + +Relevant code: + +- `frontend/src/components/pages/CompliancePage.js` renders `` (around lines 696–702) with `onClose={() => setSelectedHost(null)}` (clears selection, no list refresh) and `onNoteAdded={refresh}` (where `refresh` runs `fetchSummary` + `fetchDevices`, but `onNoteAdded` is only invoked when a note is added or deleted). +- `frontend/src/components/pages/ComplianceDetailPanel.js` has `handleSaveMetadata()` which `PATCH`es `/api/compliance/items/:hostname/metadata` then calls its own `fetchDetail()` to refresh the panel, but never invokes any parent callback after a metadata save. As a result the parent list held in `CompliancePage` state is never re-fetched after a resolution date / remediation plan edit. + +## Bug Analysis + +### Current Behavior (Defect) + +1.1 WHEN the user saves a resolution date and/or remediation plan in the sidebar via `handleSaveMetadata()` (`PATCH /api/compliance/items/:hostname/metadata` succeeds) THEN the system refreshes only the panel's own detail (`fetchDetail()`) and does not notify the parent `CompliancePage`, so `fetchDevices` is never re-issued. + +1.2 WHEN the user closes the sidebar after a successful metadata save via `onClose` THEN the system only clears `selectedHost` (`setSelectedHost(null)`) and does not trigger any list refresh, so the parent `devices` state retains the pre-edit value. + +1.3 WHEN the host list row re-renders after a sidebar metadata save without an intervening refresh THEN the system displays the stale Resolution Date / Remediation Plan value (or "—") because `devices` in `CompliancePage` state still holds the value from the last `fetchDevices` call. + +### Expected Behavior (Correct) + +2.1 WHEN the user saves a resolution date and/or remediation plan in the sidebar via `handleSaveMetadata()` and the `PATCH` succeeds THEN the system SHALL invoke a parent refresh callback (mirroring the existing `onNoteAdded` pattern) that re-issues `fetchDevices` so the parent list reflects the updated values. + +2.2 WHEN the parent list has been re-fetched following a successful sidebar metadata save THEN the system SHALL display the updated Resolution Date / Remediation Plan value in the corresponding host list row without requiring a manual filter change, team change, tab change, or manual refresh. + +### Unchanged Behavior (Regression Prevention) + +3.1 WHEN a user adds or deletes a note in the sidebar THEN the system SHALL CONTINUE TO invoke the existing `onNoteAdded` refresh callback exactly as before. + +3.2 WHEN a user changes team, changes the active/resolved tab, completes an upload, or clicks the manual refresh button THEN the system SHALL CONTINUE TO refresh the list via the existing `fetchSummary` + `fetchDevices` triggers. + +3.3 WHEN a metadata save fails (the `PATCH` returns a non-OK response and `handleSaveMetadata()` sets `metaError`) THEN the system SHALL CONTINUE TO surface the error in the panel and SHALL NOT require the list to display changed values for a save that did not persist. + +3.4 WHEN the user closes the sidebar without having saved any metadata change THEN the system SHALL CONTINUE TO clear the selection as before, and no unnecessary behavioral regression SHALL be introduced for the no-change case. + +3.5 WHEN the host list renders hostname, IP address, device type, failing metrics, and seen count THEN the system SHALL CONTINUE TO display these fields correctly with no change in behavior. + +3.6 WHEN the sidebar saves metadata THEN the system SHALL CONTINUE TO call its own `fetchDetail()` so the panel's in-place view and history remain correct. diff --git a/.kiro/specs/compliance-list-stale-after-sidebar-edit/tasks.md b/.kiro/specs/compliance-list-stale-after-sidebar-edit/tasks.md new file mode 100644 index 0000000..e43d93d --- /dev/null +++ b/.kiro/specs/compliance-list-stale-after-sidebar-edit/tasks.md @@ -0,0 +1,116 @@ +# Implementation Plan: Compliance List Stale After Sidebar Edit + +## Overview + +Fix the frontend state-refresh defect described in GitLab issue [#23](http://steam-gitlab.charterlab.com/steam/cve-dashboard/-/issues/23). After a successful metadata save in `ComplianceDetailPanel` (`PATCH /api/compliance/items/:hostname/metadata`), the host list on `CompliancePage` keeps showing the pre-edit Resolution Date / Remediation Plan until an unrelated refresh trigger fires. The backend list endpoint already returns the correct data (fixed in the predecessor spec `compliance-remediation-display-fix`); the defect is purely that the sidebar never notifies the parent after a metadata save, so `fetchDevices` is never re-issued. + +The fix mirrors the existing `onNoteAdded` → `refresh` pattern: `handleSaveMetadata()` invokes a new parent callback (`onMetadataSaved`) after a successful `PATCH`, and `CompliancePage` wires that callback to `refresh` (`fetchSummary` + `fetchDevices`). The backend is out of scope. + +All tasks below are coding/test tasks. Tests live in the frontend and run via `react-scripts test` (`CI=true npm test`), using `@testing-library/react`, `@testing-library/jest-dom`, and `fast-check` (already permitted by `jest.transformIgnorePatterns` in `frontend/package.json`). + +## Tasks + +- [x] 1. Write bug condition exploration property test + - **Property 1: Bug Condition** - List Row Stays Stale After Sidebar Metadata Save + - **CRITICAL**: This test MUST FAIL on the current unfixed code — failure confirms the bug exists + - **DO NOT attempt to fix the test or the code when it fails** — the failure is the expected, desired outcome at this step + - **NOTE**: This test encodes the expected behavior (Property 1 below) — it will validate the fix when it passes after implementation + - **GOAL**: Surface counterexamples demonstrating that after a successful sidebar metadata save the parent `CompliancePage` list does NOT reflect the saved value without a manual refresh + - **Test file**: `frontend/src/components/pages/__tests__/compliance-list-stale-after-sidebar-edit.exploration.property.test.js` + - **Header comment**: reference spec `.kiro/specs/compliance-list-stale-after-sidebar-edit/` and issue #23, and state "EXPECTED TO FAIL on unfixed code — failure confirms the bug" (mirrors `backend/__tests__/compliance-remediation-display-fix.exploration.property.test.js` tagging convention) + - **Bug Condition** (from bugfix.md Current Behavior): `isBugCondition(input) = a successful PATCH /api/compliance/items/:hostname/metadata occurred from the sidebar` — under this condition the parent list is never re-fetched (1.1), `onClose` only clears `selectedHost` (1.2), and the row shows the stale value (1.3) + - **Scoped PBT Approach**: use `fast-check` to generate varied saved-metadata values across runs and assert the universal property holds for each: + - `fc.assert(fc.asyncProperty(arbResolutionDate, arbRemediationPlan, async (resolutionDate, remediationPlan) => { ... }), { numRuns: ... })` + - `arbResolutionDate`: build `YYYY-MM-DD` strings from integer tuples (do NOT call `toISOString` on shrunk values — follow the date-generator pattern in the predecessor exploration test) + - `arbRemediationPlan`: non-empty trimmed strings, length-bounded (e.g. 1–200) + - **Property under test**: _for any_ saved metadata value, after a successful sidebar save the parent list row for that hostname displays the saved value without a manual filter/team/tab change or manual refresh click + - **Test structure**: + - Mock `useAuth` from `../../contexts/AuthContext` (e.g. `jest.mock`) to return a stub: `canWrite: () => true`, `isAdmin: () => false`, `getAvailableTeams: () => ['STEAM']`, `adminScope: null` + - Mock `global.fetch` with a URL-routing implementation responding to: + - `GET /compliance/summary?team=STEAM` → minimal summary `{ entries: [], overall_scores: {}, upload: { report_date: '2025-01-01' } }` + - `GET /compliance/items?team=STEAM&status=active` → `{ devices: [oneDevice] }` where the device starts with a stale value (e.g. `resolution_date: null`, `remediation_plan: null`) on the FIRST call, and returns the generated saved value on SUBSEQUENT calls (so a re-fetch is observable) + - `GET /compliance/items/:hostname` → detail object with one active metric so the panel renders the Resolution Date / Remediation Plan inputs and Save button + - `PATCH /compliance/items/:hostname/metadata` → `{ ok: true }` success response + - Render ` {}} />`, wait for the device row to appear, click the row to open `ComplianceDetailPanel` + - Set the Resolution Date input and/or Remediation Plan textarea to the generated value(s) and click **Save** (use `@testing-library/user-event` or `fireEvent`, wrapped in `act`/`waitFor`) + - **Primary assertion (encodes Property 1)**: `await waitFor(() => expect(within(row).getByText()).toBeInTheDocument())` — the list row reflects the saved value + - **Supporting assertion**: the list endpoint `GET /compliance/items?...` is called more than once (initial load + post-save refresh); on unfixed code it is called exactly once + - Run on UNFIXED code: `cd frontend && CI=true npm test -- --testPathPattern compliance-list-stale-after-sidebar-edit.exploration` + - **EXPECTED OUTCOME**: Test FAILS — `handleSaveMetadata()` never invokes a parent callback, so `fetchDevices` is not re-issued and the row keeps the stale value (the list GET is called only once) + - Document the counterexample found (e.g. "after saving resolution_date='2026-03-04', the list row still shows '—'; GET /compliance/items called 1 time, expected ≥ 2") + - Mark this task complete when the test is written, run, and the failure is documented + - _Requirements: 1.1, 1.2, 1.3, 2.1, 2.2_ + +- [x] 2. Fix the stale list after sidebar metadata save + + - [x] 2.1 Invoke a parent refresh callback from `handleSaveMetadata()` after a successful PATCH + - In `frontend/src/components/pages/ComplianceDetailPanel.js`, add `onMetadataSaved` to the component props: `export default function ComplianceDetailPanel({ hostname, onClose, onNoteAdded, onMetadataSaved, onNavigate })` + - In `handleSaveMetadata()`, after the `PATCH` succeeds and `await fetchDetail()` completes (inside the `try`, after the existing success handling), add: `if (onMetadataSaved) onMetadataSaved();` + - Place the call so it only runs on success (not in the `catch`), mirroring how `handleAddNote()` calls `if (onNoteAdded) onNoteAdded();` only after a successful note save + - Keep the existing `await fetchDetail()` call so the panel's own view/history still refreshes (do not replace it) + - _Bug_Condition: isBugCondition(input) = successful PATCH /api/compliance/items/:hostname/metadata from the sidebar_ + - _Expected_Behavior: a parent refresh callback is invoked on save success so the parent list re-fetches (bugfix.md 2.1)_ + - _Preservation: only the success path is touched; the `catch` path that sets `metaError` is unchanged (3.3); `fetchDetail()` still runs (3.6)_ + - _Requirements: 2.1, 3.3, 3.6_ + + - [x] 2.2 Wire `onMetadataSaved` to `refresh` in `CompliancePage` + - In `frontend/src/components/pages/CompliancePage.js`, at the `` render site (around lines 696–702), add the prop `onMetadataSaved={refresh}` alongside the existing `onNoteAdded={refresh}` + - Reuse the existing `refresh` function (`fetchSummary(activeTeam)` + `fetchDevices(activeTeam, activeTab)`) — do not introduce a new refresh path + - Leave `onClose={() => setSelectedHost(null)}` unchanged (close still just clears selection) + - Do not leave any unused variables; if a value is intentionally unused prefix it with `_` per the ESLint `no-unused-vars` warning budget + - _Bug_Condition: isBugCondition(input) = successful sidebar metadata save while the panel is open from CompliancePage_ + - _Expected_Behavior: the parent list re-fetches via fetchDevices so the updated Resolution Date / Remediation Plan appears in the row without a manual refresh (bugfix.md 2.1, 2.2)_ + - _Preservation: onNoteAdded still maps to refresh (3.1); team/tab/upload/manual-refresh triggers unchanged (3.2); onClose unchanged (3.4); other row fields unchanged (3.5)_ + - _Requirements: 2.1, 2.2_ + + - [x] 2.3 Verify the bug condition exploration test now passes + - **Property 1: Expected Behavior** - List Row Reflects Saved Value Without Manual Refresh + - **IMPORTANT**: Re-run the SAME test from Task 1 — do NOT write a new test + - The Task 1 test encodes the expected behavior; when it passes it confirms the fix satisfies Property 1 + - Run: `cd frontend && CI=true npm test -- --testPathPattern compliance-list-stale-after-sidebar-edit.exploration` + - **EXPECTED OUTCOME**: Test PASSES — after a successful save the list re-fetches (`GET /compliance/items?...` called ≥ 2 times) and the row displays the saved value + - _Requirements: 2.1, 2.2_ + +- [x] 3. Write preservation / regression property test + - **Property 2: Preservation** - Unchanged Behaviors Hold After the Fix + - **IMPORTANT**: Follow observation-first methodology — the unchanged behaviors below are observed to hold on UNFIXED code and must continue to hold after the fix; only the new "list updates after save" guard is expected to flip from failing to passing + - **Test file**: `frontend/src/components/pages/__tests__/compliance-list-stale-after-sidebar-edit.preservation.property.test.js` + - **Header comment**: reference spec `.kiro/specs/compliance-list-stale-after-sidebar-edit/` and issue #23 (mirrors the predecessor preservation test tagging convention) + - **Test structure**: reuse the same `useAuth` mock and URL-routing `global.fetch` mock approach as Task 1; use `fast-check` to generate varied note text and metadata values across runs + - **Regression guard (new behavior, post-fix)**: _for any_ saved metadata value, after a successful save the parent list row displays the saved value (same assertion family as Property 1, kept as a standing regression guard) + - **Preservation properties (must hold before and after the fix)**: + - **Note-add refresh still works**: when a note is added in the sidebar (`POST /compliance/notes` succeeds), the existing `onNoteAdded` callback still triggers a list re-fetch — assert `GET /compliance/items?...` is re-issued after a note add (bugfix.md 3.1) + - **Failed save shows error and does NOT falsely update the list**: when `PATCH /compliance/items/:hostname/metadata` returns a non-OK response, assert the panel surfaces `metaError` AND the list row value is unchanged / no spurious list re-fetch displays a value that was never persisted (bugfix.md 3.3) + - **Close-without-change just clears selection**: clicking the close (`X`) / backdrop without saving removes the detail panel (selection cleared) and does not error; no metadata save occurs (bugfix.md 3.4) + - **Other row fields render correctly**: _for any_ generated device, the row still displays hostname, IP address, device type, failing metrics, and seen count unchanged (bugfix.md 3.5) + - Run before the fix to confirm the preservation properties pass on unfixed code; run after the fix to confirm they still pass and the regression guard now passes + - Command: `cd frontend && CI=true npm test -- --testPathPattern compliance-list-stale-after-sidebar-edit.preservation` + - **EXPECTED OUTCOME (post-fix)**: All preservation properties PASS (no regressions) and the regression guard PASSES + - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6_ + +- [x] 4. Checkpoint - Run tests and confirm the production build + - Run both new tests together: `cd frontend && CI=true npm test -- --testPathPattern compliance-list-stale-after-sidebar-edit` + - **EXPECTED OUTCOME**: exploration test PASSES (bug fixed), preservation test PASSES (no regressions) + - Run the production build to confirm it compiles and stays within the ESLint 25-warning budget: `cd frontend && npm run build` + - **EXPECTED OUTCOME**: build succeeds and the warning count remains at or below 25 (the CI `lint`/`build` stage fails above 25) — prefix any intentionally-unused variables with `_` if needed + - Ensure all tests pass; ask the user if questions arise. + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1"] }, + { "id": 1, "tasks": ["2.1", "2.2"] }, + { "id": 2, "tasks": ["2.3", "3"] }, + { "id": 3, "tasks": ["4"] } + ] +} +``` + +## Notes + +- This is a frontend-only fix. The backend list endpoint (`GET /api/compliance/items`) already returns `resolution_date` and `remediation_plan` (predecessor spec `compliance-remediation-display-fix`), and `DeviceRow` in `CompliancePage.js` (lines 883–890) already renders them correctly when present. +- The fix is intentionally minimal and mirrors the existing `onNoteAdded` → `refresh` pattern rather than introducing a new refresh mechanism, satisfying the Regression Prevention clauses (3.1–3.6). +- Both tests render `CompliancePage` with a mocked `useAuth` and a URL-routing `global.fetch` mock; the list GET mock returns the stale value on first call and the saved value on later calls so a post-save re-fetch is observable in the rendered row. +- `fast-check` is already allowed through `jest.transformIgnorePatterns` in `frontend/package.json`; generate dates as `YYYY-MM-DD` strings from integer tuples (avoid `toISOString` on shrunk values) as in the predecessor exploration test. diff --git a/.kiro/specs/compliance-metric-estimated-resolution-date/.config.kiro b/.kiro/specs/compliance-metric-estimated-resolution-date/.config.kiro new file mode 100644 index 0000000..2fe653b --- /dev/null +++ b/.kiro/specs/compliance-metric-estimated-resolution-date/.config.kiro @@ -0,0 +1 @@ +{"specId": "0962d00a-cfe6-4a5d-88c6-e9e6f220b1a0", "workflowType": "requirements-first", "specType": "feature"} diff --git a/.kiro/specs/compliance-metric-estimated-resolution-date/design.md b/.kiro/specs/compliance-metric-estimated-resolution-date/design.md new file mode 100644 index 0000000..b1b5b8d --- /dev/null +++ b/.kiro/specs/compliance-metric-estimated-resolution-date/design.md @@ -0,0 +1,244 @@ +# Design Document + +## Overview + +This feature surfaces each noncompliant metric's estimated resolution date at the top of that metric's section in the asset sidebar (`ComplianceDetailPanel.js`) on the AEO Compliance page. It is a read and display oriented change confined to the React rendering layer. No backend, API, database, or migration work is required: the per-metric `resolution_date` field is already returned by `GET /api/compliance/items/:hostname` inside `detail.metrics[]`, and it is already editable through the existing **Resolution Date** metadata field and `PATCH /api/compliance/items/:hostname/metadata`. + +Today the only place a user sees a resolution date is the editable **Resolution Date** metadata input lower in the sidebar. That input applies to the currently selected metrics and collapses to "Multiple values" whenever the selected metrics disagree, so a viewer cannot see each metric's own projected compliance date at a glance. This feature adds a read-only, per-metric date line to the top of every failing (`status === 'active'`) `MetricRow`, drawn from that same metric's `resolution_date`, formatted as `YYYY-MM-DD`, with explicit placeholders for the no-date and invalid-date cases. The existing editable metadata field and its "Multiple values" logic are left unchanged. + +The only non-trivial logic is date interpretation: deciding whether a stored `resolution_date` is a valid calendar date, normalising it to `YYYY-MM-DD`, and distinguishing "no date set" from "a value is present but it is not a valid date." That logic is extracted into a small pure helper so it can be unit- and property-tested independently of React. + +**Scope summary:** + +- Add a pure date helper (`frontend/src/utils/resolutionDate.js`) that classifies and formats a raw `resolution_date` value. +- Render a read-only estimated-resolution-date line at the top of each active `MetricRow`, above the metric description and all other fields. +- Suppress that line for resolved metrics. +- Leave the editable **Resolution Date** metadata `Section`, `computeSharedValues`, `handleSaveMetadata`, and the metadata PATCH flow untouched. + +--- + +## Architecture + +The change lives entirely in the frontend rendering layer. The data already flows from the compliance API into `ComplianceDetailPanel`; this feature only adds a derived, read-only presentation of a field that is already in hand. + +```mermaid +flowchart TD + API["GET /api/compliance/items/:hostname
detail.metrics[].resolution_date"] --> Panel["ComplianceDetailPanel
(state: detail)"] + Panel --> Split["activeMetrics = status 'active'
resolvedMetrics = status 'resolved'"] + Split --> Row["MetricRow({ metric, resolved })"] + Row --> Helper["formatResolutionDate(metric.resolution_date)
(pure helper, utils/resolutionDate.js)"] + Helper --> Decide{"resolved?"} + Decide -->|"yes"| Skip["render existing row only
(no estimated date line)"] + Decide -->|"no"| Display["render read-only date line at top of section:
set -> 'YYYY-MM-DD'
none -> 'not set' placeholder
invalid -> 'invalid' placeholder"] + Panel --> Meta["Resolution Date metadata Section
(input + computeSharedValues + handleSaveMetadata)
UNCHANGED"] + Meta --> Patch["PATCH /api/compliance/items/:hostname/metadata"] + Patch -.->|"on success: fetchDetail() re-reads detail"| Panel +``` + +**Key architectural decisions:** + +- **Pure helper for all date logic.** Parsing, validation, and formatting are isolated in `formatResolutionDate`, a pure function with no React, no I/O, and no dependence on the system clock, timezone, or locale. This keeps the rendering code declarative and makes the only branching logic in the feature directly testable. Rationale: `MetricRow` is a presentational function component; mixing date parsing into JSX would be untestable without rendering, and `new Date(...).toLocaleDateString()` would make output depend on the runner's timezone and locale, which the `YYYY-MM-DD` requirement forbids. +- **Display derives from the live `detail` state.** The date line reads `metric.resolution_date` from the same `detail.metrics[]` array that feeds the editable metadata field. Because `handleSaveMetadata` already calls `fetchDetail()` on a successful save, the displayed value updates automatically after an edit with no extra wiring. Rationale: satisfies Requirements 4.2, 4.3, and 4.5 without duplicating state or adding a second source of truth. +- **No new state, no shared/collapsed value.** The per-metric line is computed inline per `MetricRow` from that row's own metric object. It deliberately does not use `computeSharedValues`, so two metrics with different dates each show their own value and never collapse to "Multiple values." Rationale: Requirements 3.2 and 3.3. +- **Read-only by construction.** The new element is plain text (a label plus a value) with no `input`, `button`, `a`, or change handler, so it is identical and non-interactive for every role. Rationale: Requirements 5.1–5.4. Role-based gating of the existing edit field is unchanged and out of scope. + +--- + +## Components and Interfaces + +### `frontend/src/utils/resolutionDate.js` (new) + +A pure helper module, following the established `frontend/src/utils/` pattern (for example `queueGrouping.js`). It exports a single classification-and-formatting function and the placeholder strings, so both the component and the tests reference the same constants. + +```javascript +/** + * Classify and format a raw per-metric resolution_date value for display. + * + * Spec: .kiro/specs/compliance-metric-estimated-resolution-date + * Requirements: 1.1, 1.4, 1.6, 2.1 + * + * Pure and deterministic: the result depends only on `raw`. It does not read + * the system clock, timezone, or locale. Validation is strict YYYY-MM-DD with + * a real-calendar-date check (correct month lengths and leap years), which + * matches how the value is produced by the editor. + * + * @param {string|null|undefined} raw - the metric's resolution_date field + * @returns {{ state: 'set', value: string } | { state: 'none' } | { state: 'invalid' }} + * - { state: 'set', value } the value is a valid calendar date; `value` is YYYY-MM-DD + * - { state: 'none' } the value is null, undefined, empty, or whitespace-only + * - { state: 'invalid' } the value is non-empty but not a valid calendar date + */ +export function formatResolutionDate(raw) { /* ... */ } + +// Display constants (single source of truth for component + tests) +export const RESOLUTION_DATE_LABEL = 'Est. Resolution'; +export const NO_DATE_PLACEHOLDER = 'not set'; +export const INVALID_DATE_PLACEHOLDER = 'invalid date'; +``` + +**Classification rules:** + +| Input condition | Result | +|---|---| +| `null` or `undefined` | `{ state: 'none' }` | +| empty string or whitespace-only (after `trim()`) | `{ state: 'none' }` | +| trimmed value matches `^\d{4}-\d{2}-\d{2}$` and is a real calendar date | `{ state: 'set', value: '' }` | +| any other non-empty value (wrong shape, `2026-13-01`, `2026-02-30`, `not-a-date`) | `{ state: 'invalid' }` | + +**Validation approach:** the helper trims, tests against the strict `^\d{4}-\d{2}-\d{2}$` shape, then verifies the year/month/day form an actual calendar date (month `1–12`, day within that month's length, leap-year aware). It avoids `new Date(string)` for the validity decision because that constructor accepts many non-`YYYY-MM-DD` shapes and applies timezone offsets, which would make a date-only field ambiguous. Because the editable field is an ``, well-formed stored values are already `YYYY-MM-DD`; the strict check simply normalises and defends against legacy or malformed values per Requirement 1.6. + +### `MetricRow({ metric, resolved, onNavigate })` (modified) + +`MetricRow` gains a single read-only block rendered as the first child of the row's content, before the existing top row (`MetricChip` + resolved label), the metric description, the Ivanti ID row, and the highlights list. + +- The block renders only when `resolved` is falsy. For resolved metrics (`resolved === true`), `MetricRow` behaves exactly as today (Requirements 3.1, 3.4). +- For active metrics, it calls `formatResolutionDate(metric.resolution_date)` and renders, per the returned `state`: + - `set` — the label and the `YYYY-MM-DD` value (Requirements 1.1, 1.4, 1.5). + - `none` — the label and the `NO_DATE_PLACEHOLDER` text (Requirements 2.1, 4.5). + - `invalid` — the label and the `INVALID_DATE_PLACEHOLDER` text; the remainder of the row still renders (Requirement 1.6). +- The block is positioned at the top of the section using the existing row layout, so it sits above `metric_desc` and all supplementary fields (Requirement 1.2). +- Every active metric renders this block regardless of the `resolution_date` state, and `MetricRow` is still invoked once per metric in `activeMetrics.map(...)`, so all noncompliant metrics continue to render (Requirement 2.2). + +No prop signature change is required: `MetricRow` already receives the full `metric` object, which carries `resolution_date`. The `resolved` prop already distinguishes the two call sites. + +### `ComplianceDetailPanel` (unchanged) + +The container is not modified. The two `MetricRow` call sites remain: + +- `activeMetrics.map(m => )` — active metrics show the date line. +- `resolvedMetrics.map(m => )` — resolved metrics do not. + +The **Resolution Date** metadata `Section`, the `resolutionDate` state, `computeSharedValues`, `handleSaveMetadata`, the "Multiple values" placeholder logic, and the metadata PATCH call are all left exactly as they are (Requirement 4.1). Because `handleSaveMetadata` already re-runs `fetchDetail()` after a successful save, the new display reflects edits automatically (Requirements 4.2, 4.3); on a failed save the `catch` block sets `metaError` and never updates `detail`, so the previously displayed value is retained (Requirement 4.4). + +--- + +## Data Models + +No new persisted data models are introduced. The feature consumes the existing metric shape returned by the compliance API. + +**Metric object (existing, as returned in `detail.metrics[]`):** + +| Field | Type | Relevance to this feature | +|---|---|---| +| `metric_id` | string (e.g. `"2.3.6i"`) | row key; identifies the metric | +| `category` | string | drives `categoryColor` accent (existing) | +| `status` | `'active'` \| `'resolved'` | `active` = noncompliant (show date line); `resolved` = compliant (suppress) | +| `metric_desc` | string | existing description; date line renders above it | +| `resolution_date` | string \| null | source value for the estimated resolution date display | +| `remediation_plan` | string \| null | unrelated; consumed by existing metadata field | +| `seen_count`, `first_seen`, `resolved_on` | mixed | existing fields, unaffected | +| `extra` | object | existing highlights source, unaffected | + +**Helper result model (new, in-memory only):** + +```javascript +// Discriminated union returned by formatResolutionDate(raw) +{ state: 'set', value: string } // value is normalised 'YYYY-MM-DD' +{ state: 'none' } // null / empty / whitespace-only +{ state: 'invalid' } // present but not a valid calendar date +``` + +This object exists only during render and is never persisted or sent to the API. + + + +--- + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +The testable logic in this feature is concentrated in the pure helper `formatResolutionDate(raw)`. The properties below are universally quantified over its input space and over lists of metrics. Acceptance criteria that concern fixed DOM placement, label presence, role-independent rendering, conditional suppression for resolved metrics, and the save/refetch round-trip are not universal properties of varying input; they are covered by example, render, and integration tests in the Testing Strategy. + +The prework consolidated nine property-amenable criteria into five non-redundant properties: criteria 1.1 and 1.4 became Property 1; criteria 2.1 and 4.5 became Property 2; criterion 1.6 became Property 3; criterion 2.2 became Property 4; and criteria 1.3, 3.2, and 3.3 became Property 5. + +### Property 1: Valid calendar dates classify as "set" and format as YYYY-MM-DD + +*For any* string that is a valid calendar date in `YYYY-MM-DD` form (correct month range, day within the month's true length, leap-year aware), `formatResolutionDate` returns `{ state: 'set', value }` where `value` is the canonical zero-padded `YYYY-MM-DD` string (matching `^\d{4}-\d{2}-\d{2}$`) equal to the input's normalized form. + +**Validates: Requirements 1.1, 1.4** + +### Property 2: Absent values classify as "none" + +*For any* input that is `null`, `undefined`, the empty string, or a string composed entirely of whitespace, `formatResolutionDate` returns `{ state: 'none' }`. + +**Validates: Requirements 2.1, 4.5** + +### Property 3: Non-empty non-calendar-date values classify as "invalid" + +*For any* non-empty, non-whitespace-only string that is not a valid `YYYY-MM-DD` calendar date — including wrong shapes, out-of-range months or days, impossible calendar days such as `2026-02-30`, and arbitrary text — `formatResolutionDate` returns `{ state: 'invalid' }`. + +**Validates: Requirements 1.6** + +### Property 4: Classification is total over any metric list + +*For any* array of metrics whose `resolution_date` fields are drawn from all input categories (valid dates, `null`, empty, whitespace-only, and malformed strings), `formatResolutionDate` applied to each metric's field never throws and returns exactly one state in `{ 'set', 'none', 'invalid' }` for every metric, and the number of classified results equals the number of metrics. + +**Validates: Requirements 2.2** + +### Property 5: Each metric's display derives only from its own field (no collapsing) + +*For any* array of metrics — including arrays where two or more metrics have different `resolution_date` values — the derived estimated-resolution display for each metric equals `formatResolutionDate` applied to that same metric's own `resolution_date`, independent of every other metric's value, and no result is replaced by a shared or "Multiple values" representation. + +**Validates: Requirements 1.3, 3.2, 3.3** + +--- + +## Error Handling + +Because the feature is read-only rendering over data already in component state, error handling is about defensive classification rather than failure recovery. + +| Condition | Handling | +|---|---| +| `resolution_date` is `null` / `undefined` | `formatResolutionDate` returns `{ state: 'none' }`; the row shows the no-date placeholder (Requirement 2.1). | +| `resolution_date` is empty or whitespace-only | Trimmed to empty → `{ state: 'none' }`; no-date placeholder. | +| `resolution_date` is present but malformed or not a real calendar date | `{ state: 'invalid' }`; the row shows the invalid placeholder and continues rendering the description and all other fields (Requirement 1.6). The helper never throws on bad input. | +| Metric object missing `resolution_date` entirely | Accessing an absent property yields `undefined`, classified as `{ state: 'none' }`. | +| Metadata save fails (existing flow) | The existing `handleSaveMetadata` `catch` sets `metaError` and does not mutate `detail`, so the previously displayed estimated resolution date is retained and an error is shown (Requirement 4.4). No change to this behavior. | +| Detail fetch fails (existing flow) | The existing `error` state renders the panel-level error block; no metric rows (and therefore no date lines) render. Unchanged. | + +The helper is the single guard point: every branch of its discriminated union maps to a defined render path, so no metric input can crash `MetricRow` or leave a row partially rendered. + +--- + +## Testing Strategy + +This feature uses both property-based tests (for the pure helper's universal behavior) and example/render tests (for fixed DOM structure, conditional suppression, role-independence, and the save round-trip). Property-based testing is appropriate here because `formatResolutionDate` is a pure function over a large input space (arbitrary strings, null, whitespace, malformed and valid dates) with clear universal invariants. It is not appropriate for the placement, label-presence, read-only-structure, and save/refetch criteria, which are fixed behaviors verified more directly by example. + +**Property-based tests (helper — `frontend/src/utils/__tests__/resolutionDate.property.test.js`):** + +- Library: `fast-check` with Jest (`react-scripts test` / `@testing-library`), matching existing `*.property.test.js` files such as `queue-grouping.property.test.js` and `metricDefinitions.property.test.js`. +- Implemented from scratch is prohibited; use `fast-check` arbitraries. +- Each property runs a minimum of 100 iterations (`{ numRuns: 100 }` or higher, consistent with existing tests that use 100–200). +- Each test is tagged with a comment referencing its design property, in the format: **Feature: compliance-metric-estimated-resolution-date, Property {number}: {property_text}**. +- Implement each of the five correctness properties with a single property-based test: + - Property 1 — generator: valid calendar dates spanning years, all months, month-length boundaries (28/29/30/31), and leap days; assert `state === 'set'` and `value` matches `^\d{4}-\d{2}-\d{2}$` and equals the canonical form. + - Property 2 — generator: `fc.constantFrom(null, undefined, '')` combined with whitespace-only strings built from spaces, tabs, and newlines of varying length; assert `state === 'none'`. + - Property 3 — generator: non-empty strings that are not valid `YYYY-MM-DD` dates (wrong shapes, month `00`/`13`+, day `00`/`32`+, `2026-02-30`, arbitrary text), filtered to exclude any accidentally-valid date; assert `state === 'invalid'`. + - Property 4 — generator: arrays mixing all categories; assert no throw, each result's `state` is in `{ 'set', 'none', 'invalid' }`, and result count equals input length. + - Property 5 — generator: arrays of metric-like objects with independently chosen `resolution_date` values (including arrays forced to contain differing dates); assert each metric's derived result deep-equals `formatResolutionDate(metric.resolution_date)` computed in isolation, and that no result is a "Multiple values" sentinel. + +**Example and edge-case unit tests (helper):** + +- Concrete fixtures that anchor the contract and double as regression cases: `'2026-07-01'` → `{ state: 'set', value: '2026-07-01' }`; `'2026-7-1'` → `invalid` (not zero-padded); `'07/01/2026'` → `invalid`; `'2024-02-29'` → `set` (leap year); `'2023-02-29'` → `invalid`; `' '` → `none`; `null` → `none`. + +**Example / render tests (component — `MetricRow` / `ComplianceDetailPanel`):** + +- Using `@testing-library/react`: + - Placement (1.2): an active row with a valid date renders the estimated-resolution element before the description in document order. + - Label presence (1.5): the `RESOLUTION_DATE_LABEL` text appears adjacent to the value for an active row with a valid date. + - No-date and invalid placeholders (2.1, 1.6): active rows with empty and malformed dates render the respective placeholder text and still render the metric description. + - Resolved suppression (3.1, 3.4): a resolved row with a populated date renders no estimated-resolution line; a mixed list shows the line only in active rows. + - Existing editor preserved (4.1): the panel still renders the editable Resolution Date `input[type=date]`. + - Read-only structure (5.3): the date-line subtree contains no `input`, `button`, `a`, or change handler. + - Role-independence (5.1, 5.2, 5.4): rendering under viewer, editor, and admin auth contexts produces identical date-line output, and the new display introduces no editing control. + +**Integration / interaction tests (existing save flow):** + +- Successful save (4.2, 4.3, 4.5): mock a successful `PATCH` followed by `fetchDetail` returning updated metrics; assert the displayed value updates to the new date, and that clearing the field renders the no-date placeholder. +- Failed save (4.4): mock a failing `PATCH`; assert `detail` is unmodified (previously displayed date retained) and an error indication is shown. + +**Build and lint verification:** + +- After implementation, run `cd frontend && npm run build` to confirm the production build compiles and ESLint warnings stay within the 25-warning budget. Prefix any intentionally-unused variables with `_` per the project lint rules. +- Run `cd frontend && CI=true npm test` (non-watch) to execute the new property and example tests alongside the existing suite. diff --git a/.kiro/specs/compliance-metric-estimated-resolution-date/requirements.md b/.kiro/specs/compliance-metric-estimated-resolution-date/requirements.md new file mode 100644 index 0000000..f27e4ee --- /dev/null +++ b/.kiro/specs/compliance-metric-estimated-resolution-date/requirements.md @@ -0,0 +1,81 @@ +# Requirements Document + +## Introduction + +This feature surfaces the estimated resolution date for each noncompliant metric directly at the top of that metric's section in the asset sidebar on the compliance page. Today, an asset's resolution date is only visible through the editable **Resolution Date** metadata field lower in the sidebar, where it is computed as a shared value across selected metrics and collapses to "Multiple values" when metrics disagree. This feature instead displays each metric's own `resolution_date` inline with the metric, so a viewer can see the projected compliance date (for example "2026-07-01") for metrics such as `2.3.6i`, `2.3.8i`, and `Vulns_Aging` without scrolling to or interacting with the metadata editor. + +This is a read and display oriented feature. It surfaces the existing per-metric `resolution_date` value already tracked in the data model; it does not introduce new persistence, new editing surfaces, or changes to how the date is stored or computed. + +Traceability: GitLab issue #20 — "[Feature Request] Show estimated resolution date in sidebar per metric" (http://steam-gitlab.charterlab.com/steam/cve-dashboard/-/issues/20). + +## Glossary + +- **Compliance_Page**: The AEO Compliance page in the frontend that lists noncompliant assets and their metrics. +- **Asset_Sidebar**: The detail panel rendered by `ComplianceDetailPanel.js` that opens for a single asset (hostname) and lists that asset's metrics, metadata, history, and notes. +- **Metric**: A single AEO compliance check tracked for an asset, identified by a `metric_id` (for example `2.3.6i`, `2.3.8i`, `Vulns_Aging`). Each metric has a `status` of `active` (noncompliant) or `resolved` (compliant). +- **Noncompliant_Metric**: A metric with `status` equal to `active`. +- **Compliant_Metric**: A metric with `status` equal to `resolved`. +- **Estimated_Resolution_Date**: The projected date by which a metric is expected to return to compliance, stored per metric in the `resolution_date` field and derived from the per-metric remediation notes/date metadata. +- **Resolution_Date_Field**: The existing per-metric `resolution_date` data field returned by the compliance API for each metric. +- **Viewer**: A user with read-only access to all data. +- **Editor**: A user with viewer permissions plus create and update operations. +- **Admin**: A user with editor permissions plus delete, user management, and audit log access. +- **Team**: A compliance business unit. Only STEAM and ACCESS-ENG are tracked. + +## Requirements + +### Requirement 1: Display estimated resolution date per noncompliant metric + +**User Story:** As a viewer reviewing an asset's compliance posture, I want to see each noncompliant metric's estimated resolution date at the top of that metric's section, so that I can understand the projected compliance date for each metric without scrolling to or interacting with the metadata editor. + +#### Acceptance Criteria + +1. WHEN the Asset_Sidebar renders a Noncompliant_Metric whose Resolution_Date_Field contains a value parseable as a calendar date, THE Asset_Sidebar SHALL display that metric's Estimated_Resolution_Date within that metric's section. +2. WHEN the Asset_Sidebar renders a Noncompliant_Metric whose Resolution_Date_Field contains a value parseable as a calendar date, THE Asset_Sidebar SHALL position that metric's Estimated_Resolution_Date at the top of that metric's section, above the metric description and above all other fields belonging to that metric. +3. THE Asset_Sidebar SHALL display the Estimated_Resolution_Date for each Noncompliant_Metric using the value of that same metric's Resolution_Date_Field. +4. THE Asset_Sidebar SHALL display the Estimated_Resolution_Date as a calendar date in `YYYY-MM-DD` format (four-digit year, two-digit month, and two-digit day separated by hyphens). +5. WHEN the Asset_Sidebar displays a Noncompliant_Metric's Estimated_Resolution_Date, THE Asset_Sidebar SHALL render a visible text label adjacent to that value identifying it as the estimated resolution date. +6. IF a Noncompliant_Metric's Resolution_Date_Field is non-empty but does not contain a value parseable as a calendar date, THEN THE Asset_Sidebar SHALL omit the Estimated_Resolution_Date value for that metric and display a placeholder indicating that no valid estimated resolution date is available, while continuing to render the remainder of that metric's section. + +### Requirement 2: Handle metrics with no estimated resolution date + +**User Story:** As a viewer, I want a clear indication when a noncompliant metric has no estimated resolution date set, so that I can distinguish metrics with a plan from metrics that still need one. + +#### Acceptance Criteria + +1. IF a Noncompliant_Metric has a Resolution_Date_Field that is null, an empty string, or contains only whitespace, THEN THE Asset_Sidebar SHALL display, at the top of that metric's section, a text placeholder that identifies the value as the estimated resolution date and indicates that no estimated resolution date is set for that metric. +2. WHEN the Asset_Sidebar opens for an asset, THE Asset_Sidebar SHALL render every Noncompliant_Metric for that asset regardless of whether that metric's Resolution_Date_Field is populated, null, empty, or whitespace-only. + +### Requirement 3: Scope display to noncompliant metrics + +**User Story:** As a viewer, I want the estimated resolution date emphasized only for noncompliant metrics, so that the sidebar focuses attention on metrics that still require remediation. + +#### Acceptance Criteria + +1. IF a Metric is a Compliant_Metric (status = resolved), THEN THE Asset_Sidebar SHALL NOT display the top-of-section Estimated_Resolution_Date (positioned above the metric description and supplementary fields) for that metric, regardless of whether that metric has a populated Resolution_Date_Field. +2. WHEN the Asset_Sidebar renders the set of Noncompliant_Metric entries, THE Asset_Sidebar SHALL derive each metric's Estimated_Resolution_Date using only that same metric's Resolution_Date_Field value. +3. WHEN two Noncompliant_Metric entries have different Resolution_Date_Field values, THE Asset_Sidebar SHALL display each metric's own Estimated_Resolution_Date without collapsing the values into a combined or "Multiple values" representation. +4. WHEN the Asset_Sidebar renders an asset containing both Compliant_Metric and Noncompliant_Metric entries, THE Asset_Sidebar SHALL display the top-of-section Estimated_Resolution_Date only within Noncompliant_Metric sections. + +### Requirement 4: Preserve existing metadata editing behavior + +**User Story:** As an editor, I want the existing resolution date editing workflow to continue functioning unchanged, so that surfacing the per-metric date for display does not regress my ability to update it. + +#### Acceptance Criteria + +1. THE Asset_Sidebar SHALL continue to render the existing editable Resolution Date metadata field. +2. WHEN an Editor successfully saves an updated Resolution_Date_Field for a Noncompliant_Metric, THE Asset_Sidebar SHALL display the updated Estimated_Resolution_Date for that metric within 2 seconds of the successful save. +3. THE Asset_Sidebar SHALL derive the displayed Estimated_Resolution_Date from the same per-metric Resolution_Date_Field used by the existing metadata editing workflow. +4. IF an Editor's save of an updated Resolution_Date_Field fails, THEN THE Asset_Sidebar SHALL retain the previously displayed Estimated_Resolution_Date and display an error indication for the failed save. +5. WHEN an Editor clears the Resolution_Date_Field for a Noncompliant_Metric and the save succeeds, THE Asset_Sidebar SHALL display the no-date placeholder for that metric consistent with Requirement 2. + +### Requirement 5: Role-based access to the display + +**User Story:** As a security stakeholder, I want the estimated resolution date display to honor existing role-based access controls, so that read-only users can see the information without gaining editing capability. + +#### Acceptance Criteria + +1. WHILE the current user is a Viewer, THE Asset_Sidebar SHALL display each Noncompliant_Metric's Estimated_Resolution_Date using the same value and `YYYY-MM-DD` format presented to Editor and Admin users. +2. WHILE the current user is an Editor or Admin, THE Asset_Sidebar SHALL display each Noncompliant_Metric's Estimated_Resolution_Date using the same value and `YYYY-MM-DD` format presented to a Viewer. +3. THE Asset_Sidebar SHALL present the top-of-section Estimated_Resolution_Date display as read-only content that contains no input field, button, link, or other interactive control capable of modifying a Resolution_Date_Field, regardless of the current user's role. +4. WHILE the current user is a Viewer, THE Asset_Sidebar SHALL present no enabled control that creates or updates a Resolution_Date_Field value. diff --git a/.kiro/specs/compliance-metric-estimated-resolution-date/tasks.md b/.kiro/specs/compliance-metric-estimated-resolution-date/tasks.md new file mode 100644 index 0000000..15214ae --- /dev/null +++ b/.kiro/specs/compliance-metric-estimated-resolution-date/tasks.md @@ -0,0 +1,138 @@ +# Implementation Plan: Compliance Metric Estimated Resolution Date + +## Overview + +This plan implements a read-only, per-metric estimated resolution date line at the top of each noncompliant metric's section in the asset sidebar (`ComplianceDetailPanel.js`). The work is frontend-only (React 19, plain JavaScript) and is built test-first: the pure date helper and its property-based tests come first, followed by the `MetricRow` rendering change and its render tests, then build and test verification. + +All date logic is isolated in a pure helper (`frontend/src/utils/resolutionDate.js`) so it can be property- and example-tested independently of React. The component change adds a single read-only block to `MetricRow` and changes no prop signatures. The editable Resolution Date metadata `Section`, `computeSharedValues`, `handleSaveMetadata`, and the metadata PATCH flow are left unchanged. + +## Tasks + +- [x] 1. Create the pure resolution-date helper module + - [x] 1.1 Implement `formatResolutionDate` and display constants in `frontend/src/utils/resolutionDate.js` + - Create `frontend/src/utils/resolutionDate.js` following the existing `frontend/src/utils/` pure-module pattern (for example `queueGrouping.js`) + - Export `formatResolutionDate(raw)` returning the discriminated union `{ state: 'set', value } | { state: 'none' } | { state: 'invalid' }` + - Return `{ state: 'none' }` for `null`, `undefined`, empty string, or whitespace-only (after `trim()`) + - Return `{ state: 'set', value }` only when the trimmed value matches `^\d{4}-\d{2}-\d{2}$` AND is a real calendar date (month `1–12`, day within the month's true length, leap-year aware); `value` is the normalized `YYYY-MM-DD` string + - Return `{ state: 'invalid' }` for any other non-empty value (wrong shape, `2026-13-01`, `2026-02-30`, arbitrary text) + - Keep the function pure and deterministic: no React, no I/O, no system clock/timezone/locale dependency, and do NOT use `new Date(string)` for the validity decision + - Export `RESOLUTION_DATE_LABEL = 'Est. Resolution'`, `NO_DATE_PLACEHOLDER = 'not set'`, and `INVALID_DATE_PLACEHOLDER = 'invalid date'` as the single source of truth for component and tests + - _Requirements: 1.1, 1.4, 1.6, 2.1_ + + - [x]* 1.2 Write property test for valid calendar date classification and formatting + - File: `frontend/src/utils/__tests__/resolutionDate.property.test.js` + - Use `fast-check` (v4) with Jest (`react-scripts test`); do not hand-roll generators + - **Property 1: Valid calendar dates classify as "set" and format as YYYY-MM-DD** + - Generator: valid calendar dates spanning years, all months, month-length boundaries (28/29/30/31), and leap days; assert `state === 'set'`, `value` matches `^\d{4}-\d{2}-\d{2}$` and equals the canonical normalized form + - Run a minimum of 100 iterations (`{ numRuns: 100 }` or higher) + - Tag the test: `// Feature: compliance-metric-estimated-resolution-date, Property 1: Valid calendar dates classify as "set" and format as YYYY-MM-DD` + - **Validates: Requirements 1.1, 1.4** + + - [x]* 1.3 Write property test for absent values classifying as "none" + - File: `frontend/src/utils/__tests__/resolutionDate.property.test.js` + - **Property 2: Absent values classify as "none"** + - Generator: `fc.constantFrom(null, undefined, '')` combined with whitespace-only strings built from spaces, tabs, and newlines of varying length; assert `state === 'none'` + - Minimum 100 iterations + - Tag the test: `// Feature: compliance-metric-estimated-resolution-date, Property 2: Absent values classify as "none"` + - **Validates: Requirements 2.1, 4.5** + + - [x]* 1.4 Write property test for non-calendar-date values classifying as "invalid" + - File: `frontend/src/utils/__tests__/resolutionDate.property.test.js` + - **Property 3: Non-empty non-calendar-date values classify as "invalid"** + - Generator: non-empty, non-whitespace-only strings that are not valid `YYYY-MM-DD` dates (wrong shapes, month `00`/`13`+, day `00`/`32`+, `2026-02-30`, arbitrary text), filtered to exclude any accidentally-valid date; assert `state === 'invalid'` + - Minimum 100 iterations + - Tag the test: `// Feature: compliance-metric-estimated-resolution-date, Property 3: Non-empty non-calendar-date values classify as "invalid"` + - **Validates: Requirements 1.6** + + - [x]* 1.5 Write property test for total classification over any metric list + - File: `frontend/src/utils/__tests__/resolutionDate.property.test.js` + - **Property 4: Classification is total over any metric list** + - Generator: arrays mixing all input categories (valid dates, `null`, empty, whitespace-only, malformed); assert `formatResolutionDate` never throws, each result's `state` is in `{ 'set', 'none', 'invalid' }`, and the result count equals the input length + - Minimum 100 iterations + - Tag the test: `// Feature: compliance-metric-estimated-resolution-date, Property 4: Classification is total over any metric list` + - **Validates: Requirements 2.2** + + - [x]* 1.6 Write property test for per-metric independence (no collapsing) + - File: `frontend/src/utils/__tests__/resolutionDate.property.test.js` + - **Property 5: Each metric's display derives only from its own field (no collapsing)** + - Generator: arrays of metric-like objects with independently chosen `resolution_date` values (including arrays forced to contain differing dates); assert each metric's derived result deep-equals `formatResolutionDate` applied to that same metric's own `resolution_date` computed in isolation, and that no result is replaced by a shared/"Multiple values" sentinel + - Minimum 100 iterations + - Tag the test: `// Feature: compliance-metric-estimated-resolution-date, Property 5: Each metric's display derives only from its own field (no collapsing)` + - **Validates: Requirements 1.3, 3.2, 3.3** + + - [x]* 1.7 Write example and edge-case unit tests for the helper + - File: `frontend/src/utils/__tests__/resolutionDate.test.js` + - Concrete fixtures: `'2026-07-01'` → `{ state: 'set', value: '2026-07-01' }`; `'2026-7-1'` → `invalid` (not zero-padded); `'07/01/2026'` → `invalid`; `'2024-02-29'` → `set` (leap year); `'2023-02-29'` → `invalid`; `' '` → `none`; `null` → `none` + - _Requirements: 1.1, 1.4, 1.6, 2.1_ + +- [x] 2. Checkpoint - Ensure helper tests pass + - Run `cd frontend && CI=true npm test -- resolutionDate` to confirm the helper and its property tests pass + - Ensure all tests pass, ask the user if questions arise. + +- [x] 3. Render the estimated-resolution-date line in `MetricRow` + - [x] 3.1 Add the read-only date block to `MetricRow` in `frontend/src/components/pages/ComplianceDetailPanel.js` + - Import `formatResolutionDate`, `RESOLUTION_DATE_LABEL`, `NO_DATE_PLACEHOLDER`, and `INVALID_DATE_PLACEHOLDER` from `../../utils/resolutionDate` + - Render the new block as the first child of the row content, above the existing top row (`MetricChip`), the metric description, the Ivanti ID row, and the highlights list (Requirement 1.2) + - Render the block only when `resolved` is falsy; for `resolved === true`, `MetricRow` must behave exactly as today (Requirements 3.1, 3.4) + - For active metrics, call `formatResolutionDate(metric.resolution_date)` and render by state: `set` → label + `YYYY-MM-DD` value; `none` → label + `NO_DATE_PLACEHOLDER`; `invalid` → label + `INVALID_DATE_PLACEHOLDER`, with the rest of the row still rendering + - Render `RESOLUTION_DATE_LABEL` as a visible text label adjacent to the value/placeholder (Requirement 1.5) + - Use plain text only: no `input`, `button`, `a`, or change handler in the new subtree (Requirements 5.3, 5.4) + - Do NOT change the `MetricRow` prop signature; read `resolution_date` from the existing `metric` object + - Leave `computeSharedValues`, `handleSaveMetadata`, the editable Resolution Date metadata `Section`, and the metadata PATCH flow unchanged (Requirement 4.1) + - Prefix any intentionally-unused variables with `_` per the project lint rules + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 2.1, 2.2, 3.1, 3.2, 3.3, 3.4, 4.1, 5.3, 5.4_ + + - [x]* 3.2 Write render tests for placement, labels, and placeholders + - File: `frontend/src/components/pages/__tests__/ComplianceDetailPanel.metricRow.test.js` using `@testing-library/react` + - Placement (1.2): an active row with a valid date renders the estimated-resolution element before the description in document order + - Label presence (1.5): `RESOLUTION_DATE_LABEL` text appears adjacent to the value for an active row with a valid date + - Set value (1.1, 1.4): an active row with `'2026-07-01'` renders `2026-07-01` + - No-date placeholder (2.1, 4.5): active rows with `null`/empty/whitespace render `NO_DATE_PLACEHOLDER` and still render the metric description + - Invalid placeholder (1.6): an active row with a malformed date renders `INVALID_DATE_PLACEHOLDER` and still renders the metric description + - _Requirements: 1.1, 1.2, 1.4, 1.5, 1.6, 2.1, 4.5_ + + - [x]* 3.3 Write render tests for resolved suppression, read-only structure, and role-independence + - File: `frontend/src/components/pages/__tests__/ComplianceDetailPanel.metricRow.test.js` + - Resolved suppression (3.1, 3.4): a resolved row with a populated date renders no estimated-resolution line; a mixed list shows the line only in active rows + - Read-only structure (5.3): the date-line subtree contains no `input`, `button`, `a`, or change handler + - Role-independence (5.1, 5.2, 5.4): rendering under viewer, editor, and admin auth contexts produces identical date-line output and introduces no editing control + - Existing editor preserved (4.1): the panel still renders the editable Resolution Date `input[type=date]` + - _Requirements: 3.1, 3.4, 4.1, 5.1, 5.2, 5.3, 5.4_ + + - [x]* 3.4 Write interaction tests for the existing save round-trip + - File: `frontend/src/components/pages/__tests__/ComplianceDetailPanel.metricRow.test.js` + - Successful save (4.2, 4.3, 4.5): mock a successful `PATCH` followed by `fetchDetail` returning updated metrics; assert the displayed value updates to the new date, and that clearing the field renders the no-date placeholder + - Failed save (4.4): mock a failing `PATCH`; assert `detail` is unmodified (previously displayed date retained) and an error indication is shown + - _Requirements: 4.2, 4.3, 4.4, 4.5_ + +- [ ] 4. Final checkpoint - Build and test verification + - [x] 4.1 Verify production build and ESLint budget + - Run `cd frontend && npm run build` and confirm the build compiles + - Confirm ESLint warnings stay within the 25-warning budget; prefix any intentionally-unused variables with `_` + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 2.1, 2.2, 3.1, 3.2, 3.3, 3.4, 4.1_ + + - [x] 4.2 Run the full test suite + - Run `cd frontend && CI=true npm test` (non-watch) to execute the new property and render tests alongside the existing suite + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional test sub-tasks and can be skipped for a faster MVP, but they validate the five correctness properties and the fixed-DOM acceptance criteria and are recommended. +- Each task references specific requirements (and, for property tests, the design property number) for traceability. +- Test-driven ordering: the pure helper and its property tests (task 1) come before the component change (task 3) so the only branching logic is validated first. +- Property tests use `fast-check` with a minimum of 100 iterations and are tagged with their feature and property number per the design's Testing Strategy. +- Checkpoints (tasks 2 and 4) ensure incremental validation and a clean production build within the lint budget. + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1"] }, + { "id": 1, "tasks": ["1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "3.1"] }, + { "id": 2, "tasks": ["3.2", "3.3", "3.4"] }, + { "id": 3, "tasks": ["4.1"] }, + { "id": 4, "tasks": ["4.2"] } + ] +} +``` diff --git a/.kiro/specs/compliance-remediation-display-fix/.config.kiro b/.kiro/specs/compliance-remediation-display-fix/.config.kiro new file mode 100644 index 0000000..991e0de --- /dev/null +++ b/.kiro/specs/compliance-remediation-display-fix/.config.kiro @@ -0,0 +1 @@ +{"specId": "bfe6410f-227a-4c38-8e56-e6e6fe3bab50", "workflowType": "requirements-first", "specType": "bugfix"} diff --git a/.kiro/specs/compliance-remediation-display-fix/bugfix.md b/.kiro/specs/compliance-remediation-display-fix/bugfix.md new file mode 100644 index 0000000..652a057 --- /dev/null +++ b/.kiro/specs/compliance-remediation-display-fix/bugfix.md @@ -0,0 +1,41 @@ +# Bugfix Requirements Document + +## Introduction + +The compliance main table (CompliancePage.js) always displays "—" for the Resolution Date and Remediation Plan columns, even after a user edits these fields in the ComplianceDetailPanel. The detail panel correctly saves and reads per-device metadata via `PATCH /api/compliance/items/:hostname/metadata` and `GET /api/compliance/items/:hostname`, but the list endpoint `GET /api/compliance/items?team=X&status=Y` omits `resolution_date` and `remediation_plan` from its SELECT query, and the `groupByHostname()` helper does not propagate these fields into the device objects returned to the frontend. + +## Bug Analysis + +### Current Behavior (Defect) + +1.1 WHEN the main compliance table fetches devices via `GET /api/compliance/items?team=X&status=Y` THEN the system does not include `resolution_date` or `remediation_plan` in the SQL SELECT clause, so these columns are absent from the query result rows. + +1.2 WHEN `groupByHostname()` constructs per-device objects from the query rows THEN the system does not include `resolution_date` or `remediation_plan` fields in the device objects, resulting in `undefined` values on the frontend. + +1.3 WHEN the frontend renders the Resolution Date column using `device.resolution_date || '—'` THEN the system always displays "—" because the field is `undefined`. + +1.4 WHEN the frontend renders the Remediation Plan column using `truncateText(device.remediation_plan)` THEN the system always displays "—" because the field is `undefined`. + +### Expected Behavior (Correct) + +2.1 WHEN the main compliance table fetches devices via `GET /api/compliance/items?team=X&status=Y` THEN the system SHALL include `ci.resolution_date` and `ci.remediation_plan` in the SQL SELECT clause so these values are present in the query result rows. + +2.2 WHEN `groupByHostname()` constructs per-device objects from the query rows THEN the system SHALL include `resolution_date` and `remediation_plan` fields in each device object, aggregated as the first non-null value encountered across the hostname's metric rows (or null if all are null). + +2.3 WHEN the frontend renders the Resolution Date column and `device.resolution_date` contains a valid date string THEN the system SHALL display that date value instead of "—". + +2.4 WHEN the frontend renders the Remediation Plan column and `device.remediation_plan` contains a non-empty string THEN the system SHALL display the truncated plan text instead of "—". + +### Unchanged Behavior (Regression Prevention) + +3.1 WHEN the detail panel saves metadata via `PATCH /api/compliance/items/:hostname/metadata` THEN the system SHALL CONTINUE TO persist `resolution_date` and `remediation_plan` to the `compliance_items` table correctly. + +3.2 WHEN the detail panel reads a single device via `GET /api/compliance/items/:hostname` THEN the system SHALL CONTINUE TO return `resolution_date` and `remediation_plan` for that device's metrics. + +3.3 WHEN `groupByHostname()` processes rows for devices that have no `resolution_date` or `remediation_plan` set (all null) THEN the system SHALL CONTINUE TO return `null` for those fields, and the frontend SHALL CONTINUE TO display "—". + +3.4 WHEN the VCL reporting endpoints read `resolution_date` from `compliance_items` for burndown forecasts THEN the system SHALL CONTINUE TO function identically, as this fix only affects the list endpoint's SELECT and the grouping helper. + +3.5 WHEN the main compliance table displays hostname, IP address, device type, failing metrics, and seen count THEN the system SHALL CONTINUE TO display these fields correctly without any change in behavior. + +3.6 WHEN per-metric metadata scoping is used (different resolution_date values per metric on the same hostname) THEN the system SHALL CONTINUE TO store per-metric values correctly; the list endpoint SHALL aggregate by selecting the first non-null value across the hostname's metrics for display purposes. diff --git a/.kiro/specs/compliance-remediation-display-fix/tasks.md b/.kiro/specs/compliance-remediation-display-fix/tasks.md new file mode 100644 index 0000000..b44c998 --- /dev/null +++ b/.kiro/specs/compliance-remediation-display-fix/tasks.md @@ -0,0 +1,111 @@ +# Implementation Plan: Compliance Remediation Display Fix + +## Overview + +Fix the compliance main table to display Resolution Date and Remediation Plan values. The `GET /items` endpoint omits `ci.resolution_date` and `ci.remediation_plan` from its SELECT clause, and `groupByHostname()` does not propagate these fields into device objects. The frontend already renders them correctly when present. This is a backend-only fix: add the columns to the SELECT and update the grouping helper to aggregate them as first-non-null across each hostname's metric rows. + +## Tasks + +- [x] 1. Write bug condition exploration test + - **Property 1: Bug Condition** - Resolution Date and Remediation Plan Missing from GET /items Response + - **CRITICAL**: This test MUST FAIL on unfixed code - failure confirms the bug exists + - **DO NOT attempt to fix the test or the code when it fails** + - **NOTE**: This test encodes the expected behavior - it will validate the fix when it passes after implementation + - **GOAL**: Surface counterexamples that demonstrate the bug exists (resolution_date and remediation_plan are undefined in grouped device objects) + - **Scoped PBT Approach**: Generate compliance_items rows where resolution_date and/or remediation_plan are non-null, pass them through groupByHostname(), and assert the output device objects contain those fields + - **Bug Condition**: `isBugCondition(row) = row.resolution_date != null OR row.remediation_plan != null` — any row with metadata set will lose it through groupByHostname() + - **Test file**: `backend/__tests__/compliance-remediation-display-fix.exploration.property.test.js` + - **Test structure**: + - Use fast-check to generate arbitrary hostname strings, date strings for resolution_date, and non-empty strings for remediation_plan + - Construct rows array with generated values present in resolution_date and remediation_plan columns + - Call groupByHostname() with those rows + - Assert: `device.resolution_date` equals the first non-null resolution_date across the hostname's rows + - Assert: `device.remediation_plan` equals the first non-null remediation_plan across the hostname's rows + - Run test on UNFIXED code + - **EXPECTED OUTCOME**: Test FAILS because groupByHostname() does not propagate resolution_date or remediation_plan (they are undefined on the output objects) + - Document counterexamples found (e.g., "groupByHostname([{hostname:'H1', resolution_date:'2025-06-01', ...}]) returns device with resolution_date === undefined") + - Mark task complete when test is written, run, and failure is documented + - _Requirements: 1.1, 1.2, 2.2_ + +- [x] 2. Write preservation property tests (BEFORE implementing fix) + - **Property 2: Preservation** - Existing groupByHostname Fields Unchanged + - **IMPORTANT**: Follow observation-first methodology + - **Test file**: `backend/__tests__/compliance-remediation-display-fix.preservation.property.test.js` + - Observe: groupByHostname() correctly aggregates hostname, ip_address, device_type, team, status, failing_metrics, seen_count, first_seen, last_seen, resolved_on, has_notes on unfixed code + - Observe: groupByHostname() deduplicates metrics by metric_id (no duplicate failing_metrics entries) + - Observe: groupByHostname() picks the maximum seen_count across rows for the same hostname + - Observe: groupByHostname() picks the earliest first_seen and latest last_seen across rows + - **Test structure**: + - Use fast-check to generate arrays of compliance rows with varying hostnames, metric_ids, seen_counts, first_seen/last_seen dates + - All generated rows have resolution_date = null and remediation_plan = null (non-bug-condition inputs) + - Call groupByHostname() and assert: + - Property: each device.hostname appears exactly once in output + - Property: device.failing_metrics contains no duplicate metric_ids + - Property: device.seen_count >= every row's seen_count for that hostname + - Property: device.first_seen <= every row's first_seen for that hostname + - Property: device.last_seen >= every row's last_seen for that hostname + - Property: device.has_notes matches noteHostnames membership + - Verify test passes on UNFIXED code + - **EXPECTED OUTCOME**: Tests PASS (confirms baseline behavior to preserve) + - Mark task complete when tests are written, run, and passing on unfixed code + - _Requirements: 3.3, 3.5_ + +- [x] 3. Fix for resolution_date and remediation_plan missing from compliance list endpoint + + - [x] 3.1 Add resolution_date and remediation_plan to the GET /items SQL SELECT clause + - In `backend/routes/compliance.js` around line 601, add `ci.resolution_date, ci.remediation_plan` to the SELECT column list + - The columns already exist in the `compliance_items` table — they just need to be selected + - _Bug_Condition: isBugCondition(row) = row.resolution_date != null OR row.remediation_plan != null — these columns are not selected so they are always absent_ + - _Expected_Behavior: The query result rows include resolution_date and remediation_plan values from the database_ + - _Preservation: All other selected columns remain unchanged; query WHERE/ORDER BY clauses unchanged_ + - _Requirements: 1.1, 2.1_ + + - [x] 3.2 Update groupByHostname() to propagate resolution_date and remediation_plan + - In `backend/routes/compliance.js` around line 220, add `resolution_date: null, remediation_plan: null` to the initial device object + - In the row iteration loop, aggregate as first-non-null: `if (row.resolution_date && !dev.resolution_date) dev.resolution_date = row.resolution_date;` + - Same for remediation_plan: `if (row.remediation_plan && !dev.remediation_plan) dev.remediation_plan = row.remediation_plan;` + - This matches the "first non-null value across the hostname's metric rows" aggregation strategy from the requirements + - _Bug_Condition: isBugCondition(device) = device has rows with non-null resolution_date or remediation_plan but groupByHostname() drops them_ + - _Expected_Behavior: device.resolution_date = first non-null resolution_date across hostname's rows (or null if all null); same for remediation_plan_ + - _Preservation: All other device fields (hostname, ip_address, device_type, team, status, failing_metrics, seen_count, first_seen, last_seen, resolved_on, has_notes) remain unchanged_ + - _Requirements: 1.2, 2.2, 3.3, 3.5, 3.6_ + + - [x] 3.3 Verify bug condition exploration test now passes + - **Property 1: Expected Behavior** - Resolution Date and Remediation Plan Present in GET /items Response + - **IMPORTANT**: Re-run the SAME test from task 1 — do NOT write a new test + - The test from task 1 encodes the expected behavior (resolution_date and remediation_plan propagated through groupByHostname) + - When this test passes, it confirms the expected behavior is satisfied + - Run: `npx jest backend/__tests__/compliance-remediation-display-fix.exploration.property.test.js --run` + - **EXPECTED OUTCOME**: Test PASSES (confirms bug is fixed) + - _Requirements: 2.1, 2.2, 2.3, 2.4_ + + - [x] 3.4 Verify preservation tests still pass + - **Property 2: Preservation** - Existing groupByHostname Fields Unchanged + - **IMPORTANT**: Re-run the SAME tests from task 2 — do NOT write new tests + - Run: `npx jest backend/__tests__/compliance-remediation-display-fix.preservation.property.test.js --run` + - **EXPECTED OUTCOME**: Tests PASS (confirms no regressions — existing field aggregation unchanged) + - Confirm all tests still pass after fix (no regressions) + +- [x] 4. Checkpoint - Ensure all tests pass + - Run full test suite: `npx jest backend/__tests__/compliance-remediation-display-fix --run` + - Verify no other compliance tests regressed: `npx jest backend/__tests__/compliance --run` + - Ensure all tests pass, ask the user if questions arise. + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1", "2"] }, + { "id": 1, "tasks": ["3.1", "3.2"] }, + { "id": 2, "tasks": ["3.3", "3.4"] }, + { "id": 3, "tasks": ["4"] } + ] +} +``` + +## Notes + +- This is a backend-only fix. The frontend (CompliancePage.js lines 883-890) already renders resolution_date and remediation_plan correctly when the data is present. +- The `groupByHostname()` function is a pure helper (no DB access), making it ideal for direct property-based testing without HTTP mocking. +- The "first non-null" aggregation strategy handles per-metric metadata scoping: different metrics on the same hostname may have different resolution_date values, but the list view shows the first encountered non-null for display purposes. diff --git a/.kiro/specs/flexible-jira-ticket-creation/.config.kiro b/.kiro/specs/flexible-jira-ticket-creation/.config.kiro new file mode 100644 index 0000000..7af275d --- /dev/null +++ b/.kiro/specs/flexible-jira-ticket-creation/.config.kiro @@ -0,0 +1 @@ +{"specId": "244dcf03-8538-4bc6-b47a-dde02022e4ab", "workflowType": "requirements-first", "specType": "feature"} diff --git a/.kiro/specs/flexible-jira-ticket-creation/design.md b/.kiro/specs/flexible-jira-ticket-creation/design.md new file mode 100644 index 0000000..877dc6d --- /dev/null +++ b/.kiro/specs/flexible-jira-ticket-creation/design.md @@ -0,0 +1,370 @@ +# Design Document: Flexible Jira Ticket Creation + +## Overview + +This feature decouples Jira ticket creation from the CVE-only workflow by making CVE ID and Vendor optional, adding a `source_context` field to track ticket origin, and exposing "Create Jira Ticket" actions from the Ivanti queue and Archer detail views. The changes span a database migration, backend validation updates, and frontend modal/list enhancements. + +The design preserves backward compatibility — existing tickets retain their CVE ID and Vendor values, and the new `source_context` column defaults to `manual` for legacy rows. + +## Architecture + +```mermaid +flowchart TD + subgraph Frontend + JiraPage[JiraPage.js
Creation Modal + Ticket List] + IvantiQueue[Ivanti Queue Page
Create Jira Ticket action] + ArcherView[App.js Archer Detail
Create Jira Ticket action] + end + + subgraph Backend + Route[POST /api/jira-tickets/create-in-jira] + Validation[Input Validation Layer] + JiraAPI[jiraApi.js → Jira REST API] + DB[(PostgreSQL
jira_tickets table)] + end + + IvantiQueue -->|opens modal with pre-populated fields| JiraPage + ArcherView -->|opens modal with pre-populated fields| JiraPage + JiraPage -->|POST payload| Route + Route --> Validation + Validation -->|valid| JiraAPI + JiraAPI -->|issue created| DB + Validation -->|invalid| Route + Route -->|400/201| JiraPage +``` + +The modal component is reused across all three entry points (Jira page, Ivanti queue, Archer detail). Each entry point passes pre-populated field values and a locked `source_context` value to the modal. + +## Components and Interfaces + +### Backend: `POST /api/jira-tickets/create-in-jira` (Updated) + +**Request Body:** + +| Field | Type | Required | Validation | +|---|---|---|---| +| `summary` | string | Yes | Non-empty after trim, max 255 chars | +| `cve_id` | string \| null | No | If present and non-empty, must match `CVE-YYYY-NNNN+`. Empty string treated as null. | +| `vendor` | string \| null | No | If present and non-empty, 1–200 chars after trim. Empty string treated as null. | +| `source_context` | string \| null | No | Must be one of: `cve`, `archer`, `ivanti_queue`, `email`, `manual`. Defaults to `manual` if absent. | +| `description` | string \| null | No | Free text, passed to Jira issue body | +| `project_key` | string \| null | No | Falls back to `JIRA_PROJECT_KEY` env var | +| `issue_type` | string \| null | No | Falls back to `JIRA_ISSUE_TYPE` env var | + +**Response (201):** + +```json +{ + "id": 42, + "ticket_key": "VULN-789", + "jira_url": "https://jira.example.com/browse/VULN-789", + "source_context": "ivanti_queue", + "message": "Jira issue created and linked successfully" +} +``` + +**Validation Logic (pseudocode):** + +``` +if cve_id is empty string → treat as null +if cve_id is non-null and non-empty: + if not matching /^CVE-\d{4}-\d{4,}$/ → 400 +if vendor is empty string or whitespace-only → treat as null +if vendor is non-null and non-empty: + trim whitespace + if length > 200 → 400 +if source_context is present: + if not in ALLOWED_SET → 400 +else: + source_context = 'manual' +if summary is empty or > 255 chars → 400 +``` + +### Backend: `PUT /api/jira-tickets/:id` (Updated) + +The update endpoint rejects any attempt to change `source_context`: + +``` +if request body contains source_context field → 400 "source_context is immutable after creation" +``` + +### Backend: `GET /api/jira-tickets` (Updated) + +Response now includes `source_context` on each ticket object. No other changes to the GET endpoint. + +### Frontend: Creation Modal (Updated) + +The existing `showCreateJira` modal in `JiraPage.js` is updated: + +- CVE ID label: `"CVE ID (optional)"` with placeholder `"e.g. CVE-2024-12345"` +- Vendor label: `"Vendor (optional)"` with placeholder `"e.g. Microsoft"` +- New Source Context dropdown: options map display labels to API values +- Summary remains required with inline validation error on empty submit +- Form state extended with `source_context` field + +**Source Context Options Mapping:** + +| Display Label | API Value | +|---|---| +| CVE | `cve` | +| Archer Request | `archer` | +| Ivanti Queue | `ivanti_queue` | +| Email | `email` | +| Manual | `manual` | + +When opened from Ivanti queue or Archer context, the source_context dropdown is pre-selected and read-only. + +### Frontend: Ivanti Queue Integration + +A "Create Jira Ticket" button is added to each queue item row (or action menu). When activated: + +1. Opens the Creation Modal +2. Pre-populates `summary` with `finding_title` (truncated to 255 chars) +3. Pre-populates `cve_id` with first element of `cves_json` array (if non-empty) +4. Pre-populates `vendor` with queue item's `vendor` value (if present) +5. Sets `source_context` to `ivanti_queue` (locked) + +### Frontend: Archer Detail Integration + +A "Create Jira Ticket" button is added to the Archer ticket detail view (visible to editor/admin roles). When activated: + +1. Opens the Creation Modal +2. Pre-populates `summary` with `exc_number` (e.g., "EXC-1234") +3. Pre-populates `cve_id` with Archer ticket's `cve_id` (if present) +4. Pre-populates `vendor` with Archer ticket's `vendor` (if present) +5. Sets `source_context` to `archer` (locked) + +### Frontend: Ticket List Updates + +The ticket list table gains: + +- A new "Source" column between Vendor and Summary, displaying a color-coded badge +- A source_context dropdown filter (matching the existing status filter pattern) +- Search includes `source_context` in the filterable fields + +**Badge Color Mapping:** + +| Source Context | Badge Color | +|---|---| +| `cve` | `#0EA5E9` (blue) | +| `archer` | `#8B5CF6` (purple) | +| `ivanti_queue` | `#F59E0B` (amber) | +| `email` | `#10B981` (green) | +| `manual` | `#94A3B8` (gray) | + +Null/empty source_context displays as "CVE" badge (blue) for backward compatibility with legacy tickets. + +## Data Models + +### `jira_tickets` Table (Updated Schema) + +```sql +CREATE TABLE IF NOT EXISTS jira_tickets ( + id SERIAL PRIMARY KEY, + cve_id TEXT, -- Changed: NULL allowed + vendor TEXT, -- Changed: NULL allowed + ticket_key TEXT NOT NULL, + url TEXT, + summary TEXT, + status TEXT DEFAULT 'Open' CHECK (status IN ('Open', 'In Progress', 'Closed')), + source_context TEXT DEFAULT 'manual', -- New column + jira_id TEXT, + jira_status TEXT, + last_synced_at TIMESTAMPTZ, + created_by INTEGER REFERENCES users(id), + created_at TIMESTAMPTZ DEFAULT NOW(), + updated_at TIMESTAMPTZ DEFAULT NOW() +); +``` + +**Constraints:** +- `source_context` has a CHECK constraint limiting values to `('cve', 'archer', 'ivanti_queue', 'email', 'manual')` +- `cve_id` allows NULL (NOT NULL constraint dropped) +- `vendor` allows NULL (NOT NULL constraint dropped) + +### Migration: `add_flexible_jira_ticket_creation.js` + +```javascript +const pool = require('../db'); + +async function run() { + console.log('Starting flexible Jira ticket creation migration...'); + + // Verify table exists + const { rows } = await pool.query(` + SELECT 1 FROM information_schema.tables + WHERE table_name = 'jira_tickets' + `); + if (rows.length === 0) { + console.error('✗ jira_tickets table does not exist. Cannot proceed.'); + process.exit(1); + } + + // Drop NOT NULL on cve_id + await pool.query(`ALTER TABLE jira_tickets ALTER COLUMN cve_id DROP NOT NULL`); + console.log('✓ cve_id NOT NULL constraint dropped (or was already nullable)'); + + // Drop NOT NULL on vendor + await pool.query(`ALTER TABLE jira_tickets ALTER COLUMN vendor DROP NOT NULL`); + console.log('✓ vendor NOT NULL constraint dropped (or was already nullable)'); + + // Add source_context column with default + await pool.query(` + ALTER TABLE jira_tickets + ADD COLUMN IF NOT EXISTS source_context TEXT DEFAULT 'manual' + `); + console.log('✓ source_context column added (or already exists)'); + + // Add CHECK constraint (idempotent via IF NOT EXISTS pattern) + await pool.query(` + DO $$ + BEGIN + IF NOT EXISTS ( + SELECT 1 FROM pg_constraint WHERE conname = 'jira_tickets_source_context_check' + ) THEN + ALTER TABLE jira_tickets + ADD CONSTRAINT jira_tickets_source_context_check + CHECK (source_context IN ('cve', 'archer', 'ivanti_queue', 'email', 'manual')); + END IF; + END $$; + `); + console.log('✓ source_context CHECK constraint added (or already exists)'); + + // Backfill existing rows (DEFAULT handles this, but explicit for clarity) + await pool.query(` + UPDATE jira_tickets SET source_context = 'manual' WHERE source_context IS NULL + `); + console.log('✓ Existing rows backfilled with source_context = manual'); + + // Index for filtering + await pool.query(` + CREATE INDEX IF NOT EXISTS idx_jira_tickets_source_context + ON jira_tickets(source_context) + `); + console.log('✓ source_context index created (or already exists)'); + + console.log('Migration complete.'); + process.exit(0); +} + +run().catch(err => { + console.error('Migration failed:', err.message); + process.exit(1); +}); +``` + +**Idempotency:** `ALTER COLUMN DROP NOT NULL` is safe to run multiple times (no-op if already nullable). `ADD COLUMN IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS` are inherently idempotent. The CHECK constraint uses a `DO $$ ... IF NOT EXISTS` guard. + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +### Property 1: CVE ID validation and storage + +*For any* create-ticket request payload, if `cve_id` is absent, null, or an empty string, the service SHALL accept the request and store NULL for `cve_id`; if `cve_id` is a non-empty string matching `CVE-YYYY-NNNN+`, the service SHALL store that exact value. + +**Validates: Requirements 1.1, 1.2, 1.4** + +### Property 2: Invalid CVE ID rejection + +*For any* non-empty string that does not match the pattern `CVE-YYYY-NNNN+` (four-digit year, four or more digit sequence), the service SHALL reject the create-ticket request with HTTP 400. + +**Validates: Requirements 1.3** + +### Property 3: Vendor validation and storage + +*For any* create-ticket request payload, if `vendor` is absent, null, empty, or whitespace-only, the service SHALL store NULL; if `vendor` is a non-empty string of 1–200 characters after trimming, the service SHALL store the trimmed value. + +**Validates: Requirements 2.1, 2.2** + +### Property 4: Over-length vendor rejection + +*For any* string that, after trimming whitespace, exceeds 200 characters, the service SHALL reject the create-ticket request with a validation error. + +**Validates: Requirements 2.3** + +### Property 5: Invalid source_context rejection + +*For any* string not in the set `{cve, archer, ivanti_queue, email, manual}`, the service SHALL reject the create-ticket request with HTTP 400 when that string is provided as `source_context`. + +**Validates: Requirements 3.3** + +### Property 6: source_context round-trip persistence + +*For any* valid `source_context` value in `{cve, archer, ivanti_queue, email, manual}`, creating a ticket with that value and then fetching the ticket via GET SHALL return the same `source_context` value. + +**Validates: Requirements 3.4** + +### Property 7: source_context immutability + +*For any* existing ticket and any `source_context` value (valid or invalid), an update request that includes a `source_context` field SHALL be rejected with HTTP 400. + +**Validates: Requirements 3.6** + +### Property 8: Summary pre-population truncation from Ivanti queue + +*For any* Ivanti queue item with a `finding_title` of arbitrary length, the pre-populated summary in the Creation Modal SHALL be at most 255 characters and SHALL equal the first 255 characters of `finding_title`. + +**Validates: Requirements 5.2** + +### Property 9: Search includes source_context + +*For any* ticket whose `source_context` value contains the search term as a substring, that ticket SHALL appear in the filtered results when the user searches by that term. + +**Validates: Requirements 8.4** + +## Error Handling + +| Scenario | HTTP Status | Error Message | Client Behavior | +|---|---|---|---| +| Invalid CVE ID format | 400 | "CVE ID format is invalid. Expected CVE-YYYY-NNNN+." | Display inline error, preserve form | +| Vendor exceeds 200 chars | 400 | "Vendor exceeds maximum length of 200 characters." | Display inline error, preserve form | +| Invalid source_context | 400 | "source_context must be one of: cve, archer, ivanti_queue, email, manual." | Display inline error, preserve form | +| source_context update attempt | 400 | "source_context is immutable after creation." | Display inline error | +| Empty summary | 400 | "Summary is required (max 255 chars)." | Prevent submission, show inline error | +| Jira API unavailable | 502 | "Failed to create Jira issue." | Display error banner, preserve form values for retry | +| Jira rate limit exceeded | 429 | "Jira rate limit exceeded. Try again later." | Display error banner, preserve form values | +| Jira API not configured | 503 | "Jira API is not configured." | Display error banner | +| DB insert fails after Jira creation | 207 | Warning with Jira key/URL | Display warning with Jira link | +| jira_tickets table missing (migration) | Exit code 1 | Console error | Migration aborts cleanly | + +**Error preservation:** When the Jira API call fails or returns an error, the frontend modal retains all user-entered form data so the user can retry without re-entering information. This applies to all three entry points (Jira page, Ivanti queue, Archer detail). + +## Testing Strategy + +### Unit Tests (Example-Based) + +- Modal renders with "(optional)" labels and correct placeholders +- Modal allows submission with empty CVE ID and Vendor +- Source Context dropdown contains all five options with no default selection +- Source Context dropdown sends correct API value for each display label +- Summary validation prevents empty submission with inline error +- Ivanti queue "Create Jira Ticket" button is visible on queue items +- Archer detail "Create Jira Ticket" button visible for editor/admin, hidden for viewer +- Archer pre-populates summary with `exc_number` +- Ticket list displays source_context badge between Vendor and Summary columns +- Null source_context displays "CVE" badge text +- Source context filter dropdown includes "All" and all distinct values +- API error preserves form field values in modal + +### Property-Based Tests + +Property-based testing is appropriate for this feature because the backend validation logic operates on a wide input space (arbitrary strings for CVE IDs, vendors, and source_context values) where universal properties must hold. + +**Library:** fast-check (already available in the project's test infrastructure via Jest) + +**Configuration:** +- Minimum 100 iterations per property test +- Each test tagged with: `Feature: flexible-jira-ticket-creation, Property {N}: {title}` + +Properties 1–7 test backend validation logic (pure input → output behavior, mockable Jira API). +Property 8 tests frontend pre-population logic (pure string truncation). +Property 9 tests frontend filtering logic (pure array filter). + +### Integration Tests + +- Migration runs successfully on a database with existing `jira_tickets` rows, preserving all data +- Migration is idempotent (running twice produces same result) +- Migration fails gracefully when `jira_tickets` table doesn't exist +- End-to-end: create ticket from Ivanti queue context → verify stored with `source_context = 'ivanti_queue'` +- End-to-end: create ticket from Archer context → verify stored with `source_context = 'archer'` diff --git a/.kiro/specs/flexible-jira-ticket-creation/requirements.md b/.kiro/specs/flexible-jira-ticket-creation/requirements.md new file mode 100644 index 0000000..09fe3ae --- /dev/null +++ b/.kiro/specs/flexible-jira-ticket-creation/requirements.md @@ -0,0 +1,117 @@ +# Requirements Document + +## Introduction + +The STEAM Security Dashboard's Jira ticket creation flow currently requires a CVE ID and Vendor for every ticket. This locks the workflow to vulnerability-driven contexts only. In practice, the team creates Jira work items from multiple sources — Archer risk acceptance requests, email-driven tasks, and Ivanti queue items — that may not relate to a specific CVE. This feature makes CVE ID and Vendor optional, adds a source context field to track where a ticket originated, and exposes a "Create Jira Ticket" action from the Ivanti queue and Archer ticket views. + +## Glossary + +- **Ticket_Creation_Service**: The backend endpoint (`POST /api/jira-tickets/create-in-jira`) responsible for creating issues in Jira and storing the local record. +- **Jira_Tickets_Table**: The PostgreSQL `jira_tickets` table storing local records of Jira issues. +- **Creation_Modal**: The frontend modal dialog used to create a new Jira ticket. +- **Source_Context**: A metadata field indicating where a Jira ticket originated (e.g., `cve`, `archer`, `ivanti_queue`, `email`, `manual`). +- **Ivanti_Queue_Page**: The frontend page where users work through their Ivanti finding queue items. +- **Archer_Detail_View**: The frontend view displaying Archer ticket details. +- **Dashboard**: The STEAM Security Dashboard application. + +## Requirements + +### Requirement 1: Make CVE ID Optional for Ticket Creation + +**User Story:** As a security analyst, I want to create Jira tickets without providing a CVE ID, so that I can track work items that originate from non-vulnerability contexts. + +#### Acceptance Criteria + +1. WHEN a create-ticket request is submitted without a cve_id field or with cve_id set to null, THE Ticket_Creation_Service SHALL accept the request, create the Jira issue, store the local ticket record with a NULL cve_id, and return an HTTP 201 response. +2. WHEN a create-ticket request includes a cve_id matching the pattern CVE-YYYY-NNNN+ (four-digit year, four or more digit sequence number), THE Ticket_Creation_Service SHALL store the cve_id on the local ticket record. +3. IF a create-ticket request includes a cve_id that is a non-empty string not matching the pattern CVE-YYYY-NNNN+ (four-digit year, four or more digit sequence number), THEN THE Ticket_Creation_Service SHALL reject the request with an HTTP 400 response and an error message indicating the CVE ID format is invalid. +4. IF a create-ticket request includes a cve_id that is an empty string, THEN THE Ticket_Creation_Service SHALL treat it as absent (NULL) and accept the request. +5. THE Jira_Tickets_Table SHALL allow NULL values in the cve_id column. + +### Requirement 2: Make Vendor Optional for Ticket Creation + +**User Story:** As a security analyst, I want to create Jira tickets without providing a Vendor, so that I can create work items for tasks that are not vendor-specific. + +#### Acceptance Criteria + +1. WHEN a create-ticket request is submitted with the vendor field absent, null, or an empty string, THE Ticket_Creation_Service SHALL accept the request and create the Jira issue with a NULL vendor value stored in the local ticket record. +2. WHEN a create-ticket request includes a vendor that is a non-empty string of 1 to 200 characters (after trimming whitespace), THE Ticket_Creation_Service SHALL store the trimmed vendor value on the local ticket record. +3. IF a create-ticket request includes a vendor that, after trimming, is longer than 200 characters, THEN THE Ticket_Creation_Service SHALL reject the request with a validation error indicating the vendor exceeds the maximum length. +4. THE Jira_Tickets_Table SHALL allow NULL values in the vendor column. +5. WHEN a create-ticket request is submitted without a vendor, THE Ticket_Creation_Service SHALL NOT send the vendor value to the Jira API and SHALL still create the Jira issue successfully. + +### Requirement 3: Add Source Context Tracking + +**User Story:** As a security analyst, I want each Jira ticket to record where it was created from, so that I can understand the origin of work items when reviewing the ticket list. + +#### Acceptance Criteria + +1. THE Jira_Tickets_Table SHALL include a source_context column that accepts one of: `cve`, `archer`, `ivanti_queue`, `email`, `manual`. +2. WHEN a create-ticket request does not include a source_context value, THE Ticket_Creation_Service SHALL default source_context to `manual`. +3. IF a create-ticket request includes a source_context value not in the allowed set, THEN THE Ticket_Creation_Service SHALL reject the request with an HTTP 400 response containing an error message indicating the provided value is not in the allowed set. +4. THE Ticket_Creation_Service SHALL store the source_context value on the local ticket record and return it in all GET responses that include ticket data. +5. WHEN the source_context column is added, THE Migration SHALL set existing rows to `manual` so that no ticket has a null source_context value. +6. IF an update-ticket request includes a source_context value, THEN THE Ticket_Creation_Service SHALL reject the change with an HTTP 400 response containing an error message indicating that source_context is immutable after creation. + +### Requirement 4: Update the Creation Modal for Optional Fields + +**User Story:** As a security analyst, I want the Create Jira Ticket modal to clearly indicate that CVE ID and Vendor are optional, so that I am not blocked from creating tickets without them. + +#### Acceptance Criteria + +1. THE Creation_Modal SHALL display CVE ID and Vendor fields each with a label containing the text "(optional)" and placeholder text indicating example input rather than a required value. +2. THE Creation_Modal SHALL allow submission when CVE ID and Vendor are both empty. +3. THE Creation_Modal SHALL include a Source Context selector with the options: CVE, Archer Request, Ivanti Queue, Email, Manual, and no option pre-selected by default. +4. WHEN the user selects a source context, THE Creation_Modal SHALL send the selected option's corresponding value (`cve`, `archer`, `ivanti_queue`, `email`, `manual`) in the create request payload. +5. THE Creation_Modal SHALL require Summary as a mandatory field with a maximum length of 255 characters. +6. IF the user attempts to submit with an empty Summary, THEN THE Creation_Modal SHALL prevent submission and display an inline error message indicating that Summary is required. +7. IF the user does not select a source context, THEN THE Creation_Modal SHALL omit the source_context field from the create request payload, allowing the backend to default to `manual`. + +### Requirement 5: Create Jira Tickets from Ivanti Queue Context + +**User Story:** As a security analyst, I want to create a Jira ticket directly from an Ivanti queue item, so that I can escalate findings to Jira without navigating away from my workflow. + +#### Acceptance Criteria + +1. WHILE viewing an Ivanti queue item, THE Ivanti_Queue_Page SHALL display a "Create Jira Ticket" action. +2. WHEN the user activates the "Create Jira Ticket" action on a queue item, THE Ivanti_Queue_Page SHALL open the Creation_Modal with the Summary field pre-populated with the queue item's finding_title (truncated to 255 characters) and source_context set to `ivanti_queue`. +3. WHEN the queue item has a non-empty cves_json array, THE Creation_Modal SHALL pre-populate the CVE ID field with the first element of the cves_json array. +4. IF the queue item has an empty or null cves_json value, THEN THE Creation_Modal SHALL leave the CVE ID field blank. +5. WHEN the queue item has a vendor value, THE Creation_Modal SHALL pre-populate the Vendor field with that value. +6. IF the Jira API is unavailable or returns an error when the user submits the Creation_Modal, THEN THE system SHALL display an error indication describing the failure and preserve the form field values so the user can retry without re-entering data. + +### Requirement 6: Create Jira Tickets from Archer Ticket Context + +**User Story:** As a security analyst, I want to create a Jira ticket from an Archer ticket view, so that I can create related Jira work items for risk acceptance exceptions. + +#### Acceptance Criteria + +1. WHILE viewing Archer ticket details, IF the user has editor or admin role, THEN THE Archer_Detail_View SHALL display a "Create Jira Ticket" action. +2. WHEN the user activates the "Create Jira Ticket" action on an Archer ticket, THE Archer_Detail_View SHALL open the Creation_Modal with the summary field pre-populated with the Archer ticket exc_number (e.g., "EXC-1234") and source_context set to `archer`. +3. IF the Archer ticket has an associated CVE ID, THEN THE Creation_Modal SHALL pre-populate the CVE ID field with that value. +4. IF the Archer ticket has an associated vendor, THEN THE Creation_Modal SHALL pre-populate the Vendor field with that value. +5. IF the Jira ticket creation API call fails, THEN THE Creation_Modal SHALL display an error message indicating the failure reason and SHALL retain all user-entered form data. + +### Requirement 7: Database Migration for Schema Changes + +**User Story:** As a database administrator, I want the schema changes applied via a migration, so that existing deployments can upgrade without data loss. + +#### Acceptance Criteria + +1. THE Dashboard SHALL include a migration that alters the jira_tickets table to allow NULL in the cve_id column by dropping the NOT NULL constraint. +2. THE Dashboard SHALL include a migration that alters the jira_tickets table to allow NULL in the vendor column by dropping the NOT NULL constraint. +3. THE Dashboard SHALL include a migration that adds a source_context TEXT column to the jira_tickets table with a default value of `manual`, where existing rows receive the default value upon column addition. +4. WHEN the migration runs on a database with existing jira_tickets rows, THE migration SHALL preserve all existing row values in the cve_id, vendor, ticket_key, url, summary, status, created_at, and updated_at columns without modification. +5. THE migration SHALL be idempotent — running it multiple times SHALL produce the same schema state and data as running it once, without raising errors on subsequent executions. +6. IF the jira_tickets table does not exist when the migration runs, THEN THE migration SHALL exit with an error message indicating the table is missing and no schema changes were applied. + +### Requirement 8: Display Source Context in Ticket List + +**User Story:** As a security analyst, I want to see the source context of each ticket in the Jira tickets list, so that I can quickly identify where work items originated. + +#### Acceptance Criteria + +1. THE Dashboard SHALL display the source_context value as a badge on each ticket row in the Jira tickets list view, positioned as a column between the VENDOR and SUMMARY columns. +2. IF a ticket has no source_context value (null or empty string), THEN THE Dashboard SHALL display "CVE" as the default badge text since all legacy tickets were CVE-linked. +3. THE Dashboard SHALL provide a dropdown filter for source_context values, consistent with the existing status filter dropdown, that includes an "All" option showing all tickets and one option per distinct source_context value present in the ticket data. +4. WHEN the user enters a search term in the ticket search input, THE Dashboard SHALL include the source_context value in the searchable fields alongside ticket_key, cve_id, vendor, and summary. diff --git a/.kiro/specs/flexible-jira-ticket-creation/tasks.md b/.kiro/specs/flexible-jira-ticket-creation/tasks.md new file mode 100644 index 0000000..3db3a3b --- /dev/null +++ b/.kiro/specs/flexible-jira-ticket-creation/tasks.md @@ -0,0 +1,139 @@ +# Implementation Plan: Flexible Jira Ticket Creation + +## Overview + +This plan implements flexible Jira ticket creation by making CVE ID and Vendor optional, adding source_context tracking, updating the creation modal, exposing "Create Jira Ticket" actions from Ivanti queue and Archer detail views, and updating the ticket list with source context display and filtering. The implementation proceeds from database migration → backend validation → frontend modal updates → integration points → list view enhancements. + +## Tasks + +- [x] 1. Database migration for schema changes + - [x] 1.1 Create migration file `backend/migrations/add_flexible_jira_ticket_creation.js` + - Drop NOT NULL constraint on `cve_id` column + - Drop NOT NULL constraint on `vendor` column + - Add `source_context TEXT DEFAULT 'manual'` column with `IF NOT EXISTS` + - Add CHECK constraint for allowed source_context values (`cve`, `archer`, `ivanti_queue`, `email`, `manual`) with idempotent guard + - Backfill existing rows with `source_context = 'manual'` where NULL + - Add index on `source_context` column + - Verify `jira_tickets` table exists before proceeding; exit with error if missing + - Ensure full idempotency — safe to run multiple times + - _Requirements: 1.5, 2.4, 3.1, 3.5, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6_ + +- [x] 2. Update backend validation and creation endpoint + - [x] 2.1 Update `POST /api/jira-tickets/create-in-jira` validation logic in `backend/routes/jiraTickets.js` + - Make `cve_id` optional: accept absent, null, or empty string as NULL; validate format `CVE-YYYY-NNNN+` only when non-empty + - Make `vendor` optional: accept absent, null, empty, or whitespace-only as NULL; trim and validate max 200 chars when non-empty + - Add `source_context` validation: must be in allowed set if provided, default to `manual` if absent + - Update INSERT query to include `source_context` column + - Update 201 response to include `source_context` in returned JSON + - Update audit log details to include `source_context` + - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.1, 2.2, 2.3, 2.5, 3.2, 3.3, 3.4_ + + - [x] 2.2 Update `PUT /api/jira-tickets/:id` to reject source_context changes + - If request body contains `source_context` field, return 400 with "source_context is immutable after creation" + - _Requirements: 3.6_ + + - [x] 2.3 Update `GET /api/jira-tickets` to include source_context in response + - Ensure `source_context` is included in SELECT query and returned in ticket objects + - _Requirements: 3.4_ + + - [ ]* 2.4 Write property tests for CVE ID validation (Property 1 and Property 2) + - **Property 1: CVE ID validation and storage** — For any payload with absent/null/empty cve_id, service accepts and stores NULL; for valid CVE format, stores exact value + - **Property 2: Invalid CVE ID rejection** — For any non-empty string not matching `CVE-YYYY-NNNN+`, service rejects with 400 + - **Validates: Requirements 1.1, 1.2, 1.3, 1.4** + - File: `backend/__tests__/jira-flexible-cve-validation.property.test.js` + + - [ ]* 2.5 Write property tests for Vendor validation (Property 3 and Property 4) + - **Property 3: Vendor validation and storage** — For any payload with absent/null/empty/whitespace vendor, stores NULL; for 1–200 char string after trim, stores trimmed value + - **Property 4: Over-length vendor rejection** — For any string exceeding 200 chars after trim, service rejects with validation error + - **Validates: Requirements 2.1, 2.2, 2.3** + - File: `backend/__tests__/jira-flexible-vendor-validation.property.test.js` + + - [ ]* 2.6 Write property tests for source_context validation (Property 5, Property 6, Property 7) + - **Property 5: Invalid source_context rejection** — For any string not in allowed set, service rejects with 400 + - **Property 6: source_context round-trip persistence** — For any valid source_context, creating then fetching returns same value + - **Property 7: source_context immutability** — For any existing ticket, update with source_context field is rejected with 400 + - **Validates: Requirements 3.3, 3.4, 3.6** + - File: `backend/__tests__/jira-flexible-source-context.property.test.js` + +- [x] 3. Checkpoint - Ensure all tests pass + - Ensure all tests pass, ask the user if questions arise. + +- [x] 4. Update frontend Creation Modal for optional fields + - [x] 4.1 Update the create Jira ticket modal in `frontend/src/components/pages/JiraPage.js` + - Change CVE ID label to "CVE ID (optional)" with placeholder "e.g. CVE-2024-12345" + - Change Vendor label to "Vendor (optional)" with placeholder "e.g. Microsoft" + - Add Source Context dropdown with options: CVE → `cve`, Archer Request → `archer`, Ivanti Queue → `ivanti_queue`, Email → `email`, Manual → `manual`; no default selection + - Allow form submission when CVE ID and Vendor are both empty + - Keep Summary as required with inline error on empty submit (max 255 chars) + - Send `source_context` in payload only when selected; omit if no selection (backend defaults to `manual`) + - Support pre-populated field values and locked source_context (read-only when set externally) + - Preserve form field values on API error for retry + - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 5.6, 6.5_ + +- [x] 5. Add "Create Jira Ticket" action from Ivanti Queue + - [x] 5.1 Add "Create Jira Ticket" button to Ivanti queue items in `frontend/src/components/pages/IvantiTodoQueuePage.js` + - Add button/action to each queue item row + - On click, open Creation Modal with: summary pre-populated from `finding_title` (truncated to 255 chars), source_context locked to `ivanti_queue`, cve_id from first element of `cves_json` (if non-empty), vendor from queue item's vendor (if present) + - Leave CVE ID blank if `cves_json` is empty or null + - _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5_ + + - [ ]* 5.2 Write property test for summary truncation (Property 8) + - **Property 8: Summary pre-population truncation from Ivanti queue** — For any finding_title of arbitrary length, pre-populated summary is at most 255 chars and equals first 255 chars of finding_title + - **Validates: Requirements 5.2** + - File: `backend/__tests__/jira-flexible-summary-truncation.property.test.js` + +- [x] 6. Add "Create Jira Ticket" action from Archer Detail View + - [x] 6.1 Add "Create Jira Ticket" button to Archer ticket detail view in `frontend/src/components/pages/ArcherPage.js` + - Show button only for users with editor or admin role + - On click, open Creation Modal with: summary pre-populated with `exc_number` (e.g., "EXC-1234"), source_context locked to `archer`, cve_id from Archer ticket's cve_id (if present), vendor from Archer ticket's vendor (if present) + - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_ + +- [x] 7. Update ticket list with source context display and filtering + - [x] 7.1 Add source context badge column to ticket list in `frontend/src/components/pages/JiraPage.js` + - Add "Source" column between Vendor and Summary columns + - Display color-coded badge per source_context value: cve → blue (#0EA5E9), archer → purple (#8B5CF6), ivanti_queue → amber (#F59E0B), email → green (#10B981), manual → gray (#94A3B8) + - Display "CVE" badge (blue) for null/empty source_context (legacy tickets) + - _Requirements: 8.1, 8.2_ + + - [x] 7.2 Add source context dropdown filter to ticket list + - Add dropdown filter consistent with existing status filter pattern + - Include "All" option showing all tickets plus one option per distinct source_context value + - _Requirements: 8.3_ + + - [x] 7.3 Include source_context in ticket search + - Add source_context to the set of searchable fields alongside ticket_key, cve_id, vendor, and summary + - _Requirements: 8.4_ + + - [ ]* 7.4 Write property test for search includes source_context (Property 9) + - **Property 9: Search includes source_context** — For any ticket whose source_context contains the search term as a substring, that ticket appears in filtered results + - **Validates: Requirements 8.4** + - File: `backend/__tests__/jira-flexible-search-source-context.property.test.js` + +- [x] 8. Final checkpoint - Ensure all tests pass + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional and can be skipped for faster MVP +- Each task references specific requirements for traceability +- Checkpoints ensure incremental validation +- Property tests validate universal correctness properties from the design document +- Unit tests validate specific examples and edge cases +- The migration must run before backend changes are tested against a real database +- The Creation Modal is shared across all three entry points (Jira page, Ivanti queue, Archer detail) — task 4.1 builds the reusable foundation, tasks 5.1 and 6.1 wire it from their respective contexts + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1"] }, + { "id": 1, "tasks": ["2.1", "2.2", "2.3"] }, + { "id": 2, "tasks": ["2.4", "2.5", "2.6"] }, + { "id": 3, "tasks": ["4.1"] }, + { "id": 4, "tasks": ["5.1", "6.1"] }, + { "id": 5, "tasks": ["5.2", "7.1", "7.2", "7.3"] }, + { "id": 6, "tasks": ["7.4"] } + ] +} +``` diff --git a/.kiro/specs/forecast-burndown-chart/.config.kiro b/.kiro/specs/forecast-burndown-chart/.config.kiro new file mode 100644 index 0000000..8ee1c3e --- /dev/null +++ b/.kiro/specs/forecast-burndown-chart/.config.kiro @@ -0,0 +1 @@ +{"specId": "13977c27-d996-4859-a09b-f0162bb13be0", "workflowType": "requirements-first", "specType": "feature"} diff --git a/.kiro/specs/forecast-burndown-chart/design.md b/.kiro/specs/forecast-burndown-chart/design.md new file mode 100644 index 0000000..ab1d65a --- /dev/null +++ b/.kiro/specs/forecast-burndown-chart/design.md @@ -0,0 +1,328 @@ +# Design Document: Forecast Burndown Chart + +## Overview + +This feature adds a per-metric forecast burndown chart to the CCP Metrics page. It combines historical compliance data (from `compliance_snapshots`) with forward-looking remediation projections (derived from `compliance_items` resolution dates) into a single stacked bar + line chart. A metric selector allows users to switch between individual compliance metrics (e.g., 2.3.5, 2.3.6, 5.2.5). + +The design separates concerns into: +1. A **pure helper function** (`computeMetricForecastBurndown`) for testable forecast computation +2. Two **API endpoints** (forecast-burndown data + metrics-list) added to the existing VCL multi-vertical router +3. A **React chart component** using recharts `ComposedChart` with `Bar` + `Line` + `ReferenceLine` + +Key design decisions: +- **No caching**: The API queries `compliance_items` directly on each request so edits on the AEO Compliance page are immediately reflected. +- **Pure computation**: The forecast logic is a stateless helper function that accepts pre-fetched data, making it testable in isolation without database mocks. +- **Existing patterns**: Routes use the `createVCLMultiVerticalRouter` factory pattern; the chart component follows the same inline-style + recharts patterns as `AggregatedBurndownChart` and `TrendChart`. + +## Architecture + +```mermaid +flowchart TD + subgraph Frontend + MS[MetricSelector] -->|selected metric_id| FBC[ForecastBurndownChart] + FBC -->|fetch| API1[GET /metric/:metricId/forecast-burndown] + MS -->|fetch on mount| API2[GET /metrics-list] + end + + subgraph Backend + API1 --> RH1[Route Handler] + API2 --> RH2[Route Handler] + RH1 -->|query| DB[(PostgreSQL)] + RH1 -->|compute| HF[computeMetricForecastBurndown] + RH2 -->|query| DB + end + + subgraph Database + DB --> CI[compliance_items] + DB --> CS[compliance_snapshots] + end +``` + +The data flow: +1. On page load, `MetricSelector` fetches `/api/compliance/vcl-multi/metrics-list` to populate the dropdown. +2. When a metric is selected, `ForecastBurndownChart` fetches `/api/compliance/vcl-multi/metric/:metricId/forecast-burndown`. +3. The route handler queries `compliance_items` for active devices and `compliance_snapshots` for historical data, then passes both to `computeMetricForecastBurndown`. +4. The helper returns structured data that the frontend renders as a `ComposedChart`. + +## Components and Interfaces + +### Backend: Helper Function + +**File:** `backend/helpers/vclHelpers.js` + +```javascript +/** + * Computes per-metric forecast burndown from device records and historical snapshots. + * + * @param {Array<{hostname: string, resolution_date: string|null}>} currentDevices + * Active non-compliant devices for the metric + * @param {number} totalAssets + * Total device count in scope for this metric (from snapshot or summary) + * @param {Array<{month: string, total_assets: number, non_compliant: number, compliance_pct: number}>} historicalSnapshots + * Pre-computed historical data points (up to 4 months) + * @returns {{ + * historical: Array<{month: string, total_assets: number, non_compliant: number, compliance_pct: number}>, + * forecast: Array<{month: string, total_assets: number, non_compliant: number, compliance_pct: number}>, + * current_snapshot: {total_assets: number, non_compliant: number, compliant: number, compliance_pct: number, blockers: number, with_dates: number} + * }} + */ +function computeMetricForecastBurndown(currentDevices, totalAssets, historicalSnapshots) { ... } +``` + +### Backend: API Endpoints + +**File:** `backend/routes/vclMultiVertical.js` (added to existing router) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/api/compliance/vcl-multi/metric/:metricId/forecast-burndown` | `requireAuth()` | Returns combined historical + forecast data for a metric | +| GET | `/api/compliance/vcl-multi/metrics-list` | `requireAuth()` | Returns list of metrics with active non-compliant device counts | + +**Forecast Burndown Response Shape:** +```json +{ + "metric_id": "2.3.5", + "historical": [ + { "month": "2025-01", "total_assets": 500, "non_compliant": 45, "compliance_pct": 91.0 }, + { "month": "2025-02", "total_assets": 510, "non_compliant": 38, "compliance_pct": 92.5 } + ], + "forecast": [ + { "month": "2025-04", "total_assets": 520, "non_compliant": 25, "compliance_pct": 95.2 }, + { "month": "2025-05", "total_assets": 520, "non_compliant": 15, "compliance_pct": 97.1 } + ], + "current_snapshot": { + "total_assets": 520, + "non_compliant": 30, + "compliant": 490, + "compliance_pct": 94.2, + "blockers": 8, + "with_dates": 22 + } +} +``` + +**Metrics List Response Shape:** +```json +[ + { "metric_id": "2.3.5", "device_count": 30 }, + { "metric_id": "2.3.6", "device_count": 12 }, + { "metric_id": "5.2.5", "device_count": 45 } +] +``` + +### Frontend: Components + +**File:** `frontend/src/components/pages/CCPMetricsPage.js` (added to existing file) + +1. **`MetricSelector`** — Dropdown component that fetches and displays available metrics. Triggers forecast data fetch on selection change. + +2. **`ForecastBurndownChart`** — Renders a `ComposedChart` with: + - Blue `Bar` for `total_assets` (left Y-axis) + - Orange `Bar` for `non_compliant` (left Y-axis, stacked appearance) + - Green `Line` for `compliance_pct` (right Y-axis, 0–100%) + - `ReferenceLine` as vertical divider between historical and forecast sections + - Forecast data points rendered at 50% opacity + +## Data Models + +### Database Tables (existing, no schema changes required) + +**`compliance_items`** (relevant columns): +| Column | Type | Description | +|--------|------|-------------| +| hostname | TEXT | Device hostname | +| metric_id | TEXT | Compliance metric identifier (e.g., "2.3.5") | +| status | TEXT | `'active'` or `'resolved'` | +| resolution_date | DATE | Planned remediation date (nullable) | +| vertical | TEXT | Organizational vertical (e.g., "NTS_AEO") | +| team | TEXT | Team assignment | + +**`compliance_snapshots`** (relevant columns): +| Column | Type | Description | +|--------|------|-------------| +| snapshot_month | TEXT | Month in YYYY-MM format | +| vertical | TEXT | Organizational vertical | +| total_devices | INTEGER | Total devices in scope | +| compliant | INTEGER | Compliant device count | +| non_compliant | INTEGER | Non-compliant device count | +| compliance_pct | NUMERIC(5,2) | Compliance percentage | + +### Internal Data Structures + +**Chart data point (shared between historical and forecast):** +```javascript +{ + month: "2025-04", // YYYY-MM + total_assets: 520, // Total devices in scope + non_compliant: 25, // Non-compliant count + compliance_pct: 95.2, // Percentage (0-100, 1 decimal) + isForecast: true // Frontend flag for opacity styling +} +``` + +**Current snapshot:** +```javascript +{ + total_assets: 520, + non_compliant: 30, + compliant: 490, + compliance_pct: 94.2, + blockers: 8, // Devices with no resolution_date + with_dates: 22 // Devices with a resolution_date +} +``` + +### Query Patterns + +**Metrics list query:** +```sql +SELECT metric_id, COUNT(DISTINCT hostname) AS device_count +FROM compliance_items +WHERE status = 'active' AND vertical IS NOT NULL +GROUP BY metric_id +ORDER BY metric_id ASC +``` + +**Active devices for a metric:** +```sql +SELECT hostname, resolution_date, vertical +FROM compliance_items +WHERE metric_id = $1 AND status = 'active' AND vertical IS NOT NULL +``` + +**Historical snapshots for a vertical (3 months prior):** +```sql +SELECT snapshot_month AS month, total_devices AS total_assets, + non_compliant, compliance_pct::numeric AS compliance_pct +FROM compliance_snapshots +WHERE vertical = $1 AND snapshot_month >= $2 AND snapshot_month < $3 +ORDER BY snapshot_month ASC +``` + + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +The `computeMetricForecastBurndown` helper function is a pure function with clear input/output behavior and universal properties that hold across a wide input space. This makes it an ideal candidate for property-based testing. + +### Property 1: Forecast structure invariant + +*For any* valid `currentDevices` array and non-negative `totalAssets` integer, the `computeMetricForecastBurndown` function SHALL return an object with `historical`, `forecast`, and `current_snapshot` fields, where every element in the `forecast` array contains `month` (YYYY-MM string), `total_assets` (equal to the input `totalAssets` or the current snapshot's total), `non_compliant` (non-negative integer), and `compliance_pct` (number between 0 and 100), and all `total_assets` values in the forecast array are identical. + +**Validates: Requirements 1.4, 3.1, 3.6** + +### Property 2: Blocker and with_dates partition invariant + +*For any* valid `currentDevices` array and non-negative `totalAssets` integer, the `computeMetricForecastBurndown` function SHALL produce a `current_snapshot` where `blockers + with_dates = non_compliant`. + +**Validates: Requirements 3.2** + +### Property 3: Compliance percentage formula correctness + +*For any* valid inputs where `totalAssets > 0`, the `computeMetricForecastBurndown` function SHALL compute `compliance_pct` for each forecast and current_snapshot data point as `ROUND((total_assets - non_compliant) / total_assets * 100, 1)`. When `totalAssets` is 0, all `compliance_pct` values SHALL be 0. + +**Validates: Requirements 3.3, 3.10** + +### Property 4: Forecast non_compliant monotonicity + +*For any* valid `currentDevices` array and non-negative `totalAssets` integer, the `forecast` array produced by `computeMetricForecastBurndown` SHALL have monotonically non-increasing `non_compliant` values (each month's `non_compliant` is less than or equal to the previous month's `non_compliant`). + +**Validates: Requirements 3.4** + +### Property 5: Per-month non_compliant computation correctness + +*For any* valid `currentDevices` array and non-negative `totalAssets` integer, for each month in the `forecast` array, the `non_compliant` value SHALL equal the count of devices whose `resolution_date` is after that month (not yet remediated) plus the count of devices with no `resolution_date` (blockers). + +**Validates: Requirements 3.8** + +### Property 6: Forecast horizon bound + +*For any* valid `currentDevices` array and non-negative `totalAssets` integer, the `forecast` array produced by `computeMetricForecastBurndown` SHALL contain at most 12 elements, and SHALL terminate either when all devices with resolution dates are projected to be remediated or at the 12-month maximum, whichever comes first. + +**Validates: Requirements 1.9, 3.9** + +### Property 7: Past-due resolution dates treated as current month + +*For any* device in `currentDevices` whose `resolution_date` is in a month that has already passed relative to the current date, the `computeMetricForecastBurndown` function SHALL treat that device as projected to be remediated in the current month (i.e., it is excluded from `non_compliant` in the first forecast month if that month is after the current month, but remains in `non_compliant` for the current month's snapshot). + +**Validates: Requirements 6.5** + +## Error Handling + +### Backend API Errors + +| Scenario | HTTP Status | Response Body | Behavior | +|----------|-------------|---------------|----------| +| Unauthenticated request | 401 | `{ "error": "Unauthorized" }` | `requireAuth()` middleware rejects | +| Invalid/unknown metricId | 200 | `{ metric_id, historical: [], forecast: [], current_snapshot: {zeros} }` | Graceful empty response | +| Database query failure | 500 | `{ "error": "Failed to compute forecast burndown" }` | Catch block logs error, returns 500 | +| Database failure on metrics-list | 500 | `{ "error": "Failed to fetch metrics list" }` | Catch block logs error, returns 500 | + +### Frontend Error States + +| Scenario | UI Behavior | +|----------|-------------| +| Metrics list fetch fails | Inline error with AlertCircle icon, red border, error message text | +| Forecast data fetch fails | Inline error in chart area with AlertCircle icon and error description | +| Empty metrics list | "No metrics with active non-compliant devices" message | +| Empty forecast + empty historical | "No data available for this metric" message in chart area | +| Race condition (rapid metric switching) | Discard stale responses; only render data from the most recent selection | + +### Helper Function Edge Cases + +| Input Condition | Behavior | +|-----------------|----------| +| `currentDevices` is empty | Return empty `forecast`, zeroed `current_snapshot` (except `total_assets`) | +| `totalAssets` is 0 | All `compliance_pct` values are 0 (avoid division by zero) | +| `totalAssets` < `currentDevices.length` | Use `currentDevices.length` as `non_compliant` without clamping | +| All devices are blockers (no resolution dates) | Return empty `forecast` array (no month-by-month projection possible) | +| All resolution dates are in the past | Treat as remediated in current month; forecast shows only blockers remaining | + +## Testing Strategy + +### Property-Based Tests (via fast-check) + +The `computeMetricForecastBurndown` helper function is a pure function with well-defined invariants, making it ideal for property-based testing. Each property test runs a minimum of 100 iterations with randomly generated inputs. + +**Library:** `fast-check` (already available in the project's test dependencies based on existing `.property.test.js` files) + +**Test file:** `backend/__tests__/forecast-burndown-chart.property.test.js` + +**Generator strategy:** +- `currentDevices`: Array of `{ hostname: arbitraryString, resolution_date: oneOf(null, arbitraryFutureDate, arbitraryPastDate) }` +- `totalAssets`: Non-negative integer (including 0 and values less than device count) +- `historicalSnapshots`: Array of 0–4 elements with valid month strings and non-negative counts + +Each test is tagged with: `Feature: forecast-burndown-chart, Property {N}: {title}` + +Properties to implement: +1. Forecast structure invariant +2. Blocker/with_dates partition invariant +3. Compliance percentage formula correctness +4. Forecast non_compliant monotonicity +5. Per-month non_compliant computation correctness +6. Forecast horizon bound +7. Past-due resolution dates treated as current month + +### Unit Tests (example-based) + +**Test file:** `backend/__tests__/forecast-burndown-chart.test.js` + +- Specific examples with known inputs and expected outputs +- Edge cases: empty devices, zero totalAssets, all blockers, all past-due dates +- Integration with route handlers (mocked database) + +### Integration Tests + +- Verify API endpoints return correct response shapes with seeded database data +- Verify authentication middleware is applied +- Verify correct SQL filtering (vertical scoping, active-only devices) +- Verify historical data derivation from `compliance_snapshots` + +### Frontend Tests + +- Component rendering with mock data (snapshot tests) +- Loading and error states +- Metric selector interaction (selection triggers fetch) +- Race condition handling (rapid metric switching discards stale responses) diff --git a/.kiro/specs/forecast-burndown-chart/requirements.md b/.kiro/specs/forecast-burndown-chart/requirements.md new file mode 100644 index 0000000..82ef012 --- /dev/null +++ b/.kiro/specs/forecast-burndown-chart/requirements.md @@ -0,0 +1,132 @@ +# Requirements Document + +## Introduction + +This feature adds a per-metric forecast burndown chart to the CCP Metrics page. Unlike the existing aggregated burndown (which rolls up across all verticals by hostname), this chart focuses on individual compliance metrics (e.g., 2.3.5, 2.3.6, 5.2.5) and combines historical compliance data with forward-looking remediation projections in a single stacked bar + line chart. The chart displays actual historical data on the left side and forecasted future data on the right side, separated by a bold vertical divider. A metric selector allows users to switch between metrics. The chart reads from the same `compliance_items` table that the AEO Compliance page writes to, so device-level edits (resolution_date changes) dynamically feed into the forecast. + +## Glossary + +- **Forecast_Burndown_API**: The backend endpoint that returns combined historical and forecast burndown data for a specific compliance metric, identified by `metric_id`. +- **Metric_Selector**: A dropdown or picker UI component that allows the user to choose which compliance metric to display in the chart. +- **Historical_Data**: Monthly compliance snapshots from the past 3 months plus the current month, showing actual total asset counts, non-compliant device counts, and compliance percentages as recorded at each snapshot point. +- **Forecast_Data**: Projected future monthly data points computed from active non-compliant devices with scheduled `resolution_date` values, assuming total asset count remains constant and all remediation plans complete on schedule. +- **Compliance_Percentage**: The ratio of compliant devices to total devices for a given metric, expressed as a percentage (0%–100%). +- **Total_Assets**: The total number of devices in scope for a given metric at a point in time. +- **Non_Compliant_Count**: The number of devices that are non-compliant for a given metric at a point in time. +- **Divider_Line**: A bold vertical line on the chart separating actual historical data (left) from forecasted future data (right). +- **CCP_Metrics_Page**: The page component (`CCPMetricsPage.js`) where the forecast burndown chart is displayed. +- **Compliance_Items_Table**: The `compliance_items` PostgreSQL table that tracks non-compliant devices with fields including `hostname`, `metric_id`, `status`, `resolution_date`, `vertical`, and `team`. + +## Requirements + +### Requirement 1: Per-Metric Forecast Burndown API Endpoint + +**User Story:** As a compliance analyst, I want an API endpoint that returns combined historical and forecast burndown data for a specific metric, so that the frontend can render a chart showing past compliance trends and projected future remediation. + +#### Acceptance Criteria + +1. WHEN a GET request is made to `/api/compliance/vcl-multi/metric/:metricId/forecast-burndown`, THE Forecast_Burndown_API SHALL return a JSON response containing `metric_id`, `historical`, `forecast`, and `current_snapshot` fields within 3 seconds. +2. THE Forecast_Burndown_API SHALL compute `historical` as an array of exactly 4 monthly data points (the current month plus the 3 preceding months in chronological order), where each data point contains `month` (YYYY-MM), `total_assets`, `non_compliant`, and `compliance_pct`. +3. THE Forecast_Burndown_API SHALL derive historical `total_assets` and `non_compliant` counts from the `compliance_snapshots` table filtered by the vertical associated with the requested `metricId` in `compliance_items` (determined by querying the `vertical` column of active devices matching that `metric_id`), combined with per-metric device counts from `compliance_items` for that snapshot period. +4. THE Forecast_Burndown_API SHALL compute `forecast` as an array of projected future monthly data points, where each data point contains `month` (YYYY-MM), `total_assets`, `non_compliant`, and `compliance_pct`. +5. THE Forecast_Burndown_API SHALL project forecast data by assuming the current `total_assets` count remains constant and that each non-compliant device with a `resolution_date` in a future month will become compliant in that month. +6. THE Forecast_Burndown_API SHALL compute `current_snapshot` containing `total_assets`, `non_compliant`, `compliant`, `compliance_pct`, `blockers` (count of devices with no resolution_date), and `with_dates` (count of devices with a resolution_date). +7. THE Forecast_Burndown_API SHALL require authentication via `requireAuth()` middleware. +8. IF the `metricId` parameter does not match any active devices in `compliance_items` (where `status = 'active'` and `vertical IS NOT NULL`), THEN THE Forecast_Burndown_API SHALL return a 200 response with empty `historical` and `forecast` arrays and `current_snapshot` with all numeric fields set to 0. +9. THE Forecast_Burndown_API SHALL return forecast data extending forward until all devices with resolution dates are projected to be remediated, or for a maximum of 12 months from the current date, whichever comes first. +10. IF the database query fails or an internal error occurs while processing the request, THEN THE Forecast_Burndown_API SHALL return a 500 response with a JSON body containing an `error` field indicating the failure reason. + +### Requirement 2: Available Metrics List Endpoint + +**User Story:** As a frontend developer, I want an API endpoint that returns the list of distinct metrics with active non-compliant devices, so that the metric selector can be populated with valid options. + +#### Acceptance Criteria + +1. WHEN a GET request is made to `/api/compliance/vcl-multi/metrics-list`, THE Forecast_Burndown_API SHALL return a 200 response containing a JSON array of objects, each containing `metric_id` (string) and `device_count` (integer representing the number of distinct hostnames with status `active` for that metric) fields. +2. THE Forecast_Burndown_API SHALL include only metrics that have at least one active non-compliant device in `compliance_items` where `vertical IS NOT NULL`. +3. THE Forecast_Burndown_API SHALL sort the returned array by `metric_id` in ascending alphanumeric order. +4. THE Forecast_Burndown_API SHALL require authentication via `requireAuth()` middleware. +5. IF no metrics have active non-compliant devices with a non-null vertical, THEN THE Forecast_Burndown_API SHALL return a 200 response containing an empty JSON array. +6. IF the database query fails, THEN THE Forecast_Burndown_API SHALL return a 500 response with a JSON object containing an `error` field. + +### Requirement 3: Forecast Computation Logic + +**User Story:** As a developer, I want a pure helper function that computes forecast burndown data for a given metric from device records and historical snapshots, so that the computation is testable in isolation. + +#### Acceptance Criteria + +1. THE `computeMetricForecastBurndown` helper function SHALL accept a `currentDevices` array (each element containing at minimum `hostname` and `resolution_date` fields, where `resolution_date` is either a YYYY-MM-DD string or null), a `totalAssets` count (non-negative integer), and a `historicalSnapshots` array (each element containing `month` as YYYY-MM, `total_assets`, `non_compliant`, and `compliance_pct`), and return an object with `historical`, `forecast`, and `current_snapshot` fields. +2. FOR ALL valid inputs, THE `computeMetricForecastBurndown` function SHALL satisfy the invariant: `current_snapshot.blockers + current_snapshot.with_dates = current_snapshot.non_compliant`. +3. FOR ALL valid inputs where `total_assets > 0`, THE `computeMetricForecastBurndown` function SHALL compute `compliance_pct` as `ROUND((total_assets - non_compliant) / total_assets * 100, 1)` for each data point. +4. FOR ALL forecast data points, THE `computeMetricForecastBurndown` function SHALL produce monotonically non-increasing `non_compliant` counts (each month has equal or fewer non-compliant devices than the previous month). +5. FOR ALL forecast data points, THE `computeMetricForecastBurndown` function SHALL produce monotonically non-decreasing `compliance_pct` values. +6. THE `computeMetricForecastBurndown` function SHALL hold `total_assets` constant across all forecast data points, using the current month's total as the baseline. +7. WHEN the `currentDevices` array is empty, THE `computeMetricForecastBurndown` function SHALL return an empty `forecast` array and `current_snapshot` with all zero values except `total_assets`. +8. FOR ALL forecast months, THE `computeMetricForecastBurndown` function SHALL compute `non_compliant` as the count of devices whose `resolution_date` is after that month (devices not yet remediated) plus `blockers` (devices with no resolution_date). +9. THE `computeMetricForecastBurndown` function SHALL generate forecast data points extending forward month-by-month until all devices with resolution dates are projected to be remediated, or for a maximum of 12 months from the current date, whichever comes first. +10. IF `totalAssets` is 0, THEN THE `computeMetricForecastBurndown` function SHALL return `compliance_pct` of 0 for all data points in `historical`, `forecast`, and `current_snapshot`. +11. IF `totalAssets` is less than the number of elements in `currentDevices`, THEN THE `computeMetricForecastBurndown` function SHALL use the count of `currentDevices` as `non_compliant` without clamping to `totalAssets`. + +### Requirement 4: Forecast Burndown Chart Component + +**User Story:** As a senior leader viewing the CCP Metrics page, I want to see a combined historical and forecast burndown chart for each metric, so that I can visualize past compliance trends and projected future remediation timelines. + +#### Acceptance Criteria + +1. THE Forecast_Burndown_Chart SHALL be displayed on the CCP_Metrics_Page in a dedicated section below the Metric_Selector. +2. THE Forecast_Burndown_Chart SHALL render a stacked bar chart where blue bars represent Total_Assets (100% height baseline) and orange bars represent Non_Compliant_Count. +3. THE Forecast_Burndown_Chart SHALL render a green line overlay showing Compliance_Percentage as a trend line with percentage labels at each data point. +4. THE Forecast_Burndown_Chart SHALL display a left Y-axis scaled to the maximum Total_Assets value for device counts, and a right Y-axis scaled from 0% to 100% for Compliance_Percentage. +5. THE Forecast_Burndown_Chart SHALL display the X-axis labeled with months (YYYY-MM format), showing up to 16 data points (up to 4 historical months plus up to 12 forecast months). +6. THE Forecast_Burndown_Chart SHALL render a bold vertical Divider_Line separating Historical_Data (left side) from Forecast_Data (right side), positioned between the last historical data point and the first forecast data point. +7. THE Forecast_Burndown_Chart SHALL render forecast bars and line segments with 50% opacity to visually distinguish projections from actuals. +8. THE Forecast_Burndown_Chart SHALL display raw device count labels inside the bars (total assets count in blue bars, non-compliant count in orange bars). +9. THE Forecast_Burndown_Chart SHALL display compliance percentage values as labels on the green trend line at each data point. +10. WHEN the Forecast_Burndown_API returns both empty `historical` and empty `forecast` arrays for a selected metric, THE Forecast_Burndown_Chart SHALL display a message indicating no data is available for that metric. +11. WHEN the user selects a different metric from the Metric_Selector, THE Forecast_Burndown_Chart SHALL fetch updated data from the Forecast_Burndown_API and re-render the chart with the new metric's data. +12. WHILE the Forecast_Burndown_API request is in flight, THE Forecast_Burndown_Chart SHALL display a loading indicator in place of the chart content. +13. IF the Forecast_Burndown_API request fails, THEN THE Forecast_Burndown_Chart SHALL display an inline error message with an AlertCircle icon and the error description, styled with a red border consistent with the existing error display pattern on the CCP_Metrics_Page. +14. WHEN the Forecast_Burndown_API returns a non-empty `historical` array but an empty `forecast` array, THE Forecast_Burndown_Chart SHALL render only the historical bars and trend line without a Divider_Line or forecast section. + +### Requirement 5: Metric Selector Component + +**User Story:** As a compliance analyst, I want a metric picker that lets me choose which metric's forecast burndown to view, so that I can analyze remediation progress for individual compliance requirements. + +#### Acceptance Criteria + +1. THE Metric_Selector SHALL be displayed above or adjacent to the Forecast_Burndown_Chart on the CCP_Metrics_Page. +2. THE Metric_Selector SHALL populate its options by fetching the available metrics list from the metrics-list endpoint on page load. +3. THE Metric_Selector SHALL display each metric option showing the `metric_id` and the count of active non-compliant devices. +4. WHEN the user selects a metric, THE Metric_Selector SHALL trigger a data fetch for that metric's forecast burndown and update the chart. +5. WHEN the Metric_Selector finishes loading the metrics list and defaults to the first metric in the sorted list, THE Metric_Selector SHALL automatically trigger a data fetch for that default metric's forecast burndown and update the chart. +6. WHILE the metrics list is loading, THE Metric_Selector SHALL display a loading indicator and remain non-interactive until the fetch completes or fails. +7. IF the metrics list endpoint returns an empty array, THEN THE Metric_Selector SHALL display a message indicating no metrics with active non-compliant devices exist. +8. IF the metrics list endpoint request fails, THEN THE Metric_Selector SHALL display an inline error message consistent with the existing error display pattern on the CCP_Metrics_Page. +9. WHILE the forecast burndown data is loading after a metric selection, THE Metric_Selector SHALL remain interactive, and if the user selects a different metric before the previous fetch completes, THE Metric_Selector SHALL discard the previous in-flight response and use only the result from the most recent selection. + +### Requirement 6: Dynamic Data Integration with AEO Compliance Page + +**User Story:** As a compliance analyst, I want the forecast burndown chart to reflect device-level edits made on the AEO Compliance page in real time, so that when I update a device's resolution_date for a metric, the forecast projection updates accordingly on the next chart load. + +#### Acceptance Criteria + +1. THE Forecast_Burndown_API SHALL query the `compliance_items` table directly (not a cached copy), so that edits made via the AEO Compliance page's `PATCH /items/:hostname/metadata` endpoint are reflected in the next API call made after the PATCH response completes. +2. WHEN a user changes a device's `resolution_date` on the AEO Compliance page for a specific metric, THE Forecast_Burndown_API SHALL include that updated date in its forecast computation on the next request for that metric. +3. WHEN a device's status changes from `active` to `resolved` on the AEO Compliance page, THE Forecast_Burndown_API SHALL exclude that device from the non-compliant count on the next request. +4. IF a device's `resolution_date` is cleared (set to NULL) on the AEO Compliance page, THEN THE Forecast_Burndown_API SHALL count that device as a blocker in the `current_snapshot.blockers` count and exclude it from month-by-month forecast remediation projections on the next request. +5. IF a device's `resolution_date` is changed to a date in a month that has already passed relative to the current date, THEN THE Forecast_Burndown_API SHALL treat that device as projected to be remediated in the current month (not exclude it from non-compliant count until its status changes to `resolved`). +6. THE Forecast_Burndown_API SHALL use the current state of `compliance_items` as the source of truth for the current snapshot and forecast projections, with no application-level caching between requests. + +### Requirement 7: Historical Data Derivation + +**User Story:** As a compliance analyst, I want the chart to show actual historical compliance data for the past 3 months, so that I can see the real trend leading into the forecast. + +#### Acceptance Criteria + +1. THE Forecast_Burndown_API SHALL derive historical data from the `compliance_snapshots` table, which records monthly total_devices, compliant, and non_compliant counts per vertical, for the 3 calendar months immediately preceding the current month. +2. THE Forecast_Burndown_API SHALL compute per-metric historical non_compliant count at each snapshot point by multiplying the vertical's snapshot non_compliant count by the ratio of the metric's non-compliant devices to the vertical's total non-compliant devices (as recorded in `compliance_items` for that period), rounding to the nearest integer. +3. IF the vertical's total non_compliant count is 0 at a historical snapshot point, THEN THE Forecast_Burndown_API SHALL set the metric's non_compliant count to 0 for that data point. +4. THE Forecast_Burndown_API SHALL use the vertical's snapshot total_devices as the metric's total_assets for each historical data point. +5. WHEN no historical snapshots exist for a metric's vertical, THE Forecast_Burndown_API SHALL return an empty `historical` array. +6. THE Forecast_Burndown_API SHALL include the current month as the most recent historical data point, computed from live `compliance_items` data rather than a stored snapshot. +7. THE Forecast_Burndown_API SHALL return historical data points in chronological order (oldest first). diff --git a/.kiro/specs/forecast-burndown-chart/tasks.md b/.kiro/specs/forecast-burndown-chart/tasks.md new file mode 100644 index 0000000..33398b8 --- /dev/null +++ b/.kiro/specs/forecast-burndown-chart/tasks.md @@ -0,0 +1,158 @@ +# Implementation Plan: Forecast Burndown Chart + +## Overview + +Add a per-metric forecast burndown chart to the CCP Metrics page. A pure helper function (`computeMetricForecastBurndown`) computes forecast projections from device records and historical snapshots. Two new API endpoints serve the metrics list and forecast data. A React frontend renders a metric selector and a ComposedChart (Bar + Line + ReferenceLine) using recharts. No database migrations are needed — the feature reads from existing `compliance_items` and `compliance_snapshots` tables. + +## Tasks + +- [x] 1. Implement the computeMetricForecastBurndown helper function + - [x] 1.1 Add computeMetricForecastBurndown to backend/helpers/vclHelpers.js + - Implement the pure function accepting `currentDevices`, `totalAssets`, and `historicalSnapshots` + - Return object with `historical`, `forecast`, and `current_snapshot` fields + - Compute `current_snapshot.blockers` (devices with no resolution_date) and `current_snapshot.with_dates` (devices with a resolution_date) + - Compute `compliance_pct` as `ROUND((total_assets - non_compliant) / total_assets * 100, 1)`, returning 0 when totalAssets is 0 + - Generate forecast months by iterating forward from current month, decrementing non_compliant as devices reach their resolution_date month + - Treat past-due resolution dates as remediated in the current month + - Hold total_assets constant across all forecast data points + - Terminate forecast when all dated devices are remediated or at 12-month maximum + - Return empty forecast array when all devices are blockers (no resolution dates) + - Export the function for use in route handlers and tests + - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 6.5_ + + - [ ]* 1.2 Write property test: Forecast structure invariant + - **Property 1: Forecast structure invariant** + - **Validates: Requirements 1.4, 3.1, 3.6** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + + - [ ]* 1.3 Write property test: Blocker and with_dates partition invariant + - **Property 2: Blocker and with_dates partition invariant** + - **Validates: Requirements 3.2** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + + - [ ]* 1.4 Write property test: Compliance percentage formula correctness + - **Property 3: Compliance percentage formula correctness** + - **Validates: Requirements 3.3, 3.10** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + + - [ ]* 1.5 Write property test: Forecast non_compliant monotonicity + - **Property 4: Forecast non_compliant monotonicity** + - **Validates: Requirements 3.4** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + + - [ ]* 1.6 Write property test: Per-month non_compliant computation correctness + - **Property 5: Per-month non_compliant computation correctness** + - **Validates: Requirements 3.8** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + + - [ ]* 1.7 Write property test: Forecast horizon bound + - **Property 6: Forecast horizon bound** + - **Validates: Requirements 1.9, 3.9** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + + - [ ]* 1.8 Write property test: Past-due resolution dates treated as current month + - **Property 7: Past-due resolution dates treated as current month** + - **Validates: Requirements 6.5** + - Test file: `backend/__tests__/forecast-burndown-chart.property.test.js` + +- [x] 2. Checkpoint - Helper function verified + - Ensure all tests pass, ask the user if questions arise. + +- [x] 3. Implement backend API endpoints + - [x] 3.1 Add GET /metrics-list endpoint to backend/routes/vclMultiVertical.js + - Add route handler at `/metrics-list` with `requireAuth()` middleware + - Query `compliance_items` for distinct metric_ids with active non-compliant devices where vertical IS NOT NULL + - Return JSON array of `{ metric_id, device_count }` sorted by metric_id ascending + - Return empty array when no metrics have active devices + - Return HTTP 500 with `{ "error": "Failed to fetch metrics list" }` on database failure + - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6_ + + - [x] 3.2 Add GET /metric/:metricId/forecast-burndown endpoint to backend/routes/vclMultiVertical.js + - Add route handler with `requireAuth()` middleware + - Query active devices for the metric from `compliance_items` (status = 'active', vertical IS NOT NULL) + - Determine the vertical from active devices and query `compliance_snapshots` for 3 months of historical data + - Compute per-metric historical non_compliant using the ratio method from Requirement 7.2 + - Include current month as the most recent historical data point computed from live data + - Pass data to `computeMetricForecastBurndown` helper + - Return response with `metric_id`, `historical`, `forecast`, and `current_snapshot` + - Return 200 with empty arrays and zeroed snapshot when metricId has no active devices + - Return HTTP 500 with `{ "error": "Failed to compute forecast burndown" }` on database failure + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7_ + + - [ ]* 3.3 Write unit tests for API endpoints + - Test metrics-list returns correct shape with mocked database + - Test forecast-burndown returns correct shape with mocked database + - Test authentication middleware is applied to both endpoints + - Test empty/error states + - Test file: `backend/__tests__/forecast-burndown-chart.test.js` + - _Requirements: 1.7, 1.8, 1.10, 2.4, 2.5, 2.6_ + +- [x] 4. Checkpoint - Backend complete + - Ensure all tests pass, ask the user if questions arise. + +- [x] 5. Implement frontend MetricSelector and ForecastBurndownChart components + - [x] 5.1 Add MetricSelector component to frontend/src/components/pages/CCPMetricsPage.js + - Fetch metrics list from `/api/compliance/vcl-multi/metrics-list` on mount + - Display dropdown showing each metric_id with active non-compliant device count + - Auto-select first metric and trigger forecast data fetch on load + - Handle loading state (non-interactive while fetching) + - Handle empty state ("No metrics with active non-compliant devices") + - Handle error state (inline error with AlertCircle icon, red border) + - On selection change, trigger `onMetricSelect` callback + - _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8_ + + - [x] 5.2 Add ForecastBurndownChart component to frontend/src/components/pages/CCPMetricsPage.js + - Fetch forecast data from `/api/compliance/vcl-multi/metric/:metricId/forecast-burndown` when metric changes + - Render recharts ComposedChart with: + - Blue Bar for total_assets (left Y-axis) + - Orange Bar for non_compliant (left Y-axis) + - Green Line for compliance_pct (right Y-axis, 0-100%) + - ReferenceLine as vertical divider between historical and forecast sections + - Render forecast data points at 50% opacity + - Display raw device count labels inside bars + - Display compliance percentage labels on the trend line + - X-axis labeled with months (YYYY-MM format) + - Left Y-axis scaled to max total_assets, right Y-axis 0-100% + - Handle loading state (loading indicator in chart area) + - Handle error state (inline error with AlertCircle icon and description) + - Handle empty data state ("No data available for this metric") + - Handle historical-only state (no divider line when forecast is empty) + - Discard stale responses on rapid metric switching (race condition handling) + - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9, 4.10, 4.11, 4.12, 4.13, 4.14, 5.9_ + + - [x] 5.3 Wire MetricSelector and ForecastBurndownChart into CCPMetricsPage layout + - Add state for selected metric + - Place MetricSelector above ForecastBurndownChart in a dedicated section + - Connect selection change to chart data fetch + - Follow existing inline-style patterns from AggregatedBurndownChart and TrendChart + - _Requirements: 4.1, 5.1, 5.4_ + +- [x] 6. Final checkpoint - Ensure all tests pass + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional and can be skipped for faster MVP +- Each task references specific requirements for traceability +- Checkpoints ensure incremental validation +- Property tests validate universal correctness properties from the design document +- The helper function is pure and stateless — all 7 property tests exercise it in isolation without database mocks +- All backend changes are in `backend/helpers/vclHelpers.js` and `backend/routes/vclMultiVertical.js` +- All frontend changes are within `frontend/src/components/pages/CCPMetricsPage.js` +- Property-based tests use `fast-check` (already in project dependencies) +- No database migrations needed — uses existing `compliance_items` and `compliance_snapshots` tables + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1"] }, + { "id": 1, "tasks": ["1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8"] }, + { "id": 2, "tasks": ["3.1", "3.2"] }, + { "id": 3, "tasks": ["3.3"] }, + { "id": 4, "tasks": ["5.1", "5.2"] }, + { "id": 5, "tasks": ["5.3"] } + ] +} +``` diff --git a/.kiro/specs/granite-loader-export/.config.kiro b/.kiro/specs/granite-loader-export/.config.kiro new file mode 100644 index 0000000..346c93f --- /dev/null +++ b/.kiro/specs/granite-loader-export/.config.kiro @@ -0,0 +1 @@ +version = "1.0" diff --git a/.kiro/specs/granite-loader-export/design.md b/.kiro/specs/granite-loader-export/design.md new file mode 100644 index 0000000..75683c8 --- /dev/null +++ b/.kiro/specs/granite-loader-export/design.md @@ -0,0 +1,319 @@ +# Design Document: Granite Loader Sheet Export + +## Overview + +This feature adds a Granite Team_Device Loader xlsx generator to the STEAM Security Dashboard. It integrates with the existing Ivanti Queue (for CARD/GRANITE items) and provides a standalone mode for ad-hoc device lists. The system enriches device data from the CARD API, allows bulk defaults with per-row overrides in an editable preview table, and generates a properly formatted xlsx for upload to SNIP XperLoad. + +Key design decisions: +- **Frontend-driven xlsx generation**: The xlsx is generated client-side using the `xlsx` library (already a project dependency) to avoid backend file I/O and temp file cleanup. +- **Backend CARD enrichment endpoint**: A single batch endpoint accepts an array of IPs and returns enriched Granite fields from CARD. This keeps the frontend simple and centralizes CARD API token management. +- **Reuse queue data model**: Queue items already have `ip_address`, `hostname`, and `workflow_type` — no schema changes needed. +- **Modal-based UI**: The loader configuration lives in a modal (like the Consolidation Modal pattern) to avoid adding a new page while keeping the queue page clean. + +## Architecture + +```mermaid +flowchart TD + subgraph Frontend + QP[IvantiTodoQueuePage] -->|selected CARD/GRANITE items| LM[LoaderModal] + NAV[Nav Drawer / Standalone] -->|manual IP list| LM + LM -->|enrich request| API[POST /api/card/enrich-batch] + LM -->|generate xlsx| XLSX[xlsx library - client-side] + XLSX -->|download| FILE[Loader_Change_TEAM_DATE.xlsx] + end + + subgraph Backend + API --> CARD[CARD API Helper] + CARD -->|per-IP lookup| EXT[card.charter.com] + end +``` + +## Components and Interfaces + +### Backend: Batch Enrichment Endpoint + +**File:** `backend/routes/cardApi.js` (added to existing router) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/api/card/enrich-batch` | Admin, Standard_User | Batch lookup IPs in CARD, return Granite-relevant fields | + +**Request:** +```json +{ + "ips": ["10.240.78.110", "10.240.78.111", "172.16.5.20"] +} +``` + +**Response:** +```json +{ + "results": [ + { + "ip": "10.240.78.110", + "found": true, + "equip_inst_id": "1931008", + "hostname": "NDCW-SW-CORE-01", + "site_name": "ENWDCOCD-PEAKVIEW-SRDC", + "mgmt_ip_asn": "11427", + "responsible_team": "NTS-AEO-STEAM", + "equipment_class": "S", + "equip_template": "DISC-CISCO NEXUS 9300", + "equip_status": "Active" + }, + { + "ip": "172.16.5.20", + "found": false, + "equip_inst_id": null, + "hostname": null, + "error": "IP not found in CARD" + } + ], + "enriched_count": 1, + "not_found_count": 1, + "total": 2 +} +``` + +**Implementation notes:** +- Accepts up to 200 IPs per request (matches Requirement 1.4 limit). +- For each IP, constructs asset ID candidates with known suffixes (CTEC, NATL, CHTR, etc.) and queries `GET /api/v1/owner/{assetId}`. +- Falls back to team asset search if direct owner lookup fails. +- Extracts fields from `ncim_discovery`, `netops_granite_allips`, `card_flags`, and `owner` on the asset record. +- Returns partial results on CARD API errors (best-effort enrichment). + +### Frontend: LoaderModal Component + +**File:** `frontend/src/components/LoaderModal.js` + +A modal component that handles the full loader sheet workflow: + +**Props:** +```javascript +{ + isOpen: boolean, + onClose: () => void, + // Pre-populated from queue selection (null in standalone mode) + initialDevices: Array<{ ip_address: string, hostname: string }> | null, +} +``` + +**Internal State:** +```javascript +{ + operationType: 'Change' | 'Add' | 'Delete' | 'Move', + selectedColumns: Set, // checked column IDs + devices: Array, // the editable row data + bulkDefaults: Record, // column → default value + overrides: Record>, // rowIndex → column → value + enriching: boolean, + enrichErrors: Array<{ ip: string, error: string }>, +} +``` + +**DeviceRow shape:** +```javascript +{ + ip_address: string, + hostname: string, + // CARD-enriched fields (populated after enrichment) + equip_inst_id: string | null, + site_name: string | null, + mgmt_ip_asn: string | null, + responsible_team: string | null, + equipment_class: string | null, + equip_template: string | null, + equip_status: string | null, +} +``` + +### Frontend: Column Configuration + +**File:** `frontend/src/utils/graniteLoaderConfig.js` + +Pure data module defining the 41 columns, their groupings, and operation-type requirements: + +```javascript +export const LOADER_COLUMNS = [ + { id: 'DELETE', label: 'DELETE', group: 'Identification', requiredFor: ['Delete'] }, + { id: 'SET_CONFIRMED', label: 'SET_CONFIRMED', group: 'Identification', requiredFor: [] }, + { id: 'EQUIPMENT_CLASS', label: 'EQUIPMENT CLASS', group: 'Identification', requiredFor: ['Add'] }, + { id: 'EQUIP_INST_ID', label: 'EQUIP_INST_ID', group: 'Identification', requiredFor: ['Change', 'Move', 'Delete'] }, + { id: 'SITE_NAME', label: 'SITE_NAME', group: 'Identification', requiredFor: ['Add', 'Move'] }, + { id: 'EQUIP_NAME', label: 'EQUIP_NAME', group: 'Identification', requiredFor: ['Add'] }, + { id: 'EQUIP_TEMPLATE', label: 'EQUIP_TEMPLATE', group: 'Identification', requiredFor: ['Add'] }, + { id: 'EQUIP_STATUS', label: 'EQUIP_STATUS', group: 'Identification', requiredFor: ['Add'] }, + { id: 'RESPONSIBLE_TEAM', label: 'UDA#RESPONSIBLE ORGANIZATION#RESPONSIBLE TEAM', group: 'Responsible Org', requiredFor: ['Add'] }, + { id: 'IPV4_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV4_ADDRESS', group: 'IP Addressing', requiredFor: ['Add'] }, + { id: 'MAC_ADDRESS', label: 'UDA#IP_ADDRESSING#MAC ADDRESS', group: 'IP Addressing', requiredFor: [] }, + { id: 'MGMT_IP_ASN', label: 'UDA#IP_ADDRESSING#MGMT_IP_ASN', group: 'IP Addressing', requiredFor: ['Add'] }, + { id: 'SERIALNUMBER', label: 'SERIALNUMBER', group: 'Equipment Info', requiredFor: [] }, + { id: 'EXCLUDED_DISCOVERY', label: 'UDA#AUTO DISCOVERY#EXCLUDED FROM DISCOVERY', group: 'Discovery', requiredFor: [] }, + { id: 'EXCLUDED_REASON', label: 'UDA#AUTO DISCOVERY#EXCLUDED FROM DISCOVERY REASON', group: 'Discovery', requiredFor: [] }, + { id: 'IPV6_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV6_ADDRESS', group: 'IP Addressing', requiredFor: [] }, + { id: 'ILOM_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV4_ILOM_ADDRESS', group: 'IP Addressing', requiredFor: [] }, + { id: 'APP_ID_ASSET_TAG', label: 'UDA#CHERWELL_CMDB#APP ID ASSET TAG', group: 'Cyber Metrics', requiredFor: [] }, + { id: 'DEVICE_FUNCTION', label: 'UDA#CHERWELL_CMDB#DEVICE_FUNCTION', group: 'Cyber Metrics', requiredFor: [] }, + { id: 'ENVIRONMENT', label: 'UDA#CHERWELL_CMDB#ENVIRONMENT', group: 'Cyber Metrics', requiredFor: [] }, + { id: 'SECONDARY_MGMT_IP', label: 'UDA#IP_ADDRESSING#SECONDARY_MGMT_IP_ADDRESS', group: 'IP Addressing', requiredFor: [] }, + { id: 'VIP', label: 'UDA#IP_ADDRESSING#VIP', group: 'IP Addressing', requiredFor: [] }, + { id: 'FLOATING_IP', label: 'UDA#IP_ADDRESSING#FLOATING IP ADDRESS', group: 'IP Addressing', requiredFor: [] }, + { id: 'SCAN_IP_1', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 1', group: 'IP Addressing', requiredFor: [] }, + { id: 'SCAN_IP_2', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 2', group: 'IP Addressing', requiredFor: [] }, + { id: 'SCAN_IP_3', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 3', group: 'IP Addressing', requiredFor: [] }, + { id: 'EQUIP_MODEL', label: 'EQUIP_MODEL', group: 'Equipment Info', requiredFor: [] }, + { id: 'EQUIP_COMMENTS', label: 'EQUIP_COMMENTS', group: 'Equipment Info', requiredFor: [] }, + { id: 'EQUIP_PARTNUMBER', label: 'EQUIP_PARTNUMBER', group: 'Equipment Info', requiredFor: [] }, + { id: 'OS', label: 'UDA#CONTROLLER CONFIG#OS', group: 'Equipment Info', requiredFor: [] }, + { id: 'OS_VERSION', label: 'UDA#CONTROLLER CONFIG#OS VERSION', group: 'Equipment Info', requiredFor: [] }, + { id: 'CPU_CORES', label: 'UDA#CONTROLLER CONFIG#TOTAL CPU CORES', group: 'Equipment Info', requiredFor: [] }, + { id: 'RAM_GB', label: 'UDA#CONTROLLER CONFIG#RAM IN GB', group: 'Equipment Info', requiredFor: [] }, + { id: 'STORAGE_GB', label: 'UDA#CONTROLLER CONFIG#STORAGE IN GB', group: 'Equipment Info', requiredFor: [] }, + { id: 'ARCHER_ID', label: 'UDA#WIFI EQUIP INFO#ARCHER ID', group: 'Other', requiredFor: [] }, + { id: 'INSTALL_LOCATION', label: 'UDA#WIFI EQUIP INFO#INSTALL LOCATION', group: 'Other', requiredFor: [] }, + { id: 'SYSNAME', label: 'UDA#EQUIPMENT_INFO#SYSNAME', group: 'Other', requiredFor: [] }, + { id: 'LATITUDE', label: 'UDA#WIFI EQUIP INFO#LATITUDE', group: 'Other', requiredFor: [] }, + { id: 'LONGITUDE', label: 'UDA#WIFI EQUIP INFO#LONGITUDE', group: 'Other', requiredFor: [] }, + { id: 'OSTYPE', label: 'UDA#EQUIP MIGRATION#OSTYPE', group: 'Other', requiredFor: [] }, + { id: 'OSVERSION', label: 'UDA#EQUIP MIGRATION#OSVERSION', group: 'Other', requiredFor: [] }, +]; + +export const COLUMN_GROUPS = [ + 'Identification', + 'IP Addressing', + 'Responsible Org', + 'Discovery', + 'Cyber Metrics', + 'Equipment Info', + 'Other', +]; + +export const OPERATION_TYPES = ['Change', 'Add', 'Delete', 'Move']; +``` + +### Frontend: XLSX Generation + +**File:** `frontend/src/utils/graniteLoaderExport.js` + +Pure function that takes the configured state and produces an xlsx workbook: + +```javascript +/** + * @param {Object} config + * @param {string} config.operationType - 'Change' | 'Add' | 'Delete' | 'Move' + * @param {Array} config.columnIds - ordered list of selected column IDs + * @param {Array} config.rows - device rows with resolved values (bulk + overrides merged) + * @returns {Blob} xlsx file as a Blob for download + */ +export function generateLoaderXlsx(config) { ... } +``` + +Uses the `xlsx` library (already in `frontend/package.json`) to create a workbook with a single "Load_Sheet" worksheet. + +## Data Flow + +### Queue-Initiated Flow + +1. User selects CARD/GRANITE items on IvantiTodoQueuePage +2. User clicks "Generate Loader Sheet" in floating action bar +3. LoaderModal opens with `initialDevices` populated from selected items +4. User selects Operation Type (defaults to "Change") +5. User checks desired columns (required columns pre-checked) +6. User optionally clicks "Enrich from CARD" → `POST /api/card/enrich-batch` +7. EQUIP_INST_ID and other fields populate in the preview table +8. User sets bulk defaults and per-row overrides as needed +9. User clicks "Download" → client-side xlsx generation → browser download + +### Standalone Flow + +1. User navigates to standalone access point (nav drawer link or CARD page section) +2. LoaderModal opens with empty device list +3. User pastes IPs (textarea, one per line or comma-separated) → rows populate +4. Steps 4–9 same as above + +### CARD Enrichment Flow (Backend) + +1. Frontend sends `POST /api/card/enrich-batch` with array of IPs +2. Backend iterates IPs, for each: + a. Try `GET /api/v1/owner/{ip}-CTEC`, then `-NATL`, then `-CHTR` (known suffixes) + b. If found, extract fields from the asset record + c. If not found via owner lookup, search team assets for the IP +3. Return results array with found/not-found status per IP + +## Error Handling + +| Scenario | Behavior | +|----------|----------| +| CARD API not configured | "Enrich from CARD" button hidden; tooltip explains why | +| CARD API timeout on individual IP | Mark that IP as not-found, continue with others | +| CARD API auth failure | Show error toast, abort enrichment, preserve any already-enriched data | +| All IPs not found in CARD | Show warning banner "No devices found in CARD — enter EQUIP_INST_ID manually" | +| Required field missing on download | Highlight cells, show warning count, allow download with acknowledgment | +| xlsx generation failure | Show error toast with message | +| More than 200 IPs submitted | Frontend truncates to 200 with warning message | + +## UI Layout + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Generate Granite Loader Sheet [X] │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ Operation: [Change ▾] Devices: 24 items │ +│ │ +│ ┌─ Columns ──────────────────────────────────────────────────┐ │ +│ │ ▸ Identification (4 selected) │ │ +│ │ ▸ IP Addressing (2 selected) │ │ +│ │ ▸ Responsible Org (1 selected) │ │ +│ │ ▸ Cyber Metrics (0 selected) │ │ +│ │ ▸ Equipment Info (0 selected) │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ [Enrich from CARD] │ +│ │ +│ ┌─ Bulk Defaults ────────────────────────────────────────────┐ │ +│ │ RESPONSIBLE TEAM: [NTS-AEO-STEAM ] │ │ +│ │ EQUIP_STATUS: [Active ] │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌─ Preview (24 rows) ────────────────────────────────────────┐ │ +│ │ IP Address │ EQUIP_INST_ID │ RESP TEAM │ STATUS │ │ +│ │───────────────┼───────────────┼────────────────┼───────────│ │ +│ │ 10.240.78.110 │ 1931008 │ NTS-AEO-STEAM │ Active │ │ +│ │ 10.240.78.111 │ 1931009 │ NTS-AEO-STEAM │ Active │ │ +│ │ 172.16.5.20 │ ⚠ (not found) │ ACCESS-ENG ● │ Active │ │ +│ │ 172.16.5.21 │ 2045112 │ NTS-AEO-STEAM │ Active │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ⚠ 1 row missing EQUIP_INST_ID │ +│ │ +│ [Cancel] [Download Loader Sheet] │ +└─────────────────────────────────────────────────────────────────┘ +``` + +- ● = per-row override indicator (amber dot) +- ⚠ = missing required field or CARD lookup failure + +## Testing Strategy + +### Property-Based Tests + +**File:** `backend/__tests__/granite-loader-enrichment.property.test.js` + +- **Property 1: Enrichment result count** — For any array of N IPs (1 ≤ N ≤ 200), the response contains exactly N result objects. +- **Property 2: Found results have required fields** — For any result where `found === true`, `equip_inst_id` is a non-empty string. +- **Property 3: Not-found results have null fields** — For any result where `found === false`, `equip_inst_id` is null. + +### Unit Tests + +- Column configuration: required columns for each operation type +- XLSX generation: correct headers, correct row data, empty cells handled +- Bulk default + override merge logic +- IP validation +- EQUIP_INST_ID numeric validation + +### Integration Tests + +- `POST /api/card/enrich-batch` with mocked CARD API responses +- Auth requirement enforcement +- 200-IP limit enforcement diff --git a/.kiro/specs/granite-loader-export/requirements.md b/.kiro/specs/granite-loader-export/requirements.md new file mode 100644 index 0000000..1df8718 --- /dev/null +++ b/.kiro/specs/granite-loader-export/requirements.md @@ -0,0 +1,122 @@ +# Requirements Document: Granite Loader Sheet Export + +## Introduction + +The STEAM Security Dashboard's Ivanti Queue already stages findings for CARD and GRANITE workflows. Once those items are worked, the next step is often submitting changes to Granite via the Team_Device Loader xlsx uploaded through SNIP XperLoad. Currently this requires manually downloading the 5MB template, finding EQUIP_INST_IDs via SNIP, and hand-filling rows — a tedious process for batches of 20+ devices. + +This feature adds a "Generate Loader Sheet" action accessible from the Ivanti Queue that produces a properly formatted Team_Device Loader xlsx pre-populated with device data from the queue items, optionally enriched with EQUIP_INST_ID and other fields from the CARD API. Users select which columns they need, set bulk defaults with per-row overrides, and download a ready-to-upload xlsx. + +## Glossary + +- **Loader_Sheet**: The first worksheet ("Load_Sheet") in the Team_Device Loader xlsx workbook, containing the 41-column format that SNIP XperLoad accepts. +- **EQUIP_INST_ID**: The unique Granite identifier for an asset record. Required for Changes, Moves, and Deletes. +- **CARD_Enrichment**: The process of looking up an IP address in the CARD API to retrieve EQUIP_INST_ID, hostname, ASN, site, and other Granite-relevant fields. +- **Bulk_Default**: A value applied to all rows in a selected column. Can be overridden per-row. +- **Operation_Type**: One of Add, Change, Move, or Delete — determines which columns are required. +- **XperLoad**: The SNIP bulk loader tool that accepts the Team_Device Loader xlsx. + +## Requirements + +### Requirement 1: Generate Loader Sheet Action from Ivanti Queue + +**User Story:** As a security analyst, I want to generate a Granite Loader Sheet directly from my CARD/GRANITE queue items, so that I don't have to manually look up device data and fill the template by hand. + +#### Acceptance Criteria + +1. WHEN the user has one or more CARD or GRANITE queue items selected (pending or completed), THE Queue page SHALL display a "Generate Loader Sheet" action button in the floating action bar. +2. WHEN the user clicks "Generate Loader Sheet", THE system SHALL open a Loader Configuration Modal pre-populated with the IP addresses and hostnames from the selected queue items. +3. THE action SHALL be available to users in the Admin and Standard_User groups. +4. THE action SHALL support any number of selected items from 1 to 200. +5. IF no CARD or GRANITE items are selected, THE "Generate Loader Sheet" button SHALL NOT appear. + +### Requirement 2: Operation Type Selection + +**User Story:** As a security analyst, I want to specify what type of Granite operation I'm performing, so that the generated sheet includes the correct required columns. + +#### Acceptance Criteria + +1. THE Loader Configuration Modal SHALL present an Operation Type selector with options: Change, Add, Delete, Move. +2. WHEN "Change" is selected, THE system SHALL require EQUIP_INST_ID (or SITE_NAME + EQUIP_NAME) as an identifier and allow any other column as an optional change field. +3. WHEN "Add" is selected, THE system SHALL require: EQUIPMENT CLASS, SITE_NAME, EQUIP_NAME, EQUIP_TEMPLATE, EQUIP_STATUS, UDA#RESPONSIBLE ORGANIZATION#RESPONSIBLE TEAM, UDA#IP_ADDRESSING#IPV4_ADDRESS, UDA#IP_ADDRESSING#MGMT_IP_ASN. +4. WHEN "Delete" is selected, THE system SHALL require DELETE column (auto-filled with "X") and EQUIP_INST_ID (or SITE_NAME + EQUIP_NAME). +5. WHEN "Move" is selected, THE system SHALL require EQUIP_INST_ID and SITE_NAME (the new site). +6. THE default Operation Type SHALL be "Change" since queue items are typically existing assets needing updates. + +### Requirement 3: Column Selection + +**User Story:** As a security analyst, I want to choose which columns to include in my loader sheet, so that I only fill in what's relevant to my specific task. + +#### Acceptance Criteria + +1. THE Loader Configuration Modal SHALL display a list of available columns as checkboxes, grouped by category (Identification, IP Addressing, Discovery, Responsible Org, Cyber Metrics, Equipment Info, Other). +2. Columns required by the selected Operation Type SHALL be pre-checked and non-dismissable. +3. THE user SHALL be able to check additional optional columns beyond the required set. +4. THE generated xlsx SHALL only include columns that are checked (required + user-selected optional). +5. THE column order in the generated xlsx SHALL match the canonical Team_Device Loader column order (DELETE first, then SET_CONFIRMED, EQUIPMENT CLASS, EQUIP_INST_ID, etc.). + +### Requirement 4: Device List with Bulk Defaults and Per-Row Overrides + +**User Story:** As a security analyst, I want to set a default value for a column that applies to all rows, but override it on specific rows when devices need different values. + +#### Acceptance Criteria + +1. THE Loader Configuration Modal SHALL display a preview table showing all devices (rows) and selected columns. +2. FOR each selected column, THE modal SHALL provide a "Bulk Default" input above the table that sets the value for all rows in that column. +3. WHEN a Bulk Default is set or changed, ALL rows in that column that have not been individually overridden SHALL update to the new default value. +4. THE user SHALL be able to click any cell in the preview table to edit its value inline, creating a per-row override. +5. Cells with per-row overrides SHALL display a visual indicator (amber dot) distinguishing them from bulk-defaulted cells. +6. THE user SHALL be able to clear a per-row override to revert a cell back to the bulk default. +7. THE preview table SHALL be scrollable for large device lists (20+ items) while keeping column headers and the bulk default row sticky. + +### Requirement 5: CARD API Enrichment + +**User Story:** As a security analyst, I want to automatically look up EQUIP_INST_ID and other Granite fields from the CARD API using the device IP, so that I don't have to manually search SNIP for each device. + +#### Acceptance Criteria + +1. THE Loader Configuration Modal SHALL include an "Enrich from CARD" toggle/button. +2. WHEN "Enrich from CARD" is activated, THE system SHALL look up each device's IP address via the CARD API and populate available fields: EQUIP_INST_ID, EQUIP_NAME (hostname), UDA#IP_ADDRESSING#MGMT_IP_ASN, SITE_NAME (if discoverable from CARD data). +3. THE enrichment SHALL display a progress indicator while lookups are in progress. +4. IF a device IP is not found in CARD, THE system SHALL leave the EQUIP_INST_ID cell empty and display a warning indicator on that row. +5. CARD-enriched values SHALL be treated as pre-populated defaults that can still be overridden per-row. +6. THE enrichment SHALL NOT overwrite values the user has already manually entered or overridden. +7. THE system SHALL handle CARD API errors gracefully — partial enrichment is acceptable (enrich what you can, warn on failures). +8. THE CARD enrichment endpoint SHALL require the CARD API to be configured (CARD_API_URL, CARD_API_USER, CARD_API_PASS set in .env). If not configured, the "Enrich from CARD" option SHALL be hidden. + +### Requirement 6: XLSX Generation and Download + +**User Story:** As a security analyst, I want to download a properly formatted xlsx file that I can upload directly to SNIP XperLoad without further editing. + +#### Acceptance Criteria + +1. WHEN the user clicks "Download", THE system SHALL generate an xlsx file with a single worksheet named "Load_Sheet". +2. THE first row SHALL contain column headers matching the exact canonical Team_Device Loader column names (e.g., "UDA#RESPONSIBLE ORGANIZATION#RESPONSIBLE TEAM", not abbreviated). +3. Each subsequent row SHALL contain the device data as configured in the preview table (bulk defaults + per-row overrides). +4. THE DELETE column SHALL be auto-populated with "X" for every row when Operation Type is "Delete". +5. THE EQUIPMENT CLASS column SHALL default to "S" (Shelf) unless overridden. +6. THE generated file SHALL be named with a descriptive pattern: `Loader_{operation}_{team}_{date}.xlsx` (e.g., `Loader_Change_NTS-AEO-STEAM_2026-05-27.xlsx`). +7. THE xlsx SHALL NOT include the reference data sheets (site names, templates, etc.) — only the Load_Sheet with data. This keeps the file small and focused. +8. Empty cells SHALL remain empty (not "null" or "undefined"). + +### Requirement 7: Standalone Access (Outside Queue Context) + +**User Story:** As a security analyst, I want to access the Loader Sheet generator without going through the queue, for cases where I have a list of devices that aren't in my queue. + +#### Acceptance Criteria + +1. THE Loader Sheet generator SHALL be accessible as a standalone tool from the navigation drawer or a dedicated page section. +2. IN standalone mode, THE user SHALL be able to paste a list of IP addresses (one per line or comma-separated) to populate the device list. +3. IN standalone mode, THE user SHALL be able to manually add rows and fill in device identifiers (IP, hostname, or EQUIP_INST_ID). +4. ALL other functionality (operation type, column selection, bulk defaults, per-row overrides, CARD enrichment, xlsx download) SHALL work identically in standalone mode and queue-initiated mode. + +### Requirement 8: Validation and Error Prevention + +**User Story:** As a security analyst, I want the tool to warn me about missing required fields before I download, so that I don't upload an incomplete sheet to XperLoad. + +#### Acceptance Criteria + +1. BEFORE generating the xlsx, THE system SHALL validate that all required columns for the selected Operation Type have values in every row. +2. IF required fields are missing, THE system SHALL highlight the empty cells in red and display a summary warning (e.g., "3 rows missing EQUIP_INST_ID"). +3. THE user SHALL be able to proceed with download despite warnings (XperLoad will reject invalid rows anyway), but must explicitly acknowledge the warning. +4. THE system SHALL validate IP address format (IPv4 pattern) for the IPV4_ADDRESS column if populated. +5. THE system SHALL validate EQUIP_INST_ID is numeric when populated. diff --git a/.kiro/specs/granite-loader-export/tasks.md b/.kiro/specs/granite-loader-export/tasks.md new file mode 100644 index 0000000..d0685e4 --- /dev/null +++ b/.kiro/specs/granite-loader-export/tasks.md @@ -0,0 +1,151 @@ +# Implementation Plan: Granite Loader Sheet Export + +## Overview + +Add a Granite Team_Device Loader xlsx generator accessible from the Ivanti Queue (for CARD/GRANITE items) and as a standalone tool. The system enriches device data from the CARD API, allows bulk defaults with per-row overrides in an editable preview table, and generates a properly formatted xlsx for upload to SNIP XperLoad. Implementation proceeds from column configuration utility → backend enrichment endpoint → frontend modal → queue integration → standalone access. + +## Tasks + +- [ ] 1. Create column configuration utility module + - [ ] 1.1 Create `frontend/src/utils/graniteLoaderConfig.js` + - Define `LOADER_COLUMNS` array with all 41 columns: id, label (exact Granite header name), group, requiredFor array + - Define `COLUMN_GROUPS` ordered array for UI grouping + - Define `OPERATION_TYPES` array: Change, Add, Delete, Move + - Export `getRequiredColumns(operationType)` helper that returns column IDs required for the given operation + - Export `getColumnsByGroup(group)` helper that returns columns in a group + - _Requirements: 2.1–2.6, 3.1–3.5_ + + - [ ] 1.2 Create `frontend/src/utils/graniteLoaderExport.js` + - Export `generateLoaderXlsx(config)` function that accepts `{ operationType, columnIds, rows }` and returns a Blob + - Use the `xlsx` library (already in frontend dependencies) to create a workbook with a single "Load_Sheet" worksheet + - First row contains exact canonical column headers from LOADER_COLUMNS (matched by columnIds, in canonical order) + - Subsequent rows contain device data; empty values become empty cells (not "null") + - DELETE column auto-filled with "X" when operationType is "Delete" + - EQUIPMENT CLASS defaults to "S" unless overridden + - Export `generateFilename(operationType, teamName)` helper returning `Loader_{op}_{team}_{YYYY-MM-DD}.xlsx` + - _Requirements: 6.1–6.8_ + +- [ ] 2. Backend CARD enrichment endpoint + - [ ] 2.1 Add `POST /api/card/enrich-batch` endpoint in `backend/routes/cardApi.js` + - Accept `{ ips: string[] }` in request body + - Validate: ips is a non-empty array, max 200 items, each item is a non-empty string + - Require Admin or Standard_User group + - Require CARD API to be configured (return 503 if not) + - For each IP, attempt owner lookup with known suffixes (CTEC, NATL, CHTR, etc.) + - Extract from asset record: equip_inst_id (from ncim_discovery, netops_granite_allips, or ise_granite_equipment), hostname, site_name, mgmt_ip_asn, responsible_team, equipment_class, equip_template, equip_status + - Return `{ results: [...], enriched_count, not_found_count, total }` + - Handle per-IP errors gracefully (mark as not-found, continue with others) + - Handle CARD API auth failures (return 502 with error message) + - _Requirements: 5.1–5.8_ + + - [ ] 2.2 Add `GET /api/card/configured` endpoint (or extend existing `/api/card/status`) + - Return `{ configured: boolean }` so the frontend knows whether to show the "Enrich from CARD" option + - This may already exist as `GET /api/card/status` — verify and reuse if so + - _Requirements: 5.8_ + +- [ ] 3. Frontend LoaderModal component + - [ ] 3.1 Create `frontend/src/components/LoaderModal.js` + - Accept props: `isOpen`, `onClose`, `initialDevices` (array of `{ ip_address, hostname }` or null) + - Render modal overlay with header "Generate Granite Loader Sheet" + - Include Operation Type selector (dropdown, defaults to "Change") + - Include Column Selection panel with collapsible groups and checkboxes + - Required columns for selected operation are pre-checked and disabled + - Include "Enrich from CARD" button (hidden if CARD not configured, checked via `/api/card/status` on mount) + - Include Bulk Defaults section: one input per selected column, setting value applies to all non-overridden rows + - Include editable Preview Table: rows = devices, columns = selected columns + - Cells are inline-editable on click; overridden cells show amber dot indicator + - Right-click or clear button on overridden cell reverts to bulk default + - Sticky column headers and bulk default row when scrolling + - Validation: highlight missing required fields in red, show warning count + - Download button: merges bulk defaults + overrides into final row data, calls `generateLoaderXlsx`, triggers browser download + - Cancel button closes modal + - _Requirements: 1.2, 2.1–2.6, 3.1–3.5, 4.1–4.7, 6.1–6.8, 8.1–8.5_ + + - [ ] 3.2 Implement CARD enrichment flow in LoaderModal + - On "Enrich from CARD" click, collect all device IPs, POST to `/api/card/enrich-batch` + - Show progress indicator during request + - On response, populate enriched fields into device rows (equip_inst_id, hostname, site_name, mgmt_ip_asn, etc.) + - Do NOT overwrite values the user has already manually entered + - Show warning indicators on rows where IP was not found + - Show error toast if CARD API auth fails + - _Requirements: 5.1–5.7_ + + - [ ] 3.3 Implement standalone mode (paste IPs) + - When `initialDevices` is null, show a textarea for pasting IPs (one per line or comma-separated) + - Parse input into device rows on "Load" button click + - Allow manually adding/removing rows via + and trash icons + - _Requirements: 7.1–7.4_ + +- [ ] 4. Integrate with Ivanti Queue page + - [ ] 4.1 Add "Generate Loader Sheet" button to IvantiTodoQueuePage floating action bar + - Show button when one or more selected items have workflow_type CARD or GRANITE + - Button label: "Generate Loader Sheet" with a FileSpreadsheet icon + - On click, open LoaderModal with `initialDevices` populated from selected items' ip_address and hostname + - _Requirements: 1.1–1.5_ + + - [ ] 4.2 Add standalone access point + - Add "Granite Loader" link in the navigation drawer under Tools section (or similar) + - Clicking opens LoaderModal in standalone mode (initialDevices = null) + - Alternatively, add a "Generate Loader Sheet" button on the CARD status section if one exists + - _Requirements: 7.1_ + +- [ ] 5. Checkpoint — Verify build and basic functionality + - Build frontend: `cd frontend && npm run build` + - Verify no lint errors or build failures + - Ensure all existing tests still pass + - Ask the user if questions arise + +- [ ]* 6. Property-based tests for enrichment endpoint + - [ ]* 6.1 Write property test: Enrichment result count + - **Property 1: Result count equals input count** — For any array of N IPs (1 ≤ N ≤ 200), the response `results` array has exactly N elements + - File: `backend/__tests__/granite-loader-enrichment.property.test.js` + - **Validates: Requirements 5.1, 5.2** + + - [ ]* 6.2 Write property test: Found results have equip_inst_id + - **Property 2: Found results have required fields** — For any result where `found === true`, `equip_inst_id` is a non-empty string + - **Validates: Requirements 5.2** + + - [ ]* 6.3 Write property test: Not-found results have null fields + - **Property 3: Not-found results have null equip_inst_id** — For any result where `found === false`, `equip_inst_id` is null + - **Validates: Requirements 5.4** + +- [ ]* 7. Unit tests for xlsx generation + - [ ]* 7.1 Write unit tests for `generateLoaderXlsx` + - Test correct column headers in canonical order + - Test DELETE column auto-fill for Delete operation + - Test EQUIPMENT CLASS defaults to "S" + - Test empty values produce empty cells (not "null" string) + - Test bulk default + override merge produces correct row values + - File: `backend/__tests__/granite-loader-xlsx-generation.test.js` + - **Validates: Requirements 6.1–6.8** + +- [ ] 8. Final checkpoint + - Build frontend and verify no regressions + - Ensure all tests pass + - Ask the user if questions arise + +## Notes + +- Tasks marked with `*` are optional property-based and unit tests that can be skipped for faster MVP +- The `xlsx` library is already a frontend dependency — no new packages needed for xlsx generation +- The CARD API enrichment reuses the existing `cardApi.js` helper (token management, TLS skip, etc.) +- No database schema changes are required — this feature reads from queue items and CARD API only +- The LoaderModal follows the same pattern as ConsolidationModal (modal overlay, form state, action buttons) +- The preview table follows the same inline-edit pattern as the Reporting page (click to edit, amber dot for overrides) +- Maximum 200 devices per batch aligns with CARD API pagination limits and practical XperLoad batch sizes + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1", "1.2"] }, + { "id": 1, "tasks": ["2.1", "2.2"] }, + { "id": 2, "tasks": ["3.1"] }, + { "id": 3, "tasks": ["3.2", "3.3"] }, + { "id": 4, "tasks": ["4.1", "4.2"] }, + { "id": 5, "tasks": ["5"] }, + { "id": 6, "tasks": ["6.1", "6.2", "6.3", "7.1"] } + ] +} +``` diff --git a/.kiro/specs/ivanti-queue-clear-completed-fix/.config.kiro b/.kiro/specs/ivanti-queue-clear-completed-fix/.config.kiro new file mode 100644 index 0000000..fa022fe --- /dev/null +++ b/.kiro/specs/ivanti-queue-clear-completed-fix/.config.kiro @@ -0,0 +1 @@ +{"specId": "3e04c019-5516-446d-867c-bdb52b4f516a", "workflowType": "requirements-first", "specType": "bugfix"} diff --git a/.kiro/specs/ivanti-queue-clear-completed-fix/bugfix.md b/.kiro/specs/ivanti-queue-clear-completed-fix/bugfix.md new file mode 100644 index 0000000..22b7f2f --- /dev/null +++ b/.kiro/specs/ivanti-queue-clear-completed-fix/bugfix.md @@ -0,0 +1,29 @@ +# Bugfix Requirements Document + +## Introduction + +The "Clear Completed" button in the Ivanti Queue panel fails silently when clicked. Completed queue items that have associated rows in the `jira_ticket_queue_items` junction table cannot be deleted because the foreign key constraint (`queue_item_id REFERENCES ivanti_todo_queue(id)`) lacks `ON DELETE CASCADE`. PostgreSQL rejects the deletion with a FK violation, the backend returns a 500, and the frontend discards the error — leaving the user with no feedback and no action taken. + +Queue items are marked complete once a Jira ticket has been opened from them. The Jira ticket continues to live independently in the `jira_tickets` table — the junction table link is purely historical at that point. Clearing completed items should remove both the queue item and its junction table references without affecting the Jira ticket itself. + +## Bug Analysis + +### Current Behavior (Defect) + +1.1 WHEN a user clicks "Clear Completed" and one or more completed queue items have associated rows in `jira_ticket_queue_items` THEN the system returns a 500 error due to a foreign key violation and no items are deleted + +1.2 WHEN the backend DELETE query fails with a FK constraint error THEN the system returns a generic 500 response and the frontend silently ignores the failure, providing no user feedback + +### Expected Behavior (Correct) + +2.1 WHEN a user clicks "Clear Completed" and one or more completed queue items have associated rows in `jira_ticket_queue_items` THEN the system SHALL first delete the associated `jira_ticket_queue_items` rows and then delete the completed queue items successfully within a transaction + +2.2 WHEN the "Clear Completed" operation succeeds THEN the system SHALL return a success response and the frontend SHALL remove all completed items from the displayed list + +### Unchanged Behavior (Regression Prevention) + +3.1 WHEN a user clicks "Clear Completed" and no completed queue items have associated rows in `jira_ticket_queue_items` THEN the system SHALL CONTINUE TO delete those items directly and return a success response + +3.2 WHEN a user clicks "Clear Completed" and there are no completed queue items THEN the system SHALL CONTINUE TO return a success response with zero deleted count + +3.3 WHEN queue items are in a non-complete status (pending, in_progress) THEN the system SHALL CONTINUE TO leave those items untouched regardless of whether they have associated `jira_ticket_queue_items` rows diff --git a/.kiro/specs/ivanti-queue-clear-completed-fix/design.md b/.kiro/specs/ivanti-queue-clear-completed-fix/design.md new file mode 100644 index 0000000..ebd4ed4 --- /dev/null +++ b/.kiro/specs/ivanti-queue-clear-completed-fix/design.md @@ -0,0 +1,199 @@ +# Ivanti Queue Clear Completed Bugfix Design + +## Overview + +The "Clear Completed" endpoint (`DELETE /api/ivanti/todo-queue/completed`) fails with a 500 error when completed queue items have associated rows in the `jira_ticket_queue_items` junction table. The foreign key constraint on `queue_item_id REFERENCES ivanti_todo_queue(id)` lacks `ON DELETE CASCADE`, so PostgreSQL rejects the deletion. The fix wraps the operation in a transaction that deletes junction table references before deleting the queue items themselves. + +## Glossary + +- **Bug_Condition (C)**: The condition that triggers the bug — when one or more completed queue items have associated rows in `jira_ticket_queue_items` +- **Property (P)**: The desired behavior — all completed items (with or without junction table links) are deleted atomically, along with their junction table references +- **Preservation**: Existing behavior for items without junction table links, empty result sets, and non-complete items must remain unchanged +- **`pool`**: The PostgreSQL connection pool exported from `backend/db.js` +- **`jira_ticket_queue_items`**: Junction table linking `jira_tickets` to `ivanti_todo_queue` items (created by `add_multi_item_jira_ticket.js` migration) +- **`ivanti_todo_queue`**: Main queue table storing user work items with `status` column (`pending`, `complete`) + +## Bug Details + +### Bug Condition + +The bug manifests when a user clicks "Clear Completed" and at least one of their completed queue items has a row in `jira_ticket_queue_items` referencing it. The current handler issues a bare `DELETE FROM ivanti_todo_queue WHERE user_id = $1 AND status = 'complete'` which PostgreSQL rejects because child rows exist in the junction table. + +**Formal Specification:** +``` +FUNCTION isBugCondition(input) + INPUT: input of type ClearCompletedRequest + OUTPUT: boolean + + completedItems := SELECT id FROM ivanti_todo_queue + WHERE user_id = input.userId AND status = 'complete' + + linkedItems := SELECT queue_item_id FROM jira_ticket_queue_items + WHERE queue_item_id IN completedItems + + RETURN linkedItems.length > 0 +END FUNCTION +``` + +### Examples + +- User has 3 completed items, 2 have junction table links → DELETE fails with FK violation, 0 items deleted (bug) +- User has 1 completed item with a junction table link → DELETE fails, item remains (bug) +- User has 5 completed items, all have junction table links → DELETE fails, all remain (bug) +- User has 3 completed items, none have junction table links → DELETE succeeds (no bug, preservation case) + +## Expected Behavior + +### Preservation Requirements + +**Unchanged Behaviors:** +- Completed items without junction table links are deleted directly and a success response is returned +- When no completed items exist, the endpoint returns `{ message: 'Completed items cleared.', deleted: 0 }` +- Pending/in-progress items are never touched regardless of junction table links +- The `jira_tickets` table is never modified (tickets persist independently of queue items) +- Auth middleware (`requireAuth`, `requireGroup`) continues to enforce access control +- Response shape remains `{ message: string, deleted: number }` + +**Scope:** +All inputs where `isBugCondition` returns false should be completely unaffected by this fix. This includes: +- Users with no completed items +- Users with completed items that have no junction table references +- Any request targeting non-complete statuses +- Mouse/UI interactions unrelated to the "Clear Completed" button + +## Hypothesized Root Cause + +Based on the bug description and code inspection, the root cause is confirmed: + +1. **Missing cascade handling in application code**: The `DELETE FROM ivanti_todo_queue WHERE user_id = $1 AND status = 'complete'` query does not account for the FK constraint added by the `add_multi_item_jira_ticket.js` migration. The junction table was added after the original endpoint was written. + +2. **FK constraint without ON DELETE CASCADE**: The migration creates `queue_item_id INTEGER NOT NULL REFERENCES ivanti_todo_queue(id)` without specifying `ON DELETE CASCADE`. This is a deliberate design choice (junction table links should be explicitly managed), but the delete endpoint was never updated to handle it. + +3. **No transaction wrapping**: The current handler uses a single query without a transaction. Even if it attempted to delete junction rows first, without a transaction there would be a race condition window. + +## Correctness Properties + +Property 1: Bug Condition - Clear Completed With Junction Table Links + +_For any_ set of completed queue items belonging to a user where at least one item has associated rows in `jira_ticket_queue_items`, the fixed clear completed operation SHALL delete all associated `jira_ticket_queue_items` rows first, then delete all completed queue items for that user, atomically within a transaction, and return a success response with the correct deleted count. + +**Validates: Requirements 2.1, 2.2** + +Property 2: Preservation - Clear Completed Without Junction Table Links + +_For any_ set of completed queue items belonging to a user where NO items have associated rows in `jira_ticket_queue_items` (or where no completed items exist at all), the fixed clear completed operation SHALL produce the same result as the original function — deleting all completed items and returning a success response with the correct deleted count. + +**Validates: Requirements 3.1, 3.2, 3.3** + +## Fix Implementation + +### Changes Required + +**File**: `backend/routes/ivantiTodoQueue.js` + +**Function**: `router.delete('/completed', ...)` handler + +**Specific Changes**: + +1. **Acquire a dedicated client from the pool**: Replace `pool.query(...)` with `pool.connect()` to get a client that supports transactions. + +2. **Begin a transaction**: Issue `BEGIN` before any data-modifying queries. + +3. **Select completed item IDs**: Query `SELECT id FROM ivanti_todo_queue WHERE user_id = $1 AND status = 'complete'` to get the set of IDs to delete. + +4. **Early exit for empty set**: If no completed items exist, commit and return `{ deleted: 0 }` immediately. + +5. **Delete junction table references**: Issue `DELETE FROM jira_ticket_queue_items WHERE queue_item_id = ANY($1::int[])` with the collected IDs. + +6. **Delete queue items by ID**: Issue `DELETE FROM ivanti_todo_queue WHERE id = ANY($1::int[])` using the same ID set (more precise than re-filtering by status). + +7. **Commit transaction**: Issue `COMMIT` on success. + +8. **Rollback on error**: Wrap in try/catch, issue `ROLLBACK` on any failure, then return 500. + +9. **Release client**: Always release the client back to the pool in a `finally` block. + +## Testing Strategy + +### Validation Approach + +The testing strategy follows a two-phase approach: first, surface counterexamples that demonstrate the bug on unfixed code, then verify the fix works correctly and preserves existing behavior. + +### Exploratory Bug Condition Checking + +**Goal**: Surface counterexamples that demonstrate the bug BEFORE implementing the fix. Confirm the FK violation root cause. + +**Test Plan**: Mock the `pool.connect()` / `pool.query()` pattern to simulate the FK constraint violation. Write tests that attempt to clear completed items when junction table references exist and assert the operation fails on unfixed code. + +**Test Cases**: +1. **Single linked item**: One completed item with a junction table reference — DELETE fails (will fail on unfixed code) +2. **Mixed linked/unlinked items**: Some completed items have links, some don't — DELETE fails for all (will fail on unfixed code) +3. **All items linked**: Every completed item has junction table references — DELETE fails (will fail on unfixed code) +4. **Multiple links per item**: One completed item with multiple junction table rows — DELETE fails (will fail on unfixed code) + +**Expected Counterexamples**: +- The simple DELETE query throws a FK violation error +- The catch block returns 500 and no items are deleted +- Possible cause confirmed: missing junction table cleanup before parent row deletion + +### Fix Checking + +**Goal**: Verify that for all inputs where the bug condition holds, the fixed function produces the expected behavior. + +**Pseudocode:** +``` +FOR ALL input WHERE isBugCondition(input) DO + result := clearCompleted_fixed(input) + ASSERT result.status = 200 + ASSERT result.body.deleted = count(completedItems) + ASSERT jira_ticket_queue_items has no rows for deleted IDs + ASSERT ivanti_todo_queue has no completed rows for user + ASSERT pending items unchanged + ASSERT jira_tickets table unchanged +END FOR +``` + +### Preservation Checking + +**Goal**: Verify that for all inputs where the bug condition does NOT hold, the fixed function produces the same result as the original function. + +**Pseudocode:** +``` +FOR ALL input WHERE NOT isBugCondition(input) DO + ASSERT clearCompleted_original(input) = clearCompleted_fixed(input) +END FOR +``` + +**Testing Approach**: Property-based testing is recommended for preservation checking because: +- It generates many test cases automatically across the input domain +- It catches edge cases (empty sets, single items, large batches) +- It provides strong guarantees that behavior is unchanged for all non-buggy inputs + +**Test Plan**: Observe behavior on UNFIXED code for cases without junction table links (these succeed today), then write property-based tests capturing that behavior. + +**Test Cases**: +1. **No completed items**: Verify returns `{ deleted: 0 }` — same as before +2. **Completed items without links**: Verify all are deleted and count is correct — same as before +3. **Pending items untouched**: Verify non-complete items are never affected — same as before +4. **Response shape preserved**: Verify `{ message, deleted }` structure unchanged + +### Unit Tests + +- Mock `pool.connect()` and verify correct query sequence within transaction (BEGIN → SELECT → DELETE junction → DELETE queue → COMMIT) +- Verify ROLLBACK is called on any query failure +- Verify client is always released in finally block +- Test edge case: empty completed set triggers early COMMIT and returns `{ deleted: 0 }` + +### Property-Based Tests + +- Generate random sets of queue items (mix of pending/complete, with/without junction links) and verify the transaction deletes exactly the right rows +- Generate random configurations without junction links and verify identical behavior to original code +- Generate random user IDs and verify isolation (one user's clear doesn't affect another's items) + +### Integration Tests + +- End-to-end test with actual FK constraints: insert queue items + junction rows, call endpoint, verify both tables cleaned up +- Verify atomicity: if the junction DELETE succeeds but queue DELETE fails, nothing is committed +- Verify the endpoint still works for the simple case (no junction rows) + +**Test file**: `backend/__tests__/ivanti-queue-clear-completed-fix.test.js` diff --git a/.kiro/specs/ivanti-queue-clear-completed-fix/tasks.md b/.kiro/specs/ivanti-queue-clear-completed-fix/tasks.md new file mode 100644 index 0000000..7697a2e --- /dev/null +++ b/.kiro/specs/ivanti-queue-clear-completed-fix/tasks.md @@ -0,0 +1,86 @@ +# Implementation Plan + +- [x] 1. Write bug condition exploration test + - **Property 1: Bug Condition** - FK Violation on Clear Completed With Junction Table Links + - **CRITICAL**: This test MUST FAIL on unfixed code — failure confirms the bug exists + - **DO NOT attempt to fix the test or the code when it fails** + - **NOTE**: This test encodes the expected behavior — it will validate the fix when it passes after implementation + - **GOAL**: Surface counterexamples that demonstrate the FK violation bug exists + - **Scoped PBT Approach**: Scope the property to the concrete failing case — completed queue items that have associated `jira_ticket_queue_items` rows + - Bug condition from design: `isBugCondition(input)` returns true when `linkedItems.length > 0` (completed items have junction table references) + - Test file: `backend/__tests__/ivanti-queue-clear-completed-fix.property.test.js` + - Mock `pool.query` to simulate FK constraint violation when DELETE is issued against `ivanti_todo_queue` while child rows exist in `jira_ticket_queue_items` + - Assert that the current (unfixed) handler returns 500 and deletes zero items + - Run test on UNFIXED code + - **EXPECTED OUTCOME**: Test FAILS (confirms the bug — the handler crashes with FK violation instead of succeeding) + - Document counterexamples: "DELETE FROM ivanti_todo_queue fails with FK violation when junction rows reference completed items" + - Mark task complete when test is written, run, and failure is documented + - _Requirements: 1.1_ + +- [x] 2. Write preservation property tests (BEFORE implementing fix) + - **Property 2: Preservation** - Clear Completed Without Junction Table Links + - **IMPORTANT**: Follow observation-first methodology + - **GOAL**: Verify that the unfixed code already handles the non-bug-condition cases correctly, establishing a baseline to preserve + - Test file: `backend/__tests__/ivanti-queue-clear-completed-fix.property.test.js` + - Observe: When no completed items have junction table links, the simple DELETE succeeds and returns correct count + - Observe: When no completed items exist, the endpoint returns `{ message: 'Completed items cleared.', deleted: 0 }` + - Observe: Pending/in-progress items are never touched by the DELETE + - Write property-based tests generating random sets of completed items WITHOUT junction table links and verify: + - All completed items for the user are deleted + - Response is `{ message: 'Completed items cleared.', deleted: N }` where N matches count + - Non-complete items remain untouched + - Other users' items remain untouched + - Run tests on UNFIXED code + - **EXPECTED OUTCOME**: Tests PASS (confirms baseline behavior to preserve) + - Mark task complete when tests are written, run, and passing on unfixed code + - _Requirements: 3.1, 3.2, 3.3_ + +- [x] 3. Fix for FK violation on clear completed queue items + + - [x] 3.1 Implement the fix + - File: `backend/routes/ivantiTodoQueue.js` + - Replace the simple `pool.query(DELETE...)` in the `router.delete('/completed', ...)` handler with a transaction-based approach: + - 1. Acquire a dedicated client via `pool.connect()` + - 2. Issue `BEGIN` + - 3. Select completed item IDs: `SELECT id FROM ivanti_todo_queue WHERE user_id = $1 AND status = 'complete'` + - 4. If no IDs found, `COMMIT` and return `{ deleted: 0 }` early + - 5. Delete junction table references: `DELETE FROM jira_ticket_queue_items WHERE queue_item_id = ANY($1::int[])` + - 6. Delete queue items: `DELETE FROM ivanti_todo_queue WHERE id = ANY($1::int[])` + - 7. `COMMIT` on success + - 8. `ROLLBACK` on any error, then return 500 + - 9. Always release client in `finally` block + - _Bug_Condition: isBugCondition(input) where completedItems have rows in jira_ticket_queue_items_ + - _Expected_Behavior: All completed items and their junction references deleted atomically, returns success with correct count_ + - _Preservation: Items without junction links still deleted; empty sets return deleted: 0; pending items untouched_ + - _Requirements: 2.1, 2.2, 3.1, 3.2, 3.3_ + + - [x] 3.2 Verify bug condition exploration test now passes + - **Property 1: Expected Behavior** - FK Violation on Clear Completed With Junction Table Links + - **IMPORTANT**: Re-run the SAME test from task 1 — do NOT write a new test + - The test from task 1 encodes the expected behavior (successful deletion with junction cleanup) + - When this test passes, it confirms the expected behavior is satisfied + - Run: `npx jest backend/__tests__/ivanti-queue-clear-completed-fix.property.test.js --run` + - **EXPECTED OUTCOME**: Test PASSES (confirms bug is fixed — transaction deletes junction rows then queue items) + - _Requirements: 2.1, 2.2_ + + - [x] 3.3 Verify preservation tests still pass + - **Property 2: Preservation** - Clear Completed Without Junction Table Links + - **IMPORTANT**: Re-run the SAME tests from task 2 — do NOT write new tests + - Run preservation property tests from step 2 + - Run: `npx jest backend/__tests__/ivanti-queue-clear-completed-fix.property.test.js --run` + - **EXPECTED OUTCOME**: Tests PASS (confirms no regressions — non-linked items still deleted correctly) + - Confirm all tests still pass after fix (no regressions) + +- [x] 4. Write unit tests for transaction logic + - Test file: `backend/__tests__/ivanti-queue-clear-completed-fix.test.js` + - Mock `pool.connect()` and verify correct query sequence: BEGIN → SELECT IDs → DELETE junction → DELETE queue → COMMIT + - Verify ROLLBACK is called when any query in the transaction fails + - Verify client is always released in the `finally` block (even on error) + - Test edge case: empty completed set triggers early COMMIT and returns `{ deleted: 0 }` + - Test that response shape remains `{ message: 'Completed items cleared.', deleted: N }` + - _Requirements: 2.1, 2.2, 3.1, 3.2_ + +- [x] 5. Checkpoint — Ensure all tests pass + - Run full test suite: `npx jest backend/__tests__/ivanti-queue-clear-completed-fix --run` + - Ensure all property tests and unit tests pass + - Ask the user if questions arise diff --git a/.kiro/specs/multi-item-jira-ticket/.config.kiro b/.kiro/specs/multi-item-jira-ticket/.config.kiro new file mode 100644 index 0000000..315d675 --- /dev/null +++ b/.kiro/specs/multi-item-jira-ticket/.config.kiro @@ -0,0 +1 @@ +{"specId": "0b0784a9-768e-42f0-ab8f-70b799b7ab67", "workflowType": "requirements-first", "specType": "feature"} diff --git a/.kiro/specs/multi-item-jira-ticket/design.md b/.kiro/specs/multi-item-jira-ticket/design.md new file mode 100644 index 0000000..fcb8c7f --- /dev/null +++ b/.kiro/specs/multi-item-jira-ticket/design.md @@ -0,0 +1,408 @@ +# Design Document: Multi-Item Jira Ticket Creation + +## Overview + +This feature adds multi-select capability to the Ivanti Queue page and a consolidation modal that aggregates information from multiple selected queue items into a single Jira ticket. The design builds on the existing `flexible-jira-ticket-creation` infrastructure — reusing the same backend endpoint (`POST /api/jira-tickets/create-in-jira`) and extending the frontend with selection state management, aggregation logic, and a junction table for tracking which queue items contributed to a ticket. + +The backend endpoint contract remains unchanged. All new logic lives in: +1. Frontend selection state and UI (checkboxes, floating action bar) +2. Frontend aggregation functions (summary/description generation) +3. A new junction table and its associated insert logic (post-creation) +4. A backend endpoint to record and query junction associations + +## Architecture + +```mermaid +flowchart TD + subgraph Frontend + QueuePage[Ivanti Queue Page
Selection Mode + Action Bar] + ConsolidationModal[Consolidation Modal
Aggregated Summary/Description] + SingleModal[Existing Creation Modal
Single-item flow] + end + + subgraph Backend + CreateEndpoint[POST /api/jira-tickets/create-in-jira
Unchanged contract] + JunctionEndpoint[POST /api/jira-tickets/:id/queue-items
New: store associations] + GetAssocEndpoint[GET /api/ivanti/todo-queue/ticket-links
New: fetch linked tickets] + JiraAPI[jiraApi.js → Jira REST API] + DB[(PostgreSQL)] + end + + QueuePage -->|1 item selected| SingleModal + QueuePage -->|2+ items selected| ConsolidationModal + ConsolidationModal -->|POST create ticket| CreateEndpoint + CreateEndpoint --> JiraAPI + JiraAPI -->|issue created| DB + ConsolidationModal -->|POST associations| JunctionEndpoint + JunctionEndpoint --> DB + QueuePage -->|fetch linked tickets| GetAssocEndpoint + GetAssocEndpoint --> DB +``` + +## Components and Interfaces + +### Frontend: Selection State Management + +Selection state is managed in the Ivanti Queue Page component using React state: + +```javascript +const [selectionMode, setSelectionMode] = useState(false); +const [selectedIds, setSelectedIds] = useState(new Set()); +``` + +**Selection Mode Toggle:** +- A "Select" button in the page toolbar toggles `selectionMode` +- When deactivated, `selectedIds` is cleared + +**Select All:** +- A header checkbox toggles all currently visible (filtered) queue item IDs into/out of `selectedIds` + +**Floating Action Bar:** +- Rendered when `selectionMode === true && selectedIds.size > 0` +- Contains: selection count badge, "Create Jira Ticket" button, "Cancel" button + +### Frontend: Consolidation Modal + +The Consolidation Modal is a new component that receives the selected queue items as props and handles aggregation. + +**Props:** + +| Prop | Type | Description | +|---|---|---| +| `items` | Array | The selected queue items (full objects) | +| `onClose` | Function | Close handler | +| `onSuccess` | Function | Called with created ticket data on success | + +**Internal State:** + +| Field | Initial Value | Editable | +|---|---|---| +| `summary` | Generated (see aggregation) | Yes | +| `description` | Generated (see aggregation) | Yes | +| `cve_id` | First CVE from items | Yes | +| `vendor` | Common vendor or empty | Yes | +| `source_context` | `ivanti_queue` | No (locked) | +| `selectedItems` | Copy of `items` prop | Yes (removable) | + +### Frontend: Aggregation Functions (Pure) + +These are pure functions that can be unit tested and property tested independently. + +**`generateConsolidatedSummary(items)`** + +```javascript +function generateConsolidatedSummary(items) { + const count = items.length; + const vendors = [...new Set(items.map(i => i.vendor).filter(Boolean))]; + const vendorLabel = vendors.length === 1 ? vendors[0] : 'Multiple Vendors'; + const firstTitle = items[0]?.finding_title || 'Untitled'; + const raw = `[${count} findings] ${vendorLabel} - ${firstTitle}`; + return raw.slice(0, 255); +} +``` + +**`generateConsolidatedDescription(items)`** + +```javascript +function generateConsolidatedDescription(items) { + const header = `Consolidated Jira ticket covering ${items.length} Ivanti queue findings.\n\n`; + + // Group by vendor + const grouped = {}; + for (const item of items) { + const vendor = item.vendor || 'Unknown Vendor'; + if (!grouped[vendor]) grouped[vendor] = []; + grouped[vendor].push(item); + } + + let body = ''; + for (const [vendor, vendorItems] of Object.entries(grouped)) { + body += `== ${vendor} ==\n`; + for (const item of vendorItems) { + const cves = item.cves_json ? JSON.parse(item.cves_json).join(', ') : 'None'; + body += `- ${item.finding_title}\n`; + body += ` CVEs: ${cves}\n`; + body += ` Host: ${item.hostname || 'N/A'} (${item.ip_address || 'N/A'})\n\n`; + } + } + + return header + body; +} +``` + +**`extractFirstCve(items)`** + +```javascript +function extractFirstCve(items) { + for (const item of items) { + if (item.cves_json) { + const cves = JSON.parse(item.cves_json); + if (Array.isArray(cves) && cves.length > 0) return cves[0]; + } + } + return ''; +} +``` + +**`extractCommonVendor(items)`** + +```javascript +function extractCommonVendor(items) { + const vendors = [...new Set(items.map(i => i.vendor).filter(Boolean))]; + return vendors.length === 1 ? vendors[0] : ''; +} +``` + +### Backend: Junction Table Endpoints (New) + +**`POST /api/jira-tickets/:id/queue-items`** + +Records the association between a Jira ticket and the queue items that contributed to it. + +| Field | Type | Required | Description | +|---|---|---|---| +| `queue_item_ids` | number[] | Yes | Array of ivanti_todo_queue IDs | + +**Request:** +```json +{ + "queue_item_ids": [12, 15, 18, 22] +} +``` + +**Response (201):** +```json +{ + "message": "Queue items linked to ticket", + "ticket_id": 42, + "linked_count": 4 +} +``` + +**Validation:** +- `queue_item_ids` must be a non-empty array of integers +- All referenced queue items must exist +- The jira_ticket must exist +- Duplicate associations are ignored (ON CONFLICT DO NOTHING) + +**`GET /api/ivanti/todo-queue/ticket-links`** + +Returns ticket associations for the current user's queue items. + +**Response (200):** +```json +{ + "links": { + "12": { "ticket_key": "VULN-789", "jira_url": "https://jira.example.com/browse/VULN-789" }, + "15": { "ticket_key": "VULN-789", "jira_url": "https://jira.example.com/browse/VULN-789" }, + "22": { "ticket_key": "VULN-801", "jira_url": "https://jira.example.com/browse/VULN-801" } + } +} +``` + +Returns a map of queue_item_id → ticket info for all queue items belonging to the authenticated user that have an associated Jira ticket. + +### Frontend: Submission Flow + +1. User clicks "Create Jira Ticket" in Consolidation Modal +2. Frontend sends `POST /api/jira-tickets/create-in-jira` with: + - `summary`: user-edited summary + - `description`: user-edited aggregated description + - `cve_id`: first CVE (or null) + - `vendor`: common vendor (or null) + - `source_context`: `ivanti_queue` +3. On 201 response, frontend sends `POST /api/jira-tickets/:id/queue-items` with the selected queue item IDs +4. On success, modal closes, success toast shown, queue page refreshes ticket link badges + +### Frontend: Ticket Link Badge on Queue Items + +When the queue page loads, it fetches `GET /api/ivanti/todo-queue/ticket-links` and stores the result in state. For each queue item row, if a link exists, a small badge (e.g., `VULN-789`) is rendered. Clicking the badge opens the Jira URL in a new tab. + +## Data Models + +### `jira_ticket_queue_items` Table (New) + +```sql +CREATE TABLE IF NOT EXISTS jira_ticket_queue_items ( + id SERIAL PRIMARY KEY, + jira_ticket_id INTEGER NOT NULL REFERENCES jira_tickets(id), + queue_item_id INTEGER NOT NULL REFERENCES ivanti_todo_queue(id), + created_at TIMESTAMPTZ DEFAULT NOW(), + UNIQUE (jira_ticket_id, queue_item_id) +); + +CREATE INDEX IF NOT EXISTS idx_jira_ticket_queue_items_queue_item +ON jira_ticket_queue_items(queue_item_id); + +CREATE INDEX IF NOT EXISTS idx_jira_ticket_queue_items_ticket +ON jira_ticket_queue_items(jira_ticket_id); +``` + +**Foreign Key Behavior:** +- `jira_ticket_id` references `jira_tickets(id)` — no CASCADE delete (association preserved even if ticket is deleted from local DB, though this is unlikely) +- `queue_item_id` references `ivanti_todo_queue(id)` — no CASCADE delete (association preserved even if queue item is completed/deleted) + +### Migration: `add_multi_item_jira_ticket.js` + +```javascript +const pool = require('../db'); + +async function run() { + console.log('Starting multi-item Jira ticket migration...'); + + // Verify prerequisite tables exist + const { rows: jiraTable } = await pool.query(` + SELECT 1 FROM information_schema.tables WHERE table_name = 'jira_tickets' + `); + if (jiraTable.length === 0) { + console.error('✗ jira_tickets table does not exist. Cannot proceed.'); + process.exit(1); + } + + const { rows: queueTable } = await pool.query(` + SELECT 1 FROM information_schema.tables WHERE table_name = 'ivanti_todo_queue' + `); + if (queueTable.length === 0) { + console.error('✗ ivanti_todo_queue table does not exist. Cannot proceed.'); + process.exit(1); + } + + // Create junction table + await pool.query(` + CREATE TABLE IF NOT EXISTS jira_ticket_queue_items ( + id SERIAL PRIMARY KEY, + jira_ticket_id INTEGER NOT NULL REFERENCES jira_tickets(id), + queue_item_id INTEGER NOT NULL REFERENCES ivanti_todo_queue(id), + created_at TIMESTAMPTZ DEFAULT NOW(), + UNIQUE (jira_ticket_id, queue_item_id) + ) + `); + console.log('✓ jira_ticket_queue_items table created (or already exists)'); + + // Add indexes + await pool.query(` + CREATE INDEX IF NOT EXISTS idx_jira_ticket_queue_items_queue_item + ON jira_ticket_queue_items(queue_item_id) + `); + console.log('✓ queue_item_id index created (or already exists)'); + + await pool.query(` + CREATE INDEX IF NOT EXISTS idx_jira_ticket_queue_items_ticket + ON jira_ticket_queue_items(jira_ticket_id) + `); + console.log('✓ jira_ticket_id index created (or already exists)'); + + console.log('Migration complete.'); + process.exit(0); +} + +run().catch(err => { + console.error('Migration failed:', err.message); + process.exit(1); +}); +``` + +## Correctness Properties + +### Property 1: Summary generation format and truncation + +*For any* non-empty array of queue items (each with optional vendor and finding_title of arbitrary length), `generateConsolidatedSummary` SHALL return a string that: +- Starts with `[N findings]` where N equals the array length +- Is at most 255 characters long +- Contains the common vendor name when all items share the same vendor, or "Multiple Vendors" when vendors differ + +**Validates: Requirements 3.1, 3.2** + +### Property 2: Description includes all items + +*For any* non-empty array of queue items, `generateConsolidatedDescription` SHALL produce a string that contains the `finding_title` of every item in the input array. + +**Validates: Requirements 4.1, 4.2** + +### Property 3: Description groups by vendor + +*For any* array of queue items with two or more distinct vendors, `generateConsolidatedDescription` SHALL produce a string where all items sharing the same vendor appear in a contiguous block (not interleaved with items from other vendors). + +**Validates: Requirements 4.3** + +### Property 4: Description header contains item count + +*For any* non-empty array of queue items of length N, `generateConsolidatedDescription` SHALL produce a string containing the substring representing N (the count of items). + +**Validates: Requirements 4.4** + +### Property 5: First CVE extraction + +*For any* array of queue items, `extractFirstCve` SHALL return the first element of the first non-empty `cves_json` array encountered in item order, or an empty string if no items have CVEs. + +**Validates: Requirements 4.6** + +### Property 6: Common vendor extraction + +*For any* array of queue items where all items have the same non-null vendor value, `extractCommonVendor` SHALL return that vendor value. For any array where items have two or more distinct vendor values, `extractCommonVendor` SHALL return an empty string. + +**Validates: Requirements 4.7** + +### Property 7: Junction table row count equals selected item count + +*For any* successful consolidated ticket creation with N selected queue items (N >= 2), the `jira_ticket_queue_items` table SHALL contain exactly N rows with the created ticket's ID. + +**Validates: Requirements 5.3, 6.2** + +## Error Handling + +| Scenario | HTTP Status | Error Message | Client Behavior | +|---|---|---|---| +| No queue items selected | N/A (frontend) | "Select at least one item" | Button disabled, no request sent | +| Fewer than 2 items in consolidation modal | N/A (frontend) | "At least 2 items required for consolidation" | Submit button disabled | +| Empty summary on submit | 400 | "Summary is required (max 255 chars)." | Inline error, preserve form | +| Jira API unavailable | 502 | "Failed to create Jira issue." | Error banner, preserve form for retry | +| Jira rate limit exceeded | 429 | "Jira rate limit exceeded. Try again later." | Error banner, preserve form | +| Junction insert fails after ticket creation | 207 (partial) | Warning with ticket key/URL | Show ticket link, warn that associations failed | +| Invalid queue_item_ids in junction request | 400 | "queue_item_ids must be a non-empty array of integers" | Display error | +| Referenced queue items do not exist | 400 | "One or more queue items not found" | Display error | +| Prerequisite tables missing (migration) | Exit code 1 | Console error | Migration aborts cleanly | + +## Testing Strategy + +### Unit Tests (Example-Based) + +- Selection mode toggle shows/hides checkboxes +- Select All selects all visible items +- Deactivating selection mode clears selections +- Floating action bar appears when items selected, hidden when none selected +- "Create Jira Ticket" opens consolidation modal for 2+ items +- "Create Jira Ticket" opens single-item modal for exactly 1 item +- Consolidation modal renders with correct initial summary, description, CVE, vendor +- Consolidation modal allows editing summary and description +- Consolidation modal prevents submission with empty summary +- Consolidation modal source_context is read-only and set to ivanti_queue +- Removing items from modal updates preview list and regenerates summary/description +- Removing items below 2 disables submit +- Ticket link badge renders on queue items with associations +- Clicking ticket link badge opens URL in new tab +- Junction endpoint rejects empty or non-array queue_item_ids +- Junction endpoint rejects non-existent queue item IDs +- Migration creates table with correct schema +- Migration is idempotent (running twice succeeds) +- Migration fails gracefully when prerequisite tables missing + +### Property-Based Tests + +Property-based testing is appropriate for the aggregation functions because they operate on arrays of arbitrary length with arbitrary string content — a wide input space where universal format properties must hold. + +**Library:** fast-check (already available in the project's test infrastructure via Jest) + +**Configuration:** +- Minimum 100 iterations per property test +- Each test tagged with: `Feature: multi-item-jira-ticket, Property {N}: {title}` + +Properties 1–6 test pure aggregation functions (frontend logic extracted into testable modules). +Property 7 tests the backend junction insert invariant (mockable DB). + +### Integration Tests + +- End-to-end: select 3 items → create consolidated ticket → verify junction table has 3 rows +- End-to-end: verify ticket-links endpoint returns correct associations after creation +- Migration runs successfully on database with existing jira_tickets and ivanti_todo_queue rows +- Unique constraint prevents duplicate junction entries (ON CONFLICT DO NOTHING) diff --git a/.kiro/specs/multi-item-jira-ticket/requirements.md b/.kiro/specs/multi-item-jira-ticket/requirements.md new file mode 100644 index 0000000..a2fc3f6 --- /dev/null +++ b/.kiro/specs/multi-item-jira-ticket/requirements.md @@ -0,0 +1,114 @@ +# Requirements Document + +## Introduction + +The STEAM Security Dashboard currently allows users to create a Jira ticket from a single Ivanti queue item via the "Create Jira Ticket" action (implemented in the `flexible-jira-ticket-creation` spec). In practice, multiple queue items often relate to the same remediation effort — same vendor, same CVE, or same host group. Users need the ability to select multiple Ivanti queue items and create ONE consolidated Jira ticket whose summary and description aggregate information from all selected items. This is not batch creation (one ticket per item) — it is consolidation (N items into 1 ticket). + +## Glossary + +- **Ivanti_Queue_Page**: The frontend page where users work through their Ivanti finding queue items. +- **Queue_Item**: A single row in the `ivanti_todo_queue` table representing a finding assigned to a user for triage. +- **Selection_Mode**: A UI state on the Ivanti_Queue_Page where checkboxes appear on each Queue_Item, allowing the user to select multiple items. +- **Consolidation_Modal**: The frontend modal dialog used to create a single Jira ticket from multiple selected Queue_Items, extending the existing Creation_Modal with aggregation logic. +- **Ticket_Creation_Service**: The backend endpoint (`POST /api/jira-tickets/create-in-jira`) responsible for creating issues in Jira and storing the local record. +- **Aggregated_Description**: A structured text block composed from the selected Queue_Items' metadata (finding titles, CVEs, hostnames, IP addresses, vendor) formatted for the Jira issue description. +- **Dashboard**: The STEAM Security Dashboard application. + +## Requirements + +### Requirement 1: Multi-Select Mode on Ivanti Queue + +**User Story:** As a security analyst, I want to select multiple items from my Ivanti queue, so that I can act on them as a group. + +#### Acceptance Criteria + +1. THE Ivanti_Queue_Page SHALL display a "Select" toggle button that activates Selection_Mode. +2. WHILE Selection_Mode is active, THE Ivanti_Queue_Page SHALL display a checkbox on each Queue_Item row. +3. WHILE Selection_Mode is active, THE Ivanti_Queue_Page SHALL display a selection count indicator showing the number of currently selected items. +4. WHILE Selection_Mode is active, THE Ivanti_Queue_Page SHALL display a "Select All" checkbox in the table header that toggles selection of all visible Queue_Items. +5. WHEN the user deactivates Selection_Mode, THE Ivanti_Queue_Page SHALL clear all selections and hide the checkboxes. +6. THE Ivanti_Queue_Page SHALL preserve the selected items when the user scrolls or when the list re-renders due to filter changes within the same session. + +### Requirement 2: Create Consolidated Jira Ticket Action + +**User Story:** As a security analyst, I want to create a single Jira ticket from my selected queue items, so that I can track a group of related findings as one work item. + +#### Acceptance Criteria + +1. WHILE Selection_Mode is active and at least one Queue_Item is selected, THE Ivanti_Queue_Page SHALL display a "Create Jira Ticket" action button in a floating action bar. +2. WHEN no Queue_Items are selected, THE Ivanti_Queue_Page SHALL disable the "Create Jira Ticket" action button. +3. WHEN the user activates the "Create Jira Ticket" action with multiple items selected, THE Ivanti_Queue_Page SHALL open the Consolidation_Modal. +4. WHEN the user activates the "Create Jira Ticket" action with exactly one item selected, THE Ivanti_Queue_Page SHALL open the existing single-item Creation_Modal with pre-populated fields from that item (existing behavior from flexible-jira-ticket-creation spec). + +### Requirement 3: Aggregated Summary Generation + +**User Story:** As a security analyst, I want the consolidated ticket summary to clearly indicate it covers multiple findings, so that I can identify multi-item tickets at a glance. + +#### Acceptance Criteria + +1. WHEN the Consolidation_Modal opens, THE Consolidation_Modal SHALL pre-populate the Summary field with a generated value in the format: "[N findings] vendor_name - first_finding_title" where N is the count of selected items, vendor_name is the common vendor (if all items share the same vendor) or "Multiple Vendors", and first_finding_title is the finding_title of the first selected item. +2. THE Consolidation_Modal SHALL truncate the generated summary to 255 characters. +3. THE Consolidation_Modal SHALL allow the user to edit the pre-populated summary before submission. +4. IF the user clears the summary field and attempts to submit, THEN THE Consolidation_Modal SHALL prevent submission and display an inline error indicating that Summary is required. + +### Requirement 4: Aggregated Description Generation + +**User Story:** As a security analyst, I want the consolidated ticket description to list details from each selected item, so that the Jira ticket contains all relevant finding information. + +#### Acceptance Criteria + +1. WHEN the Consolidation_Modal opens, THE Consolidation_Modal SHALL generate an Aggregated_Description containing a structured list of all selected Queue_Items. +2. THE Aggregated_Description SHALL include for each Queue_Item: finding_title, CVE IDs (from cves_json), hostname, IP address, and vendor. +3. THE Aggregated_Description SHALL group items by vendor when multiple vendors are present. +4. THE Aggregated_Description SHALL include a header line stating the total count of findings included. +5. THE Consolidation_Modal SHALL display the Aggregated_Description in an editable text area, allowing the user to modify it before submission. +6. THE Consolidation_Modal SHALL pre-populate the CVE ID field with the first CVE found across all selected items (from the first item's cves_json array), or leave it blank if no items have CVEs. +7. THE Consolidation_Modal SHALL pre-populate the Vendor field with the common vendor if all selected items share the same vendor, or leave it blank if vendors differ. + +### Requirement 5: Consolidated Ticket Creation via Backend + +**User Story:** As a security analyst, I want the consolidated Jira ticket to be created through the existing ticket creation endpoint, so that it is tracked and synced like any other Jira ticket. + +#### Acceptance Criteria + +1. WHEN the user submits the Consolidation_Modal, THE Dashboard SHALL send a create-ticket request to the Ticket_Creation_Service with source_context set to `ivanti_queue`, the user-provided summary, and the Aggregated_Description as the description field. +2. THE Ticket_Creation_Service SHALL accept the request and create the Jira issue using the existing creation flow without modification to the backend endpoint contract. +3. WHEN the Jira ticket is created successfully, THE Dashboard SHALL store a local record linking the ticket to the selected Queue_Item IDs via a new junction table. +4. WHEN the Jira ticket is created successfully, THE Dashboard SHALL display a success notification with the created ticket key and a link to the Jira issue. +5. IF the Jira API returns an error, THEN THE Consolidation_Modal SHALL display an error message and preserve all form field values for retry. + +### Requirement 6: Queue Item to Ticket Association + +**User Story:** As a security analyst, I want to see which Jira ticket was created from my queue items, so that I can track the relationship between findings and tickets. + +#### Acceptance Criteria + +1. THE Dashboard SHALL include a `jira_ticket_queue_items` junction table that associates a jira_ticket ID with one or more Queue_Item IDs. +2. WHEN a consolidated Jira ticket is created, THE Dashboard SHALL insert one row per selected Queue_Item into the junction table. +3. WHILE viewing a Queue_Item that has an associated Jira ticket, THE Ivanti_Queue_Page SHALL display the linked ticket key as a badge or indicator on that item's row. +4. WHEN the user clicks the ticket key indicator on a Queue_Item, THE Ivanti_Queue_Page SHALL navigate to or open the Jira ticket URL in a new tab. +5. THE junction table association SHALL persist regardless of whether the Queue_Item is later marked complete or deleted. + +### Requirement 7: Database Migration for Junction Table + +**User Story:** As a database administrator, I want the queue-item-to-ticket association stored in a proper junction table, so that the relationship is queryable and maintainable. + +#### Acceptance Criteria + +1. THE Dashboard SHALL include a migration that creates a `jira_ticket_queue_items` table with columns: id (SERIAL PRIMARY KEY), jira_ticket_id (INTEGER NOT NULL REFERENCES jira_tickets(id)), queue_item_id (INTEGER NOT NULL REFERENCES ivanti_todo_queue(id)), and created_at (TIMESTAMPTZ DEFAULT NOW()). +2. THE migration SHALL add a unique constraint on (jira_ticket_id, queue_item_id) to prevent duplicate associations. +3. THE migration SHALL add an index on queue_item_id for efficient lookup of tickets associated with a given queue item. +4. THE migration SHALL be idempotent — running it multiple times SHALL produce the same schema state without raising errors. +5. IF the `jira_tickets` or `ivanti_todo_queue` tables do not exist when the migration runs, THEN THE migration SHALL exit with an error message indicating the prerequisite tables are missing. + +### Requirement 8: Consolidation Modal Source Context and Field Locking + +**User Story:** As a security analyst, I want the consolidated ticket to automatically track its origin as the Ivanti queue, so that I do not need to manually set the source context. + +#### Acceptance Criteria + +1. WHEN the Consolidation_Modal opens, THE Consolidation_Modal SHALL set source_context to `ivanti_queue` and display it as read-only. +2. THE Consolidation_Modal SHALL display the count of selected items in the modal header or subtitle. +3. WHEN the Consolidation_Modal opens, THE Consolidation_Modal SHALL display a scrollable preview list of the selected Queue_Items showing finding_title and hostname for each. +4. THE Consolidation_Modal SHALL allow the user to remove individual items from the selection within the modal before submission, with a minimum of 2 items required for consolidation. +5. IF the user removes items until only 1 remains, THEN THE Consolidation_Modal SHALL display a message indicating that at least 2 items are required for consolidation and disable the submit button. diff --git a/.kiro/specs/multi-item-jira-ticket/tasks.md b/.kiro/specs/multi-item-jira-ticket/tasks.md new file mode 100644 index 0000000..5565bad --- /dev/null +++ b/.kiro/specs/multi-item-jira-ticket/tasks.md @@ -0,0 +1,124 @@ +# Implementation Plan: Multi-Item Jira Ticket Creation + +## Overview + +This plan implements multi-select on the Ivanti Queue page and a consolidation modal that creates a single Jira ticket from multiple selected queue items. The implementation proceeds from database migration → backend junction endpoints → frontend aggregation logic → frontend selection UI → consolidation modal → ticket link badges. + +## Tasks + +- [x] 1. Database migration for junction table + - [x] 1.1 Create migration file `backend/migrations/add_multi_item_jira_ticket.js` + - Verify `jira_tickets` and `ivanti_todo_queue` tables exist; exit with error if missing + - Create `jira_ticket_queue_items` table with columns: id (SERIAL PRIMARY KEY), jira_ticket_id (INTEGER NOT NULL REFERENCES jira_tickets(id)), queue_item_id (INTEGER NOT NULL REFERENCES ivanti_todo_queue(id)), created_at (TIMESTAMPTZ DEFAULT NOW()) + - Add UNIQUE constraint on (jira_ticket_id, queue_item_id) + - Add index on queue_item_id + - Add index on jira_ticket_id + - Ensure full idempotency — safe to run multiple times + - _Requirements: 7.1, 7.2, 7.3, 7.4, 7.5_ + +- [x] 2. Backend junction table endpoints + - [x] 2.1 Add `POST /api/jira-tickets/:id/queue-items` endpoint in `backend/routes/jiraTickets.js` + - Validate `queue_item_ids` is a non-empty array of integers + - Verify the jira_ticket exists + - Verify all referenced queue items exist + - Insert rows into `jira_ticket_queue_items` with ON CONFLICT DO NOTHING + - Return 201 with linked_count + - Require Admin or Standard_User group + - _Requirements: 5.3, 6.1, 6.2_ + + - [x] 2.2 Add `GET /api/ivanti/todo-queue/ticket-links` endpoint in `backend/routes/ivantiTodoQueue.js` + - Join `jira_ticket_queue_items` with `jira_tickets` to get ticket_key and url + - Filter by queue items belonging to the authenticated user + - Return a map of queue_item_id → { ticket_key, jira_url } + - _Requirements: 6.3, 6.4_ + +- [x] 3. Frontend aggregation utility functions + - [x] 3.1 Create `frontend/src/utils/jiraConsolidation.js` with pure functions + - `generateConsolidatedSummary(items)` — format: `[N findings] vendor - title`, truncated to 255 chars + - `generateConsolidatedDescription(items)` — structured description grouped by vendor + - `extractFirstCve(items)` — first CVE from first item with non-empty cves_json + - `extractCommonVendor(items)` — common vendor if all same, empty string otherwise + - _Requirements: 3.1, 3.2, 4.1, 4.2, 4.3, 4.4, 4.6, 4.7_ + + - [ ]* 3.2 Write property tests for aggregation functions (Properties 1–6) + - **Property 1: Summary format and truncation** — starts with `[N findings]`, at most 255 chars, contains correct vendor label + - **Property 2: Description includes all items** — every item's finding_title appears in output + - **Property 3: Description groups by vendor** — items with same vendor are contiguous + - **Property 4: Description header contains count** — output contains the item count + - **Property 5: First CVE extraction** — returns first CVE from first item with CVEs, or empty string + - **Property 6: Common vendor extraction** — returns vendor when all same, empty when different + - **Validates: Requirements 3.1, 3.2, 4.1, 4.2, 4.3, 4.4, 4.6, 4.7** + - File: `backend/__tests__/jira-consolidation-aggregation.property.test.js` + +- [x] 4. Frontend multi-select mode on Ivanti Queue page + - [x] 4.1 Add selection mode toggle and checkbox UI to `frontend/src/components/pages/IvantiQueuePage.js` + - Add "Select" toggle button to page toolbar + - Show checkboxes on each queue item row when selection mode active + - Add "Select All" checkbox in table header + - Display selection count indicator + - Clear selections when selection mode deactivated + - Preserve selections on scroll/re-render within same session + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6_ + + - [x] 4.2 Add floating action bar with "Create Jira Ticket" button + - Render floating bar when selection mode active and at least 1 item selected + - Disable "Create Jira Ticket" button when no items selected + - Route to existing single-item modal when exactly 1 item selected + - Route to new Consolidation Modal when 2+ items selected + - _Requirements: 2.1, 2.2, 2.3, 2.4_ + +- [x] 5. Frontend Consolidation Modal + - [x] 5.1 Create `frontend/src/components/ConsolidationModal.js` + - Accept selected queue items as props + - Call aggregation functions to pre-populate summary, description, cve_id, vendor + - Lock source_context to `ivanti_queue` (read-only display) + - Display item count in modal header/subtitle + - Render scrollable preview list of selected items (finding_title + hostname) + - Allow removing individual items from selection (minimum 2 required) + - Disable submit and show message when fewer than 2 items remain + - Editable summary field with required validation (max 255 chars) + - Editable description textarea + - Editable CVE ID and Vendor fields (optional) + - On submit: POST to create-in-jira, then POST to junction endpoint + - On success: close modal, show success toast, trigger queue page refresh + - On error: display error message, preserve form values + - _Requirements: 3.1, 3.2, 3.3, 3.4, 4.1, 4.5, 5.1, 5.2, 5.4, 5.5, 8.1, 8.2, 8.3, 8.4, 8.5_ + +- [x] 6. Frontend ticket link badges on queue items + - [x] 6.1 Fetch and display ticket link badges on Ivanti Queue page + - On page load, fetch `GET /api/ivanti/todo-queue/ticket-links` + - For each queue item with an association, render a ticket key badge (e.g., "VULN-789") + - Clicking badge opens Jira URL in new tab + - Refresh links after successful consolidated ticket creation + - _Requirements: 6.3, 6.4, 6.5_ + +- [ ] 7. Backend property test for junction insert invariant + - [ ]* 7.1 Write property test for junction row count (Property 7) + - **Property 7: Junction table row count equals selected item count** — for N items, exactly N rows inserted + - **Validates: Requirements 5.3, 6.2** + - File: `backend/__tests__/jira-consolidation-junction.property.test.js` + +- [x] 8. Final checkpoint + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional property-based tests that can be skipped for faster MVP +- The existing `POST /api/jira-tickets/create-in-jira` endpoint is reused without modification +- The aggregation functions are pure and extracted into a utility module for easy testing +- The junction table insert happens as a second request after ticket creation — if it fails, the ticket still exists in Jira (partial success scenario handled with warning) +- The Consolidation Modal is a separate component from the existing Creation Modal to avoid overcomplicating the single-item flow + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1"] }, + { "id": 1, "tasks": ["2.1", "2.2", "3.1"] }, + { "id": 2, "tasks": ["3.2", "4.1"] }, + { "id": 3, "tasks": ["4.2", "5.1"] }, + { "id": 4, "tasks": ["6.1", "7.1"] } + ] +} +``` diff --git a/.kiro/specs/queue-collapsible-sections/.config.kiro b/.kiro/specs/queue-collapsible-sections/.config.kiro new file mode 100644 index 0000000..e5374af --- /dev/null +++ b/.kiro/specs/queue-collapsible-sections/.config.kiro @@ -0,0 +1 @@ +{"specId": "e9cd98e7-7a78-4795-a7a7-d5b6358dd10d", "workflowType": "fast-task", "specType": "feature"} \ No newline at end of file diff --git a/.kiro/specs/queue-collapsible-sections/design.md b/.kiro/specs/queue-collapsible-sections/design.md new file mode 100644 index 0000000..8b318dd --- /dev/null +++ b/.kiro/specs/queue-collapsible-sections/design.md @@ -0,0 +1,298 @@ +# Queue Collapsible Sections — Design + +## Overview + +This feature adds collapsible, grouped sections to the IvantiTodoQueuePage. Items are organized into a hybrid layout: an "Inventory" section (CARD, GRANITE, DECOM workflow types) at the top, followed by vendor-grouped sections for FP and Archer items. Each section has a clickable header that toggles visibility of its items. The implementation is entirely frontend — no backend or database changes are needed. + +## Architecture + +The feature modifies a single file: `frontend/src/components/pages/IvantiTodoQueuePage.js`. It introduces: + +1. A **grouping computation** (via `useMemo`) that transforms the flat `visibleItems` array into a structured array of sections. +2. A **collapse state** (via `useState`) that tracks which sections are expanded or collapsed. +3. **Section header components** rendered inline that display group labels, item counts, and chevron icons. +4. **Extended styles** added to the existing `STYLES` constant. + +No new files, components, or dependencies are introduced. The existing selection logic, floating action bar, consolidation modal, and ticket link badges continue to operate unchanged. + +## Components and Data Flow + +``` +queueItems (from API) + │ + ▼ +visibleItems = useMemo(filter pending) ← existing + │ + ▼ +groupedSections = useMemo(groupItems) ← NEW: hybrid grouping logic + │ + ▼ +collapsedSections (useState) ← NEW: collapse state + │ + ▼ +Render: map over groupedSections + ├── Section Header (clickable, toggles collapse) + └── Section Body (conditionally rendered based on collapse state) + └── Queue Item Rows (existing rendering logic) +``` + +## Data Model + +### Grouped Section Structure + +The `useMemo` grouping hook produces an array of section objects: + +```javascript +// Output of the grouping computation +const groupedSections = [ + { + key: 'inventory', // Unique section identifier + label: 'Inventory', // Display label + type: 'inventory', // Section type for styling + items: [/* queue items */], // Items belonging to this section + }, + { + key: 'vendor:Microsoft', // Unique section identifier + label: 'Microsoft', // Display label (vendor name) + type: 'vendor', // Section type for styling + items: [/* queue items */], // Items belonging to this section + }, + // ... more vendor sections, alphabetically sorted +]; +``` + +### Collapse State + +```javascript +// Key: section.key string, Value: boolean (true = collapsed) +const [collapsedSections, setCollapsedSections] = useState({}); +// Initial state: empty object → all sections default to expanded +``` + +## Interfaces + +### Grouping Function + +```javascript +/** + * Groups visible queue items into the hybrid section layout. + * + * @param {Array} visibleItems - Queue items with status 'pending' + * @returns {Array<{key: string, label: string, type: string, items: Array}>} + * + * Rules: + * - Items with workflow_type CARD, GRANITE, or DECOM → Inventory section + * - Items with workflow_type FP or Archer → grouped by vendor field + * - Items with null/undefined/empty vendor → placed in "Unknown" vendor section + * - Inventory section appears first (if non-empty) + * - Vendor sections sorted alphabetically by label + * - Sections with zero items are omitted from output + */ +const groupedSections = useMemo(() => { + const INVENTORY_TYPES = new Set(['CARD', 'GRANITE', 'DECOM']); + + const inventoryItems = []; + const vendorMap = new Map(); // vendor name → items array + + for (const item of visibleItems) { + if (INVENTORY_TYPES.has(item.workflow_type)) { + inventoryItems.push(item); + } else { + const vendor = item.vendor?.trim() || 'Unknown'; + if (!vendorMap.has(vendor)) vendorMap.set(vendor, []); + vendorMap.get(vendor).push(item); + } + } + + const sections = []; + + // Inventory section first (only if non-empty) + if (inventoryItems.length > 0) { + sections.push({ + key: 'inventory', + label: 'Inventory', + type: 'inventory', + items: inventoryItems, + }); + } + + // Vendor sections sorted alphabetically + const sortedVendors = [...vendorMap.entries()] + .sort((a, b) => a[0].localeCompare(b[0])); + + for (const [vendor, items] of sortedVendors) { + sections.push({ + key: `vendor:${vendor}`, + label: vendor, + type: 'vendor', + items, + }); + } + + return sections; +}, [visibleItems]); +``` + +### Toggle Collapse Handler + +```javascript +const toggleSection = useCallback((sectionKey) => { + setCollapsedSections((prev) => ({ + ...prev, + [sectionKey]: !prev[sectionKey], + })); +}, []); +``` + +### Section Header Rendering + +```javascript +// Within the render, for each section: +{groupedSections.map((section) => { + const isCollapsed = !!collapsedSections[section.key]; + + return ( +
+ {/* Section Header */} +
toggleSection(section.key)} + style={section.type === 'inventory' + ? STYLES.sectionHeaderInventory + : STYLES.sectionHeaderVendor} + role="button" + tabIndex={0} + onKeyDown={(e) => { + if (e.key === 'Enter' || e.key === ' ') { + e.preventDefault(); + toggleSection(section.key); + } + }} + aria-expanded={!isCollapsed} + aria-label={`${section.label} section, ${section.items.length} items`} + > + {isCollapsed + ? + : + } + {section.label} + ({section.items.length}) +
+ + {/* Section Body — only rendered when expanded */} + {!isCollapsed && section.items.map((item) => ( + // ... existing queue item row rendering + ))} +
+ ); +})} +``` + +## Styling + +New entries added to the existing `STYLES` constant: + +```javascript +sectionHeaderInventory: { + display: 'flex', + alignItems: 'center', + gap: '0.5rem', + padding: '0.5rem 0.75rem', + marginTop: '0.5rem', + borderBottom: '1px solid rgba(16, 185, 129, 0.2)', + cursor: 'pointer', + userSelect: 'none', + fontFamily: 'monospace', + fontSize: '0.7rem', + fontWeight: 700, + color: '#10B981', + textTransform: 'uppercase', + letterSpacing: '0.1em', +}, +sectionHeaderVendor: { + display: 'flex', + alignItems: 'center', + gap: '0.5rem', + padding: '0.5rem 0.75rem', + marginTop: '0.5rem', + borderBottom: '1px solid rgba(148, 163, 184, 0.15)', + cursor: 'pointer', + userSelect: 'none', + fontFamily: 'monospace', + fontSize: '0.7rem', + fontWeight: 700, + color: '#94A3B8', + textTransform: 'uppercase', + letterSpacing: '0.1em', +}, +sectionCount: { + fontFamily: 'monospace', + fontSize: '0.65rem', + fontWeight: 600, + color: '#64748B', + marginLeft: '0.25rem', +}, +``` + +## Icon Imports + +Add `ChevronDown` and `ChevronRight` to the existing lucide-react import: + +```javascript +import { ListTodo, RefreshCw, CheckSquare, Square, Loader, AlertCircle, X, Plus, CheckCircle, ChevronDown, ChevronRight } from 'lucide-react'; +``` + +## Select All Interaction with Collapsed Sections + +The existing `toggleSelectAll` and `allVisibleSelected` logic operates on the `visibleItems` array, which contains ALL pending items regardless of collapse state. This means: + +- **Select All** selects/deselects all pending items across all sections, whether those sections are collapsed or expanded. +- **Selection count** always reflects `selectedIds.size`, which includes items in collapsed sections. +- **Floating action bar** operates on `selectedQueueItems` (derived from `selectedIds`), which is independent of collapse state. + +No changes to the selection logic are needed. The collapse state is purely visual — it controls rendering, not data. + +## Error Handling + +- If `visibleItems` is empty, `groupedSections` will be an empty array, and the existing empty state renders instead. +- If an item has an unexpected `workflow_type` (not CARD, GRANITE, DECOM, FP, or Archer), it falls into the vendor grouping path and is grouped by its vendor field. This is a safe fallback. +- The `collapsedSections` state uses an object with string keys. Non-existent keys return `undefined`, which is falsy, so all sections default to expanded without explicit initialization. + +## Performance Considerations + +- The `useMemo` grouping computation runs only when `visibleItems` changes (on fetch or status update). For typical queue sizes (10–100 items), this is negligible. +- Collapse state changes trigger re-renders only of the affected section's body. React's reconciliation handles this efficiently since each section has a stable `key`. +- No additional API calls or data fetching is introduced. + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +### Property 1: Grouping Correctness + +For any array of visible queue items, every item with workflow_type CARD, GRANITE, or DECOM SHALL appear in the Inventory section, every item with workflow_type FP or Archer SHALL appear in the vendor section matching its vendor field (or "Unknown" if vendor is null/empty), and no item SHALL appear in more than one section. + +**Validates: Requirements 1.1, 1.2, 1.5** + +### Property 2: Section Ordering + +For any non-empty grouping result, the Inventory section (if present) SHALL be the first element, and all subsequent vendor sections SHALL be sorted in case-insensitive alphabetical order by their label. + +**Validates: Requirements 1.3, 1.4** + +### Property 3: Empty Section Omission + +For any array of visible queue items, the grouped output SHALL contain no sections with zero items. Specifically, if no CARD/GRANITE/DECOM items exist, no Inventory section appears; if no items exist for a given vendor, no section for that vendor appears. + +**Validates: Requirements 1.6, 1.7** + +### Property 4: Section Header Count Accuracy + +For any section in the grouped output, the item count displayed in the section header SHALL equal the actual number of items in that section's items array. + +**Validates: Requirements 3.1, 3.2** + +### Property 5: Selection Independence from Collapse State + +For any combination of selected items and collapse state, the set of selected item IDs SHALL remain unchanged when sections are collapsed or expanded. The selection count SHALL always equal the total number of selected items across all sections, and Select All SHALL toggle selection for all visible items regardless of which sections are collapsed. + +**Validates: Requirements 4.2, 4.4, 4.5** diff --git a/.kiro/specs/queue-collapsible-sections/requirements.md b/.kiro/specs/queue-collapsible-sections/requirements.md new file mode 100644 index 0000000..7039570 --- /dev/null +++ b/.kiro/specs/queue-collapsible-sections/requirements.md @@ -0,0 +1,85 @@ +# Requirements Document + +## Introduction + +The IvantiTodoQueuePage currently displays all pending queue items in a flat table. This feature introduces collapsible sections that group items using a hybrid layout: an "Inventory" section at the top containing CARD, GRANITE, and DECOM workflow type items, followed by vendor-grouped sections for FP and Archer items. Each section header is clickable to collapse or expand its contents, allowing users to focus on relevant groups while maintaining full cross-section selection and existing page functionality. + +## Glossary + +- **Queue_Page**: The IvantiTodoQueuePage React component located at `frontend/src/components/pages/IvantiTodoQueuePage.js` +- **Section**: A collapsible group of queue items sharing a common grouping criterion (either Inventory type or vendor name) +- **Section_Header**: The clickable row that displays the section label, item count, and collapse/expand toggle +- **Inventory_Section**: The top-level section containing items with workflow_type of CARD, GRANITE, or DECOM +- **Vendor_Section**: A section grouping FP and Archer workflow type items by their vendor field +- **Collapse_State**: A per-section boolean indicating whether the section body is hidden (collapsed) or visible (expanded) +- **Visible_Items**: Queue items with status "pending" that are displayed in the main list + +## Requirements + +### Requirement 1: Hybrid Grouping Layout + +**User Story:** As a queue operator, I want items grouped into logical sections (Inventory at top, then vendor groups), so that I can quickly locate items by their workflow category. + +#### Acceptance Criteria + +1. THE Queue_Page SHALL group Visible_Items into an Inventory_Section containing all items where workflow_type is CARD, GRANITE, or DECOM. +2. THE Queue_Page SHALL group remaining Visible_Items (workflow_type FP or Archer) into Vendor_Sections keyed by the item vendor field. +3. THE Queue_Page SHALL display the Inventory_Section before all Vendor_Sections. +4. THE Queue_Page SHALL sort Vendor_Sections alphabetically by vendor name. +5. WHEN a Visible_Item has no vendor value, THE Queue_Page SHALL place the item in a Vendor_Section labeled "Unknown". +6. WHEN the Inventory_Section contains zero items, THE Queue_Page SHALL omit the Inventory_Section from the layout. +7. WHEN a vendor has zero pending items, THE Queue_Page SHALL omit that Vendor_Section from the layout. + +### Requirement 2: Collapsible Section Headers + +**User Story:** As a queue operator, I want to collapse sections I am not working on, so that I can reduce visual clutter and focus on relevant items. + +#### Acceptance Criteria + +1. THE Queue_Page SHALL render a Section_Header for each displayed Section. +2. WHEN a user clicks a Section_Header, THE Queue_Page SHALL toggle the Collapse_State of that Section. +3. WHILE a Section is in expanded Collapse_State, THE Queue_Page SHALL display all items belonging to that Section. +4. WHILE a Section is in collapsed Collapse_State, THE Queue_Page SHALL hide all items belonging to that Section. +5. WHILE a Section is in collapsed Collapse_State, THE Section_Header SHALL remain visible with the section label and item count. +6. THE Section_Header SHALL display a directional chevron icon indicating the current Collapse_State (down for expanded, right for collapsed). +7. THE Queue_Page SHALL initialize all Sections in expanded Collapse_State on page load. + +### Requirement 3: Section Header Display + +**User Story:** As a queue operator, I want section headers to show the group name and item count, so that I can assess workload distribution at a glance. + +#### Acceptance Criteria + +1. THE Section_Header for the Inventory_Section SHALL display the label "Inventory" with the total count of items in that section. +2. THE Section_Header for each Vendor_Section SHALL display the vendor name with the total count of items in that section. +3. THE Section_Header SHALL use a monospace font, uppercase text, and styling consistent with the dark theme tactical intelligence aesthetic. +4. THE Section_Header SHALL use a distinct accent color for the Inventory_Section (green) and a neutral color for Vendor_Sections. +5. THE Section_Header SHALL display a bottom border to visually separate the header from section content. + +### Requirement 4: Cross-Section Selection Preservation + +**User Story:** As a queue operator, I want to select items across multiple sections without losing selections when collapsing sections, so that I can create consolidated Jira tickets from items in different groups. + +#### Acceptance Criteria + +1. WHILE selection mode is active, THE Queue_Page SHALL allow selecting items from any combination of Sections. +2. WHEN a Section is collapsed, THE Queue_Page SHALL preserve the selected state of items within that collapsed Section. +3. WHEN a Section is expanded after being collapsed, THE Queue_Page SHALL display the previously selected items as still selected. +4. THE selection count indicator SHALL reflect the total count of selected items across all Sections regardless of Collapse_State. +5. THE Select All checkbox SHALL toggle selection for all Visible_Items across all Sections regardless of Collapse_State. +6. THE floating action bar SHALL operate on all selected items across all Sections regardless of Collapse_State. + +### Requirement 5: Visual and Interaction Consistency + +**User Story:** As a queue operator, I want the collapsible sections to match the existing page aesthetic and not break existing functionality, so that the experience remains cohesive. + +#### Acceptance Criteria + +1. THE Queue_Page SHALL use inline styles consistent with the existing STYLES constant and dark theme tactical intelligence aesthetic. +2. THE Queue_Page SHALL use lucide-react icons for the collapse/expand chevron indicators. +3. THE Section_Header SHALL provide a pointer cursor to indicate clickability. +4. WHEN items are grouped into Sections, THE ticket link badges SHALL continue to display on items that have associated Jira tickets. +5. WHEN items are grouped into Sections, THE consolidation modal SHALL continue to function with the selected items. +6. WHEN items are grouped into Sections, THE floating action bar SHALL continue to appear when one or more items are selected. +7. THE Queue_Page SHALL continue to display the completed items count at the bottom of the page. +8. IF the queue contains zero Visible_Items, THEN THE Queue_Page SHALL display the existing empty state without any Section_Headers. diff --git a/.kiro/specs/queue-collapsible-sections/tasks.md b/.kiro/specs/queue-collapsible-sections/tasks.md new file mode 100644 index 0000000..c628cdd --- /dev/null +++ b/.kiro/specs/queue-collapsible-sections/tasks.md @@ -0,0 +1,112 @@ +# Implementation Plan: Queue Collapsible Sections + +## Overview + +Add collapsible, grouped sections to the IvantiTodoQueuePage. Items are organized into a hybrid layout: an "Inventory" section (CARD, GRANITE, DECOM workflow types) at the top, followed by vendor-grouped sections for FP and Archer items. Each section has a clickable header that toggles visibility. This is a frontend-only change to a single file with property-based tests for the grouping logic. + +## Tasks + +- [x] 1. Add section header styles and icon imports + - [x] 1.1 Add ChevronDown and ChevronRight to the lucide-react import statement + - Add `ChevronDown, ChevronRight` to the existing import from `lucide-react` + - _Requirements: 5.2_ + + - [x] 1.2 Add section header style entries to the STYLES constant + - Add `sectionHeaderInventory`, `sectionHeaderVendor`, and `sectionCount` style objects to the existing `STYLES` constant + - Use green accent (`#10B981`) for Inventory header, neutral (`#94A3B8`) for vendor headers + - Include `cursor: 'pointer'`, `userSelect: 'none'`, monospace font, uppercase text + - _Requirements: 3.3, 3.4, 3.5, 5.1, 5.3_ + +- [x] 2. Implement grouping computation and collapse state + - [x] 2.1 Add the `groupedSections` useMemo hook + - Add a `useMemo` that transforms `visibleItems` into an array of section objects `{ key, label, type, items }` + - Items with workflow_type CARD, GRANITE, or DECOM go into the Inventory section + - Remaining items are grouped by vendor field (null/empty vendor → "Unknown") + - Inventory section appears first (if non-empty), vendor sections sorted alphabetically + - Sections with zero items are omitted + - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7_ + + - [x] 2.2 Add collapse state and toggle handler + - Add `const [collapsedSections, setCollapsedSections] = useState({})` for collapse tracking + - Add `toggleSection` callback that flips the boolean for a given section key + - All sections default to expanded (empty object → falsy lookup) + - _Requirements: 2.2, 2.7_ + +- [x] 3. Refactor render to use grouped sections with collapsible headers + - [x] 3.1 Replace flat `visibleItems.map(...)` with grouped section rendering + - Map over `groupedSections` instead of `visibleItems` directly + - For each section, render a clickable Section Header with chevron icon, label, and item count + - Conditionally render section body (item rows) only when section is not collapsed + - Preserve existing queue item row rendering logic inside each section body + - Add `role="button"`, `tabIndex={0}`, `aria-expanded`, `aria-label` to section headers for accessibility + - _Requirements: 2.1, 2.3, 2.4, 2.5, 2.6, 3.1, 3.2_ + + - [x] 3.2 Verify cross-section selection still works correctly + - Ensure Select All checkbox still toggles all `visibleItems` across all sections regardless of collapse state + - Ensure selection count reflects total selected items across all sections + - Ensure floating action bar operates on all selected items regardless of collapse state + - Ensure collapsing a section does not clear or modify `selectedIds` + - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6_ + + - [x] 3.3 Verify existing functionality is preserved + - Ticket link badges continue to display on items with associated Jira tickets + - Consolidation modal continues to function with selected items + - Floating action bar appears when items are selected + - Completed items count still displays at the bottom + - Empty state renders without section headers when no visible items exist + - _Requirements: 5.4, 5.5, 5.6, 5.7, 5.8_ + +- [x] 4. Checkpoint - Ensure frontend builds successfully + - Ensure all tests pass, ask the user if questions arise. + +- [x] 5. Write property-based tests for grouping logic + - [x]* 5.1 Write property test: Grouping Correctness + - **Property 1: Grouping Correctness** + - Every item with workflow_type CARD, GRANITE, or DECOM appears in the Inventory section; every FP/Archer item appears in the vendor section matching its vendor field (or "Unknown" if null/empty); no item appears in more than one section + - Extract the grouping logic into a testable pure function or test it inline + - **Validates: Requirements 1.1, 1.2, 1.5** + + - [x]* 5.2 Write property test: Section Ordering + - **Property 2: Section Ordering** + - Inventory section (if present) is always first; vendor sections are sorted alphabetically by label + - **Validates: Requirements 1.3, 1.4** + + - [x]* 5.3 Write property test: Empty Section Omission + - **Property 3: Empty Section Omission** + - No section in the output has zero items; if no inventory-type items exist, no Inventory section appears + - **Validates: Requirements 1.6, 1.7** + + - [x]* 5.4 Write property test: Section Header Count Accuracy + - **Property 4: Section Header Count Accuracy** + - For every section, the items array length equals the count that would be displayed in the header + - **Validates: Requirements 3.1, 3.2** + + - [x]* 5.5 Write property test: Selection Independence from Collapse State + - **Property 5: Selection Independence from Collapse State** + - Toggling collapse state does not alter the set of selected item IDs; Select All always covers all visible items regardless of collapse + - **Validates: Requirements 4.2, 4.4, 4.5** + +- [x] 6. Final checkpoint - Ensure all tests pass + - Ensure all tests pass, ask the user if questions arise. + +## Notes + +- Tasks marked with `*` are optional and can be skipped for faster MVP +- This is a frontend-only change to a single file: `frontend/src/components/pages/IvantiTodoQueuePage.js` +- Property tests should use `fast-check` (already a project dependency) and extract the grouping logic as a pure function for testability +- The selection logic (`selectedIds`, `toggleSelectAll`, `allVisibleSelected`) operates on `visibleItems` which is independent of collapse state — no changes needed to selection logic +- The collapse state is purely visual — it controls rendering, not data + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["1.1", "1.2"] }, + { "id": 1, "tasks": ["2.1", "2.2"] }, + { "id": 2, "tasks": ["3.1"] }, + { "id": 3, "tasks": ["3.2", "3.3"] }, + { "id": 4, "tasks": ["5.1", "5.2", "5.3", "5.4", "5.5"] } + ] +} +``` diff --git a/.kiro/specs/remediation-plan-history/design.md b/.kiro/specs/remediation-plan-history/design.md index 988180e..4afc5c2 100644 --- a/.kiro/specs/remediation-plan-history/design.md +++ b/.kiro/specs/remediation-plan-history/design.md @@ -4,6 +4,8 @@ Adds an append-only audit trail for resolution_date and remediation_plan changes on compliance items. The design preserves the existing compliance_items schema (current values remain directly queryable) and introduces a new `compliance_item_history` table for historical entries. The pattern mirrors how `compliance_notes` works — separate rows with timestamps and attribution. +**Per-metric extension (Requirements 8–15):** The existing PATCH endpoint updates all active rows for a hostname uniformly. Since `compliance_items` already stores `resolution_date` and `remediation_plan` per row (each row is a hostname+metric_id pair), the extension allows targeting specific metrics via optional `metric_id`/`metric_ids` parameters. This mirrors the multi-metric notes pattern established in the `compliance-multi-metric-notes` spec — same chip selector UI, same Select All toggle, same `metric_ids` array API convention. + ## Architecture ``` @@ -59,7 +61,21 @@ Adds an append-only audit trail for resolution_date and remediation_plan changes └─────────────────────────────────────────────────────────────────┘ ``` -## Data Model +## Components and Interfaces + +### Backend Components + +- **PATCH /api/compliance/items/:hostname/metadata** — updates resolution_date and/or remediation_plan on compliance_items rows, records field-level change history. Extended with optional `metric_id`/`metric_ids` for per-metric scoping. +- **GET /api/compliance/items/:hostname** — returns device detail including history entries (with metric_id field). +- **POST /api/compliance/vcl/bulk-commit** — bulk update path that also records history per hostname. + +### Frontend Components + +- **ComplianceDetailPanel.js** — slide-out panel displaying device compliance detail, metadata editing, change history, and notes. +- **MetricChipSelector (metadata)** — chip-based multi-select for choosing which metrics a resolution_date/remediation_plan update applies to. Positioned above the metadata inputs. +- **HistoryMetricLabel** — renders a MetricChip for per-metric history entries or "All metrics" label for hostname-level entries. + +## Data Models ### New Table: compliance_item_history @@ -277,3 +293,492 @@ The `compliance_item_history` table is never referenced by any reporting query. - The LIMIT 10 on history retrieval prevents unbounded result sets for devices with many changes. - Indexes on (hostname, field_name) and (changed_at) ensure fast lookups without full table scans. - No additional queries are added to the VCL reporting paths. + +--- + +## Per-Metric Extension (Requirements 8–15) + +### Overview + +The `compliance_items` table already stores `resolution_date` and `remediation_plan` per row, where each row represents a unique (hostname, metric_id) pair. The current PATCH endpoint updates ALL active rows for a hostname uniformly. This extension adds optional metric scoping so analysts can set different resolution dates and remediation plans for individual metrics on the same device. + +The UI pattern replicates the multi-metric notes selector from the `compliance-multi-metric-notes` spec: chip-based multi-select with Select All toggle, positioned above the resolution_date and remediation_plan inputs. + +### Architecture — Per-Metric Flow + +```mermaid +sequenceDiagram + participant User + participant Panel as ComplianceDetailPanel + participant API as PATCH /items/:hostname/metadata + participant DB as PostgreSQL + + User->>Panel: Open detail panel for hostname + Panel->>Panel: Pre-select all active metrics (Select All default) + User->>Panel: Deselect some metrics, select specific ones + Panel->>Panel: Compute shared values for selected metrics + Panel->>Panel: Display shared value or "Multiple values" placeholder + + User->>Panel: Edit resolution_date / remediation_plan + User->>Panel: Click Save + + alt All metrics selected (Select All) + Panel->>API: PATCH { resolution_date, remediation_plan, change_reason } + Note over Panel,API: No metric_ids → hostname-level update (backward compat) + else Specific metrics selected + Panel->>API: PATCH { resolution_date, remediation_plan, change_reason, metric_ids: [...] } + end + + API->>API: Validate metric_ids against active items + API->>DB: SELECT current values per targeted metric + loop For each targeted metric with changed values + API->>DB: INSERT INTO compliance_item_history (hostname, metric_id, field_name, old_value, new_value, ...) + end + API->>DB: UPDATE compliance_items SET ... WHERE hostname = $1 AND metric_id = ANY($2) + API-->>Panel: { updated: N } + Panel->>Panel: Re-fetch detail, refresh history +``` + +### Schema Extension + +#### Add metric_id column to compliance_item_history + +```sql +ALTER TABLE compliance_item_history ADD COLUMN metric_id TEXT; + +CREATE INDEX IF NOT EXISTS idx_compliance_history_hostname_metric + ON compliance_item_history(hostname, metric_id); +``` + +The column is nullable. Existing rows retain `NULL` metric_id, indicating they were hostname-level changes made before this extension. New per-metric changes populate the column; hostname-level changes (no metric_ids in request) continue to insert with `NULL`. + +#### Migration File: `backend/migrations/add_compliance_history_metric_id.js` + +```javascript +const pool = require('../db'); + +async function run() { + console.log('Starting compliance_item_history metric_id migration...'); + try { + // Add nullable metric_id column (idempotent check) + const { rows } = await pool.query(` + SELECT column_name FROM information_schema.columns + WHERE table_name = 'compliance_item_history' AND column_name = 'metric_id' + `); + if (rows.length === 0) { + await pool.query(`ALTER TABLE compliance_item_history ADD COLUMN metric_id TEXT`); + console.log('✓ metric_id column added to compliance_item_history'); + } else { + console.log('✓ metric_id column already exists'); + } + + await pool.query(` + CREATE INDEX IF NOT EXISTS idx_compliance_history_hostname_metric + ON compliance_item_history(hostname, metric_id) + `); + console.log('✓ (hostname, metric_id) index created'); + + console.log('Migration complete.'); + } catch (err) { + console.error('Migration failed:', err.message); + throw err; + } +} + +module.exports = { run }; + +// Self-execute when run directly +if (require.main === module) { + run().then(() => process.exit(0)).catch(() => process.exit(1)); +} +``` + +No backfill is performed — pre-existing history rows with `NULL` metric_id correctly represent hostname-level changes. + +### API Changes — Per-Metric Scoping + +#### PATCH /api/compliance/items/:hostname/metadata (extended) + +**New optional request body fields:** + +```json +{ + "resolution_date": "2026-03-15", + "remediation_plan": "Upgrade firmware to v4.2", + "change_reason": "Vendor pushed back delivery date", + "metric_id": "2.1.1", + "metric_ids": ["2.1.1", "2.3.2"] +} +``` + +| Field | Type | Description | +|---|---|---| +| `metric_id` | string (optional) | Scope update to a single metric | +| `metric_ids` | string[] (optional) | Scope update to multiple specific metrics | + +**Precedence:** If both `metric_id` and `metric_ids` are provided, `metric_ids` wins and `metric_id` is ignored. + +**Validation:** +- Each metric_id must be a non-empty string of 100 characters or fewer +- Each metric_id must correspond to an active compliance_item for the hostname — if any provided metric_id has no matching active row, return 400 with `{ error: "Invalid metric_id: — no active compliance item found" }` + +**Behavior by scoping mode:** + +| Mode | Condition | Query target | History metric_id | +|---|---|---|---| +| Hostname-level | Neither `metric_id` nor `metric_ids` provided | All active rows for hostname | `NULL` | +| Single metric | `metric_id` provided (no `metric_ids`) | Single row matching hostname + metric_id | The metric_id value | +| Multi-metric | `metric_ids` provided | Rows matching hostname + any of the metric_ids | Per-row metric_id | + +**Per-metric history insertion logic:** + +When metric scoping is active, the endpoint queries current values per targeted metric individually (since they may differ), then inserts one history row per metric per changed field: + +```sql +-- Get current values for each targeted metric +SELECT metric_id, resolution_date, remediation_plan +FROM compliance_items +WHERE hostname = $1 AND metric_id = ANY($2) AND status = 'active' +``` + +```sql +-- Insert history per metric per changed field +INSERT INTO compliance_item_history (hostname, metric_id, field_name, old_value, new_value, change_reason, changed_by) +VALUES ($1, $2, $3, $4, $5, $6, $7) +``` + +```sql +-- Update only targeted rows +UPDATE compliance_items +SET resolution_date = $1, remediation_plan = $2 +WHERE hostname = $3 AND metric_id = ANY($4) AND status = 'active' +``` + +**Response:** unchanged (`{ updated: number }`). + +#### GET /api/compliance/items/:hostname (extended history response) + +The history query now includes `metric_id`: + +```sql +SELECT id, hostname, metric_id, field_name, old_value, new_value, change_reason, changed_by, changed_at +FROM compliance_item_history +WHERE hostname = $1 +ORDER BY changed_at DESC +LIMIT 10 +``` + +Response shape — history entries now include `metric_id`: + +```json +{ + "history": [ + { + "id": 55, + "metric_id": "2.1.1", + "field_name": "resolution_date", + "old_value": "2026-02-01", + "new_value": "2026-04-15", + "change_reason": "Vendor delay on patch", + "changed_by": "jsmith", + "changed_at": "2026-02-10T11:00:00.000Z" + }, + { + "id": 54, + "metric_id": null, + "field_name": "remediation_plan", + "old_value": "Upgrade firmware", + "new_value": "Replace hardware", + "change_reason": null, + "changed_by": "admin", + "changed_at": "2026-02-05T09:00:00.000Z" + } + ] +} +``` + +### Frontend Changes — Per-Metric Metadata Selector + +#### New State + +```javascript +const [metricSelection, setMetricSelection] = useState([]); // metric_ids for metadata editing +``` + +This is separate from the existing `selectedMetrics` state (used for notes). The metadata metric selector has its own selection state. + +#### UI Layout + +The Metric_Selector is placed above the resolution_date and remediation_plan inputs, matching the notes section pattern: + +``` +ComplianceDetailPanel +├── Header (hostname, IP, device type, team) +├── Section: Failing Metrics +├── Section: Resolved Metrics +├── Section: Remediation Plan & Resolution Date +│ ├── MetricChipSelector (for metadata) ← NEW +│ │ ├── Select All / Deselect All toggle +│ │ └── MetricChip (per active metric, clickable) +│ ├── Resolution Date input (populated from selection) +│ ├── Remediation Plan textarea (populated from selection) +│ ├── Change Reason input +│ └── Save button +├── Section: Change History +│ └── HistoryEntry (with optional MetricChip) ← EXTENDED +├── Section: Notes +│ ├── NoteCard (grouped by group_id) +│ └── Add Note Form (with its own MetricChipSelector) +``` + +#### MetricChipSelector Behavior (Metadata) + +| State | Behavior | +|---|---| +| 1 active metric | Chip is pre-selected and non-removable. No Select All toggle. | +| 2+ active metrics, panel just opened | **All metrics pre-selected** (Select All default). Select All toggle visible. | +| User clicks unselected chip | Chip added to selection | +| User clicks selected chip (2+ selected) | Chip removed from selection | +| User clicks selected chip (only 1 selected, 2+ metrics exist) | No-op — at least one must remain selected | +| Select All clicked | All active metrics selected, toggle label changes to "Deselect All" | +| Deselect All clicked | All metrics deselected except the first (minimum selection invariant) | + +**Key difference from notes selector:** The metadata selector defaults to "Select All" on panel open (preserving existing hostname-level behavior), while the notes selector defaults to the first metric only. + +#### Per-Metric Field Display Logic + +When the metric selection changes, the panel computes what to display in the resolution_date and remediation_plan inputs: + +```javascript +function computeSharedValues(metrics, selectedIds) { + const selected = metrics.filter(m => selectedIds.includes(m.metric_id)); + if (selected.length === 0) return { resolution_date: '', remediation_plan: '' }; + if (selected.length === 1) { + return { + resolution_date: selected[0].resolution_date || '', + remediation_plan: selected[0].remediation_plan || '', + }; + } + + // Multiple selected — check if values are uniform + const dates = new Set(selected.map(m => m.resolution_date || '')); + const plans = new Set(selected.map(m => m.remediation_plan || '')); + + return { + resolution_date: dates.size === 1 ? [...dates][0] : null, // null = "Multiple values" + remediation_plan: plans.size === 1 ? [...plans][0] : null, // null = "Multiple values" + }; +} +``` + +**Display rules:** + +| Computed value | Input state | +|---|---| +| Non-null string (including empty) | Input populated with that value | +| `null` (multiple different values) | Input empty, placeholder text: "Multiple values" | + +**Save behavior with "Multiple values":** + +- If the user leaves an input untouched while it shows "Multiple values" placeholder, that field is NOT included in the PATCH request body — preserving existing per-metric values. +- If the user types a new value into a "Multiple values" field, that value is sent and applied to all selected metrics. + +#### Save Flow — Metric Scoping + +```javascript +const handleSaveMetadata = async (fields) => { + const body = { ...fields }; + if (changeReason.trim()) body.change_reason = changeReason.trim(); + + // Determine if we need metric scoping + const allSelected = activeMetrics.length > 0 && + activeMetrics.every(m => metricSelection.includes(m.metric_id)); + + if (!allSelected) { + // Specific metrics selected — include metric_ids + body.metric_ids = metricSelection; + } + // If all selected, omit metric_ids → hostname-level update (backward compat) + + const res = await fetch(`${API_BASE}/compliance/items/${encodeURIComponent(hostname)}/metadata`, { + method: 'PATCH', + credentials: 'include', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body), + }); + // ... error handling, re-fetch ... +}; +``` + +### History Display — Per-Metric Extension + +History entries now optionally show which metric the change applied to: + +**Display format (extended):** + +``` +[MetricChip "2.1.1"] [field_name icon] old_value → new_value + username · 2026-01-20 [reason if present] +``` + +Or for hostname-level changes (metric_id is null): + +``` +[All metrics] [field_name icon] old_value → new_value + username · 2026-01-20 [reason if present] +``` + +**Rendering logic:** + +```javascript +function HistoryMetricLabel({ metricId, metricMap }) { + if (metricId) { + return ; + } + return ( + + All metrics + + ); +} +``` + +### Per-Metric Reporting + +The burndown and donut queries already read from individual `compliance_items` rows (each row has its own `resolution_date`). The per-metric extension does not change reporting queries — it only changes how the PATCH endpoint targets rows. + +**Existing burndown query (unchanged):** + +```sql +SELECT resolution_date, COUNT(*) as count +FROM compliance_items +WHERE status = 'active' AND resolution_date IS NOT NULL +GROUP BY resolution_date +ORDER BY resolution_date +``` + +Each row is already a (hostname, metric_id) pair, so per-metric resolution dates are naturally reflected in the burndown without query changes. + +**Aggregated hostname view (unchanged):** + +```sql +SELECT hostname, MAX(resolution_date) as resolution_date +FROM compliance_items +WHERE status = 'active' +GROUP BY hostname +``` + +When deduplicating by hostname for summary views, the latest (MAX) resolution_date among all active metrics is used. + +### Error Handling — Per-Metric Extension + +| Scenario | Response | Behavior | +|---|---|---| +| `metric_ids` contains a metric_id with no active row | 400 `{ error: "Invalid metric_id: — no active compliance item found" }` | No rows updated, no history inserted | +| `metric_ids` is empty array | 400 `{ error: "metric_ids must contain at least one entry" }` | No rows updated | +| `metric_id` or entry in `metric_ids` exceeds 100 chars | 400 `{ error: "metric_id exceeds 100 characters" }` | No rows updated | +| `metric_id` or entry in `metric_ids` is empty string | 400 `{ error: "metric_id cannot be empty" }` | No rows updated | +| Per-metric update targets metrics with different current values | N/A (success) | One history entry per metric per changed field | +| All targeted metrics already have the new value | 200 `{ updated: 0 }` | No history entries created (no-change skip) | + +### Backward Compatibility + +The per-metric extension is fully backward compatible: + +1. **API:** Requests without `metric_id`/`metric_ids` follow the existing hostname-level path — update all active rows, insert history with `NULL` metric_id. +2. **Bulk commit:** Continues to apply values to all active metrics for each hostname. No `metric_ids` support in bulk (same as before). +3. **UI default:** Panel opens with all metrics selected. Saving without changing selection omits `metric_ids` from the request, triggering hostname-level behavior. +4. **History display:** Existing history entries (NULL metric_id) display as "All metrics" — no visual regression. +5. **Reporting:** No query changes. Per-row resolution_date was already the source of truth for burndown calculations. + +## Correctness Properties + +### Property 1: History entries are created only when values change + +*For any* PATCH request to `/items/:hostname/metadata` with a resolution_date or remediation_plan value, a history entry is inserted if and only if the new value differs from the current value on the targeted compliance_items row(s). Identical values produce zero history rows. + +**Validates: Requirements 1.4, 6.3** + +### Property 2: Per-metric history records the correct old_value per row + +*For any* per-metric update targeting N metrics with potentially different current values, the system inserts one history entry per metric per changed field, and each entry's `old_value` matches the actual previous value of that specific compliance_items row — not a shared/aggregated value. + +**Validates: Requirements 11.4, 11.5** + +### Property 3: Hostname-level updates produce NULL metric_id in history + +*For any* PATCH request without `metric_id` or `metric_ids`, all resulting history entries have `metric_id = NULL`, indicating the change applied to all active metrics for the hostname. + +**Validates: Requirements 11.3, 15.1** + +### Property 4: Backward compatibility — omitting metric_ids updates all active rows + +*For any* PATCH request without `metric_id` or `metric_ids`, the number of compliance_items rows updated equals the count of active rows for that hostname. The behavior is identical to the pre-extension endpoint. + +**Validates: Requirements 8.6, 15.1, 15.2** + +### Property 5: Invalid metric_ids reject the entire request atomically + +*For any* PATCH request where at least one entry in `metric_ids` does not correspond to an active compliance_item for the hostname, the endpoint returns 400 and no rows are updated, no history entries are created. + +**Validates: Requirements 8.7** + +### Property 6: Select All default preserves hostname-level behavior + +*For any* panel open action on a hostname with N active metrics, the metadata metric selector defaults to all N metrics selected. Saving without changing the selection omits `metric_ids` from the request body, triggering the hostname-level update path. + +**Validates: Requirements 9.7, 15.3, 15.4** + +### Property 7: Shared value display is a pure function of selection + +*For any* set of selected metrics, if all share the same resolution_date (or remediation_plan), the input displays that value. If any differ, the input shows "Multiple values" placeholder. This is deterministic and independent of selection order. + +**Validates: Requirements 10.2, 10.3, 10.4, 10.5** + +## Testing Strategy + +### Unit Tests + +**Backend:** +- PATCH without metric_ids updates all active rows (backward compat) +- PATCH with `metric_id` updates only the single matching row +- PATCH with `metric_ids` updates only matching rows +- Invalid metric_id returns 400 with descriptive error +- History entries include correct metric_id (or NULL for hostname-level) +- Per-metric update with different current values creates per-row history entries +- Both `metric_id` and `metric_ids` provided — `metric_ids` wins + +**Frontend:** +- MetricChipSelector defaults to all metrics selected on panel open +- Shared value computation returns correct value when all metrics agree +- Shared value computation returns null when metrics disagree +- Save with all selected omits metric_ids from request body +- Save with subset includes metric_ids in request body +- History entries with metric_id render MetricChip +- History entries with null metric_id render "All metrics" label + +### Property-Based Tests + +Property-based tests use `fast-check` with a minimum of 100 iterations per property. + +- **Property 2: Per-metric history records correct old_value** — generate random sets of metrics with varying current values, apply a uniform update, verify each history entry's old_value matches the pre-update value for that specific metric. +- **Property 4: Backward compatibility** — generate random hostnames with N active metrics, send PATCH without metric_ids, verify all N rows are updated. +- **Property 7: Shared value display** — generate random arrays of metric objects with varying resolution_date/remediation_plan values, verify computeSharedValues returns the correct shared value or null. + +### Integration Tests + +- End-to-end: open panel → select 2 of 4 metrics → change resolution_date → save → verify only 2 rows updated, history entries have correct metric_ids +- Migration: verify metric_id column exists after running migration, existing rows retain NULL +- Backward compat: existing bulk-commit flow continues to work without metric_ids + +### Performance Considerations — Per-Metric Extension + +- Per-metric updates query current values with `WHERE metric_id = ANY($2)` — the new `(hostname, metric_id)` index on `compliance_item_history` ensures efficient lookups. +- Worst case: a device with 15 active metrics, all selected, both fields changed = 30 history inserts per save. Still lightweight within a single transaction. +- The `(hostname, metric_id)` index on the history table supports future filtering of history by metric if needed. +- No additional queries added to reporting paths. diff --git a/.kiro/specs/remediation-plan-history/requirements.md b/.kiro/specs/remediation-plan-history/requirements.md index 89a0b89..c6c2a01 100644 --- a/.kiro/specs/remediation-plan-history/requirements.md +++ b/.kiro/specs/remediation-plan-history/requirements.md @@ -4,6 +4,8 @@ Historical tracking for resolution dates and remediation plans on compliance items. When a user changes the resolution_date or remediation_plan for a device, the previous value is preserved as an audit trail entry with a timestamp and the identity of the user who made the change. The most recent values remain directly queryable on the compliance_items table so existing VCL reporting queries continue to work without modification. +This spec also covers per-metric scoping of resolution_date and remediation_plan (GitLab issue #19). Previously these fields were edited at the hostname level (one value applied uniformly to all active metrics for a device). The per-metric extension allows analysts to set different resolution dates and remediation plans for individual metrics within the same device, matching the pattern already established by compliance notes. + ## Glossary - **Compliance_Item**: A row in the `compliance_items` table representing a single non-compliant device/metric pair. @@ -14,6 +16,10 @@ Historical tracking for resolution dates and remediation plans on compliance ite - **Detail_Panel**: The ComplianceDetailPanel UI component that displays device-level compliance information and allows editing of metadata fields. - **VCL_Report**: The multi-vertical compliance reporting system that uses resolution_date for burndown forecasts and blocked/in-progress donut charts. - **Current_Value**: The value stored directly on the compliance_items row, representing the most recent resolution_date or remediation_plan. +- **Metric_Selector**: The UI control in the Detail_Panel that allows the user to choose which metric(s) a remediation plan update applies to. Displays active metrics as selectable options with category-colored chips. +- **Active_Metric**: A compliance item with `status = 'active'` for the selected hostname — a metric currently failing for that device. +- **Metric_Chip**: A small colored badge displaying a metric ID, used throughout the compliance UI to visually identify metrics by category color. +- **Metadata_API**: The `PATCH /api/compliance/items/:hostname/metadata` endpoint that updates resolution_date and remediation_plan on compliance items. ## Requirements @@ -96,3 +102,101 @@ Historical tracking for resolution dates and remediation plans on compliance ite 2. THE Migration SHALL create the required indexes on the `compliance_item_history` table. 3. THE Migration SHALL be idempotent and safe to run multiple times without error. 4. THE Migration SHALL NOT modify the existing compliance_items table structure. + +### Requirement 8: Per-Metric Metadata API + +**User Story:** As a compliance analyst, I want to set resolution dates and remediation plans for specific metrics within a device, so that I can track different remediation timelines for different compliance failures on the same hostname. + +#### Acceptance Criteria + +1. THE Metadata_API SHALL accept an optional `metric_id` field (string) in the request body to scope the update to a single metric for the given hostname. +2. THE Metadata_API SHALL accept an optional `metric_ids` field (array of strings) in the request body to scope the update to multiple specific metrics for the given hostname. +3. WHEN `metric_ids` is provided, THE Metadata_API SHALL update only the compliance_items rows matching the specified hostname AND metric_id values. +4. WHEN `metric_id` is provided (single string), THE Metadata_API SHALL update only the compliance_items row matching the specified hostname AND metric_id. +5. IF both `metric_id` and `metric_ids` are provided, THEN THE Metadata_API SHALL use `metric_ids` and ignore `metric_id`. +6. WHEN neither `metric_id` nor `metric_ids` is provided, THE Metadata_API SHALL update all active compliance_items for the hostname, preserving backward compatibility with the existing hostname-level behavior. +7. IF a provided metric_id does not correspond to an active compliance_item for the hostname, THEN THE Metadata_API SHALL return a 400 error identifying the invalid metric_id. +8. WHEN `metric_ids` is provided, THE Metadata_API SHALL validate that each entry is a non-empty string of 100 characters or fewer. + +### Requirement 9: Per-Metric Metric Selector UI + +**User Story:** As a compliance analyst, I want a metric selector in the detail panel when editing remediation plans, so that I can choose which metrics a resolution date or remediation plan applies to — matching the pattern used for notes. + +#### Acceptance Criteria + +1. WHEN the Detail_Panel is open for a hostname with more than one Active_Metric, THE Detail_Panel SHALL display a Metric_Selector above the resolution date and remediation plan inputs. +2. WHEN the Detail_Panel is open for a hostname with exactly one Active_Metric, THE Detail_Panel SHALL pre-select that metric and display the Metric_Selector as a single non-removable selection. +3. THE Metric_Selector SHALL allow the user to select one or more Active_Metrics simultaneously. +4. THE Metric_Selector SHALL display each option using the Metric_Chip component with the metric's category color, so that metrics are visually distinguishable. +5. WHEN the hostname has more than one Active_Metric, THE Metric_Selector SHALL display a "Select All" toggle that selects all Active_Metrics when activated. +6. WHEN all Active_Metrics are already selected, THE "Select All" toggle SHALL change to "Deselect All" and deselect all Active_Metrics when activated. +7. WHEN the Detail_Panel first opens for a hostname with multiple Active_Metrics, THE Metric_Selector SHALL pre-select all Active_Metrics by default, preserving the existing hostname-level editing experience. + +### Requirement 10: Per-Metric Field Display + +**User Story:** As a compliance analyst, I want to see the current resolution date and remediation plan for the selected metric(s), so that I know what values are already set before making changes. + +#### Acceptance Criteria + +1. WHEN a single metric is selected in the Metric_Selector, THE Detail_Panel SHALL populate the resolution date and remediation plan inputs with the current values from that specific compliance_item row. +2. WHEN multiple metrics are selected and all share the same resolution_date value, THE Detail_Panel SHALL display that shared value in the resolution date input. +3. WHEN multiple metrics are selected and they have different resolution_date values, THE Detail_Panel SHALL display the resolution date input as empty with placeholder text indicating "Multiple values". +4. WHEN multiple metrics are selected and all share the same remediation_plan value, THE Detail_Panel SHALL display that shared value in the remediation plan input. +5. WHEN multiple metrics are selected and they have different remediation_plan values, THE Detail_Panel SHALL display the remediation plan input as empty with placeholder text indicating "Multiple values". +6. WHEN the user saves with "Multiple values" placeholder visible and the input left empty, THE Detail_Panel SHALL NOT send that field in the PATCH request, preserving existing per-metric values. + +### Requirement 11: Per-Metric History Tracking + +**User Story:** As a compliance analyst, I want the change history to record which specific metric was changed, so that the audit trail reflects per-metric remediation plan changes accurately. + +#### Acceptance Criteria + +1. THE compliance_item_history table SHALL include a `metric_id` column (text, nullable) to record which metric the change applies to. +2. WHEN a per-metric update changes a field value, THE History_Service SHALL record the metric_id in the History_Entry. +3. WHEN a hostname-level update (no metric_id specified) changes a field value, THE History_Service SHALL record the metric_id as NULL in the History_Entry, indicating the change applied to all metrics. +4. THE History_Entry SHALL record the old_value and new_value per metric when a per-metric update is performed, reflecting the actual previous value of that specific compliance_item row. +5. WHEN a per-metric update targets multiple metrics with different current values, THE History_Service SHALL insert one History_Entry per metric that actually changed, each with its own old_value. +6. THE Compliance_API SHALL include the metric_id field in history entries returned to the client. + +### Requirement 12: Per-Metric History Display + +**User Story:** As a compliance analyst, I want the change history section to show which metric each change applied to, so that I can distinguish between hostname-wide changes and metric-specific changes. + +#### Acceptance Criteria + +1. WHEN a History_Entry has a non-null metric_id, THE Detail_Panel SHALL display the associated Metric_Chip next to the history entry. +2. WHEN a History_Entry has a null metric_id, THE Detail_Panel SHALL display "All metrics" label next to the history entry to indicate a hostname-level change. +3. THE Detail_Panel SHALL continue to display all history entries for the hostname in reverse chronological order, regardless of metric_id. + +### Requirement 13: Per-Metric Reporting + +**User Story:** As a VCL report consumer, I want burndown forecasts to use per-metric resolution dates, so that the forecast accurately reflects when each individual compliance failure is expected to be resolved. + +#### Acceptance Criteria + +1. THE VCL_Report SHALL read resolution_date from each individual compliance_items row (per hostname+metric_id pair) for burndown calculations, rather than using a single hostname-level value. +2. THE VCL_Report SHALL count each compliance_item row with a null resolution_date as a separate blocker in the donut chart. +3. THE VCL_Report SHALL bucket each compliance_item row by its own resolution_date month for the burndown forecast chart. +4. WHEN deduplicating by hostname for aggregated views, THE VCL_Report SHALL use the latest (MAX) resolution_date among all active metrics for that hostname. + +### Requirement 14: Per-Metric History Migration + +**User Story:** As a developer, I want the history table extended with a metric_id column via a migration script, so that per-metric history tracking can be deployed to existing environments. + +#### Acceptance Criteria + +1. THE Migration SHALL add a nullable `metric_id` column (text) to the `compliance_item_history` table. +2. THE Migration SHALL create an index on (hostname, metric_id) for efficient per-metric history lookups. +3. THE Migration SHALL be idempotent and safe to run multiple times without error. +4. THE Migration SHALL NOT alter or backfill existing history rows — pre-existing entries retain a NULL metric_id indicating they were hostname-level changes. + +### Requirement 15: Backward Compatibility + +**User Story:** As an existing user of the bulk upload and API workflows, I want hostname-level updates to continue working without modification, so that existing integrations and scripts are not broken by the per-metric change. + +#### Acceptance Criteria + +1. WHEN the Metadata_API receives a request without `metric_id` or `metric_ids`, THE Metadata_API SHALL update all active compliance_items for the hostname, matching the pre-existing behavior. +2. WHEN the bulk update commit endpoint processes a row, THE Metadata_API SHALL continue to apply resolution_date and remediation_plan to all active metrics for that hostname. +3. THE Detail_Panel SHALL default to "Select All" metrics when first opened, so that saving without changing the metric selection produces the same hostname-level update behavior as before. +4. WHEN all metrics are selected and the user saves, THE Metadata_API SHALL NOT include metric_ids in the request body, triggering the hostname-level update path for backward compatibility. diff --git a/.kiro/specs/remediation-plan-history/tasks.md b/.kiro/specs/remediation-plan-history/tasks.md index b803001..264a54a 100644 --- a/.kiro/specs/remediation-plan-history/tasks.md +++ b/.kiro/specs/remediation-plan-history/tasks.md @@ -1,51 +1,85 @@ -# Tasks — Remediation Plan History +# Implementation Plan: Remediation Plan History — Per-Metric Extension -## Task 1: Create migration for compliance_item_history table [Requirement 2, 7] -- [x] Create `backend/migrations/add_compliance_item_history.js` with the schema from the design doc -- [~] Table: `compliance_item_history` (id, hostname, field_name, old_value, new_value, change_reason, changed_by, changed_at) -- [~] Add CHECK constraint on field_name: IN ('resolution_date', 'remediation_plan') -- [~] Add index on (hostname, field_name) -- [~] Add index on (changed_at) -- [~] Register in `migrations/run-all.js` -- [~] Run migration and verify table exists +## Overview -## Task 2: Modify PATCH /items/:hostname/metadata to record history [Requirement 1, 6] -- [~] In `backend/routes/compliance.js`, locate the PATCH metadata handler -- [~] Accept new optional `change_reason` field (max 500 chars, validated) -- [~] Before updating compliance_items, SELECT current resolution_date and remediation_plan for the hostname -- [~] For each field where old !== new, INSERT into compliance_item_history (hostname, field_name, old_value, new_value, change_reason, changed_by) -- [~] Skip history insert if old === new (no-op changes) -- [~] Wrap history insert + item update in a transaction -- [~] Handle NULL → value and value → NULL transitions -- [~] Add audit log entry with old/new values -- [~] Verify existing response shape is preserved +Extends the existing remediation plan history system to support per-metric scoping of resolution_date and remediation_plan. The original hostname-level implementation (Tasks 1–6) is already complete. Tasks 7–12 add metric_id targeting to the PATCH endpoint, a metric selector UI in the detail panel, per-metric history tracking, and verification that reporting queries work correctly with per-metric resolution dates. This addresses GitLab issue #19. -## Task 3: Extend GET /items/:hostname to return history [Requirement 4] -- [~] In the existing `/items/:hostname` handler, add a query: `SELECT id, field_name, old_value, new_value, change_reason, changed_by, changed_at FROM compliance_item_history WHERE hostname = $1 ORDER BY changed_at DESC LIMIT 10` -- [~] Add `history` array to the response object -- [~] If query fails, return empty array (graceful degradation) and log error -- [~] Verify response includes history alongside existing metrics and notes +## Tasks -## Task 4: Modify bulk update commit to track history [Requirement 6] -- [~] In the bulk update flow (POST /vcl/bulk-commit), before updating each hostname's resolution_date or remediation_plan, query current values -- [~] For each changed field, INSERT into compliance_item_history with changed_by = req.user.username -- [~] Skip if value is unchanged -- [~] No change_reason for bulk updates (set to NULL) +- [x] 1. Create migration for compliance_item_history table + - [x] 1.1 Create `backend/migrations/add_compliance_item_history.js` with schema, CHECK constraint, indexes, register in run-all.js, and verify table exists + - _Requirements: 2, 7_ -## Task 5: Add change_reason input and history section to ComplianceDetailPanel [Requirement 5] -- [~] Add `changeReason` state and a single-line text input between the Save button and Notes section -- [~] Pass `change_reason` in the PATCH request body when saving -- [~] Clear changeReason after successful save -- [~] Add "Change History" section below the remediation plan area -- [~] Fetch history from the GET /items/:hostname response -- [~] Display entries: field icon, old → new, username, date, reason (if present) -- [~] Resolution dates formatted as YYYY-MM-DD, NULL shown as "—" -- [~] Remediation plan values truncated to 60 chars with title tooltip -- [~] Show "No changes recorded" when history is empty -- [~] Run `npm run build` after changes +- [x] 2. Modify PATCH /items/:hostname/metadata to record history + - [x] 2.1 Accept change_reason, SELECT current values, INSERT history for changed fields, wrap in transaction, handle NULL transitions, add audit log + - _Requirements: 1, 6_ -## Task 6: Verify VCL burndown is unaffected [Requirement 3] -- [~] Confirm burndown query in vclMultiVertical.js reads from compliance_items.resolution_date only -- [~] Confirm donut query uses MAX(resolution_date) grouped by hostname -- [~] Set a resolution date, change it multiple times, verify device appears once in burndown -- [~] No code changes expected — verification only +- [x] 3. Extend GET /items/:hostname to return history + - [x] 3.1 Add history query, include history array in response, graceful degradation on failure + - _Requirements: 4_ + +- [x] 4. Modify bulk update commit to track history + - [x] 4.1 Query current values before update, INSERT history for changed fields, skip unchanged, NULL change_reason + - _Requirements: 6_ + +- [x] 5. Add change_reason input and history section to ComplianceDetailPanel + - [x] 5.1 Add changeReason state and input, pass in PATCH body, display Change History section with formatted entries, run npm run build + - _Requirements: 5_ + +- [x] 6. Verify VCL burndown is unaffected + - [x] 6.1 Confirm burndown and donut queries read from compliance_items.resolution_date only, no code changes + - _Requirements: 3_ + +- [x] 7. Create migration to add metric_id column to compliance_item_history + - [x] 7.1 Create `backend/migrations/add_compliance_history_metric_id.js` that adds nullable `metric_id TEXT` column (idempotent check if column exists), creates index on (hostname, metric_id), register in run-all.js, run migration and verify column exists, verify existing rows retain NULL + - _Requirements: 14_ + +- [x] 8. Extend PATCH /items/:hostname/metadata to support per-metric scoping + - [x] 8.1 Accept optional `metric_id` (string) and `metric_ids` (array) in request body; if both provided use metric_ids; validate non-empty and max 100 chars; validate each corresponds to active compliance_item + - _Requirements: 8_ + - [x] 8.2 When metric_ids provided: SELECT current values per targeted metric, INSERT history with metric_id per changed field, UPDATE only matching rows + - _Requirements: 8, 11_ + - [x] 8.3 When neither metric_id nor metric_ids provided: preserve hostname-level behavior with NULL metric_id in history entries + - _Requirements: 15_ + +- [x] 9. Extend GET /items/:hostname to include metric_id in history entries + - [x] 9.1 Update history query to SELECT metric_id column, include metric_id in each history entry in response, verify NULL entries returned correctly + - _Requirements: 4, 11_ + +- [x] 10. Add MetricChipSelector for metadata editing in ComplianceDetailPanel + - [x] 10.1 Add metricSelection state separate from notes selector, default all active metrics selected on panel open, render MetricChipSelector above resolution_date/remediation_plan inputs with category-colored chips and Select All/Deselect All toggle + - _Requirements: 9_ + - [x] 10.2 Implement computeSharedValues: display shared value when all selected metrics agree, show "Multiple values" placeholder when they differ, omit unchanged fields from PATCH + - _Requirements: 10_ + - [x] 10.3 When all metrics selected omit metric_ids from request body (backward compat), when subset selected include metric_ids, run npm run build + - _Requirements: 15_ + +- [x] 11. Update history display to show per-metric labels + - [x] 11.1 When history entry has non-null metric_id display MetricChip, when null display "All metrics" label, build metricMap from metrics array, verify existing entries display correctly, run npm run build + - _Requirements: 12_ + +- [x] 12. Verify per-metric burndown reporting works correctly + - [x] 12.1 Confirm burndown forecast reads resolution_date per compliance_items row, donut uses per-row presence, aggregated view uses MAX(resolution_date), no code changes expected + - _Requirements: 13_ + +## Notes + +- Tasks 1–6 are already implemented (hostname-level history tracking is live in production) +- Tasks 7–12 implement the per-metric extension from GitLab issue #19 +- Task 7 must run first as it adds the metric_id column needed by all subsequent tasks +- Tasks 10 and 11 can run in parallel after Task 9 completes +- Task 12 is verification-only and depends on both frontend tasks completing + +## Task Dependency Graph + +```json +{ + "waves": [ + { "id": 0, "tasks": ["7.1"] }, + { "id": 1, "tasks": ["8.1", "8.2", "8.3"] }, + { "id": 2, "tasks": ["9.1"] }, + { "id": 3, "tasks": ["10.1", "10.2", "10.3", "11.1"] }, + { "id": 4, "tasks": ["12.1"] } + ] +} +``` diff --git a/.kiro/specs/vendor-issue-type-dropdown/.config.kiro b/.kiro/specs/vendor-issue-type-dropdown/.config.kiro new file mode 100644 index 0000000..608ea1f --- /dev/null +++ b/.kiro/specs/vendor-issue-type-dropdown/.config.kiro @@ -0,0 +1 @@ +{"specId": "02adddd3-ffc0-4935-95f2-7f8d89b9c1ff", "workflowType": "requirements-first", "specType": "feature"} diff --git a/.kiro/specs/vendor-issue-type-dropdown/design.md b/.kiro/specs/vendor-issue-type-dropdown/design.md new file mode 100644 index 0000000..21491e9 --- /dev/null +++ b/.kiro/specs/vendor-issue-type-dropdown/design.md @@ -0,0 +1,295 @@ +# Design Document: Vendor Issue Type Dropdown + +## Overview + +This feature makes the Issue Type dropdown in the Jira ticket Creation Modal context-aware. When a user enters a project key that belongs to a known vendor project (e.g., `AA_VECIMA`), the dropdown switches from STEAM issue types to vendor-specific issue types. The determination is purely frontend logic — a function that compares the trimmed, uppercased project key against a constant array of vendor keys. The backend remains agnostic to issue type values, passing any non-empty string through to the Jira API unchanged. + +The design prioritizes single-source-of-truth configuration: one constant array for vendor keys, one for vendor issue types, one for STEAM issue types. Both the JiraPage modal and the Ivanti queue modal reference the same constants and the same determination logic. + +## Architecture + +```mermaid +flowchart TD + subgraph Frontend Constants + VPK[VENDOR_PROJECT_KEYS
e.g. 'AA_VECIMA'] + VIT[VENDOR_ISSUE_TYPES
Epic, Story, Task, ...] + SIT[STEAM_ISSUE_TYPES
Story, Epic, Program, ...] + end + + subgraph Determination Logic + FN["getIssueTypes(projectKey, vendorKeys)"] + end + + subgraph Creation Modals + JM[JiraPage Modal] + IQ[Ivanti Queue Modal] + end + + subgraph Backend + EP["POST /api/jira-tickets/create-in-jira"] + JA[Jira REST API] + end + + VPK --> FN + VIT --> FN + SIT --> FN + FN --> JM + FN --> IQ + JM -->|"{ issue_type: string }"| EP + IQ -->|"{ issue_type: string }"| EP + EP -->|"issuetype: { name: value }"| JA +``` + +**Key design decisions:** + +1. **Pure function for determination** — The logic that decides which issue type list to show is a pure function (`getIssueTypes`) that takes the current project key string and returns the appropriate array. This makes it testable independently of React rendering. + +2. **Constants at module level** — All three arrays (`VENDOR_PROJECT_KEYS`, `VENDOR_ISSUE_TYPES`, `STEAM_ISSUE_TYPES`) are defined as module-level constants in a shared location importable by both JiraPage and IvantiTodoQueuePage. + +3. **Backend is type-agnostic** — The backend does not maintain a list of valid issue types. It passes whatever `issue_type` string it receives to the Jira API. If Jira rejects it, the error propagates back to the user. + +4. **Reset on context switch only** — The selected issue type resets to empty only when the project context changes (STEAM ↔ vendor). Edits that stay within the same context preserve the selection. + +## Components and Interfaces + +### Frontend: Issue Type Constants + +A new constants file or a constants block at the top of JiraPage.js (adjacent to existing STYLES): + +```javascript +// Vendor project keys — add new vendor keys here to enable vendor issue types +const VENDOR_PROJECT_KEYS = ['AA_VECIMA']; + +// Issue types for vendor projects (e.g., EoN-VECIMA) +const VENDOR_ISSUE_TYPES = [ + 'Epic', + 'Story', + 'Task', + 'Defect', + 'Production Defect/Incident Fix', + 'New Feature', + 'Spike', + 'Release Candidate', + 'Documentation', +]; + +// Issue types for STEAM projects (default) +const STEAM_ISSUE_TYPES = [ + 'Story', + 'Epic', + 'Program', + 'Project', + 'Reservation', + 'Automation Maintenance', +]; +``` + +### Frontend: Determination Function + +```javascript +/** + * Determines whether a project key belongs to a vendor project. + * @param {string} projectKey - The raw project key input value + * @param {string[]} vendorKeys - Array of known vendor project keys (uppercase) + * @returns {boolean} + */ +function isVendorProject(projectKey, vendorKeys) { + if (!projectKey || typeof projectKey !== 'string') return false; + const normalized = projectKey.trim().toUpperCase(); + if (normalized.length === 0) return false; + return vendorKeys.includes(normalized); +} + +/** + * Returns the appropriate issue type list for a given project key. + * @param {string} projectKey - The raw project key input value + * @param {string[]} vendorKeys - Array of known vendor project keys + * @param {string[]} vendorTypes - Vendor issue type options + * @param {string[]} steamTypes - STEAM issue type options + * @returns {string[]} + */ +function getIssueTypesForProject(projectKey, vendorKeys, vendorTypes, steamTypes) { + return isVendorProject(projectKey, vendorKeys) ? vendorTypes : steamTypes; +} +``` + +### Frontend: JiraPage Modal Integration + +The existing `createJiraForm.project_key` onChange handler is extended to: + +1. Call `isVendorProject()` on the new value and the old value. +2. If the context changed (vendor → STEAM or STEAM → vendor), reset `issue_type` to `''`. +3. If the context did not change, preserve the current `issue_type` selection. + +The ` + + {currentIssueTypes.map(type => ( + + ))} + +``` + +### Frontend: Ivanti Queue Modal Integration + +The Ivanti queue single-item modal (`showSingleJiraModal`) gains a Project Key input and an Issue Type dropdown that use the same constants and determination logic. The `singleJiraForm` state is extended with `project_key` and `issue_type` fields. + +If the modal is opened with a pre-populated `project_key` that matches a vendor key, the issue type dropdown shows vendor options on initial render. + +### Backend: No Changes Required + +The existing `POST /api/jira-tickets/create-in-jira` endpoint already handles `issue_type` as a passthrough: + +```javascript +const issueType = issue_type || jiraApi.JIRA_ISSUE_TYPE; +// ... +fields.issuetype = { name: issueType }; +``` + +This logic already: +- Falls back to `JIRA_ISSUE_TYPE` env var when `issue_type` is empty/null/undefined +- Passes any non-empty string directly to the Jira API +- Does not validate against a fixed list + +No backend code changes are needed. The backend is already compliant with Requirements 7.1–7.4. + +## Data Models + +No database schema changes are required. The `issue_type` value is not stored in the local `jira_tickets` table — it is only used at creation time to set the Jira issue's type via the API. + +**Frontend state model (extended):** + +```javascript +// createJiraForm state shape (JiraPage) +{ + cve_id: string, + vendor: string, + summary: string, + description: string, + project_key: string, // existing — used for vendor determination + issue_type: string, // existing — now dynamically constrained + source_context: string, +} + +// singleJiraForm state shape (IvantiTodoQueuePage) — extended +{ + cve_id: string, + vendor: string, + summary: string, + description: string, + source_context: 'ivanti_queue', + project_key: string, // NEW — enables vendor key entry + issue_type: string, // NEW — context-aware dropdown +} +``` + +## Correctness Properties + +*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.* + +### Property 1: Issue type list determination + +*For any* string input as a project key, after trimming whitespace and converting to uppercase, if the result exactly matches an entry in the VENDOR_PROJECT_KEYS array then `getIssueTypesForProject` SHALL return the VENDOR_ISSUE_TYPES array; otherwise it SHALL return the STEAM_ISSUE_TYPES array. + +**Validates: Requirements 1.2, 1.3, 1.4, 2.1, 2.3, 3.4, 3.5** + +### Property 2: Context switch resets issue type selection + +*For any* pair of project key values where one resolves to vendor context and the other resolves to STEAM context (or vice versa), changing the project key from one to the other SHALL result in the issue_type being reset to an empty string. + +**Validates: Requirements 1.5, 4.2, 6.3** + +### Property 3: Same context preserves issue type selection + +*For any* pair of project key values that both resolve to the same context (both vendor or both STEAM), and any previously selected issue_type value, changing the project key from one to the other SHALL preserve the issue_type value unchanged. + +**Validates: Requirements 4.1** + +### Property 4: Backend issue type passthrough + +*For any* non-empty string of 1 to 100 characters provided as `issue_type` in a create-ticket request, the backend SHALL pass that exact string to the Jira API as the `issuetype.name` field without modification or validation against a fixed list. + +**Validates: Requirements 7.1, 7.3** + +### Property 5: Backend issue type fallback to environment variable + +*For any* create-ticket request where `issue_type` is undefined, null, or an empty string, the backend SHALL use the value of the `JIRA_ISSUE_TYPE` environment variable as the `issuetype.name` field sent to the Jira API. + +**Validates: Requirements 7.2** + +## Error Handling + +| Scenario | Behavior | User Experience | +|---|---|---| +| User enters a project key not in vendor list | Dropdown shows STEAM types | Seamless — no error state | +| User enters a vendor key with mixed casing | Normalized to uppercase, matched correctly | Seamless — input auto-uppercases | +| User switches context with a type selected | Selection resets to empty (default) | Dropdown shows "Story (default)" placeholder | +| Jira API rejects the issue type for the target project | Backend returns HTTP 502 with Jira error details | Modal displays error message, form preserved for retry | +| VENDOR_PROJECT_KEYS is empty array | All keys resolve to STEAM types | Graceful degradation — behaves as before this feature | +| Project key is only whitespace | Trimmed to empty → STEAM types shown | Same as empty input | + +No new HTTP error codes or validation errors are introduced by this feature. The backend's existing passthrough behavior means invalid issue types are caught by the Jira API, not by our validation layer. + +## Testing Strategy + +### Unit Tests (Example-Based) + +- `isVendorProject('AA_VECIMA', ['AA_VECIMA'])` returns `true` +- `isVendorProject('STEAM-123', ['AA_VECIMA'])` returns `false` +- `isVendorProject('', ['AA_VECIMA'])` returns `false` +- `isVendorProject(null, ['AA_VECIMA'])` returns `false` +- Dropdown renders vendor issue types when project key is `AA_VECIMA` +- Dropdown renders STEAM issue types when project key is empty +- "Story (default)" placeholder option has empty string value in both contexts +- Ivanti queue modal shows project_key and issue_type fields +- Ivanti queue modal with pre-populated vendor key shows vendor types on mount +- Backend returns 502 when Jira API rejects issue type (mocked) + +### Property-Based Tests + +Property-based testing is appropriate for this feature because the determination logic is a pure function operating on arbitrary string inputs where universal properties must hold across the entire input space (any possible project key string). + +**Library:** fast-check (already available in the project's test infrastructure via Jest) + +**Configuration:** +- Minimum 100 iterations per property test +- Each test tagged with: `Feature: vendor-issue-type-dropdown, Property {N}: {title}` + +| Property | What it tests | Approach | +|---|---|---| +| 1: Issue type list determination | Pure function correctness | Generate random strings; verify output matches expected array based on membership in vendor keys | +| 2: Context switch resets selection | State transition logic | Generate vendor/non-vendor key pairs; simulate transition; verify reset | +| 3: Same context preserves selection | State preservation logic | Generate same-context key pairs + arbitrary selection; simulate transition; verify preservation | +| 4: Backend issue type passthrough | Backend field forwarding | Generate random 1-100 char strings; mock Jira API; verify exact string passed through | +| 5: Backend issue type fallback | Default value logic | Generate undefined/null/empty variants; mock Jira API; verify env var used | + +### Integration Tests + +- End-to-end: enter `AA_VECIMA` as project key, select "Defect", submit → verify Jira API receives `issuetype: { name: "Defect" }` +- End-to-end: enter `AA_VECIMA` then clear it → verify dropdown switches back to STEAM types and selection resets +- Ivanti queue: open modal, enter vendor key, select vendor type, submit → verify correct issue type in payload diff --git a/.kiro/specs/vendor-issue-type-dropdown/requirements.md b/.kiro/specs/vendor-issue-type-dropdown/requirements.md new file mode 100644 index 0000000..8b05970 --- /dev/null +++ b/.kiro/specs/vendor-issue-type-dropdown/requirements.md @@ -0,0 +1,94 @@ +# Requirements Document + +## Introduction + +The STEAM Security Dashboard's Jira ticket creation modal includes an "Issue Type" dropdown that currently only lists STEAM project issue types (Story, Epic, Program, Project, Reservation, Automation Maintenance). When users create tickets in vendor Jira projects (e.g., EoN-VECIMA / AA_VECIMA), those projects have a different set of issue types (Epic, Story, Task, Defect, Production Defect/Incident Fix, New Feature, Spike, Release Candidate, Documentation). This feature adds context-aware issue type selection so that the dropdown options change based on the target project key entered in the creation modal. + +## Glossary + +- **Creation_Modal**: The frontend modal dialog in JiraPage.js used to create a new Jira ticket. +- **Issue_Type_Dropdown**: The select element within the Creation_Modal that allows users to choose the Jira issue type for the ticket being created. +- **Project_Key_Input**: The text input field in the Creation_Modal where users can override the default JIRA_PROJECT_KEY from the .env file. +- **STEAM_Project**: The default internal Jira project identified by the JIRA_PROJECT_KEY environment variable, which uses issue types: Story, Epic, Program, Project, Reservation, Automation Maintenance. +- **Vendor_Project**: A Jira project belonging to an external vendor (e.g., AA_VECIMA), which uses issue types: Epic, Story, Task, Defect, Production Defect/Incident Fix, New Feature, Spike, Release Candidate, Documentation. +- **Vendor_Project_Keys**: A configurable list of project keys that are recognized as vendor projects, stored as a frontend constant or configuration. + +## Requirements + +### Requirement 1: Display Vendor Issue Types for Vendor Project Keys + +**User Story:** As a security analyst, I want the Issue Type dropdown to show vendor-specific issue types when I enter a vendor project key, so that I can select the correct issue type for the target Jira project. + +#### Acceptance Criteria + +1. THE Dashboard SHALL define a Vendor_Project_Keys list as a frontend constant array of project key strings that identify vendor Jira projects, configurable without a code deployment by updating the list in a single source location. +2. WHEN the Project_Key_Input value, after trimming and converting to uppercase, matches a value in the Vendor_Project_Keys list (case-insensitive comparison), THE Issue_Type_Dropdown SHALL display the vendor issue type options: Epic, Story, Task, Defect, Production Defect/Incident Fix, New Feature, Spike, Release Candidate, Documentation. +3. WHEN the Project_Key_Input value, after trimming and converting to uppercase, does not match any value in the Vendor_Project_Keys list, THE Issue_Type_Dropdown SHALL display the STEAM issue type options: Story, Epic, Program, Project, Reservation, Automation Maintenance. +4. WHEN the Project_Key_Input is empty (indicating the .env default will be used), THE Issue_Type_Dropdown SHALL display the STEAM issue type options. +5. WHEN the Project_Key_Input value changes such that the displayed issue type list switches between vendor and STEAM options, THE Issue_Type_Dropdown SHALL update to display the new option set and reset the selected issue_type value to empty within the same event cycle as the input change. +6. THE Issue_Type_Dropdown SHALL re-evaluate which option set to display on every change to the Project_Key_Input value (i.e., on each input change event, not on blur or after a delay). + +### Requirement 2: Case-Insensitive Project Key Matching + +**User Story:** As a security analyst, I want the project key matching to be case-insensitive, so that I do not need to remember the exact casing of vendor project keys. + +#### Acceptance Criteria + +1. WHEN the user enters a project key in any letter casing (e.g., "aa_vecima", "AA_VECIMA", "Aa_Vecima"), THE Issue_Type_Dropdown SHALL match the entered value against the Vendor_Project_Keys list using case-insensitive exact string comparison, where the entire entered value must equal a list entry after both are normalized to the same case. +2. THE Project_Key_Input SHALL convert the entered value to uppercase on each keystroke for display purposes, consistent with the existing toUpperCase() behavior in the onChange handler. +3. IF the Project_Key_Input contains leading or trailing whitespace, THEN THE Issue_Type_Dropdown SHALL trim the value before performing the case-insensitive comparison against the Vendor_Project_Keys list. + +### Requirement 3: Configurable Vendor Project Key List + +**User Story:** As a developer, I want the list of vendor project keys to be defined in a single location, so that adding new vendor projects in the future requires a change in only one place. + +#### Acceptance Criteria + +1. THE Creation_Modal SHALL define the Vendor_Project_Keys list as a named constant at the module level, containing all recognized vendor project keys as an array of strings (initially containing "AA_VECIMA"). +2. THE Creation_Modal SHALL define the vendor issue type options as a named constant array (initially containing "Epic", "Story", "Task", "Defect", "Production Defect/Incident Fix", "New Feature", "Spike", "Release Candidate", "Documentation") at the module level adjacent to the Vendor_Project_Keys constant. +3. THE Creation_Modal SHALL define the STEAM issue type options as a named constant array (initially containing "Story", "Epic", "Program", "Project", "Reservation", "Automation Maintenance") at the module level adjacent to the Vendor_Project_Keys constant. +4. WHEN the user selects a project key that is included in the Vendor_Project_Keys constant, THE Creation_Modal SHALL populate the issue type dropdown exclusively with the vendor issue type options. +5. WHEN the user selects a project key that is not included in the Vendor_Project_Keys constant or leaves the project key empty, THE Creation_Modal SHALL populate the issue type dropdown exclusively with the STEAM issue type options. +6. WHEN a new vendor project key needs to be supported, THE developer SHALL add the key string to the Vendor_Project_Keys constant array without modifying any rendering logic or conditional branching elsewhere in the module. + +### Requirement 4: Preserve Selected Issue Type When Project Context Does Not Change + +**User Story:** As a security analyst, I want my selected issue type to be preserved when I make minor edits to the project key that do not change the project context (STEAM vs vendor), so that I do not lose my selection unnecessarily. + +#### Acceptance Criteria + +1. WHILE the user edits the Project_Key_Input and the project context remains the same (both old and new values resolve to STEAM or both resolve to vendor), THE Issue_Type_Dropdown SHALL retain the currently selected issue_type value. +2. IF the user selects an issue type and then edits the project key such that the project context changes (STEAM to vendor or vendor to STEAM), THEN THE Issue_Type_Dropdown SHALL reset the selected issue_type value to empty because the previously selected type may not exist in the new context. + +### Requirement 5: Default Issue Type Behavior for Vendor Projects + +**User Story:** As a security analyst, I want a sensible default indicated for vendor projects, so that I know what issue type will be used if I do not explicitly select one. + +#### Acceptance Criteria + +1. WHEN the Issue_Type_Dropdown displays vendor issue types and no option has been explicitly selected by the user, THE Issue_Type_Dropdown SHALL display "Story (default)" as the visible placeholder option with an empty string as its form value, matching the existing STEAM project Issue_Type_Dropdown behavior. +2. WHEN the user submits the Creation_Modal with the Issue_Type_Dropdown value equal to an empty string while in vendor project context, THE Creation_Modal SHALL send the issue_type field as an empty string in the request payload, causing the Ticket_Creation_Service to apply the default from the JIRA_ISSUE_TYPE environment variable. +3. WHEN the user explicitly selects a non-default option from the vendor Issue_Type_Dropdown, THE Creation_Modal SHALL send the selected issue type name as the issue_type field value in the request payload, and the Ticket_Creation_Service SHALL use that value instead of the JIRA_ISSUE_TYPE environment variable default. + +### Requirement 6: Vendor Issue Types in Ivanti Queue Context + +**User Story:** As a security analyst, I want the vendor issue type dropdown to work correctly when I create a Jira ticket from the Ivanti queue, so that I can select the appropriate issue type for vendor projects without leaving my queue workflow. + +#### Acceptance Criteria + +1. WHEN the user opens the Creation_Modal from the Ivanti queue "Create Jira Ticket" action and enters a project key in the Project_Key_Input that matches a value in the Vendor_Project_Keys list (case-insensitive), THE Issue_Type_Dropdown SHALL display the vendor issue type options: Epic, Story, Task, Defect, Production Defect/Incident Fix, New Feature, Spike, Release Candidate, Documentation. +2. WHEN the user opens the Creation_Modal from the Ivanti queue and the Project_Key_Input is empty or contains a project key not in the Vendor_Project_Keys list, THE Issue_Type_Dropdown SHALL display the STEAM issue type options: Story, Epic, Program, Project, Reservation, Automation Maintenance. +3. WHEN the user changes the Project_Key_Input value in the Ivanti queue Creation_Modal such that the project context switches (STEAM to vendor or vendor to STEAM), THE Issue_Type_Dropdown SHALL reset the selected issue_type value to empty and update the displayed options to match the new project context. +4. WHEN the Creation_Modal is opened from the Ivanti queue with a pre-populated project_key that matches a value in the Vendor_Project_Keys list, THE Issue_Type_Dropdown SHALL display the vendor issue type options on initial render without requiring additional user input. +5. THE Ivanti queue Creation_Modal SHALL reference the same Vendor_Project_Keys constant and the same issue type option lists as the Jira page Creation_Modal. + +### Requirement 7: Backend Accepts Vendor Issue Types + +**User Story:** As a security analyst, I want the backend to accept vendor issue types in the create-ticket payload, so that tickets are created with the correct type in the vendor Jira project. + +#### Acceptance Criteria + +1. WHEN a create-ticket request includes an issue_type value that is a non-empty string of 1 to 100 characters, THE Ticket_Creation_Service SHALL pass that value to the Jira API as the issuetype name field without modification. +2. WHEN a create-ticket request does not include an issue_type field, or includes it as null or empty string, THE Ticket_Creation_Service SHALL use the value of the JIRA_ISSUE_TYPE environment variable as the issuetype name sent to the Jira API. +3. THE Ticket_Creation_Service SHALL NOT validate the issue_type value against a fixed list of allowed types, because the valid issue types depend on the target Jira project configuration. +4. IF the Jira API rejects the request due to an invalid issue type for the target project, THEN THE Ticket_Creation_Service SHALL return the Jira error response to the caller with an HTTP 502 status and an error message indicating the issue type was not accepted by the target project. diff --git a/.kiro/specs/vendor-issue-type-dropdown/tasks.md b/.kiro/specs/vendor-issue-type-dropdown/tasks.md new file mode 100644 index 0000000..7b32c48 --- /dev/null +++ b/.kiro/specs/vendor-issue-type-dropdown/tasks.md @@ -0,0 +1,102 @@ +# Implementation Plan: Vendor Issue Type Dropdown + +## Overview + +This plan implements context-aware issue type selection in the Jira ticket creation modals. The core logic is a pair of pure functions (`isVendorProject`, `getIssueTypesForProject`) backed by three module-level constants. Both the JiraPage modal and the Ivanti queue modal consume the same constants and functions. The backend requires no changes. + +## Tasks + +- [x] 1. Add vendor issue type constants and determination functions + - [x] 1.1 Create constants and helper functions in JiraPage.js + - Add `VENDOR_PROJECT_KEYS`, `VENDOR_ISSUE_TYPES`, and `STEAM_ISSUE_TYPES` as module-level constants at the top of `frontend/src/components/pages/JiraPage.js` (adjacent to existing STYLES) + - Add `isVendorProject(projectKey, vendorKeys)` pure function + - Add `getIssueTypesForProject(projectKey, vendorKeys, vendorTypes, steamTypes)` pure function + - Export the constants and functions so IvantiTodoQueuePage can import them + - _Requirements: 1.1, 3.1, 3.2, 3.3, 3.6_ + + - [x]* 1.2 Write property test for issue type list determination + - **Property 1: Issue type list determination** + - Generate arbitrary strings as project keys; verify that after trim+uppercase, membership in VENDOR_PROJECT_KEYS returns VENDOR_ISSUE_TYPES, otherwise STEAM_ISSUE_TYPES + - **Validates: Requirements 1.2, 1.3, 1.4, 2.1, 2.3, 3.4, 3.5** + + - [x]* 1.3 Write unit tests for isVendorProject and getIssueTypesForProject + - Test exact match: `isVendorProject('AA_VECIMA', ['AA_VECIMA'])` → true + - Test non-match: `isVendorProject('STEAM-123', ['AA_VECIMA'])` → false + - Test empty string: `isVendorProject('', ['AA_VECIMA'])` → false + - Test null/undefined: `isVendorProject(null, ['AA_VECIMA'])` → false + - Test case-insensitive: `isVendorProject('aa_vecima', ['AA_VECIMA'])` → true + - Test whitespace trimming: `isVendorProject(' AA_VECIMA ', ['AA_VECIMA'])` → true + - _Requirements: 1.2, 1.3, 1.4, 2.1, 2.3_ + +- [x] 2. Update JiraPage modal with dynamic issue type dropdown + - [x] 2.1 Update project_key onChange handler to detect context switches + - Modify the `project_key` input's onChange to call `isVendorProject()` on both old and new values + - Reset `issue_type` to `''` only when context switches (vendor ↔ STEAM) + - Preserve `issue_type` when context does not change + - _Requirements: 1.5, 1.6, 4.1, 4.2_ + + - [x] 2.2 Render Issue Type dropdown dynamically from getIssueTypesForProject + - Replace the hardcoded `` placeholder + - Map the returned array to `