From b0cb67b97580e0e54dda65b9047c0883807a559f Mon Sep 17 00:00:00 2001 From: Jordan Ramos Date: Mon, 15 Jun 2026 13:55:22 -0600 Subject: [PATCH] Update full reference manual to v2.2.0 Brings the reference manual in line with the current codebase: - Add CCP Metrics (VCL multi-vertical) feature section and full API reference - Add CARD Asset Ownership feature section with tooltip, direct actions, queue actions - Add Granite Loader Sheet feature section with CARD enrichment details - Add Atlas Action Plans feature section with cache and badge rendering - Add Finding Archive Tracking feature section with lifecycle states and anomaly detection - Add Archer Template Library feature section with hierarchy and clone - Add In-App Notifications feature section - Add Feedback (GitLab integration) feature section with webhook lifecycle - Add FP Workflow Submission feature section with lifecycle tracking - Update Ivanti Queue to document all 6 workflow types (FP, Archer, CARD, GRANITE, DECOM, Remediate) - Update Reporting section with Group by Host, CARD tooltip, multi-BU scope - Update Jira section with flexible creation, consolidation modal, raw status - Update Configuration with CARD, Atlas, GitLab, IVANTI_BU_FILTER, IVANTI_MANAGED_BUS vars - Update Architecture tree with all routes, helpers, and components - Update Database Schema with 15+ new tables - Update Migrations section with all 46 migration files - Update API Reference with Archer Templates, CARD, Atlas, VCL, Notifications, Feedback, Webhooks - Add per-user settings (bu_teams, ivanti_identity) to auth section --- docs/guides/full-reference-manual.md | 666 ++++++++++++++++++++++++--- 1 file changed, 604 insertions(+), 62 deletions(-) diff --git a/docs/guides/full-reference-manual.md b/docs/guides/full-reference-manual.md index 64eccc4..6928a16 100644 --- a/docs/guides/full-reference-manual.md +++ b/docs/guides/full-reference-manual.md @@ -1,6 +1,6 @@ # STEAM Security Dashboard -A self-hosted vulnerability management dashboard for the NTS-AEO-STEAM and NTS-AEO-ACCESS-ENG business units. Centralises CVE tracking, Ivanti host finding triage, AEO compliance posture, FP and Archer exception workflows, and internal documentation in a single interface. +A self-hosted vulnerability management dashboard for the NTS-AEO-STEAM, NTS-AEO-ACCESS-ENG, NTS-AEO-ACCESS-OPS, and NTS-AEO-INTELDEV business units. Centralises CVE tracking, Ivanti host finding triage, AEO compliance posture monitoring, CCP Metrics cross-org compliance reporting, FP/Archer/CARD/GRANITE/DECOM exception workflows, CARD asset ownership management, Granite Loader Sheet generation, Jira ticket management, and internal documentation in a single interface. --- @@ -18,11 +18,20 @@ A self-hosted vulnerability management dashboard for the NTS-AEO-STEAM and NTS-A - [Home — CVE Management](#home--cve-management) - [Reporting — Host Findings](#reporting--host-findings) - [Ivanti Queue](#ivanti-queue) + - [FP Workflow Submission](#fp-workflow-submission) - [Compliance — AEO Posture](#compliance--aeo-posture) + - [CCP Metrics — Multi-Vertical Compliance](#ccp-metrics--multi-vertical-compliance) + - [CARD Asset Ownership](#card-asset-ownership) + - [Granite Loader Sheet](#granite-loader-sheet) + - [Atlas Action Plans](#atlas-action-plans) + - [Finding Archive Tracking](#finding-archive-tracking) - [Knowledge Base](#knowledge-base) - [Exports](#exports) - [Jira Tickets](#jira-tickets) - [Archer Risk Acceptance Tickets](#archer-risk-acceptance-tickets) + - [Archer Template Library](#archer-template-library) + - [In-App Notifications](#in-app-notifications) + - [Feedback — GitLab Integration](#feedback--gitlab-integration) - [Admin Panel](#admin-panel) - [Scripts](#scripts) - [API Reference](#api-reference) @@ -43,13 +52,24 @@ The application provides: - A searchable, filterable CVE list with per-vendor tracking and document storage - NVD API integration to auto-populate CVE metadata -- **Ivanti/RiskSense integration** — sync open host findings with live FP workflow tracking -- **Reporting page** with donut charts, advanced per-column filtering, inline editing, Ivanti Queue, and CSV/XLSX export -- **Ivanti Queue** — personal staging list for batch-processing FP, Archer, and CARD workflows +- **Ivanti/RiskSense integration** — sync open host findings with live FP workflow tracking, per-BU trend lines, and archive detection +- **Reporting page** with donut charts, Group by Host toggle, CARD ownership tooltips, advanced per-column filtering, inline editing, Ivanti Queue, and CSV/XLSX export +- **Ivanti Queue** — personal staging list for batch-processing FP, Archer, CARD, GRANITE, DECOM, and Remediate workflows +- **FP Workflow Submission** — submit False Positive workflows directly to Ivanti API with attachments and lifecycle tracking - **AEO Compliance page** — weekly xlsx upload, diff preview, per-team metric health cards, device-level violation tracking with notes history +- **CCP Metrics page** — multi-vertical VCL upload, cross-org compliance reporting with forecast burndown, metric drill-down, and data management +- **CARD Asset Ownership** — owner lookup, confirm/decline/redirect actions, tooltip integration, Granite enrichment +- **Granite Loader Sheet** — generate xlsx loader sheets with CARD enrichment, searchable picklists, per-row editing +- **Atlas action plan tracking** with per-host vulnerability mapping and badge rendering +- **Jira integration** — flexible ticket creation, multi-item consolidation, vendor-specific issue types, JQL sync +- **Finding archive tracking** — automatic detection of disappeared/returned findings with anomaly logging - Archer risk acceptance ticket tracking (EXC numbers) linked to CVE/vendor pairs +- **Archer Template Library** — store and clone static content for Archer Risk Acceptance forms +- **In-app notifications** — native notification system for sync events and workflow completions +- **Feedback integration** — submit bug reports and feature requests directly to GitLab as issues - A knowledge base for internal documentation and policies - Group-based access control (Admin, Standard_User, Leadership, Read_Only) with a full audit trail +- Per-user BU team assignments and Ivanti identity for multi-tenant scoping --- @@ -180,12 +200,22 @@ IVANTI_CLIENT_ID=1550 # Optional: filter workflows to a specific person's submissions IVANTI_FIRST_NAME= IVANTI_LAST_NAME= -# Set to 'true' if your network has SSL inspection / self-signed certs -IVANTI_SKIP_TLS=false +# Comma-separated list of BU values to sync from Ivanti. +# Default if unset: NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM +IVANTI_BU_FILTER=NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM # Comma-separated list of BUs considered "managed" for archive drift classification. # Findings leaving these BUs are classified as bu_reassignment. # Default if unset: NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM IVANTI_MANAGED_BUS=NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM +# Set to 'true' if your network has SSL inspection / self-signed certs +IVANTI_SKIP_TLS=false + +# Atlas InfoSec API (required for Atlas action plan tracking) +ATLAS_API_URL=https://atlas-infosec.caas.charterlab.com +ATLAS_API_USER=your-atlas-user +ATLAS_API_PASS=your-atlas-password +# Set to true if behind Charter's SSL inspection proxy +ATLAS_SKIP_TLS=false # Jira Data Center REST API (required for Jira Tickets page) # VPN or Charter Network connection required for all Jira instances. @@ -200,6 +230,25 @@ JIRA_API_TOKEN=your-api-token JIRA_PROJECT_KEY=VULN JIRA_ISSUE_TYPE=Task JIRA_SKIP_TLS=false + +# CARD Asset Ownership API (required for CARD integration) +# OAuth Bearer token auth — service account must be onboarded with the CARD team. +# Tokens are acquired automatically via Basic Auth and cached for 1 hour. +CARD_API_URL=https://card.charter.com +CARD_API_USER=your-card-user +CARD_API_PASS=your-card-password +# Set to true if behind Charter's SSL inspection proxy +CARD_SKIP_TLS=false + +# GitLab Feedback Integration (bug reports and feature requests from the dashboard) +# PAT needs 'api' scope. Project ID is the numeric ID from GitLab project settings. +GITLAB_URL=http://steam-gitlab.charterlab.com +GITLAB_PROJECT_ID=13 +GITLAB_PAT=glpat-xxxxxxxxxxxxx +# Webhook secret — shared secret for validating incoming webhook requests. +# Set this same value in GitLab project > Settings > Webhooks > Secret Token. +# Generate with: openssl rand -hex 20 +GITLAB_WEBHOOK_SECRET=changeme_generate_a_random_secret ``` **`SESSION_SECRET` is required.** The server will exit on startup if it is not set. Generate one with `openssl rand -base64 32`. @@ -312,6 +361,10 @@ All routes require authentication. Four user groups are supported: Sessions expire after 24 hours. Session tokens are stored in `httpOnly` cookies. Login is rate-limited to 20 attempts per 15-minute window. +**Per-user settings:** +- `bu_teams` — comma-separated BU team assignments for multi-tenant scoping. Determines default BU filter on the Reporting page. +- `ivanti_first_name` / `ivanti_last_name` — per-user Ivanti identity for filtering FP workflow views to their own submissions. + **Migration from legacy roles:** The `add_user_groups.js` migration automatically maps existing users: `admin` → `Admin`, `editor` → `Standard_User`, `viewer` → `Read_Only`. Unrecognized or NULL roles default to `Read_Only`. --- @@ -358,7 +411,7 @@ The home page is the primary CVE research and tracking tool. ### Reporting — Host Findings -The Reporting page is the core operational view for remediation tracking. It integrates with Ivanti/RiskSense to show all host findings for the configured business units. +The Reporting page is the core operational view for remediation tracking. It integrates with Ivanti/RiskSense to show all host findings for the configured business units, with CARD ownership integration and multi-BU scoping. #### Syncing Data @@ -367,11 +420,38 @@ Click **Sync** (top right) to pull the latest findings from Ivanti. Sync require 2. Fetches the closed finding count separately 3. Sweeps closed findings to capture FP workflow states (including Approved FPs now closed) 4. Stores findings as individual rows in the PostgreSQL `ivanti_findings` table +5. Detects archived findings (present in previous sync, absent in current) and logs transitions +6. Records per-BU counts in `ivanti_counts_history_by_bu` for trend analysis +7. Logs sync anomalies when significant count deltas are detected Findings are also auto-synced on a 24-hour schedule. The last sync timestamp is shown at the top of the page. > `IVANTI_API_KEY` must be set in `backend/.env` for sync to work. +#### Group by Host + +The **Group by Host** toggle in the toolbar collapses duplicate assets (same hostname + IP) with multiple finding IDs into expandable host rows. Hosts with only one finding remain as flat rows. Toggle between grouped and flat views. This reduces visual clutter when a single host has dozens of findings. + +#### CARD Ownership Tooltip + +Hover over any IP address in the findings table to see CARD asset ownership data in an interactive tooltip: +- Confirmed, unconfirmed, and candidate team assignments with confidence scores +- Click "Actions" to open the CARD Action Modal for direct confirm/decline/redirect operations +- Results cached per session — re-hover displays instantly without API calls +- Quick mode uses CTEC suffix only with 15s timeout to avoid multi-minute waits +- Timeouts (504) are not cached — re-hover will retry the lookup +- When Ivanti Host ID is available, uses the faster asset-search path + +#### Multi-BU Scope + +The multi-select BU picker at the top of the page replaces the previous binary scope toggle. Select one or more BUs to filter findings: +- NTS-AEO-ACCESS-ENG +- NTS-AEO-STEAM +- NTS-AEO-ACCESS-OPS +- NTS-AEO-INTELDEV + +Per-user BU team assignments (`bu_teams` on the user record) determine the default scope. + #### Metric Charts | Chart | What it shows | @@ -417,36 +497,72 @@ Each row represents a single Ivanti host finding. ### Ivanti Queue -A personal staging list for batch-processing FP, Archer, and CARD workflows without context-switching into Ivanti mid-review. Requires Admin or Standard_User group. +A personal staging list for batch-processing FP, Archer, CARD, GRANITE, DECOM, and Remediate workflows without context-switching into Ivanti mid-review. Requires Admin or Standard_User group. **Adding items:** Check the checkbox at the far left of any finding row. A popover appears: - For **FP** and **Archer** items: enter the Vendor / Platform (e.g., "Juniper MX", "Cisco IOS-XE") -- For **CARD** items: no vendor entry required — the IP address is captured automatically -- Select the workflow type: **FP**, **Archer**, or **CARD** +- For **CARD** and **GRANITE** items: no vendor entry required — the IP address is captured automatically +- For **DECOM** items: the finding is flagged for decommission workflow +- For **Remediate** items: the finding is flagged for remediation tracking +- Select the workflow type: **FP**, **Archer**, **CARD**, **GRANITE**, **DECOM**, or **Remediate** - Click **Add to Queue** — the row checkbox turns solid blue **Queue panel:** Click the **Queue** button (top right of Reporting page) to open the slide-out panel: -- **CARD** items appear at the top in their own section with the IP address displayed -- **FP and Archer** items are grouped alphabetically by vendor below -- Badges show workflow type: amber = FP, sky = Archer, green = CARD +- Collapsible sections per workflow type +- **CARD** items appear with the IP address displayed +- **GRANITE** items show IP and hostname for Granite Loader Sheet generation +- **FP and Archer** items are grouped alphabetically by vendor +- **DECOM** items auto-note and auto-hide the finding on completion +- Badges show workflow type colour-coded by category **Working the queue:** - Check the green checkbox on an item to mark it complete (strikethrough at reduced opacity) - Delete individual items with the trash icon, or select multiple and use **Delete (N)** -- **Clear Completed** removes all marked-complete items at once +- **Clear Completed** removes all marked-complete items at once (FK-safe deletion) - **Create FP Workflow** — select pending FP items and click to open the FP Workflow modal, which submits a False Positive workflow batch directly to the Ivanti API with form fields, file attachments, and scope override. Attachments can be local file uploads or documents selected from the CVE document library — library documents are read from disk and sent to Ivanti identically to local uploads. Successful submission marks the queue items as complete and records the submission locally. +- **Create Jira Ticket** — select multiple items and use the consolidation modal to create a single Jira ticket covering all selected findings. Ticket links are displayed on completed items. +- **Loader Sheet** — select GRANITE items to generate a Granite Loader Sheet with CARD enrichment +- **Archer Template** — select Archer items to open the template selector for pre-filling Archer Risk Acceptance forms -**Redirecting completed items:** +**Redirecting items:** +- Pending items can be redirected to a different workflow type without duplication - Completed items show a redirect button (↱) next to the delete icon -- Click redirect to open a modal where you select the target workflow type (FP, Archer, or CARD) and vendor (required for FP/Archer) -- Redirecting creates a new pending queue item with the same finding data under the new workflow type — the original completed item is preserved +- Click redirect to open a modal where you select the target workflow type and vendor (required for FP/Archer) +- Redirecting creates a new pending queue item with the same finding data under the new workflow type — the original item is preserved - This is useful when a CARD inventory fix is done but the finding still needs an FP or Archer workflow, or when an item was assigned to the wrong workflow initially -- Not every completed item needs a redirect — it's an optional action for items that require further processing Queue items are stored in the database, are **personal to your login**, and persist across sessions and page refreshes. --- +### FP Workflow Submission + +Submit False Positive workflows directly to the Ivanti API with attachments and full lifecycle tracking. Accessible from the Ivanti Queue when FP items are selected. + +**Submission workflow:** +1. Select pending FP items in the queue +2. Fill in workflow name, reason, description, expiration date, and scope override +3. Attach supporting documents (local uploads or library documents, 10MB limit per file) +4. Submit — the workflow batch is created in Ivanti via API and recorded locally + +**Lifecycle tracking:** Each submission tracks its status through the lifecycle: +- `submitted` — initial state after successful API submission +- `approved` — FP workflow approved by Ivanti team +- `rejected` — FP workflow rejected, requires rework +- `rework` — submission is being edited for resubmission +- `resubmitted` — edited submission resubmitted to Ivanti + +**Managing submissions:** +- View submission history with attachment results and lifecycle status +- Edit and resubmit rejected workflows (fields, findings, and attachments) +- Dismiss rejected submissions (sets `dismissed_at` timestamp) +- Re-queue findings from rejected submissions into the todo queue under a different workflow type +- Auto-clear approved submissions from the active list +- Collapsible submissions panel with per-user filtering +- Sync lifecycle status from Ivanti `currentState` on fetch + +--- + ### Compliance — AEO Posture The Compliance page tracks NTS-AEO team posture against the AEO compliance framework using weekly xlsx reports exported from the NTS_AEO reporting system. @@ -497,6 +613,170 @@ Only **STEAM** and **ACCESS-ENG** teams are tracked. The team selector at the to --- +### CCP Metrics — Multi-Vertical Compliance + +The CCP Metrics page provides executive-level visibility into VCL (Vulnerability Compliance Level) compliance posture across multiple verticals (organisational units). It supports multi-file upload, forecast burndown charts, sub-team drill-down, and data management. + +**Upload workflow:** +1. Click **Upload** and select 1–14 xlsx files (naming convention: `_YYYY_MM_DD.xlsx`) +2. Each file is parsed, the vertical is extracted from the filename, and a scoped diff is computed against existing data for that vertical +3. The preview shows per-file new/recurring/resolved counts and unrecognised files +4. Click **Commit** to persist all files in a single transaction with vertical-scoped resolution + +**Executive overview:** +- Aggregated stats across all verticals: total devices, compliant, non-compliant, compliance percentage vs target +- Donut chart: blocked (no resolution date) vs in-progress (has resolution date) +- Per-vertical breakdown cards with compliance percentage, burndown forecast, and last upload date +- Cross-vertical metric breakdown table sorted by non-compliant count +- LIVE and LAST REPORT badges showing data freshness + +**Metric drill-down:** +- Click a metric ID to see per-vertical breakdown with sub-team data +- Per-metric summary statistics and donut breakdown +- Per-metric monthly compliance trend with linear regression forecast (3 months forward when 3+ months of history) +- Device list per vertical/metric with optional team filtering +- Compliant/total counts on metric summary cards +- Non-Compliant stat clickable with metric breakdown buttons + +**Burndown forecasts:** +- Per-vertical burndown: deduplicates devices by hostname, projects monthly resolution based on `resolution_date` fields +- Aggregated burndown: cross-vertical forecast with projected clear date +- Per-metric forecast burndown chart + +**Sub-team drill-down:** +- Each metric shows sub-team rows (e.g., STEAM, ACCESS-OPS, ACCESS-ENG) beneath the rollup row +- Intermediate view between overview and device list +- Team filtering on the device list endpoint + +**Data management (Admin only):** +- Delete a specific vertical (all items, uploads, summary, snapshots) +- Rollback a specific upload (must be the most recent for that vertical) +- Reset all multi-vertical data (nuclear option) + +**VCL metric calculations:** See `docs/guides/vcl-metric-calculations.md` for formula reference. + +--- + +### CARD Asset Ownership + +CARD API integration for managing asset ownership — confirm, decline, and redirect ownership for network assets directly from the dashboard. + +**Owner lookup:** +- Hover over any IP address in the findings table to see CARD ownership data in a tooltip (confirmed/unconfirmed/candidate teams) +- Tooltip uses quick mode (CTEC suffix only, 15s timeout) for performance +- Results cached per session for instant re-display — timeouts (504) are not cached and will retry on re-hover +- When Ivanti Host ID is available, uses the CARD asset-search endpoint for faster resolution + +**Direct actions (no queue item required):** +- Click "Actions" in the CARD tooltip to open the CARD Action Modal +- Modal displays full owner context: confirmed, unconfirmed, declined, and candidate teams +- Confirm, decline, or redirect ownership via the CARD API +- Bare IPs are auto-resolved to CARD asset IDs (via host_id fast path or suffix guessing: CTEC, NATL, CHTR, COML, RESI, WIFI, VOIP) +- IP address validation before mutation operations + +**Queue-based actions:** +- Add findings to the queue with workflow type CARD +- Confirm, decline, or redirect from the queue panel +- Queue items are marked complete on successful CARD action +- update_token handling for safe concurrent operations + +**Team assets endpoint:** +- Paginated team asset lookup by disposition (confirmed, unconfirmed, candidate) +- Used by the Granite enrichment batch endpoint for full data + +--- + +### Granite Loader Sheet + +Generate Granite Loader Sheets with CARD enrichment for network device workflows. + +**Generation workflow:** +1. Add findings to the queue with workflow type GRANITE +2. Click **Loader Sheet** in the queue panel (or use the Loader Sheet button on the Reporting page) +3. The modal fetches CARD data for each IP/host_id to enrich with NCIM, Qualys, and Netops Granite fields +4. Review and edit per-row data with searchable picklists +5. Export as formatted XLSX + +**CARD enrichment fields:** +- `equip_inst_id` — NCIM equipment instance ID +- `hostname` — resolved hostname from CARD +- `site_name` — NCIM site name +- `mgmt_ip_asn` — management IP ASN +- `responsible_team` — NCIM responsible team +- `equipment_class` — equipment classification +- `equip_template` — equipment template +- `equip_status` — equipment status +- `serial_number` — device serial number + +**Features:** +- Searchable picklists for teams, statuses, operation types (defined in `graniteLoaderPicklists.js`) +- Column groups with configurable visibility (defined in `graniteLoaderConfig.js`) +- Per-row inline editing before export +- Batch enrichment: accepts up to 200 IPs or host_ids per request +- When no team is specified, searches both NTS-AEO-STEAM and NTS-AEO-ACCESS-ENG + +--- + +### Atlas Action Plans + +Atlas InfoSec action plan tracking with per-host vulnerability mapping. Provides visibility into which hosts have active remediation, risk acceptance, or compensating control plans. + +**AtlasBadge component:** +- Appears on finding rows in the Reporting table when a host has one or more action plans +- Badge colour indicates plan type: remediation, risk acceptance, or compensating control +- Reads from local cache (`atlas_action_plans_cache` table) for instant rendering without API round-trips + +**Slide-out panel:** +- Click the AtlasBadge to open the Atlas detail panel +- Shows all plans for the host with type, status, and metadata +- Qualys vulnerability mapping per host (resolved via `/hosts/vulnerabilities` endpoint) + +**Cache management:** +- Local cache stores plan existence, count, and full plan JSON per host_id +- Manual refresh triggers a re-fetch from the Atlas API and updates the cache +- `atlas_known` flag indicates whether the host has been checked (avoids re-querying hosts with no plans) + +**Plan operations (Admin/Standard_User):** +- Create action plans: remediation, risk_acceptance, or compensating_control +- Update existing plans (PATCH) +- Refresh cache per host + +--- + +### Finding Archive Tracking + +Automatic detection of findings that disappear between Ivanti syncs, with lifecycle state tracking and anomaly logging. + +**How it works:** +- On each sync, findings present in the previous sync but absent from the current sync are classified as archived +- If a previously archived finding reappears, it transitions to RETURNED +- Findings that remain absent are eventually classified as CLOSED or CLOSED_GONE + +**Lifecycle states:** +| State | Meaning | +|-------|---------| +| ARCHIVED | Finding disappeared from Ivanti (first detection) | +| RETURNED | Previously archived finding reappeared in a subsequent sync | +| CLOSED | Finding confirmed closed by Ivanti (workflow completed) | +| CLOSED_GONE | Finding disappeared and is confirmed gone (no workflow, long absence) | + +**Anomaly detection:** +- Each sync logs open/closed count deltas, newly archived count, and returned count +- Significance threshold triggers the AnomalyBanner component on the Reporting page +- Classification JSON tracks the breakdown of archive reasons + +**BU reassignment tracking:** +- Findings that change BU ownership between syncs are logged in `ivanti_finding_bu_history` +- `IVANTI_MANAGED_BUS` env var defines which BUs are "managed" — findings leaving these BUs are classified as `bu_reassignment` +- Return classification distinguishes between original finding restoration and new duplicates + +**UI components:** +- **AnomalyBanner** — alert bar on the Reporting page when significant sync anomalies are detected +- **ArchiveSummaryBar** — state distribution summary (ARCHIVED / RETURNED / CLOSED counts) +- Archive view with transition history per finding + +--- + ### Knowledge Base A document library for internal reference material — policies, runbooks, vendor advisories, and process guides. @@ -520,16 +800,19 @@ Bulk export tools for reports and data extracts. Available to Admin, Standard_Us ### Jira Tickets -A dedicated page for managing Jira Data Center tickets linked to CVE/vendor pairs. Accessible from the navigation drawer. Requires a configured Jira API connection (see [Configuration](#configuration)). +A dedicated page for managing Jira Data Center tickets linked to CVE/vendor pairs and Ivanti queue items. Accessible from the navigation drawer. Requires a configured Jira API connection (see [Configuration](#configuration)). **Ticket list** - View all tracked Jira tickets with status, CVE ID, vendor, summary, and Jira key - Filter by status or search by keyword - Click a Jira key to open the issue in Jira Data Center +- Raw Jira status display — shows the actual Jira status field (no Open/In Progress/Closed mapping) **Jira API operations (Admin/Standard_User)** - **Lookup** — search for any Jira issue by key and view its current status, assignee, and summary -- **Create in Jira** — create a new Jira issue directly from the dashboard with project key, issue type, summary, and description; the resulting ticket is automatically linked to a CVE/vendor pair in the local database +- **Create in Jira** — create a new Jira issue directly from the dashboard with project key, issue type, summary, and description; the resulting ticket is automatically linked to a CVE/vendor pair in the local database. CVE/Vendor fields are optional — tickets can be created with source context tracking only. +- **Multi-item creation** — from the Ivanti Queue consolidation modal, create a single Jira ticket covering multiple findings +- **Save to Dashboard** — save a Jira issue found via lookup to the local database - **Sync** — refresh a single ticket's status and summary from Jira, or bulk-sync all tracked tickets via JQL search - **Create / Edit / Delete** — manage local ticket records linking Jira keys to CVE/vendor pairs @@ -539,7 +822,7 @@ A dedicated page for managing Jira Data Center tickets linked to CVE/vendor pair **Rate limit monitoring (Admin)** — view current burst and daily rate limit usage against Charter's posted limits (60/minute burst, 1 440/day). -All Jira API calls are proxied through the backend. Credentials are never exposed to the browser. Rate limits are enforced client-side with inter-request delays (1s for GETs, 2s for writes). See `docs/jira-api-use-cases.md` for the full API compliance summary. +All Jira API calls are proxied through the backend. Credentials are never exposed to the browser. Rate limits are enforced client-side with inter-request delays (1s for GETs, 2s for writes). See `docs/api/jira-api-use-cases.md` for the full API compliance summary. --- @@ -556,6 +839,75 @@ Track Archer exception tickets (EXC numbers) linked to specific CVE/vendor pairs --- +### Archer Template Library + +Template management system for Archer Risk Acceptance forms. Stores static content (Environment Overview, Segmentation, Mitigating Controls, and 5 additional sections) organised by Vendor, Platform, and Model hierarchy. + +**Capabilities:** +- Full CRUD — create, view, update, and delete templates +- Clone existing templates into new vendor/platform/model combinations +- Search/filter by vendor, platform, or model (case-insensitive) +- Hierarchy browsing endpoints: list vendors, platforms per vendor, models per vendor+platform +- Per-section copy-to-clipboard buttons in the inline view panel +- Template selector integrated into the Ivanti Queue for Archer workflow items +- Accessible from nav drawer (Template Mgr) and from Archer queue items + +**Template sections (8 content fields, each max 10,000 chars):** +- Environment Overview +- Segmentation +- Mitigating Controls +- Additional Info +- Charter Network Banner +- Data Classification +- Charter Network +- Additional Access List + +**Hierarchy:** Vendor > Platform > Model (unique constraint on the combination). Templates are sorted alphabetically by vendor, platform, model. + +--- + +### In-App Notifications + +Native notification system providing per-user alerts for system events without external dependencies (replaces previous Webex bot integration). + +**Notification types:** +- `issue_resolved` — GitLab feedback issue closed and deployed + +**UI:** +- NotificationBell component in the header with unread count badge +- Click to view list of unread notifications (newest first, limited to 50) +- Mark individual notifications as read, or mark all as read +- Only the owning user can mark their own notifications + +**Storage:** Notifications are stored in the `notifications` table with `user_id`, `username`, `type`, `title`, `message`, `issue_number`, and `read` flag. + +--- + +### Feedback — GitLab Integration + +In-app bug reports and feature requests submitted directly to the GitLab project as issues. Keeps the GitLab PAT server-side so credentials are never exposed to the browser. + +**Submission workflow:** +1. Click the feedback button in the nav drawer +2. Select type: Bug Report or Feature Request +3. Fill in title and description +4. Optionally attach up to 3 screenshots (PNG, JPG, GIF, WebP — 5MB each) +5. Submit — creates a GitLab issue with labels (`bug` or `enhancement`) and formatted description + +**Issue lifecycle:** +- GitLab webhook receiver (`POST /api/webhooks/gitlab`) listens for issue close events +- When a feedback issue is closed, the submitter username is parsed from the issue description +- An in-app notification is created for the submitter: "Your bug report has been resolved and deployed" +- Webhook secret validation prevents unauthorized requests + +**Configuration:** +- `GITLAB_URL` — GitLab instance URL +- `GITLAB_PROJECT_ID` — numeric project ID +- `GITLAB_PAT` — project access token with `api` scope +- `GITLAB_WEBHOOK_SECRET` — shared secret for webhook validation (set same value in GitLab webhook settings) + +--- + ### Admin Panel The Admin Panel is a full-page, tabbed interface accessible only to Admin-group users. It replaces the previous inline modal rendering and follows the dashboard's dark tactical intelligence theme. Three tabs provide consolidated access to administrative functions: @@ -705,7 +1057,9 @@ All endpoints are prefixed with `/api`. All endpoints except `/api/auth/login` a | Method | Path | Group | Description | |---|---|---|---| -| GET | `/api/ivanti/archive` | Any | Get finding archive data for severity score drift tracking | +| GET | `/api/ivanti/archive` | Any | List archive records; optional filters: `state` (ACTIVE/ARCHIVED/RETURNED/CLOSED), `teams` (comma-separated BU names) | +| GET | `/api/ivanti/archive/anomalies` | Any | Sync anomaly log (significant count deltas and classification data) | +| GET | `/api/ivanti/archive/transitions/:archiveId` | Any | Transition history for a specific archive record | ### Compliance @@ -744,6 +1098,91 @@ All endpoints are prefixed with `/api`. All endpoints except `/api/auth/login` a | PUT | `/api/archer-tickets/:id` | Admin, Standard_User | Update an Archer ticket | | DELETE | `/api/archer-tickets/:id` | Admin, Standard_User | Delete an Archer ticket (ownership + compliance check for Standard_User) | +### Archer Templates + +| Method | Path | Group | Description | +|---|---|---|---| +| GET | `/api/archer-templates` | Any | List templates; optional filters: `search`, `vendor`, `platform`, `model` | +| GET | `/api/archer-templates/:id` | Any | Get a single template by ID | +| POST | `/api/archer-templates` | Admin, Standard_User | Create a new template | +| PUT | `/api/archer-templates/:id` | Admin, Standard_User | Update a template (partial update supported) | +| DELETE | `/api/archer-templates/:id` | Admin, Standard_User | Delete a template | +| POST | `/api/archer-templates/:id/clone` | Admin, Standard_User | Clone a template with new vendor/platform/model | +| GET | `/api/archer-templates/hierarchy/vendors` | Any | List distinct vendor names | +| GET | `/api/archer-templates/hierarchy/platforms` | Any | List platforms for a vendor; query: `vendor` | +| GET | `/api/archer-templates/hierarchy/models` | Any | List models for vendor+platform; query: `vendor`, `platform` | + +### CARD Asset Ownership + +| Method | Path | Group | Description | +|---|---|---|---| +| GET | `/api/card/status` | Any | Check if CARD API is configured | +| GET | `/api/card/teams` | Admin, Standard_User | List all CARD teams | +| GET | `/api/card/teams/:teamName/assets` | Admin, Standard_User | Paginated team assets by disposition | +| GET | `/api/card/owner/:assetId` | Admin, Standard_User | Get owner record for an asset | +| GET | `/api/card/owner-lookup/:ip` | Admin, Standard_User | Resolve IP to asset and return owner data; `?quick=1` for tooltip mode, `?hostId=N` for fast path | +| POST | `/api/card/owner/:assetId/confirm` | Admin, Standard_User | Direct confirm ownership (no queue item) | +| POST | `/api/card/owner/:assetId/decline` | Admin, Standard_User | Direct decline ownership (no queue item) | +| POST | `/api/card/owner/:assetId/redirect` | Admin, Standard_User | Direct redirect between teams (no queue item) | +| GET | `/api/card/asset-search/:hostId` | Admin, Standard_User | Search CARD by Ivanti Host ID (deep_search) | +| POST | `/api/card/enrich-batch` | Admin, Standard_User | Batch lookup IPs/host_ids for Granite loader fields (max 200) | +| POST | `/api/card/queue/:queueItemId/confirm` | Admin, Standard_User | Confirm ownership for a queue item | +| POST | `/api/card/queue/:queueItemId/decline` | Admin, Standard_User | Decline ownership for a queue item | +| POST | `/api/card/queue/:queueItemId/redirect` | Admin, Standard_User | Redirect ownership for a queue item | + +### Atlas Action Plans + +| Method | Path | Group | Description | +|---|---|---|---| +| GET | `/api/atlas/hosts/:hostId/plans` | Any | Get action plans for a host (from cache or API) | +| PUT | `/api/atlas/hosts/:hostId/plans` | Admin, Standard_User | Create a new action plan | +| PATCH | `/api/atlas/hosts/:hostId/plans` | Admin, Standard_User | Update an existing plan | +| POST | `/api/atlas/hosts/:hostId/refresh` | Admin, Standard_User | Force cache refresh from Atlas API | + +### VCL Multi-Vertical (CCP Metrics) + +| Method | Path | Group | Description | +|---|---|---|---| +| POST | `/api/compliance/vcl-multi/preview` | Admin, Standard_User | Parse xlsx files, extract verticals, compute scoped diffs | +| POST | `/api/compliance/vcl-multi/commit` | Admin, Standard_User | Commit previewed files in a single transaction | +| GET | `/api/compliance/vcl-multi/stats` | Any | Aggregated cross-vertical executive summary | +| GET | `/api/compliance/vcl-multi/trend` | Any | Monthly compliance trend with linear regression forecast | +| GET | `/api/compliance/vcl-multi/burndown` | Any | Aggregated cross-vertical burndown forecast | +| GET | `/api/compliance/vcl-multi/verticals` | Any | List known verticals | +| GET | `/api/compliance/vcl-multi/metrics` | Any | All metrics aggregated across verticals | +| GET | `/api/compliance/vcl-multi/metrics-list` | Any | Distinct metrics with active non-compliant device counts | +| GET | `/api/compliance/vcl-multi/metric/:id/stats` | Any | Per-metric summary statistics and donut breakdown | +| GET | `/api/compliance/vcl-multi/metric/:id/verticals` | Any | Per-vertical breakdown for a metric with sub-teams | +| GET | `/api/compliance/vcl-multi/metric/:id/trend` | Any | Per-metric monthly trend with forecast | +| GET | `/api/compliance/vcl-multi/vertical/:code/metrics` | Any | Per-metric breakdown for a specific vertical | +| GET | `/api/compliance/vcl-multi/vertical/:code/metric/:metricId/devices` | Any | Device list for a vertical+metric; `?team=X` | +| GET | `/api/compliance/vcl-multi/vertical/:code/burndown` | Any | Burndown forecast for a specific vertical | +| GET | `/api/compliance/vcl-multi/uploads` | Any | Upload history (most recent 100) | +| DELETE | `/api/compliance/vcl-multi/vertical/:code` | Admin | Delete all data for a vertical | +| DELETE | `/api/compliance/vcl-multi/upload/:uploadId` | Admin | Rollback a specific upload (must be most recent for that vertical) | +| DELETE | `/api/compliance/vcl-multi/all` | Admin | Delete all multi-vertical data | + +### Notifications + +| Method | Path | Group | Description | +|---|---|---|---| +| GET | `/api/notifications` | Any | Get unread notifications for current user (max 50) | +| GET | `/api/notifications/count` | Any | Get unread count for badge display | +| PATCH | `/api/notifications/:id/read` | Any | Mark a single notification as read (own only) | +| POST | `/api/notifications/read-all` | Any | Mark all notifications as read | + +### Feedback + +| Method | Path | Group | Description | +|---|---|---|---| +| POST | `/api/feedback` | Any | Submit bug report or feature request to GitLab (multipart with optional screenshots) | + +### Webhooks + +| Method | Path | Group | Description | +|---|---|---|---| +| POST | `/api/webhooks/gitlab` | Public | GitLab issue webhook receiver (validated by `x-gitlab-token` header) | + ### Users (Admin only) | Method | Path | Group | Description | @@ -776,10 +1215,16 @@ All endpoints are prefixed with `/api`. All endpoints except `/api/auth/login` a cve-dashboard/ ├── start-servers.sh # Start backend + frontend via systemd ├── stop-servers.sh # Stop both systemd services +├── configure.js # Interactive configuration wizard ├── docker-compose.yml # PostgreSQL 16 container definition ├── package.json # Root package.json (backend dependencies) ├── scripts/ -│ └── deploy-postgres.sh # One-time deployment: container, schema, migration +│ ├── deploy-postgres.sh # One-time deployment: container, schema, migration +│ └── reset-and-migrate.sh # Dev utility: reset DB and re-run migrations +├── deploy/ +│ ├── cve-backend-production.service +│ ├── cve-backend-staging.service +│ └── setup-staging.sh ├── systemd/ # systemd unit files for auto-start on boot │ ├── cve-backend.service │ └── cve-frontend.service @@ -789,6 +1234,7 @@ cve-dashboard/ │ ├── db.js # PostgreSQL connection pool (pg) │ ├── db-schema.sql # Complete DDL for fresh Postgres setup │ ├── setup-postgres.js # Schema initializer (runs db-schema.sql) +│ ├── setup.js # One-time DB init + default admin creation │ ├── uploads/ # File storage root (gitignored) │ │ ├── // # CVE documents │ │ ├── knowledge_base/ # Knowledge base documents @@ -800,20 +1246,33 @@ cve-dashboard/ │ │ ├── nvdLookup.js # NVD API proxy │ │ ├── knowledgeBase.js # Knowledge base document management │ │ ├── archerTickets.js # Archer EXC ticket CRUD +│ │ ├── archerTemplates.js # Archer template library CRUD + clone + hierarchy │ │ ├── ivantiWorkflows.js # Ivanti workflow batch sync and cache │ │ ├── ivantiFindings.js # Ivanti host findings sync, notes, overrides, FP counts -│ │ ├── ivantiTodoQueue.js # Ivanti Queue — personal FP/Archer/CARD staging list -│ │ ├── ivantiArchive.js # Finding archive for severity score drift -│ │ ├── jiraTickets.js # Jira ticket CRUD + Jira REST API integration (lookup, sync, create) -│ │ └── compliance.js # AEO compliance upload, diff, device tracking, notes +│ │ ├── ivantiTodoQueue.js # Ivanti Queue — personal staging list +│ │ ├── ivantiArchive.js # Finding archive tracking, transitions, anomaly log +│ │ ├── ivantiFpWorkflow.js # FP workflow submission to Ivanti API + lifecycle +│ │ ├── compliance.js # AEO compliance upload, diff, device tracking, notes +│ │ ├── vclMultiVertical.js # VCL/CCP multi-vertical compliance reporting +│ │ ├── atlas.js # Atlas action plan proxy + cache +│ │ ├── jiraTickets.js # Jira ticket CRUD + Jira REST API integration +│ │ ├── cardApi.js # CARD ownership proxy, mutation, asset-search, enrich +│ │ ├── notifications.js # In-app notification system +│ │ ├── feedback.js # Bug reports/feature requests to GitLab +│ │ └── webhooks.js # GitLab webhook receiver for issue lifecycle │ ├── middleware/ │ │ └── auth.js # requireAuth and requireGroup middleware │ ├── helpers/ │ │ ├── auditLog.js # logAudit helper (fire-and-forget) -│ │ ├── driftChecker.js # Schema drift detection: compareSchemaToDrift(), loadConfig(), reconcileConfig() +│ │ ├── cardApi.js # CARD API — OAuth token, owner, confirm/decline/redirect, asset-search │ │ ├── ivantiApi.js # Ivanti API HTTP helpers (multipart, JSON, form POST) -│ │ └── jiraApi.js # Jira Data Center REST API helpers (Basic/PAT auth, rate limiting) -│ ├── migrations/ # Legacy SQLite migration scripts (not needed for Postgres) +│ │ ├── atlasApi.js # Atlas action plan API (Basic auth) +│ │ ├── jiraApi.js # Jira Data Center REST API helpers (Basic/PAT auth, rate limiting) +│ │ ├── driftChecker.js # Schema drift detection: compareSchemaToDrift(), loadConfig(), reconcileConfig() +│ │ ├── vclHelpers.js # VCL metric calculation helpers (burndown, forecast, dedup) +│ │ └── teams.js # Team validation helpers +│ ├── migrations/ # Sequential migration scripts (idempotent) +│ │ └── run-all.js # Run all migrations in order │ └── scripts/ │ ├── migrate-to-postgres.js # One-time SQLite → Postgres data migration │ ├── compliance_config.json # Shared parser config (metric_categories, core_cols, skip_sheets) @@ -828,30 +1287,60 @@ cve-dashboard/ ├── App.css # Global styles and CSS variables ├── contexts/ │ └── AuthContext.js # Auth state provider (login, logout, group helpers) + ├── utils/ + │ ├── graniteLoaderConfig.js # Granite column definitions and groups + │ ├── graniteLoaderExport.js # XLSX generation logic + │ └── graniteLoaderPicklists.js # Searchable dropdown options └── components/ ├── LoginForm.js # Login page - ├── NavDrawer.js # Side navigation drawer (pages + Admin Panel link for Admin group) - ├── UserMenu.js # User dropdown in header (shows group badge) - ├── CalendarWidget.js # Due-date calendar with Ivanti finding indicators - ├── UserManagement.js # Admin user management modal (quick-access from UserMenu) - ├── AuditLog.js # Admin audit log modal (quick-access from UserMenu) + ├── NavDrawer.js # Side navigation drawer + ├── UserMenu.js # User dropdown (shows group badge) + ├── UserProfilePanel.js # User profile and password change + ├── CalendarWidget.js # Due-date calendar with finding indicators + ├── UserManagement.js # Admin user management modal + ├── AuditLog.js # Admin audit log modal ├── NvdSyncModal.js # Bulk NVD sync dialog ├── KnowledgeBaseModal.js # Knowledge base upload/list modal - ├── KnowledgeBaseViewer.js # Inline document viewer (sandboxed iframe, sanitized markdown) - ├── ConfirmModal.js # Themed confirmation dialog (replaces window.confirm) - ├── CveTooltip.js # Hover tooltip for CVE badges (portal-rendered, cached) - ├── RedirectModal.js # Queue item redirect modal (workflow type + vendor selection) + ├── KnowledgeBaseViewer.js # Inline document viewer + ├── CardOwnerTooltip.js # CARD ownership hover tooltip + ├── CardDetailModal.js # CARD asset detail (from reporting) + ├── CardActionModal.js # CARD confirm/decline/redirect (from queue) + ├── RedirectModal.js # Queue item redirect modal + ├── LoaderModal.js # Granite Loader Sheet generator + ├── SearchableSelect.js # Reusable searchable dropdown + ├── AtlasBadge.js # Atlas action plan indicator badge + ├── AtlasIcon.js # Atlas icon component + ├── AtlasSlideOutPanel.js # Atlas plan detail panel + ├── AdminScopeToggle.js # BU scope toggle + ├── ConfirmModal.js # Themed confirmation dialog + ├── ConsolidationModal.js # Multi-item Jira ticket consolidation + ├── CveTooltip.js # Hover tooltip for CVE badges + ├── DeleteConfirmModal.js # Delete confirmation with details + ├── FeedbackModal.js # Bug report/feature request submission + ├── NotificationBell.js # Notification bell with unread count + ├── RemediationModal.js # Remediation plan editing + ├── TemplateFormModal.js # Archer template create/edit form + ├── TemplateSelector.js # Archer template picker for queue items └── pages/ - ├── AdminPage.js # Admin panel: user management, audit log, system info + ├── AdminPage.js # Admin panel: users, audit log, system info ├── ReportingPage.js # Host findings: charts, table, queue, export + ├── IvantiTodoQueuePage.js # Full-page queue view + ├── IvantiCountsChart.js # Ivanti counts history chart + ├── AnomalyBanner.js # Sync anomaly alert banner + ├── ArchiveSummaryBar.js # Finding archive state distribution ├── CompliancePage.js # AEO compliance: metric cards, device table - ├── ComplianceUploadModal.js # xlsx upload with diff preview + ├── ComplianceUploadModal.js # xlsx upload with drift check + diff preview ├── ComplianceDetailPanel.js # Per-device metrics, history, notes ├── ComplianceChartsPanel.js # Compliance trend charts - ├── IvantiCountsChart.js # Ivanti counts history chart - ├── ArchiveSummaryBar.js # Finding archive summary - ├── JiraPage.js # Jira ticket management and Jira API integration - ├── KnowledgeBasePage.js # Knowledge base page + ├── CCPMetricsPage.js # CCP Metrics: multi-vertical executive view + ├── VCLReportPage.js # VCL exec report page + ├── MetricInfoPanel.js # Metric detail drill-down panel + ├── BulkUploadModal.js # Bulk VCL upload + ├── MultiVerticalUploadModal.js # Multi-vertical upload modal + ├── ArcherPage.js # Archer tickets management + ├── ArcherTemplatePage.js # Archer template library + ├── JiraPage.js # Jira ticket management + API integration + ├── KnowledgeBasePage.js # Knowledge base page └── ExportsPage.js # Exports page (group-gated) ``` @@ -891,9 +1380,31 @@ All tables are defined in `backend/db-schema.sql` and created by `setup-postgres **`ivanti_finding_overrides`** — Editor-applied overrides for `hostName` and `dns` fields. `UNIQUE(finding_id, field)`. -**`ivanti_todo_queue`** — Personal per-user queue of findings staged for FP, Archer, or CARD processing. Keyed by `(user_id, finding_id)`. Completed items can be redirected to a different workflow type via `POST /:id/redirect`, which creates a new pending item preserving the original finding data. +**`ivanti_todo_queue`** — Personal per-user queue of findings staged for FP, Archer, CARD, GRANITE, DECOM, or Remediate processing. Keyed by `(user_id, finding_id)`. Workflow type constraint: `FP`, `Archer`, `CARD`, `GRANITE`, `DECOM`. Completed items can be redirected to a different workflow type via `POST /:id/redirect`, which creates a new pending item preserving the original finding data. -**`ivanti_fp_submissions`** — Record of FP workflow submissions to the Ivanti API. Tracks user, workflow batch ID, form fields, finding IDs, queue item IDs, attachment results, and submission status (success/partial/failed). Rejected submissions can be dismissed (`dismissed_at`) or re-queued to the todo queue under a different workflow type (`requeued_at`). +**`ivanti_fp_submissions`** — Record of FP workflow submissions to the Ivanti API. Tracks user, workflow batch ID, form fields, finding IDs, queue item IDs, attachment results, and submission status (success/partial/failed). Lifecycle status tracks the workflow through: submitted, approved, rejected, rework, resubmitted. Rejected submissions can be dismissed (`dismissed_at`) or re-queued to the todo queue under a different workflow type (`requeued_at`). + +**`ivanti_fp_submission_history`** — Edit history for FP submissions. Tracks change_type (created, fields_updated, findings_added, attachments_added, status_changed) with change_details_json. + +**`ivanti_finding_archives`** — Archived finding records with lifecycle state tracking. States: ARCHIVED, RETURNED, CLOSED, CLOSED_GONE. Tracks severity at time of archival and transition timestamps. + +**`ivanti_archive_transitions`** — State transition history for archived findings. Records from_state, to_state, severity_at_transition, and reason for each transition. + +**`ivanti_sync_anomaly_log`** — Sync anomaly detection log. Records count deltas, newly archived/returned counts, classification breakdown, and significance flag per sync. + +**`ivanti_finding_bu_history`** — BU reassignment history per finding. Records previous_bu, new_bu, and detection timestamp. + +**`ivanti_counts_history_by_bu`** — Per-BU historical open/closed counts, enabling per-BU trend lines on the findings chart. + +**`atlas_action_plans_cache`** — Cached Atlas action plan data for badge rendering. Stores host_id, has_action_plan flag, plan_count, plans_json, and atlas_known flag. Indexed on host_id. + +**`archer_templates`** — Archer template library. Vendor/platform/model hierarchy with 8 section content fields (environment_overview, segmentation, mitigating_controls, additional_info, charter_network_banner, data_classification, charter_network, additional_access_list). `UNIQUE(vendor, platform, model)`. + +**`notifications`** — In-app notifications. Per-user notifications with type, title, message, issue_number, and read flag. Used by the NotificationBell component. + +**`vcl_multi_vertical_summary`** — Per-metric summary data from VCL multi-vertical uploads. Stores metric_id, metric_desc, category, team, priority, non_compliant, compliant, total, compliance_pct, target, and status per upload per vertical. Used for executive reporting without recalculating from items. + +**`compliance_snapshots`** — Monthly compliance snapshots per vertical. Used for trend charts and linear regression forecasting. **`compliance_uploads`** — Record of each compliance xlsx upload: filename, report date, uploader, timestamp, and new/resolved/recurring counts. @@ -1022,32 +1533,63 @@ After upgrading, clear your browser cookies and log in fresh — session format ## Migrations -> **Note:** The migration scripts in `backend/migrations/` are legacy SQLite migrations. They are not needed for PostgreSQL deployments — the complete schema is defined in `backend/db-schema.sql` and applied by `setup-postgres.js`. These scripts are retained for reference and for any remaining SQLite-based environments. - -For deployments still on SQLite, run them in the listed order. All are idempotent and safe to re-run. +> **Note:** The migration scripts in `backend/migrations/` are used for both PostgreSQL and legacy SQLite deployments. Run them via `node migrations/run-all.js` which executes all migrations in order. All are idempotent and safe to re-run. ```bash cd backend +node migrations/run-all.js +``` + +For manual execution or debugging, the individual scripts in order: + +```bash +cd backend +node migrations/add_user_groups.js +node migrations/add_user_ivanti_identity.js +node migrations/add_user_bu_teams.js node migrations/add_knowledge_base_table.js -node migrations/add_archer_tickets_table.js node migrations/add_ivanti_sync_table.js node migrations/add_ivanti_findings_tables.js -node migrations/add_ivanti_todo_queue_table.js -node migrations/add_card_workflow_type.js -node migrations/add_todo_queue_ip_address.js -node migrations/add_todo_queue_hostname.js -node migrations/add_compliance_tables.js -node migrations/add_finding_archive_tables.js -node migrations/add_archer_tickets_timestamps.js +node migrations/add_ivanti_findings_ipv6_columns.js node migrations/add_ivanti_counts_history_table.js +node migrations/add_ivanti_todo_queue_table.js +node migrations/add_todo_queue_hostname.js +node migrations/add_todo_queue_ip_address.js +node migrations/add_granite_workflow_type.js +node migrations/add_card_workflow_type.js +node migrations/add_decom_workflow_type.js +node migrations/add_remediate_workflow_type.js +node migrations/add_compliance_tables.js +node migrations/add_compliance_notes_group_id.js +node migrations/add_compliance_history_metric_id.js +node migrations/add_compliance_item_history.js node migrations/add_fp_submissions_table.js -node migrations/add_user_groups.js -node migrations/add_created_by_columns.js node migrations/add_fp_submission_editing.js node migrations/add_fp_submissions_dismissed.js node migrations/add_fp_submissions_requeued_at.js -node migrations/add_granite_workflow_type.js -node migrations/add_compliance_notes_group_id.js +node migrations/add_archer_tickets_table.js +node migrations/add_archer_tickets_timestamps.js +node migrations/add_archer_templates_table.js +node migrations/add_atlas_action_plans_cache.js +node migrations/add_atlas_known_column.js +node migrations/add_finding_archive_tables.js +node migrations/add_closed_gone_state.js +node migrations/add_return_classification.js +node migrations/add_sync_anomaly_tables.js +node migrations/add_flexible_jira_ticket_creation.js +node migrations/add_multi_item_jira_ticket.js +node migrations/add_jira_sync_columns.js +node migrations/add_jira_sync_columns_pg.js +node migrations/drop_jira_status_check_constraint.js +node migrations/add_notifications_table.js +node migrations/add_created_by_columns.js +node migrations/add_queue_remediation_notes_table.js +node migrations/add_vcl_multi_vertical.js +node migrations/add_vcl_reporting_columns.js +node migrations/add_vcl_vertical_metadata.js +node migrations/backfill_anomaly_log.js +node migrations/backfill_return_classification.js +node migrations/reclassify_bu_roundtrips.js ``` For deployments upgrading from an older schema, the following legacy migration scripts are also available in `backend/`: