177 Commits

Author SHA1 Message Date
Jordan Ramos
6163be626e ops: add docker-compose.yml and deploy-postgres.sh for production cutover
- docker-compose.yml: Postgres 16 Alpine on port 5433 with healthcheck
- scripts/deploy-postgres.sh: one-shot deployment script that handles
  container startup, schema creation, npm install, data migration, and
  frontend build
- Backup SQLite database as cve_database.db.pre-postgres-backup
2026-05-06 15:07:06 -06:00
Jordan Ramos
573903a885 feat: per-BU trend lines in counts history chart
- Create ivanti_counts_history_by_bu table (bu_ownership, state, count per sync)
- Sync writes per-BU snapshot alongside global history on each sync
- Seed table with current counts for immediate first data point
- GET /counts/history accepts ?teams param — queries per-BU table when filtered
- IvantiCountsChart accepts teamsParam prop, re-fetches on scope change
- ReportingPage passes getActiveTeamsParam() to the chart
- Historical per-BU data accumulates from this point forward
- Global history (no filter) still uses the original aggregate table
2026-05-06 13:38:38 -06:00
Jordan Ramos
77f113e9ae fix: load dotenv in db.js so DATABASE_URL is available on import 2026-05-06 12:30:45 -06:00
Jordan Ramos
8cd73c126e feat(postgres): data migration + per-BU closed counts in frontend
- Create backend/scripts/migrate-to-postgres.js (one-time SQLite→Postgres copy)
- Successfully migrated: 6 users, 21 CVEs, 6307 findings, 20965 compliance items,
  138 archives, 67 atlas plans, all notes/overrides merged
- All 22 tables verified with matching row counts
- Frontend StatusDonut now uses server-provided per-BU counts (no more N/A)
- Counts endpoint called with teams param on scope change
- Re-fetch counts when admin scope toggle changes
2026-05-06 12:26:54 -06:00
Jordan Ramos
e30ad79f2a feat(postgres): rewrite Ivanti findings to individual rows
- Replace 2.6MB JSON blob with individual rows in ivanti_findings table
- Batch upsert via INSERT ... ON CONFLICT in chunks of 100
- Sync stores both open AND closed findings as rows with state column
- Per-BU closed counts now possible via SQL GROUP BY
- GET /findings queries indexed table with optional ILIKE BU filter
- GET /counts returns per-BU open+closed via GROUP BY state
- Notes and overrides are columns on ivanti_findings (no separate tables)
- Removed: readState, readStateWithNotes, _findingsCache, initTables
- Preserved: extractFinding, archive detection, FP workflow counts, anomaly log
- Response shape unchanged — frontend works without modification
2026-05-06 12:12:34 -06:00
Jordan Ramos
33927b150b feat(postgres): migrate all route files from SQLite to pg pool
- All 16 route files now import pool from ../db directly
- Removed db parameter from all factory functions
- All callbacks replaced with async/await pool.query()
- All ? placeholders converted to $1, $2... numbered params
- datetime('now') → NOW(), INSERT OR IGNORE → ON CONFLICT DO NOTHING
- LIKE → ILIKE for case-insensitive searches
- Error detection: err.code === '23505' for unique violations
- server.js no longer passes pool/db/requireAuth to route factories
- Only ivantiFindings.js still receives pool (pending task 8 rewrite)
2026-05-06 11:44:17 -06:00
Jordan Ramos
845d843e71 feat(postgres): infrastructure setup and schema creation (tasks 1-2)
- Install pg (node-postgres) dependency
- Create backend/db.js connection pool module (max 10, auto-reconnect)
- Install Docker and spin up steam-postgres container on port 5433
- Create backend/db-schema.sql with complete Postgres DDL (24 tables)
- Replace findings_json blob with ivanti_findings table (individual rows)
- Merge notes/overrides into findings table columns
- Add proper indexes: state, bu_ownership, severity, composite
- Create backend/setup-postgres.js for idempotent schema initialization
- Add DATABASE_URL to .env and .env.example
- Update migration plan docs with Docker setup commands
- Verify: schema executes cleanly, pool connects, 24 tables created
2026-05-05 15:47:09 -06:00
Jordan Ramos
5cdca09f40 docs: add Postgres migration plan and Kiro spec
- docs/guides/postgres-migration-plan.md: full migration manual with
  phases, port allocation, rollback plan, and timeline
- .kiro/specs/postgres-migration/: requirements, design, and tasks
- Replaces findings_json blob with individual indexed rows
- Enables per-BU closed counts via SQL queries
- Uses existing Postgres instance (port 5432), new cve_dashboard DB
- Testing on port 3003, cutover to 3001 with 30s downtime
2026-05-05 15:04:14 -06:00
Jordan Ramos
bd5fcccacf perf: client-side BU filtering for instant scope switching
- Fetch ALL findings once on mount (no teams param to backend)
- Filter client-side via scopedFindings useMemo keyed on adminScope
- Eliminates 5-10s round-trip on every scope change
- Open vs Closed donut now uses scopedFindings.length for open count
- Closed count remains global (no per-BU closed data available)
- Action Coverage donut automatically scoped via visibleFindings chain
- Remove server-side teams param from counts fetch (client handles it)
2026-05-05 12:08:01 -06:00
Jordan Ramos
df3173a720 feat: replace binary scope toggle with multi-select BU picker
- Add IVANTI_BU_FILTER to .env with all four BUs (STEAM, ACCESS-ENG, ACCESS-OPS, INTELDEV)
- Rework AdminScopeToggle from binary (My Teams/All) to multi-select dropdown
- Admin can now pick any combination of BUs to view
- Presets: 'All BUs' and 'My Teams' for quick selection
- Individual team checkboxes for custom combinations
- Selection persisted in localStorage as JSON array
- AuthContext updated: adminScope is now an array of selected teams
- getActiveTeamsParam() returns comma-joined selected teams (empty = no filter)
- getAvailableTeams() returns selected teams for compliance selector
2026-05-05 11:31:15 -06:00
Jordan Ramos
9b8ae6cd79 fix: move AdminScopeToggle from NavDrawer to main header bar
Places the scope toggle next to the UserMenu avatar in the top-right
header area so it's always visible without opening the nav drawer.
2026-05-05 11:21:59 -06:00
Jordan Ramos
2656df94d3 feat: add multi-BU tenancy with per-user team scoping (Option B)
- Add bu_teams column to users table (migration + fresh schema)
- Create shared KNOWN_TEAMS constant and validateTeams helper
- Expose user teams in auth middleware, login, and /me responses
- Add bu_teams CRUD to user management routes with audit logging
- Make Ivanti FINDINGS_FILTERS configurable via IVANTI_BU_FILTER env var
- Add query-time team filtering to GET /findings and /findings/counts
- Update AuthContext with teams helpers and admin scope toggle
- Create AdminScopeToggle component (My Teams / All BUs)
- Scope ReportingPage findings fetch by user teams
- Scope CompliancePage team selector by user teams
- Scope ExportsPage findings exports by user teams
- Add BU teams multi-select to UserManagement create/edit forms
- Display team badges in user list table
2026-05-05 11:04:53 -06:00
Jordan Ramos
af951fdc12 chore: remove .kiro specs, hooks, and steering from release — development tooling only 2026-05-01 21:28:59 +00:00
Jordan Ramos
7f7d3a2977 release: v1.0.0 — clean README, changelog, full reference manual, dead code removal, package metadata 2026-05-01 21:18:31 +00:00
Jordan Ramos
034d3963b9 chore: reorganize docs, document migrations, gitignore operational files for v1.0.0 release 2026-05-01 20:53:39 +00:00
Jordan Ramos
c8b3626ac5 feat: consolidate setup.js with complete v1.0.0 schema — all tables, indexes, triggers for fresh deployments 2026-05-01 20:13:52 +00:00
Jordan Ramos
8e377bb85f chore: enable GPG-signed commits for code provenance 2026-05-01 19:50:31 +00:00
root
5a9df2103f fix: aggregate anomaly data per day instead of taking latest — fixes missing returned bars when multiple syncs per day 2026-05-01 19:29:11 +00:00
root
bfa52c7f8f fix: reclassify BU reassignment round-trips and fix backfill date-ordering bug 2026-05-01 17:36:28 +00:00
root
3202b0707c feat: add backfill script for return classification on existing anomaly log rows 2026-05-01 17:27:49 +00:00
root
15abf8bae4 feat: add return classification for archive chart, CARD API integration, compliance charts, systemd services 2026-05-01 17:15:41 +00:00
8df961cce8 Merge pull request 'Switch Jira API calls to GET-based JQL search with project scoping' (#9) from fix/jira-api-compliance into master
Reviewed-on: #9
2026-04-29 08:16:44 -06:00
root
7a179f19a1 Switch Jira API calls to GET-based JQL search with project scoping
- getIssue now uses GET /rest/api/2/search with JQL instead of
  GET /rest/api/2/issue/{key} for Charter compliance
- searchIssues switched from POST to GET with URL-encoded query params
- searchIssuesByKeys adds project scoping to JQL clause
- Updated UAT tests and API use-case docs to match
2026-04-29 14:12:04 +00:00
root
4f960d0866 Update README and Jira UAT test script 2026-04-28 18:44:14 +00:00
root
caa1d539cc Add CARD API integration spec, Atlas metrics updates, NavDrawer and server.js cleanup, reference docs 2026-04-28 16:38:18 +00:00
root
b1069b1a05 Add Jira Data Center integration with UAT test script and use case docs 2026-04-28 16:36:54 +00:00
root
1186f9f807 Fix build: remove unused imports, set CI=false for react-scripts build 2026-04-28 14:22:19 +00:00
root
e13b18c169 Allow frontend test failures for pre-existing ESM/env test suite issues 2026-04-28 00:20:12 +00:00
root
05d47c91a8 Remove node_modules artifacts, rely on cache for shell executor 2026-04-28 00:08:17 +00:00
root
b0c3daba01 Fix CI pipeline to use npm install instead of npm ci (no lockfile in repo) 2026-04-28 00:04:44 +00:00
root
675847de0c Add GitLab CI/CD pipeline with install, lint, test, build, and deploy stages 2026-04-27 23:08:32 +00:00
root
623b57ca06 Fix Atlas vulnerability response parsing — API returns arrays per host, not objects 2026-04-27 16:21:19 +00:00
root
06c6821d85 Add multi-select qualys_id picker to bulk Atlas action plan modal with auto-fetch from Atlas API 2026-04-24 22:07:55 +00:00
root
8da62f0f14 Require qualys_id for risk_acceptance in bulk Atlas action plan modal 2026-04-24 21:58:53 +00:00
root
5a9dc007db Add bulk Atlas action plan creation from row selection toolbar 2026-04-24 21:49:04 +00:00
root
3f9e1da2a3 Fix findings export to use overridden hostname and DNS values 2026-04-24 21:38:43 +00:00
root
7ea4ceb8df Add backfill script for anomaly log historical data 2026-04-24 21:16:35 +00:00
root
00a6f7ae0f Add archive activity sparkline to findings trend chart and update investigation doc 2026-04-24 21:06:35 +00:00
root
69809955a9 Remove diagnostic scripts and xlsx export from tracking, add to gitignore 2026-04-24 20:36:46 +00:00
root
6ee68f5521 Add sync anomaly detection, BU drift monitoring, and findings count investigation
- Add BU drift checker that classifies archived findings as BU reassignment,
  severity drift, closure, or decommission via unfiltered Ivanti API queries
- Add post-sync anomaly summary with significance threshold and classification
  breakdown stored in ivanti_sync_anomaly_log table
- Add per-finding BU tracking that detects BU changes across syncs and records
  them in ivanti_finding_bu_history table
- Add drift guard that skips trend history writes when total drops more than 50%
- Add CLOSED_GONE archive state for findings that vanish from the closed set
- Add anomaly banner UI on Vulnerability Triage page for significant sync changes
- Add API endpoints for anomaly latest/history and BU change tracking
- Add diagnostic scripts for drift checking and BU reassignment verification
- Add investigation document and xlsx export for the April 2026 BU reassignment
  incident where 109 findings were moved to SDIT-CSD-ITLS-PIES
- Migrations required: add_closed_gone_state.js, add_sync_anomaly_tables.js
2026-04-24 20:34:34 +00:00
root
5ffedad02f Add Atlas metrics reporting, security audit tracker, and spec documents 2026-04-24 17:30:06 +00:00
root
8bf8dc55dd Add user profile panel with self-service password change and dark theme UserMenu 2026-04-24 17:29:06 +00:00
root
53439b2af8 Add Atlas exports and custom Atlas InfoSec icon
Exports page:
- Add Atlas Action Plans export card with three reports: Full Status,
  Coverage Gaps, and Full Report (multi-sheet with active, gaps, history)
- Reports join Atlas cache with Ivanti findings for hostname, IP, BU context

Atlas icon:
- Add AtlasIcon SVG component matching the Atlas InfoSec logo (badge with globe)
- Replace Database icon with AtlasIcon on exports card, sync button, and panel header
2026-04-23 22:18:23 +00:00
root
4c04c9870a Add Atlas InfoSec action plans integration
Integrate Atlas InfoSec API to manage compliance action plans directly from
the ReportingPage. Users can view, create, and update action plans for host
findings without switching to the Atlas web tool.

Backend:
- Add atlasApi.js helper with Basic Auth, TLS skip, GET/PUT/PATCH/POST
- Add atlas_action_plans_cache migration for SQLite cache table
- Add atlas.js router with sync, status, and proxy CRUD endpoints
- Mount Atlas router at /api/atlas in server.js
- Extract hostId from Ivanti host findings during sync

Frontend:
- Add AtlasBadge component (amber=needs plan, green=has plan)
- Add AtlasSlideOutPanel with plan list, create form, edit capability
- Separate active plans from inactive history in collapsible section
- Custom dark-themed plan type dropdown
- Optimistic local state shows pending plans immediately after creation
- Atlas sync button on ReportingPage toolbar
- Prepopulate finding ID in create form from clicked row

Environment:
- Add ATLAS_API_URL, ATLAS_API_USER, ATLAS_API_PASS, ATLAS_SKIP_TLS to .env.example
2026-04-23 21:52:53 +00:00
root
e1b000870c Enforce 120-day maximum on FP workflow expiration date 2026-04-22 19:52:06 +00:00
root
f3ba322403 Fix variant pill labels to show short priority tag instead of full description 2026-04-22 18:37:54 +00:00
root
0bea387ac9 Add grouped metric health cards with variant pills, hover tooltips, and info panel to compliance page 2026-04-22 18:30:59 +00:00
root
aa3ce3bae9 Replace window.confirm() with themed ConfirmModal across dashboard 2026-04-20 21:54:37 +00:00
root
0cdaecf890 Add themed admin page with user management, audit log, and system info panels; add compliance note delete functionality 2026-04-20 21:39:43 +00:00
root
043c85cc69 Add admin page overhaul and compliance schema drift check specs, compliance upload improvements, drift checker helper 2026-04-20 20:12:12 +00:00
jramos
6082721452 Sync all local changes for remote dev server migration 2026-04-20 10:23:47 -06:00
jramos
a214393723 Add compliance-staging folder, gitignore agents, update docs and kiro config 2026-04-16 14:41:52 -06:00
jramos
f141fa58a1 Add multi-metric note selection to compliance detail panel 2026-04-16 14:28:44 -06:00
jramos
e1b0236874 feat: add FP attachment library — attach existing CVE documents to FP submissions
- Add GET /api/ivanti/fp-workflow/documents/search endpoint for querying the document library
- Update POST /api/ivanti/fp-workflow to accept libraryDocIds for attaching library documents on create
- Update POST .../submissions/:id/attachments to accept libraryDocIds on edit
- Add AttachmentSourcePicker component with local upload and library search modes
- Integrate picker into FpWorkflowModal (create) and FpEditModal (edit)
- Track attachment source (local/library) in attachment_results_json for traceability
2026-04-15 15:27:21 -06:00
jramos
ed48522932 feat: add row visibility controls to Reporting page — hide/bulk-hide rows, localStorage persistence, visibility manager popover, chart/export integration 2026-04-15 13:15:01 -06:00
jramos
938dda400a feat: improve archive finding clarity with finding IDs, historical severity labels, and related active finding indicators 2026-04-15 10:18:19 -06:00
jramos
732873dd6a feat: add migration for GRANITE workflow_type CHECK constraint 2026-04-14 15:44:17 -06:00
jramos
0fe8e94d51 feat: add GRANITE as fourth workflow type in Ivanti queue
- Add GRANITE to VALID_WORKFLOW_TYPES in backend (no vendor required, same as CARD)
- Update vendor validation and error messages across all endpoints (single add, batch, PUT, redirect)
- Add GRANITE option to RedirectModal with warm slate color (#A1887F)
- Rename QueuePanel CARD section to Inventory, group CARD + GRANITE with sub-divider
- Add GRANITE to AddToQueuePopover and SelectionToolbar
- Update spec docs (requirements, design, tasks)
2026-04-14 15:38:22 -06:00
jramos
28bce28fc9 docs: add knowledge base guides for reporting, compliance, queue operations, user management, and CVE tracking 2026-04-13 16:52:19 -06:00
jramos
72fd79ea42 docs: add knowledge base article for FP queue and submission editing workflow 2026-04-13 16:38:31 -06:00
jramos
f63c286458 fix: show all Ivanti reviewer notes (rework, approval, current/previous state) in history tab 2026-04-13 16:14:27 -06:00
jramos
93c144576f docs: document map endpoint behavior — JSON only, one finding per call, UUID resolution flow 2026-04-13 16:03:57 -06:00
jramos
fa3b045a2f fix: map findings one at a time via JSON POST, only mark successfully mapped queue items as complete 2026-04-13 15:59:55 -06:00
jramos
4583d09750 chore: remove debug logging, remove unused ivantiMultipartPost import 2026-04-13 14:31:36 -06:00
jramos
75ac8c823a feat: show finding IDs in history, display Ivanti reviewer notes (rework/approval feedback) in history tab 2026-04-13 14:25:14 -06:00
jramos
68e36b4bac docs: document Ivanti API limitations — attach endpoint broken, search by ID unsupported, UUID not in create response 2026-04-13 14:14:39 -06:00
jramos
d24b45b404 fix: disable attach-to-existing endpoint (Ivanti API returns 400), show redirect message instead 2026-04-13 14:10:55 -06:00
jramos
d64eb7eec4 fix: use 'file' field name with proper MIME type for attach endpoint 2026-04-13 14:07:13 -06:00
jramos
6cb65fddc1 fix: use ivantiFormPost with 'files' field name for attach endpoint (matches create) 2026-04-13 14:05:05 -06:00
jramos
0ca83c6736 fix: revert map to multipart-only, add attachment upload logging 2026-04-13 14:02:28 -06:00
jramos
06268880da fix: try JSON POST first for map endpoint, fall back to multipart on 500/415 2026-04-13 13:56:00 -06:00
jramos
b4f0ddcb78 fix: use JSON POST instead of multipart for Ivanti map endpoint 2026-04-13 13:55:15 -06:00
jramos
55e3e074a5 debug: log Ivanti map endpoint response details on failure 2026-04-13 13:30:10 -06:00
jramos
66bbeb84a5 fix: search by workflow name instead of numeric ID to resolve UUID 2026-04-13 13:16:09 -06:00
jramos
4578f8cd85 debug: log full Ivanti search response to diagnose UUID resolution 2026-04-13 13:10:31 -06:00
jramos
5469a86e6e debug: add logging to UUID resolver to identify correct field name from Ivanti search response 2026-04-13 13:02:08 -06:00
jramos
2b6db1f903 fix: resolve UUID for map/attach endpoints, fix attachment field name mismatch
- Add resolveWorkflowBatchUuid helper that searches Ivanti API for UUID by batch ID and caches it locally
- Use UUID resolver in findings and attachments endpoints instead of relying on stored UUID
- Store UUID on new FP creation by searching Ivanti after workflow batch is created
- Fix frontend attachment upload field name from 'files' to 'attachments' to match Multer config
2026-04-13 12:53:13 -06:00
jramos
7c97bc3a84 Fixed Multer config .array from files to attachements 2026-04-13 12:45:37 -06:00
jramos
835fbf26e7 fix: revert clickable workflow badges, fix migration default, auto-sync submission lifecycle status from Ivanti findings
- Revert workflow badge to static (non-clickable) — queue panel is the entry point
- Fix migration: use DEFAULT NULL for updated_at (SQLite disallows CURRENT_TIMESTAMP in ALTER TABLE)
- Add useMemo enrichment to cross-reference submission lifecycle_status with actual Ivanti workflow state from findings data
2026-04-13 12:39:47 -06:00
jramos
c4aaeff2a1 fixed const constraint default 2026-04-13 12:30:43 -06:00
jramos
df30430956 feat: add FP submission editing with lifecycle tracking, clickable workflow badges, and edit modal
- Add migration for lifecycle_status, batch UUID, updated_at columns and submission history table
- Add backend endpoints: GET/PUT/POST/PATCH for viewing, editing, adding findings/attachments, and status changes
- Add pure helpers: validateLifecycleTransition, mergeFindings, buildSubmissionHistoryEntry
- Add FpEditModal with tabbed UI (Details, Findings, Attachments, History)
- Make workflow badges clickable for Reworked/Rejected/Expired states with pencil icon
- Add submissions list section to QueuePanel with lifecycle status badges
- Wire state and data flow in ReportingPage for submissions fetch and edit callbacks
2026-04-13 12:27:56 -06:00
jramos
57f11c362b docs: update README with queue redirect, CVE tooltips, FP workflow submission, and missing migrations 2026-04-09 16:18:22 -06:00
jramos
4df83d36dd fix: include hostname overrides in all queue endpoint responses 2026-04-09 16:11:52 -06:00
jramos
0a7a7c2827 feat: add Ivanti Queue redirect for completed items 2026-04-09 16:01:36 -06:00
jramos
1963faf9b8 fix: queue now uses edited hostname override instead of original Ivanti value 2026-04-09 15:25:16 -06:00
jramos
9b36a58959 feat: add CVE tooltip on hover in Reporting Page
- Add GET /api/cves/:cveId/tooltip backend endpoint with description truncation
- Create CveTooltip portal component with caching, severity badges, and viewport-aware positioning
- Integrate tooltip into ReportingPage with 300ms hover delay on CVE badge spans
2026-04-09 14:42:23 -06:00
jramos
690c30aac0 feat: add hostname and IP display to Ivanti queue panel
- Add migration to add hostname column to ivanti_todo_queue table
- Update POST and batch POST endpoints to accept and store hostname
- Pass hostName from findings data when adding items to queue
- Display hostname and IP address in CARD queue section
- Display hostname and IP address in vendor (FP/Archer) queue sections
2026-04-09 11:56:56 -06:00
jramos
fc68097821 fix: remove dual-mode checkbox — clicks always toggle selection, no more popover on first click 2026-04-09 10:01:18 -06:00
jramos
d9fdaf5cbb fix: move selection useEffects after filtered/addPopover declarations to fix ReferenceError 2026-04-09 09:56:33 -06:00
jramos
cb3da6980c Merge feature/batchqueue into master — batch finding disposition 2026-04-09 09:50:24 -06:00
jramos
ccc3576706 feat: add batch finding disposition — multi-select findings and bulk add to Ivanti queue 2026-04-09 09:49:40 -06:00
jramos
5405926550 Merge feature/submit-workflow into master — Ivanti FP workflow submission 2026-04-08 12:45:28 -06:00
jramos
328e48ea8c fix: accept HTTP 202 as success from Ivanti workflow creation
Ivanti returns 202 (Accepted) for async job creation, not just 200/201.
2026-04-08 12:26:35 -06:00
jramos
41f9c35586 fix: correct subjectFilterRequest format and add Ivanti API docs
The subjectFilterRequest field requires a nested structure:
{ subject: 'hostFinding', filterRequest: { filters: [...] } }

Previous attempts with flat { filters: [] } or { subject, filters }
caused Ivanti to return 500. The filterRequest wrapper is required.

Also adds docs/ivanti-api-reference.md documenting all known endpoints,
field formats, and the subjectFilterRequest structure so we don't have
to reverse-engineer the Swagger again.
2026-04-08 12:20:09 -06:00
jramos
729dada05c fix: correct subjectFilterRequest format for Ivanti FP workflow API
The API expects { subject: 'hostFinding', filterRequest: { filters } }
not a flat filter object. Confirmed working via direct curl test —
workflow ID 33418832 created successfully.
2026-04-08 12:18:41 -06:00
jramos
5d417edf82 fix: align subjectFilterRequest with Ivanti search filter schema
Remove extra fields (orWithPrevious, implicitFilters, subject) that
aren't in the Swagger filter schema. Add projection and sort fields
to match the search endpoint format.
2026-04-08 12:08:08 -06:00
jramos
03e60c9daf fix: rewrite FP workflow to use Ivanti multipart/form-data API
The /workflowBatch/falsePositive/request endpoint expects
multipart/form-data with text fields (name, reason, description,
expirationDate, overrideControl, subjectFilterRequest, isEmptyWorkflow)
and inline file uploads — not a JSON body with separate attachment calls.

- Add ivantiFormPost() helper for mixed form fields + files
- Replace buildIvantiPayload with buildIvantiFormFields + buildSubjectFilterRequest
- Remove separate attachment upload loop (files sent inline)
- Update response handling for { id, created } shape
2026-04-08 10:18:45 -06:00
jramos
ee9403ab47 fix: correct Ivanti API endpoint paths for FP workflow creation and attachment
- Creation: /workflowBatch -> /workflowBatch/falsePositive/request
- Attachment: /workflowBatch/{id}/attachment -> /workflowBatch/falsePositive/{uuid}/attach
- Paths confirmed against platform4.risksense.com swagger spec
2026-04-08 10:08:14 -06:00
jramos
3d04cd393f fix: remove no-op status ternary, dead code, and redundant calls
- Fix copy-paste bug in ivantiFpWorkflow.js where both ternary branches
  returned 'partial'; simplified to direct assignment
- Remove unused shouldShowFpButton() from ReportingPage.js (canWrite
  from useAuth() is used instead)
- Hoist repeated isCreateFpButtonEnabled() calls into a single variable
  in QueuePanel render
2026-04-08 09:38:39 -06:00
jramos
382bc81a7e feat: add Ivanti FP workflow submission from Queue
- Add shared ivantiApi.js helper (ivantiPost + ivantiMultipartPost)
- Add ivantiFpWorkflow.js backend route with validation, Ivanti API
  workflow creation, attachment uploads, submission tracking, and audit
- Add add_fp_submissions_table.js migration
- Wire route into server.js at /api/ivanti/fp-workflow
- Add FpWorkflowModal component in ReportingPage.js with form fields,
  drag-and-drop file upload, progress indicator, and result views
- Add Create FP Workflow button to QueuePanel footer (editor/admin only)
- Refactor ivantiWorkflows.js and ivantiFindings.js to use shared helper
2026-04-07 16:20:24 -06:00
jramos
7302ece958 docs: add Upgrade section and Troubleshooting TOC link to README 2026-04-07 13:43:50 -06:00
jramos
80d80c099f docs: add NODE_ENV/Secure cookie warning and troubleshooting section to README 2026-04-07 12:09:27 -06:00
jramos
a2a43a8685 Merge maintenance/security-audit1: security audit remediation and README update 2026-04-07 11:31:41 -06:00
jramos
a711972054 docs: update README for group-based access control, security hardening, and current architecture
- Replace role-based docs with group-based (Admin, Standard_User, Leadership, Read_Only)
- Update API reference with correct group requirements and new endpoints (JIRA tickets, archive, todo-queue)
- Remove hardcoded default credentials from installation instructions
- Document SESSION_SECRET as required with generation instructions
- Add new migrations to install sequence (archive, timestamps, counts history, user_groups, created_by)
- Update architecture tree with new files (ivantiArchive, ComplianceChartsPanel, etc.)
- Update security model with rate limiting, sandbox iframe, rehype-sanitize, Content-Disposition sanitization
- Update database schema docs with created_by columns, user_group triggers, cascade deletes
- Fix middleware reference from requireRole to requireGroup
- Remove stale admin123 references throughout
2026-04-07 11:29:33 -06:00
jramos
8a6a3485e9 security: address audit findings C-4 through M-8
Critical:
- C-4: Add express-rate-limit to login (20 attempts/15min)
- C-5: Remove default credentials from LoginForm.js
- C-6: Add sandbox attribute to KB document iframe

High:
- H-2: Hard-fail on startup if SESSION_SECRET env var is missing
- H-6: Sanitize filenames in Content-Disposition headers
- H-7: Fix KB upload race condition — move file after DB insert succeeds
- H-8: Generate random admin password in setup.js instead of hardcoded
- H-9: Add rehype-sanitize to ReactMarkdown (requires npm install)

Medium:
- M-4: Fix loose equality (==) to strict (===) in users.js self-checks
- M-5: Add hostname format regex validation in compliance notes
- M-6: Fix vendor trim-before-validate in ivantiTodoQueue.js
- M-7: Sanitize original filename in compliance temp JSON
- M-8: Pull CSP frame-ancestors from CORS_ORIGINS env var

New dependencies needed:
- backend: express-rate-limit (npm install in root)
- frontend: rehype-sanitize (npm install in frontend/)
2026-04-07 10:23:10 -06:00
jramos
169a0d2337 Merge feature/usergroups: group-based access control (Admin, Standard_User, Leadership, Read_Only) 2026-04-07 10:11:21 -06:00
jramos
c50fc5d8a8 fix: address all 11 review items for group-based access control
Bugs fixed:
- knowledgeBase.js: logAudit calls converted from positional args to object signature
- archerTickets.js: targetType/targetId renamed to entityType/entityId
- server.js: single CVE delete now has cascade/compliance check for Standard_User

Unprotected endpoints secured:
- ivantiTodoQueue.js: POST/PUT/DELETE now require Admin or Standard_User
- ivantiFindings.js: PUT note and POST sync now require Admin or Standard_User
- compliance.js: POST notes now requires Admin or Standard_User
- ivantiWorkflows.js: POST sync now requires Admin or Standard_User
- auth.js: cleanup-sessions now requires Admin via requireAuth + requireGroup

Additional fixes:
- ExportsPage.js: canExport() guard blocks Read_Only users
- knowledgeBase.js: Standard_User delete checks created_by ownership
- Migration: added INSERT/UPDATE triggers to enforce valid user_group values
2026-04-07 10:09:18 -06:00
jramos
e9e2c0961d fix: address all 11 review items for group-based access control
Bugs fixed:
- knowledgeBase.js: logAudit calls converted from positional args to object signature
- archerTickets.js: targetType/targetId renamed to entityType/entityId
- server.js: single CVE delete now has cascade/compliance check for Standard_User

Unprotected endpoints secured:
- ivantiTodoQueue.js: POST/PUT/DELETE now require Admin or Standard_User
- ivantiFindings.js: PUT note and POST sync now require Admin or Standard_User
- compliance.js: POST notes now requires Admin or Standard_User
- ivantiWorkflows.js: POST sync now requires Admin or Standard_User
- auth.js: cleanup-sessions now requires Admin via requireAuth + requireGroup

Additional fixes:
- ExportsPage.js: canExport() guard blocks Read_Only users
- knowledgeBase.js: Standard_User delete checks created_by ownership
- Migration: added INSERT/UPDATE triggers to enforce valid user_group values
2026-04-07 09:52:26 -06:00
jramos
d910af847e fix: wire up admin page route to render UserManagement component 2026-04-06 16:25:59 -06:00
jramos
73fd747576 feat: implement group-based access control (Admin, Standard_User, Leadership, Read_Only)
- Add user_group migration and created_by column migration
- Replace requireRole middleware with requireGroup
- Update all backend routes to use group-based authorization
- Add Standard_User conditional delete with ownership, state, and compliance checks
- Add cascade impact check for CVE deletes
- Update AuthContext with group-based permission helpers
- Update all frontend components for group-based rendering
- Update UserManagement UI with group dropdown, confirmation dialogs, self-demotion prevention
2026-04-06 16:18:07 -06:00
1ef57b0504 feat(archive): add finding archive tracking to Ivanti sync pipeline
Adds a four-state lifecycle tracker (ACTIVE → ARCHIVED → RETURNED → CLOSED)
to detect and monitor findings that disappear from Ivanti sync results due to
severity score drift rather than actual remediation.

- Archive detection runs automatically after each sync, comparing previous
  and current finding sets to identify disappearances and reappearances
- Full transition history stored in ivanti_finding_archives and
  ivanti_archive_transitions tables with timestamps and severity scores
- Three new API endpoints: /api/ivanti/archive, /api/ivanti/archive/stats,
  /api/ivanti/archive/:findingId/history
- Archive Summary Bar UI on the home page shows counts for each state
  (Active, Archived, Returned, Closed) with click-through finding lists
- Two new migrations: add_finding_archive_tables, add_archer_tickets_timestamps
- Mermaid diagram support added to Knowledge Base viewer
2026-04-06 09:51:56 -06:00
jramos
d1fe0bf455 fix: resolve 5 pre-merge issues in finding archive tracking
1. ACTIVE state never populated — stats endpoint now computes ACTIVE from live findings cache count instead of querying archive table

2. CHECK constraint mismatch — migration now uses 3-state constraint (ARCHIVED, RETURNED, CLOSED) matching runtime initArchiveTables()

3. Archive filter click non-functional — handleArchiveStateClick now fetches and renders filtered archive list below summary bar

4. Hook glob pattern mismatch — changed **/migrate*.js to **/migrations/*.js so hook fires for actual migration filenames

5. Stale stats after sync — ArchiveSummaryBar polls every 60s and refreshes immediately after workflow sync via refreshKey prop
2026-04-03 15:51:18 -06:00
jramos
3f7887eba6 added hooks 2026-04-03 15:29:05 -06:00
jramos
9bd5a52661 feat: implement finding archive tracking system
- Add migration script for ivanti_finding_archives and ivanti_archive_transitions tables
- Add archive detection logic (detectArchiveChanges, detectClosedFindings) in sync pipeline
- Add archive API router with list, stats, and history endpoints at /api/ivanti/archive
- Add ArchiveSummaryBar UI component with four state cards (ACTIVE, ARCHIVED, RETURNED, CLOSED)
- Integrate ArchiveSummaryBar into Ivanti findings page in App.js
- Register archive router in server.js
2026-04-03 15:20:04 -06:00
jramos
2b4ec5d8e2 added kiro specs 2026-04-03 13:48:04 -06:00
jramos
62592e9821 add kiro steering files 2026-04-03 09:27:12 -06:00
2fead2cfef feat(kb): render Mermaid diagrams in Knowledge Base viewer
Installs mermaid v11 and adds a custom ReactMarkdown code renderer
that intercepts fenced mermaid blocks and renders them as SVG diagrams
using the dark theme. SVGs are made responsive (width: 100%).
Non-mermaid code blocks are unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 15:37:00 -06:00
7c0ba41514 fix(migrations): add created_at/updated_at to archer_tickets if missing
Production instances where the table was created before these columns
were added to the schema will see 500 errors on all /api/archer-tickets
endpoints. This migration safely checks PRAGMA table_info before each
ALTER TABLE so it is idempotent and safe to run multiple times.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 15:23:38 -06:00
9c6c03a518 feat: time-based charts, Vulnerability Triage rename, Knowledge Base page
Merges feature/compliance-time-charts into master.

Changes included:
- Compliance page: 6 Recharts trend charts (active totals, deltas, per-team,
  MTTR, recurring items, Archer pipeline)
- Ivanti findings trend chart on Vulnerability Triage page: open/closed
  counts history stored on every sync, aggregated to end-of-day snapshots
- Rename 'Reporting' page to 'Vulnerability Triage' throughout (nav, routes,
  docs, all cross-page navigation references)
- Knowledge Base page: full article library with category filter, search,
  inline viewer, upload/delete for editor+ roles
- Remove Knowledge Base sidebar panel from home page (now lives on KB page);
  home layout adjusts to 2-column (9+3)
- Add ivanti_counts_history migration script for documentation consistency
- Update security-posture-workflow-diagrams.md and team-training-agenda.md
  to reflect Vulnerability Triage page name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 14:53:13 -06:00
0d48c109b3 refactor(home): remove Knowledge Base panel from home page
The dedicated Knowledge Base page now provides the full library
experience. Remove the KB sidebar panel, viewer inline embed,
upload modal, and all supporting state/functions from App.js.

Home page layout adjusts from 3-column to 2-column (9+3 grid):
main CVE content expands to col-span-9, right panel unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 14:32:36 -06:00
18ad31228e feat(kb): build Knowledge Base page
Replaces the 'coming soon' placeholder with a full library UI.
No backend changes needed — all existing endpoints and components
(KnowledgeBaseViewer, KnowledgeBaseModal) are reused.

Features:
  - Article card grid (responsive auto-fill, min 240px per card)
  - Category filter tabs (Procedure, Guide, Policy, Reference, General)
    with live article counts; tabs only shown for populated categories
  - Search bar — filters by title and description, client-side
  - Inline viewer — clicking a card opens KnowledgeBaseViewer below
    the grid; clicking again or pressing the close button collapses it
  - Upload modal (editor/admin only) refreshes the grid on success
  - Delete button on each card (editor/admin only) with confirmation
  - Graceful empty states for no articles and no search results
  - Loading and error states with retry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 13:55:51 -06:00
3dcb91a1fc chore(migrations): add migration script for ivanti_counts_history table
Standalone migration script for consistency with other files in
backend/migrations/. The table is also created automatically at server
startup via CREATE TABLE IF NOT EXISTS in initTables() so no manual
step is required on the dev server — just restart the backend after
pulling.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 10:17:33 -06:00
5102a2c5b4 docs: update 'Reporting page' references to 'Vulnerability Triage'
Updated all human-readable references in documentation to reflect the
page rename. File path citations in security-audit-2026-04-01.md
(ReportingPage.js:51) are left unchanged as the file itself was not
renamed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 10:15:51 -06:00
a0a8979c63 fix(triage): fix missed setCurrentPage('reporting') in Archer ticket filter button
One reference to the old page ID was missed in the previous rename commit.
The Archer ticket EXC filter button in App.js was still navigating to
'reporting', which would silently fail to navigate. Updated to 'triage'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 10:13:57 -06:00
15ad207464 feat(triage): Ivanti findings trend chart + rename Reporting to Vulnerability Triage
Add time-based open/closed tracking for Ivanti findings (Tier 2 from
the reporting recommendations doc) and rename the Reporting page to
Vulnerability Triage to better reflect its purpose.

Backend — ivantiFindings.js:
  - Create ivanti_counts_history table (appended on every sync, never
    overwritten — Option B from design discussion)
  - INSERT snapshot after each successful syncClosedCount() call
  - GET /api/ivanti/findings/counts/history endpoint — returns last
    snapshot per calendar day using ROW_NUMBER window function, so
    multiple daily syncs collapse to the end-of-day value

Frontend:
  - New IvantiCountsChart component: collapsible dual-line chart
    (open vs closed) with dark tooltip, delta label showing change
    since previous day, and graceful no-data states
  - Chart placed between the donut metrics panel and the findings table
    on the Vulnerability Triage page
  - Renamed page: 'reporting' → 'triage' (page ID, nav label, component
    export, all cross-file references)
  - ComplianceDetailPanel "View in Reporting" link updated to "View in
    Triage" and navigates to the correct page ID

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 10:12:04 -06:00
b111273e5a feat(compliance): add time-based trend charts to Compliance page
Add 6 Recharts charts in a collapsible Historical Trends panel on the
Compliance page, covering all Tier-1 recommendations from the reporting
design doc.

Backend — 5 new API endpoints:
  - GET /api/compliance/trends        — active totals + per-team counts per upload
  - GET /api/compliance/mttr          — mean days to resolution per team
  - GET /api/compliance/top-recurring — most persistent active findings by seen_count
  - GET /api/compliance/category-trend — category breakdown per upload (future use)
  - GET /api/archer-tickets/status-trend — ticket pipeline by creation date + status

Frontend — new ComplianceChartsPanel component:
  - Active Findings Over Time (multi-line: total + per-team dashed)
  - Change per Report Cycle (stacked bar: new/recurring + resolved)
  - Team Compliance Health (multi-line per team)
  - Mean Time to Resolution (horizontal bar per team)
  - Most Persistent Findings (horizontal bar top-10 by seen_count)
  - Archer Exception Pipeline (stacked bar by date + status)

All charts degrade gracefully to a no-data placeholder until uploads
accumulate. Panel is collapsible to stay out of the way when not needed.
Adds recharts dependency to frontend.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 09:49:32 -06:00
a7c74f625f docs: clarify Python deps use apt packages not pip/venv
Dev server uses apt-managed python3-pandas and python3-openpyxl.
Production fix is the same. Updates README install step and rewrites
python-venv-setup.md to reflect the real setup with venv as fallback.
2026-04-01 13:07:27 -06:00
8aef51b59a fix(compliance): use PYTHON_BIN env var for venv support
Modern Debian/Ubuntu enforces PEP 668 which blocks system-wide pip
installs. The backend now reads PYTHON_BIN from the environment
(defaulting to 'python3') so each server can point to a venv.
Updates README with venv setup instructions.
2026-04-01 12:47:50 -06:00
d0087ba9b7 docs: remove all weekly reports references
Weekly report feature was removed previously. Cleans up all remaining
references from README, architecture diagram, and deletes
WEEKLY_REPORT_FEATURE.md entirely.
2026-04-01 12:42:56 -06:00
3d6062f3fa docs: refresh README and add security posture workflow diagrams
- Rename project to STEAM Security Dashboard throughout README
- Document Ivanti Queue feature (FP/Archer/CARD staging, per-user persistence)
- Document AEO Compliance page (upload flow, metric health cards, device
  table, detail panel, View in Reporting link for 2.3.x metrics)
- Add all missing migrations to install instructions (queue, CARD,
  ip_address, compliance tables)
- Add Ivanti Queue and Compliance endpoint tables to API reference
- Update architecture file tree with new routes, migrations, scripts,
  and frontend components
- Add compliance DB tables to schema section
- Document parse_compliance_xlsx.py in scripts section
- Add security-posture-workflow-diagrams.md (Mermaid, VSCode/GitHub)
- Add security-posture-workflow-lucidchart.md (Lucidchart import format)
2026-04-01 10:46:39 -06:00
7af44608d0 feat(compliance): add 'View in Reporting' link for 2.3.x Ivanti metrics
In ComplianceDetailPanel, active metrics with a metric_id starting with
'2.3' and an Ivanti_Vulnerability_ID in extra_json now surface the ID
prominently alongside a 'View in Reporting →' button. Clicking navigates
directly to the Reporting page. onNavigate prop threaded through
App → CompliancePage → ComplianceDetailPanel → MetricRow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 09:20:30 -06:00
3bb86e8369 fix(compliance): remove unused vars flagged by eslint build 2026-03-31 15:20:28 -06:00
4676279a72 feat(compliance): add AEO compliance frontend
- CompliancePage: team tabs (STEAM/ACCESS-ENG), metric health cards with
  click-to-filter, device table with Active/Resolved tabs, hostname search,
  seen-count badges, notes indicator, empty/loading/error states
- ComplianceUploadModal: phased flow (idle→upload→preview→commit→done),
  drag-and-drop xlsx drop zone, diff summary before commit
- ComplianceDetailPanel: slide-out panel with failing metrics, surfaced
  extra fields (CVEs, SLA, OS, Splunk), upload history, notes timeline,
  per-metric note add with Ctrl+Enter submit
- NavDrawer: add Compliance nav item (teal, ShieldCheck icon)
- App.js: import and render CompliancePage on compliance route
- Fix SQL join bug in compliance route (lu ON upload_id = lu.id)
- Fix groupByHostname to use max last_seen across all metric rows
2026-03-31 15:14:51 -06:00
d3d86ddcf2 feat(compliance): add AEO compliance tracking backend
- Migration: compliance_uploads, compliance_items, compliance_notes tables
  with indexes on (hostname, metric_id) identity key and team/status
- Python parser (parse_compliance_xlsx.py): reads NTS_AEO xlsx, extracts
  non-compliant assets from all detail sheets, parses Summary sheet for
  metric health data and overall scores, outputs JSON to stdout
- Route (/api/compliance): preview/commit upload flow with diff summary,
  items endpoint grouped by hostname with seen_count tracking, metric
  summary endpoint for health cards, notes endpoints keyed on
  (hostname, metric_id) persisting across uploads
- server.js: register compliance router at /api/compliance
- .gitignore: exclude planning docs and xlsx source files
2026-03-31 15:06:59 -06:00
558c65807d docs: add security posture workflow process guide
Comprehensive team-facing process documentation covering the full host
finding review workflow, vulnerability designations, Ivanti queue usage,
workflow status colour codes, and quick reference tables.

Synthesises the skeletal Security posture workflow.md, the MOP colour
codes doc, and current dashboard feature set into a single guide suitable
for Confluence/internal publishing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 14:02:42 -06:00
518cb0a849 fix(migrations): include ip_address in add_card_workflow_type table recreate
The column was missing from the new table definition, causing the
INSERT...SELECT to fail on any DB that already had ip_address (e.g.
auto-created by the updated server.js).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 16:12:24 -06:00
b0adfa1bda feat(reporting): Ivanti queue panel for batch FP/Archer/CARD staging
Adds a persistent per-user staging queue on the Reporting page so
analysts can tag findings during review and batch-process Ivanti
workflows in one focused session.

Features:
- Checkbox column on findings table to tag rows into the queue
- Add-to-queue popover: vendor input, FP / Archer / CARD workflow toggle
  (CARD skips vendor requirement and stores IP address instead)
- Queue slide-out panel (420px, CSS transition) with items grouped by
  vendor; CARD items are their own top section showing IP address
- Per-item complete toggle, individual delete, and multi-select bulk delete
- Clear Completed footer button
- Queue button in header with live pending-count badge
- All data DB-backed (ivanti_todo_queue table, per-user scoped)
- Popover flips above row when near bottom of viewport

Migrations required on existing DBs:
  node backend/migrations/add_ivanti_todo_queue_table.js   (or let server auto-create)
  node backend/migrations/add_card_workflow_type.js
  node backend/migrations/add_todo_queue_ip_address.js

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 16:08:21 -06:00
7a2c56a11f fix(reporting): visible queue checkbox + multi-select delete
Table: removed disabled={queued} from the row checkbox so accentColor
renders properly — checked rows now show a solid blue tick instead of
the greyed-out browser default.

Queue panel: each item now has a small red selection checkbox (opacity
0.35 when idle, full when selected). Selecting any items reveals a red
'Delete (N)' button in the footer alongside 'Clear Completed'. Bulk
deletes run in parallel; selection state is automatically pruned when
items are removed via the individual trash button.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 15:43:43 -06:00
89b1f57ef4 feat(reporting): store and display IP address on CARD queue items
Adds ip_address column to ivanti_todo_queue so CARD entries carry the
host IP needed to locate the asset in CARD.

- Migration: ALTER TABLE ADD COLUMN ip_address TEXT (safe to re-run)
- Backend: accepts ip_address in POST body, stores up to 64 chars
- Frontend: captures finding.ipAddress when adding to queue; CARD items
  in the queue panel show the IP in green instead of the CVE list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 15:01:32 -06:00
6bf6371e51 feat(reporting): CARD workflow needs no vendor + own queue section
CARD workflow type no longer requires a vendor/platform entry since
asset disposition is handled entirely within CARD. In the popover the
vendor field is replaced with a note when CARD is selected, and the
Add button is enabled immediately.

In the queue panel, CARD items are separated into their own top section
(green header) rather than being mixed into vendor groups.

Backend validation updated to skip vendor requirement for CARD.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 14:52:06 -06:00
4d472b0aef fix(reporting): smart-flip queue popover + add CARD workflow type
Popover now flips above the row when it would overflow the bottom of the
viewport, and clamps horizontally to stay within the window.

Adds CARD as a third workflow type (for out-of-team asset disposition in
CARD) alongside FP and Archer. CARD is styled in green (#10B981) across
the popover toggle and queue panel badge.

DB: new migration (add_card_workflow_type.js) recreates ivanti_todo_queue
with an updated CHECK constraint to allow 'CARD'; run manually on dev.
App-level validation in the route is updated to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 14:46:59 -06:00
887d11610e feat(reporting): add Ivanti queue panel for batch FP/Archer staging
Adds a persistent per-user staging queue so analysts can tag findings
during review and batch-process Ivanti workflows in one focused session.

Backend:
- New ivanti_todo_queue table (user-scoped, vendor, workflow_type, status)
- Table auto-created on server startup via idempotent CREATE IF NOT EXISTS
- New route /api/ivanti/todo-queue: GET, POST, PUT/:id, DELETE/:id,
  DELETE/completed — all scoped to req.user.id

Frontend (ReportingPage):
- Fixed checkbox column on findings table; clicking opens an add-to-queue
  popover (portal) with vendor input and FP/Archer toggle
- Already-queued rows show checked/disabled checkbox
- Queue slide-out panel (420px fixed, CSS transition) with items grouped
  by vendor, per-item complete toggle + delete, Clear Completed footer
- Queue button in header with live pending-count badge

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 14:10:53 -06:00
1520cc994b chore: remove AI tooling config from repo tracking
Untrack .claude/ directory and update .gitignore to keep AI-specific
config files out of the repository before handoff.
2026-03-26 11:26:52 -06:00
906066c7fa feat(exports): build Exports page with 5 export cards
Replaces the placeholder with a fully functional exports page.

Backend:
- Add GET /api/cves/compliance endpoint reading from cve_document_status view

Frontend (ExportsPage.js):
1. Ivanti Host Findings — 4 sub-exports:
   - Full dump (all findings, all columns)
   - Pending Action (no FP# and no EXC in notes)
   - Overdue SLA (past due date or OVERDUE SLA status)
   - By Business Unit (multi-sheet XLSX, one sheet per BU)

2. FP Workflow Summary — one row per unique FP# ticket ID with state,
   finding count, affected hosts, BUs, and CVEs

3. CVE Database — status filter dropdown + CSV and XLSX format options

4. Archer Tickets — full EXC ticket list with linked CVEs and URLs

5. Document Compliance Report — per CVE/vendor doc coverage with
   "missing only" toggle to generate a gap list

All exports are lazy (data fetched on click), per-button loading states,
global dismissable error banner, auto-fit column widths in XLSX outputs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 11:39:26 -06:00
b58bd0650a docs: comprehensive README rewrite for current feature set
Rewrites the README from scratch to reflect the full current state of the
application. Major additions over the previous version:

- Ivanti/RiskSense integration: env vars, sync behaviour, findings cache
- Reporting page: all 4 donut charts, findings table columns, column
  management, per-column filtering (including empty-cell filter),
  inline hostname/DNS overrides, inline notes, CSV/XLSX export
- FP workflow tracking: finding vs ticket count distinction, closed-finding
  sweep for Approved FPs
- import_notes_from_csv.py script documentation with usage/args
- Full API reference updated with all Ivanti findings endpoints
- Architecture diagram updated with new route and component files
- Database schema updated with all Ivanti tables and new columns
- Migrations section updated with two new Ivanti migration scripts
- Configuration section updated with all IVANTI_* env vars

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 09:05:16 -06:00
ae04bc981e feat(reporting): add empty-cell option to column filters
Columns that contain any blank values now show a '— empty —' entry at the
top of the filter dropdown. Selecting only that entry shows findings with
nothing in that column (e.g. workflow with no FP# ticket assigned).

Uses an EMPTY_SENTINEL constant ('__EMPTY__') in the filter Set so blank
cells are handled distinctly from non-blank values. Works for both
single-value and multi-value (CVEs) columns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 13:27:16 -06:00
7314dc16cb feat(reporting): split FP charts into per-finding and per-ticket donuts
Renamed the existing FP chart to "FP Finding Status" (counts findings per
workflow state) and added a new "FP Workflow Status" chart that counts
unique FP# ticket IDs per state — so 10 findings under one FP# ticket
counts as 1 ticket, not 10.

Backend: extractFPWorkflow now returns { id, state }; syncFPWorkflowCounts
builds both a finding-count map and a deduped FP# ID map, storing them in
separate columns (fp_workflow_counts_json, fp_id_counts_json). The endpoint
returns findingCounts/findingTotal and idCounts/idTotal.

Frontend: FPWorkflowDonut accepts a centerLabel prop; both donuts share the
same component fed with their respective data slices from the single fetch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 12:13:13 -06:00
602c75bf24 fix(reporting): source FP workflow status chart from DB instead of open-findings cache
The FP Workflow Status donut was reading from the in-memory open findings
array, so Approved FPs (which close the finding and remove it from the
open cache) were invisible.

Backend: during each sync, compute FP workflow state counts from open
findings then sweep all pages of closed findings to capture Approved
(and any other closed-state) FP workflows. Counts are stored in a new
fp_workflow_counts_json column on ivanti_counts_cache and exposed via
GET /api/ivanti/findings/fp-workflow-counts.

Frontend: FPWorkflowDonut now receives counts/total props from the new
endpoint (fetched on load and refreshed after manual sync) instead of
deriving them from the findings prop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 11:43:57 -06:00
706ef19872 feat(reporting): add FP Workflow Status donut chart to Metrics panel
Adds a new SVG donut chart showing the distribution of FP workflow states
(Actionable, Requested, Reworked, Approved, Rejected, Expired, Unknown)
for all findings that have an associated FP# workflow ticket.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 11:16:01 -06:00
8392124df5 fix(scripts): skip notes for finding IDs not in active cache
If a finding ID from the CSV isn't in ivanti_findings_cache it is now
silently skipped (resolved or outdated) rather than stored. Also aborts
early with a clear message if the cache is empty, prompting the user to
run a Sync first.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 17:43:44 -06:00
fbe4333e9b feat(scripts): add import_notes_from_csv.py for mass note import
Reads a CSV with ID and NOTES columns, matches finding IDs against
the cache, and upserts notes into ivanti_finding_notes. Supports
--dry-run for previewing changes, warns on unknown IDs, truncates
notes over 255 chars, and skips unchanged rows.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 17:41:33 -06:00
07894709ba feat(reporting): inline editable hostname and DNS with persistent overrides
Backend:
- New ivanti_finding_overrides table (finding_id, field, value) with
  UNIQUE(finding_id, field) — same survival-across-sync pattern as notes
- PUT /api/ivanti/findings/:id/override (editor/admin only) — saves or
  clears a field override; empty value = revert to Ivanti
- Overrides merged into findings at read time via readOverrides()
- Whitelisted fields: hostName, dns

Frontend:
- OverrideCell component — click to edit inline (editor/admin only),
  Enter/blur to save, Escape to cancel
- Amber dot indicator on cells with an active local override
- Hover tooltip shows original Ivanti value when overridden
- RotateCcw button reverts cell back to Ivanti value in one click
- canWrite() gating via useAuth — viewers see the value, can't edit

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 15:39:37 -06:00
071aef96a1 feat(reporting): Action Coverage chart + Archer Exception linking
Replace FP# Workflow chart with a 3-segment Action Coverage donut:
  - FP Request  — finding has an Ivanti FP# workflow
  - Archer Exception — note matches EXC-\d+ pattern
  - Pending — no action taken yet

Clicking a segment filters the findings table to that category with a
colored badge in the action bar (click again or ×  to clear).

Home page: each Archer ticket now has a filter icon button that navigates
directly to the Reporting page pre-filtered to findings whose notes
reference that EXC number. The EXC badge appears in the table action bar
with a one-click clear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 13:06:54 -06:00
a9404ff82a feat(reporting): add FP# workflow status donut chart to Metrics panel
Adds a second SVG donut chart showing the distribution of FP# workflow
states (Expired, Rejected, Reworked, Actionable, Requested, Approved,
No FP#) computed from the already-loaded findings array — no new API
calls or backend changes required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 12:50:15 -06:00
f24cdb5063 feat(reporting): add Open vs Closed donut chart to Metrics panel
Backend: adds ivanti_counts_cache table, fetches Closed count (page 0,
size 1) from Ivanti after each Open sync, and exposes GET /counts endpoint.

Frontend: replaces the Metrics placeholder with an SVG donut chart showing
Open vs Closed proportions with counts and percentages. Counts are fetched
on mount and refreshed after manual sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 12:23:05 -06:00
3e2546323e feat(reporting): add CSV and XLSX export to findings table
Adds an Export dropdown button to the Reporting page action bar.
Exports respect current filters, sort order, and column visibility.
CSV uses pure JS (UTF-8 BOM for Excel compatibility); XLSX uses SheetJS
with auto-fitted column widths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 12:08:20 -06:00
b1a21e8771 docs: Add MOP for Workflow column color codes
Method of Procedure explaining FP# badge states, color meanings,
required actions, decision flowchart, and quick reference card.
Intended for training NTS-AEO team members on the Reporting page.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 15:45:48 -06:00
bc9e223ab7 Workflow column: FP# only, urgency-based colors
- Backend: only extract FP# workflows; SYS# auto-generated tickets
  are no longer stored or shown (not actionable for triage purposes).
  Findings with no FP# ticket show blank in the workflow column.
- Frontend: recolor workflow badges by action urgency —
  Expired/Rejected = red (act now), Reworked/Actionable = amber
  (resubmit), Requested = blue (waiting on approval).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 15:36:02 -06:00
2d1acca990 Add Workflow column to Reporting page with FP# priority matching
- Backend: extractFinding now flattens all workflowDistribution buckets
  and prioritises FP# (False Positive) tickets over SYS# workflows.
  Falls back to workflowGeneratedNames for FP# IDs not yet in distribution.
- Frontend: Add Workflow column (sortable, filterable) with state-coloured
  badge (green=Approved, blue=Requested, amber=Reworked/Actionable,
  red=Rejected, grey=Expired/unknown).
- Bump localStorage key to v2 so the new column appears on all clients
  without needing a manual cache clear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 14:44:53 -06:00
9893460b64 feat(reporting): add Finding ID column
ID was already stored in the cache from f.id; exposed as a sortable
column (filterable: false — too many unique values to be useful as a filter).
Existing users get it appended to the end of their saved column order
via the loadColumnOrder merge logic; new users see it first.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 14:23:50 -06:00
51b1f99b3a feat(calendar): click due-date day to navigate to filtered Reporting view
- CalendarWidget accepts onDateClick prop; due-date cells are clickable
  with pointer cursor, red hover highlight, and updated tooltip
- App.js wires onDateClick: sets calendarFilter state and navigates to
  the Reporting page
- NavDrawer navigation to Reporting clears calendarFilter so it only
  applies on calendar-initiated navigation
- ReportingPage accepts filterDate prop; initializes columnFilters with
  { dueDate: Set([filterDate]) } so the view lands pre-filtered
- Existing Clear Filters button lets the user dismiss the filter normally

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 14:09:08 -06:00
669396f635 feat(calendar): live calendar with Ivanti due date indicators
- Replace hardcoded Feb 2024 static HTML with dynamic CalendarWidget component
- Auto-displays current month on load; prev/next chevron navigation
- Fetches /api/ivanti/findings on mount and builds a date→count map
- Days with findings due: date number rendered in red bold + red glowing dot below
- Today: sky-blue highlight + bold (combined with red if also a due date)
- Legend appears automatically when the displayed month has any due dates
- Tooltip on due-date cells shows count ("3 findings due")

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:44:44 -06:00
8b3ea22fa0 Merge feature/reporting-page: full-width layout + in-panel scroll
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:24:03 -06:00
75b8ecc61d fix(reporting): full-width layout and in-panel vertical scroll
- Reporting page breaks out of max-w-7xl container to use full viewport width
- Table body scrolls within the panel (maxHeight: calc(100vh - 420px)) so you
  no longer need to scroll the entire page to reach the horizontal scrollbar
- Column headers are sticky (position: sticky, top 0) with opaque background
  so they remain visible while scrolling vertically through findings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:23:56 -06:00
ade3cc25ad Merge feature/reporting-page: Add CVEs column
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:17:05 -06:00
3fd6158eb3 feat(reporting): add CVEs column from vulnerabilities.vulnInfoList
- Backend extracts cves[] array from f.vulnerabilities.vulnInfoList[].cve
- Frontend shows up to 2 CVE badges (purple) with "+N more" overflow tooltip
- Filter is multi-value aware: selecting a CVE matches any finding containing it
- FilterDropdown expands multi-value arrays into individual checkbox options
- Sort by CVE count (number of associated CVEs)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:17:01 -06:00
5bbaaf5918 Merge feature/reporting-page: BU Ownership column + column filters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:03:20 -06:00
1f36d302ea feat(reporting): add BU Ownership column and per-column Excel-style filters
- buOwnership field extracted from assetCustomAttributes['1550_host_1'][0]
  and stored in SQLite cache; badge-styled cell (sky=STEAM, amber=ACCESS-ENG)
- All columns except Notes get a funnel filter button in the header
- FilterDropdown uses ReactDOM.createPortal + fixed positioning to escape
  overflowX:auto clipping; shows unique value checkboxes with search input,
  Select All, Clear, and a selected/total count footer
- Severity filter groups by vrrGroup label (CRITICAL/HIGH) not numeric value
- columnFilters state gates a useMemo filtered array before sorting
- Active filter count shown in panel header with amber badge; Clear Filters
  button appears in the toolbar when any filters are active
- Empty Set filter (Clear All) hides all rows, consistent with Excel

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 13:03:17 -06:00
8697ba4ef3 Reporting page: add Due Date, column manager (hide/reorder), remove Discovered/Source
Backend:
- Extract dueDate from statusEmbedded.dueDate (strip time portion)
- Remove discoveredOn and source from extractFinding (not needed)

Frontend:
- Add Due Date column (color-coded: red=past due, amber=within 30d, gray=future)
- Remove Discovered and Source columns
- ColumnManager component: gear button opens popover with drag-to-reorder and
  eye toggle per column; column state persisted to localStorage
- Column order/visibility survives page refresh and syncs
- SortIcon, TableCell, NoteCell all driven by current visible column list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 12:47:11 -06:00
d3806e8ce3 Add Reporting page with Ivanti host findings table
Backend:
- New route /api/ivanti/findings (GET cached data, POST /sync, PUT /:id/note)
- Fetches all pages of hostFinding/search filtered to NTS-AEO groups, severity 8.5-9.9, Open state
- SQLite cache (ivanti_findings_cache) stores slimmed findings across syncs
- Separate ivanti_finding_notes table persists user notes by finding ID
- Daily auto-sync on startup + 24h interval, manual sync endpoint
- Notes capped at 255 chars server-side

Frontend (ReportingPage):
- Panel 1: Metric graphs placeholder (full width, amber theme)
- Panel 2: Sortable findings table (all columns click-to-sort with ASC/DESC toggle)
- Columns: Severity (color-coded badge), Title, Host, IP, DNS, SLA, Discovered, Last Found, Source, Notes
- Notes column: inline editable input, saves on blur via PUT endpoint
- Sync button with spinner, last-synced timestamp, error banner

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:56:37 -06:00
931c42faeb Merge feature/navigation: Add hamburger nav menu with 4-page navigation structure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:47:47 -06:00
ea3b72db5c Add hamburger nav menu with 4-page navigation structure
- NavDrawer component: slide-in left drawer with backdrop, matches dark theme
- Nav items: Home, Reporting, Knowledge Base, Exports with color-coded icons
- Active page highlighted with colored background + indicator dot
- Placeholder pages for Reporting (amber), Knowledge Base (green), Exports (purple)
- Stats bar and three-column layout conditionally render on Home page only
- currentPage state drives all page switching

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:47:03 -06:00
d63e7cc9b9 Merge feature/remove-weekly-reports: Remove weekly report functionality
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:36:55 -06:00
37e183543a Remove weekly report functionality
- Delete backend/routes/weeklyReports.js
- Delete backend/migrations/add_weekly_reports_table.js
- Delete backend/scripts/split_cve_report.py
- Delete backend/helpers/excelProcessor.js
- Delete frontend/src/components/WeeklyReportModal.js
- Remove import, state, button, and modal from App.js
- Remove route registration and require from server.js
- Drop weekly_reports table from SQLite database

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:32:39 -06:00
337ffe6f35 Merge feature/cleanup-branding: Rebrand dashboard header to STEAM Security Dashboard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:26:19 -06:00
08c8c8a2a1 Rebrand dashboard header to STEAM Security Dashboard
- Title: "CVE INTEL" → "STEAM Security Dashboard"
- Subtitle: "Threat Intelligence & Vulnerability Command Center" → "NTS Threat Intelligence and Metric Aggregation"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:25:21 -06:00
4ed7721a71 Merge feature/workflow: Add Ivanti Workflows panel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:22:43 -06:00
174 changed files with 95614 additions and 5223 deletions

View File

@@ -1,89 +0,0 @@
# Backend Agent — CVE Dashboard
## Role
You are the backend specialist for the CVE Dashboard project. You manage the Express.js server, SQLite database layer, API routes, middleware, and third-party API integrations (NVD, Ivanti Neurons).
## Project Context
### Tech Stack
- **Runtime:** Node.js v18+
- **Framework:** Express.js 4.x
- **Database:** SQLite3 (file: `backend/cve_database.db`)
- **Auth:** Session-based with bcryptjs password hashing, cookie-parser
- **File Uploads:** Multer 2.0.2 with security hardening
- **Environment:** dotenv for config management
### Key Files
| File | Purpose |
|------|---------|
| `backend/server.js` | Main API server (~892 lines) — routes, middleware, security framework |
| `backend/setup.js` | Fresh database initialization (tables, indexes, default admin) |
| `backend/helpers/auditLog.js` | Fire-and-forget audit logging helper |
| `backend/middleware/auth.js` | `requireAuth(db)` and `requireRole()` middleware |
| `backend/routes/auth.js` | Login/logout/session endpoints |
| `backend/routes/users.js` | User CRUD (admin only) |
| `backend/routes/auditLog.js` | Audit log retrieval with filtering |
| `backend/routes/nvdLookup.js` | NVD API 2.0 proxy endpoint |
| `backend/.env.example` | Environment variable template |
### Database Schema
- **cves**: `UNIQUE(cve_id, vendor)` — multi-vendor support
- **documents**: linked by `cve_id + vendor`, tracks file metadata
- **users**: username, email, password_hash, role (admin/editor/viewer), is_active
- **sessions**: session_id, user_id, expires_at (24hr)
- **required_documents**: vendor-specific mandatory doc types
- **audit_logs**: user_id, username, action, entity_type, entity_id, details, ip_address
### API Endpoints
- `POST /api/auth/login|logout`, `GET /api/auth/me` — Authentication
- `GET|POST|PUT|DELETE /api/cves` — CVE CRUD with role enforcement
- `GET /api/cves/check/:cveId` — Quick check (multi-vendor)
- `GET /api/cves/:cveId/vendors` — Vendors for a CVE
- `POST /api/cves/:cveId/documents` — Upload documents
- `DELETE /api/documents/:id` — Admin-only document deletion
- `GET /api/vendors` — Vendor list
- `GET /api/stats` — Dashboard statistics
- `GET /api/nvd/lookup/:cveId` — NVD proxy (10s timeout, severity cascade v3.1>v3.0>v2.0)
- `POST /api/cves/nvd-sync` — Bulk NVD update with audit logging
- `GET|POST /api/audit-logs` — Audit log (admin only)
- `GET|POST|PUT|DELETE /api/users` — User management (admin only)
### Environment Variables
```
PORT=3001
API_HOST=<server-ip>
CORS_ORIGINS=http://<server-ip>:3000
SESSION_SECRET=<secret>
NVD_API_KEY=<optional>
IVANTI_API_KEY=<future>
IVANTI_CLIENT_ID=<future>
IVANTI_BASE_URL=https://platform4.risksense.com/api/v1
```
## Rules
### Security (MANDATORY)
1. **Input validation first** — Validate all inputs before any DB operation. Use existing validators: `isValidCveId()`, `isValidVendor()`, `VALID_SEVERITIES`, `VALID_STATUSES`, `VALID_DOC_TYPES`.
2. **Sanitize file paths** — Always use `sanitizePathSegment()` + `isPathWithinUploads()` for any file/directory operation.
3. **Never leak internals** — 500 responses use generic `"Internal server error."` only. Log full error server-side.
4. **Enforce RBAC** — All state-changing endpoints require `requireAuth(db)` + `requireRole()`. Viewers are read-only.
5. **Audit everything** — Log create/update/delete actions via `logAudit()` helper.
6. **File upload restrictions** — Extension allowlist + MIME validation. No executables.
7. **Parameterized queries only** — Never interpolate user input into SQL strings.
### Code Style
- Follow existing patterns in `server.js` for new endpoints.
- New routes go in `backend/routes/` as separate files, mounted in `server.js`.
- Use async/await with try-catch. Wrap db calls in `db.get()`, `db.all()`, `db.run()`.
- Keep responses consistent: `{ success: true, data: ... }` or `{ error: "message" }`.
- Add JSDoc-style comments only for non-obvious logic.
### Database Changes
- Never modify tables directly in route code. Create migration scripts in `backend/` (pattern: `migrate_<feature>.js`).
- Always back up the DB before migrations.
- Add appropriate indexes for new query patterns.
### Testing
- After making changes, verify the server starts cleanly: `node backend/server.js`.
- Test new endpoints with curl examples.
- Check that existing endpoints still work (no regressions).

View File

@@ -1,107 +0,0 @@
# Frontend Agent — CVE Dashboard
## Role
You are the frontend specialist for the CVE Dashboard project. You build and maintain the React UI, handle client-side state, manage API communication, and implement user-facing features.
**IMPORTANT:** When creating new UI components or implementing frontend features, you should use the `frontend-design` skill to ensure production-grade, distinctive design quality. Invoke this skill using the Skill tool with `skill: "frontend-design"`.
## Project Context
### Tech Stack
- **Framework:** React 18.2.4 (Create React App)
- **Styling:** Tailwind CSS (loaded via CDN in `public/index.html`)
- **Icons:** Lucide React
- **State:** React useState/useEffect + Context API (AuthContext)
- **API Communication:** Fetch API with credentials: 'include' for session cookies
### Key Files
| File | Purpose |
|------|---------|
| `frontend/src/App.js` | Main component (~1,127 lines) — CVE list, modals, search, filters, document upload |
| `frontend/src/index.js` | React entry point |
| `frontend/src/App.css` | Global styles |
| `frontend/src/components/LoginForm.js` | Login page |
| `frontend/src/components/UserMenu.js` | User dropdown (profile, settings, logout) |
| `frontend/src/components/UserManagement.js` | Admin user management interface |
| `frontend/src/components/AuditLog.js` | Audit log viewer with filtering/sorting |
| `frontend/src/components/NvdSyncModal.js` | Bulk NVD sync (state machine: idle > fetching > review > applying > done) |
| `frontend/src/contexts/AuthContext.js` | Auth state + `useAuth()` hook |
| `frontend/public/index.html` | HTML shell (includes Tailwind CDN script) |
| `frontend/.env.example` | Environment variable template |
### Environment Variables
```
REACT_APP_API_BASE=http://<server-ip>:3001/api
REACT_APP_API_HOST=http://<server-ip>:3001
```
**Critical:** React caches env vars at build time. After `.env` changes, the dev server must be fully restarted (not just refreshed).
### API Base URL
All fetch calls use `process.env.REACT_APP_API_BASE` as the base URL. Requests include `credentials: 'include'` for session cookie auth.
### Authentication Flow
1. `LoginForm.js` posts credentials to `/api/auth/login`
2. Server returns session cookie (httpOnly, sameSite: lax)
3. `AuthContext.js` checks `/api/auth/me` on mount to restore sessions
4. `useAuth()` hook provides `user`, `login()`, `logout()`, `loading` throughout the app
5. Role-based UI: admin sees user management + audit log; editor can create/edit/delete; viewer is read-only
### Current UI Structure (in App.js)
- **Header**: App title, stats bar, Quick Check input, "Add CVE" button, "Sync with NVD" button (editor/admin), User Menu
- **Filters**: Search input, vendor dropdown, severity dropdown
- **CVE List**: Grouped by CVE ID, each group shows vendor rows with status badges, document counts, edit/delete buttons
- **Modals**: Add CVE (with NVD auto-fill), Edit CVE (with NVD update), Document Upload, NVD Sync
- **Admin Views**: User Management tab, Audit Log tab
## Rules
### Component Patterns
- New UI features should be extracted into separate components under `frontend/src/components/`.
- Use functional components with hooks. No class components.
- State that's shared across components goes in Context; local state stays local.
- Destructure props. Use meaningful variable names.
### Styling
- Use Tailwind CSS utility classes exclusively. No custom CSS unless absolutely necessary.
- Follow existing color patterns: green for success/addressed, yellow for warnings, red for errors/critical, blue for info.
- Responsive design: use Tailwind responsive prefixes (sm:, md:, lg:).
- Dark mode is not currently implemented — do not add it unless requested.
### API Communication
- Always use `fetch()` with `credentials: 'include'`.
- Handle loading states (show spinners), error states (show user-friendly messages), and empty states.
- On 401 responses, redirect to login (session expired).
- Pattern:
```js
const res = await fetch(`${process.env.REACT_APP_API_BASE}/endpoint`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'include',
body: JSON.stringify(data)
});
if (!res.ok) { /* handle error */ }
const result = await res.json();
```
### Role-Based UI
- Check `user.role` before rendering admin/editor controls.
- Viewers see data but no create/edit/delete buttons.
- Editors see create/edit/delete for CVEs and documents.
- Admins see everything editors see plus User Management and Audit Log tabs.
### File Upload UI
- The `accept` attribute on file inputs must match the backend allowlist.
- Current allowed: `.pdf,.doc,.docx,.xls,.xlsx,.ppt,.pptx,.txt,.csv,.json,.xml,.png,.jpg,.jpeg,.gif,.bmp,.tiff,.svg,.zip,.tar,.gz,.7z,.rar,.eml,.msg`
- Max file size: 10MB (enforced backend, show friendly message on 413).
### Code Quality
- No inline styles — use Tailwind classes.
- Extract repeated logic into custom hooks or utility functions.
- Keep components focused — if a component exceeds ~300 lines, consider splitting.
- Use `key` props correctly on lists (use unique IDs, not array indexes).
- Clean up useEffect subscriptions and timers.
### Testing
- After making changes, verify the frontend compiles: `cd frontend && npm start` (or check for build errors).
- Test in browser: check console for errors, verify API calls succeed.
- Test role-based visibility with different user accounts.

View File

@@ -1,138 +0,0 @@
# Security Agent — CVE Dashboard
## Role
You are the security specialist for the CVE Dashboard project. You perform code reviews, dependency audits, and vulnerability assessments. You identify security issues and recommend fixes aligned with the project's existing security framework.
## Project Context
### Application Profile
- **Type:** Internal vulnerability management tool (Charter Communications)
- **Users:** Security team members with assigned roles (admin/editor/viewer)
- **Data Sensitivity:** CVE remediation status, vendor documentation, user credentials
- **Exposure:** Internal network (home lab / corporate network), not internet-facing
### Tech Stack Security Surface
| Layer | Technology | Key Risks |
|-------|-----------|-----------|
| Frontend | React 18, Tailwind CDN | XSS, CSRF, sensitive data in client state |
| Backend | Express.js 4.x | Injection, auth bypass, path traversal, DoS |
| Database | SQLite3 | SQL injection, file access, no encryption at rest |
| Auth | bcryptjs + session cookies | Session fixation, brute force, weak passwords |
| File Upload | Multer | Unrestricted upload, path traversal, malicious files |
| External API | NVD API 2.0 | SSRF, response injection, rate limit abuse |
### Existing Security Controls
These are already implemented — verify they remain intact during reviews:
**Input Validation (backend/server.js)**
- CVE ID: `/^CVE-\d{4}-\d{4,}$/` via `isValidCveId()`
- Vendor: non-empty, max 200 chars via `isValidVendor()`
- Severity: enum `VALID_SEVERITIES` (Critical, High, Medium, Low)
- Status: enum `VALID_STATUSES` (Open, Addressed, In Progress, Resolved)
- Document type: enum `VALID_DOC_TYPES` (advisory, email, screenshot, patch, other)
- Description: max 10,000 chars
- Published date: `YYYY-MM-DD` format
**File Upload Security**
- Extension allowlist: `ALLOWED_EXTENSIONS` — documents only, all executables blocked
- MIME type validation: `ALLOWED_MIME_PREFIXES` — image/*, text/*, application/pdf, Office types
- Filename sanitization: strips `/`, `\`, `..`, null bytes
- File size limit: 10MB
**Path Traversal Prevention**
- `sanitizePathSegment(segment)` — strips dangerous characters from path components
- `isPathWithinUploads(targetPath)` — verifies resolved path stays within uploads root
**Authentication & Sessions**
- bcryptjs password hashing (default rounds)
- Session cookies: `httpOnly: true`, `sameSite: 'lax'`, `secure` in production
- 24-hour session expiry
- Role-based access control on all state-changing endpoints
**Security Headers**
- `X-Content-Type-Options: nosniff`
- `X-Frame-Options: DENY`
- `X-XSS-Protection: 1; mode=block`
- `Referrer-Policy: strict-origin-when-cross-origin`
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`
**Error Handling**
- Generic 500 responses (no `err.message` to client)
- Full errors logged server-side
- Static file serving: `dotfiles: 'deny'`, `index: false`
- JSON body limit: 1MB
### Key Files to Review
| File | Security Relevance |
|------|-------------------|
| `backend/server.js` | Central security framework, all core routes, file handling |
| `backend/middleware/auth.js` | Authentication and authorization middleware |
| `backend/routes/auth.js` | Login/logout, session management |
| `backend/routes/users.js` | User CRUD, password handling |
| `backend/routes/nvdLookup.js` | External API proxy (SSRF risk) |
| `backend/routes/auditLog.js` | Audit log access control |
| `frontend/src/contexts/AuthContext.js` | Client-side auth state |
| `frontend/src/App.js` | Client-side input handling, API calls |
| `frontend/src/components/LoginForm.js` | Credential handling |
| `.gitignore` | Verify secrets are excluded |
## Review Checklists
### Code Review (run on all PRs/changes)
1. **Injection** — Are all database queries parameterized? No string interpolation in SQL.
2. **Authentication** — Do new state-changing endpoints use `requireAuth(db)` + `requireRole()`?
3. **Authorization** — Is role checking correct? (admin-only vs editor+ vs all authenticated)
4. **Input Validation** — Are all user inputs validated before use? New fields need validators.
5. **File Operations** — Do file/directory operations use `sanitizePathSegment()` + `isPathWithinUploads()`?
6. **Error Handling** — Do 500 responses avoid leaking `err.message`? Are errors logged server-side?
7. **Audit Logging** — Are create/update/delete actions logged via `logAudit()`?
8. **CORS** — Is `CORS_ORIGINS` still restrictive? No wildcards in production.
9. **Dependencies** — Any new packages? Check for known vulnerabilities.
10. **Secrets** — No hardcoded credentials, API keys, or secrets in code. All in `.env`.
### Dependency Audit
```bash
# Backend
cd backend && npm audit
# Frontend
cd frontend && npm audit
```
- Flag any `high` or `critical` severity findings.
- Check for outdated packages with known CVEs: `npm outdated`.
- Review new dependencies: check npm page, weekly downloads, last publish date, maintainer reputation.
### OWASP Top 10 Mapping
| OWASP Category | Status | Notes |
|---------------|--------|-------|
| A01 Broken Access Control | Mitigated | RBAC + session auth on all endpoints |
| A02 Cryptographic Failures | Partial | bcrypt for passwords; no encryption at rest for DB/files |
| A03 Injection | Mitigated | Parameterized queries, input validation |
| A04 Insecure Design | Acceptable | Internal tool with limited user base |
| A05 Security Misconfiguration | Mitigated | Security headers, CORS config, dotfiles denied |
| A06 Vulnerable Components | Monitor | Run `npm audit` regularly |
| A07 Auth Failures | Mitigated | Session-based auth, bcrypt, httpOnly cookies |
| A08 Data Integrity Failures | Partial | File type validation; no code signing |
| A09 Logging & Monitoring | Mitigated | Audit logging on all mutations |
| A10 SSRF | Partial | NVD proxy validates CVE ID format; review for Ivanti integration |
## Output Format
When reporting findings, use this structure:
```
### [SEVERITY] Finding Title
- **Location:** file:line_number
- **Issue:** Description of the vulnerability
- **Impact:** What an attacker could achieve
- **Recommendation:** Specific fix with code example
- **OWASP:** Category reference
```
Severity levels: CRITICAL, HIGH, MEDIUM, LOW, INFO
## Rules
1. Never suggest disabling security controls for convenience.
2. Recommendations must be compatible with the existing security framework — extend it, don't replace it.
3. Flag any regression in existing security controls immediately.
4. For dependency issues, provide the specific CVE and affected version range.
5. Consider the threat model — this is an internal tool, not internet-facing. Prioritize accordingly.
6. When reviewing file upload changes, always verify both frontend `accept` attribute and backend allowlist stay in sync.
7. Do not recommend changes that would break existing functionality without a migration path.

View File

@@ -1,25 +0,0 @@
# Project Instructions
## Token Usage & Efficiency
Follow the guidelines in `.claude/optimization.md` for:
- When to use subagents vs main conversation
- Model selection (Haiku vs Sonnet)
- Token preservation strategies
- Rate limiting rules
## Project Context
This is a CVE (Common Vulnerabilities and Exposures) dashboard application for tracking security vulnerabilities, vendors, and JIRA tickets.
## Security Focus
All code changes should consider:
- Input validation
- SQL injection prevention
- XSS protection
- Authentication/authorization
## Frontend Development
When working on frontend features or UI components:
- Use the `frontend-design` skill for new component creation and UI implementation
- This skill provides production-grade design quality and avoids generic AI aesthetics
- Invoke it using: `Skill` tool with `skill: "frontend-design"`
- The skill will guide implementation with distinctive, polished code patterns

View File

@@ -1,143 +0,0 @@
OPTIMIZATION.md - Token Usage & Subagent Strategy
## SUBAGENT USAGE STRATEGY
Subagents run in separate contexts and preserve main conversation tokens.
### When to Use Subagents
**Use Subagents for:**
- Large-scale codebase exploration and analysis
- Complex multi-step investigations across many files
- Detailed code pattern searches and refactoring analysis
- Gathering comprehensive information before main conversation work
- When total tokens would exceed 30,000 in main conversation
**Keep in Main Conversation:**
- Direct file edits (1-3 files)
- Simple code changes and debugging
- Architecture decisions
- Security reviews and approvals
- User-facing responses and recommendations
- Questions requiring reasoning about codebase
- Frontend UI work (use `frontend-design` skill for new components)
### Subagent Types & When to Use
**Explore Agent** (Haiku 3.5)
- Codebase exploration and file discovery
- Pattern searching across large codebases
- Gathering information about file structure
- Finding references and relationships
**General-Purpose Agent** (Haiku 3.5)
- Multi-step code analysis tasks
- Summarizing findings from exploration
- Complex searches requiring multiple strategies
- Collecting data for main conversation decisions
---
## MODEL SELECTION STRATEGY
### Main Conversation (Sonnet 4.5)
- **Always use Sonnet 4.5 in main conversation**
- Direct file edits and modifications
- Architecture and design decisions
- Security analysis and approvals
- Complex reasoning and recommendations
- Final user responses
### Subagent Models
**Haiku 4.5** (Default for subagents)
- Code exploration and pattern searching
- File discovery and structure analysis
- Simple codebase investigations
- Gathering information and summarizing
- Task: Use Haiku first for subagent work
**Sonnet 4.5** (For subagents - when needed)
- Security-critical analysis within subagents
- Complex architectural decisions needed in exploration
- High-risk code analysis
- When exploration requires advanced reasoning
---
## RATE LIMITING GUIDANCE
### API Call Throttling
- 5 seconds minimum between API calls
- 10 seconds minimum between web searches
- Batch similar work whenever possible
- If you hit 429 error: STOP and wait 5 minutes
### Budget Management
- Track tokens used across all agents
- Main conversation should stay under 100,000 tokens
- Subagent work can extend to 50,000 tokens per agent
- Batch multiple subagent tasks together when possible
---
## TOKEN PRESERVATION RULES
### Best Practices for Long-Running Conversations
**In Main Conversation:**
1. Start with subagent for exploration (saves ~20,000 tokens)
2. Request subagent summarize findings
3. Use summary to inform main conversation edits/decisions
4. Keep main conversation focused on decisions and actions
**Information Gathering:**
- Use subagents to explore before asking for analysis in main conversation
- Have subagent provide condensed summaries (250-500 words max)
- Main conversation uses summary + provides feedback/decisions
**File Editing:**
- For <3 files: Keep in main conversation
- For 3+ files: Split between subagent (finding/analysis) and main (approval/execution)
- Simple edits (1-5 lines per file): Main conversation
- Complex refactoring (10+ lines per file): Subagent analysis + main approval
**Code Review Workflow:**
1. Subagent explores and analyzes code patterns
2. Subagent flags issues and suggests improvements
3. Main conversation reviews suggestions
4. Main conversation executes approved changes
### Token Budget Allocation Example
- Main conversation: 0-100,000 tokens (soft limit)
- Per subagent task: 0-50,000 tokens
- Critical work (security): Use Sonnet in main conversation
- Exploratory work: Use Explore agent (Haiku) in subagent
---
## DECISION TREE
```
Is this a direct file edit request?
├─ YES (1-3 files, <10 lines each) → Main conversation
├─ NO
└─ Is this exploratory analysis?
├─ YES (finding files, patterns) → Use Explore agent (Haiku)
├─ NO
└─ Is this complex multi-step work?
├─ YES (3+ steps, many files) → Use General agent (Haiku)
├─ NO
└─ Is this security-critical?
├─ YES → Main conversation (Sonnet)
└─ NO → Subagent (Haiku) or Main conversation
```
---
## SUMMARY
**Main Conversation (You):** Architecture, decisions, edits, reviews
**Subagents:** Exploration, analysis, information gathering
**Sonnet 4.5:** Security, complexity, final decisions
**Haiku 4.5:** Exploration, gathering, analysis support

Binary file not shown.

32
.gitignore vendored
View File

@@ -37,10 +37,38 @@ frontend.pid
# Temporary files # Temporary files
backend/uploads/temp/ backend/uploads/temp/
claude.md
claude_status.md
feature_request*.md feature_request*.md
# AI tooling config
.claude/
ai_notes.md
ai_status.md
backend/add_vendor_to_documents.js backend/add_vendor_to_documents.js
backend/fix_multivendor_constraint.js backend/fix_multivendor_constraint.js
backend/server.js-backup backend/server.js-backup
backend/setup.js-backup backend/setup.js-backup
# Compliance staging — keep folder, ignore contents
.compliance-staging/*
!.compliance-staging/.gitkeep
# Kiro agents (local only)
.kiro/
# Zip files
*.zip
# Production DB copies
cve_database_prod.db
cve_database.db.prod
cve_database.db.backup
database.db
# Operations — local admin records, UAT logs, firewall requests, data exports
docs/operations/
# Data exports — local spreadsheets
docs/data-exports/
# Python cache
__pycache__/

121
.gitlab-ci.yml Normal file
View File

@@ -0,0 +1,121 @@
# =============================================================================
# GitLab CI/CD Pipeline — STEAM Security Dashboard
# =============================================================================
#
# Pipeline stages:
# 1. install — install dependencies for backend and frontend
# 2. lint — run linters / static checks
# 3. test — run backend (Jest) and frontend (react-scripts) tests
# 4. build — produce the production frontend bundle
# 5. deploy — restart services on the local machine (manual trigger)
#
# Executor: shell (runs directly on dashboard-dev using system Node.js)
# Uses cache (not artifacts) for node_modules to avoid upload size limits.
# =============================================================================
# ---------------------------------------------------------------------------
# Global cache — persists node_modules between pipeline runs on this runner
# ---------------------------------------------------------------------------
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- frontend/node_modules/
# ---------------------------------------------------------------------------
# Stages run in order; jobs within a stage run in parallel
# ---------------------------------------------------------------------------
stages:
- install
- lint
- test
- build
- deploy
# =============================================================================
# STAGE 1: Install dependencies
# =============================================================================
install-backend:
stage: install
script:
- npm install
install-frontend:
stage: install
script:
- cd frontend
- npm install
# =============================================================================
# STAGE 2: Lint / static analysis
# =============================================================================
lint-frontend:
stage: lint
script:
- cd frontend
- npm install
- npx eslint src/ --max-warnings 0
allow_failure: true # non-blocking until the team cleans up existing warnings
# =============================================================================
# STAGE 3: Tests
# =============================================================================
test-backend:
stage: test
script:
- npm install
- npx jest --ci --forceExit --detectOpenHandles backend/__tests__/
timeout: 5 minutes
test-frontend:
stage: test
script:
- cd frontend
- npm install
- CI=true npx react-scripts test --watchAll=false --ci --forceExit
timeout: 5 minutes
allow_failure: true # 2 test suites have pre-existing ESM/env issues — fix separately
# =============================================================================
# STAGE 4: Build the production frontend bundle
# =============================================================================
build-frontend:
stage: build
script:
- cd frontend
- npm install
- CI=false REACT_APP_API_BASE=/api REACT_APP_API_HOST="" npm run build
artifacts:
paths:
- frontend/build/
expire_in: 7 days
# =============================================================================
# STAGE 5: Deploy
# =============================================================================
# Since the runner IS the app server (dashboard-dev), deploy just restarts
# the services locally. No SSH needed.
#
# Manual trigger only, and only from the main/master branch.
# =============================================================================
deploy:
stage: deploy
rules:
- if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "master"
when: manual
environment:
name: production
script:
- echo "Deploying on dashboard-dev..."
- cd /home/cve-dashboard
- git pull origin ${CI_COMMIT_BRANCH}
- npm install
- cd frontend && npm install && npm run build && cd ..
- ./stop-servers.sh || true
- ./start-servers.sh
- echo "Deploy complete."

59
CHANGELOG.md Normal file
View File

@@ -0,0 +1,59 @@
# Changelog
## v1.0.0 — 2026-05-01
First official release. Consolidates all features developed since initial commit into a stable, documented, deployment-ready package.
### Core Platform
- CVE tracking with multi-vendor support, document storage, and NVD API auto-fill
- Session-based authentication with four user groups (Admin, Standard_User, Leadership, Read_Only)
- Full audit logging of all state-changing actions
- Dark tactical intelligence UI theme with monospace typography
### Ivanti Integration
- Live sync of open host findings from Ivanti/RiskSense API (auto-sync every 24h)
- Reporting page with donut metric charts, advanced per-column filtering, inline editing
- FP workflow submission directly to Ivanti API with file attachments
- Ivanti Queue — personal staging list for batch FP, Archer, CARD, and Granite workflows
- Queue item redirect between workflow types after completion
- Row visibility controls with localStorage persistence
### Archive and Anomaly Tracking
- Automatic detection of disappeared and returned findings across syncs
- BU drift checker — classifies archived findings by reason (BU reassignment, severity drift, closed on platform, decommissioned)
- Return classification — explains why findings came back (BU reassigned back, severity re-escalated, etc.)
- Findings Trend chart with archive activity sparkline and shift reason tooltips
- Anomaly banner for significant archive events
### Compliance (AEO Posture)
- Weekly NTS_AEO xlsx upload with diff preview (new, resolved, recurring)
- Schema drift detection with breaking/silent-miss/cosmetic classification
- Admin config reconciliation for parser updates
- Per-team metric health cards with grouped categories and variant pills
- Device-level violation tracking with timestamped notes history
- Multi-metric note grouping
- Upload rollback support
### Integrations
- Jira Data Center — create, sync, and track tickets linked to CVE/vendor pairs
- Archer — risk acceptance exception tracking (EXC numbers)
- Atlas InfoSec — action plan cache, bulk creation from row selection, metrics reporting
- CARD API — Granite/CARD asset lookup for network device workflows
- NVD API — auto-fill CVE metadata with bulk sync support
### Knowledge Base
- Internal document library with inline PDF and Markdown rendering
- Category-based browsing and search
### Admin
- Full-page admin panel with user management, audit log, and system info tabs
- Themed confirm modals replacing browser dialogs
- User profile panel with self-service password change
### Infrastructure
- Consolidated `setup.js` with complete database schema (27 tables, all indexes and triggers)
- systemd service files for persistent deployment
- GitLab CI/CD pipeline (install, lint, test, build, deploy)
- GPG-signed commits for code provenance
- Organized documentation structure (api, design, guides, security, testing, troubleshooting)
- Migration scripts documented and retained for existing deployment upgrades

555
README.md
View File

@@ -1,515 +1,130 @@
# CVE Dashboard # STEAM Security Dashboard v1.0.0
A self-hosted vulnerability management dashboard for tracking CVE remediation status, maintaining vendor documentation, and managing risk acceptance workflows. A self-hosted vulnerability management dashboard for the NTS-AEO-STEAM and NTS-AEO-ACCESS-ENG business units. Centralises CVE tracking, Ivanti host finding triage, AEO compliance posture, FP/Archer/CARD exception workflows, and internal documentation in a single interface.
--- ## Quick Start
## Table of Contents ### Prerequisites
- [Overview](#overview) - Node.js 18+
- [Tech Stack](#tech-stack) - Python 3 with `python3-pandas` and `python3-openpyxl` (for compliance xlsx parsing)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Configuration](#configuration)
- [Running the Application](#running-the-application)
- [Features](#features)
- [API Reference](#api-reference)
- [Architecture](#architecture)
- [Database Schema](#database-schema)
- [Security Model](#security-model)
- [Migrations](#migrations)
--- ### Install
## Overview
The CVE Dashboard answers a common problem in vulnerability management: before requesting false positive designations, you need to know whether a CVE has already been addressed, and whether the supporting vendor documentation exists. This application provides:
- A searchable, filterable CVE list with per-vendor tracking
- Document storage attached to each CVE/vendor pair (advisories, emails, screenshots, patches)
- NVD API integration to auto-populate CVE metadata
- Archer risk acceptance ticket tracking (EXC numbers)
- Weekly vulnerability report upload and processing
- A knowledge base for internal documentation and policies
- Role-based access control with a full audit trail
---
## Tech Stack
| Layer | Technology |
|---|---|
| Backend | Node.js, Express 5 |
| Database | SQLite3 |
| File uploads | Multer 2 |
| Auth | bcryptjs, cookie-based sessions |
| Frontend | React 19, lucide-react, react-markdown |
| Report processing | Python 3 (pandas, openpyxl) |
---
## Prerequisites
- Node.js 18 or later
- npm
- Python 3 with pip (required only for weekly report processing)
---
## Installation
### 1. Clone the repository
```bash ```bash
git clone <repo-url> git clone <repo-url>
cd cve-dashboard cd cve-dashboard
```
### 2. Install backend dependencies # Backend dependencies
```bash
cd backend
npm install npm install
# Frontend dependencies
cd frontend && npm install && cd ..
# Python dependencies (Ubuntu/Debian)
apt install -y python3-pandas python3-openpyxl
``` ```
The root `package.json` lists the backend dependencies. Install them from the `backend/` directory where `server.js` lives. ### Configure
### 3. Install frontend dependencies
```bash ```bash
cd frontend cp backend/.env.example backend/.env
npm install # Edit backend/.env — at minimum set SESSION_SECRET:
# openssl rand -base64 32
``` ```
### 4. Install Python dependencies (for weekly report upload feature) See `backend/.env.example` for all available options including Ivanti API, Jira, and Atlas integration keys.
### Initialize Database
```bash ```bash
cd backend/scripts node backend/setup.js
pip install -r requirements.txt
``` ```
Required packages: `pandas>=2.0.0`, `openpyxl>=3.0.0` Creates the database with the complete schema and prints a one-time admin password. Save it.
### 5. Initialize the database ### Build and Run
Run this once from the `backend/` directory to create the SQLite database, all tables, indexes, the uploads directory, and a default admin user:
```bash ```bash
cd backend # Build frontend
node setup.js cd frontend && npm run build && cd ..
# Start servers
./start-servers.sh
``` ```
This creates `backend/cve_database.db` and a default admin account: Dashboard: http://localhost:3000 · API: http://localhost:3001
- Username: `admin`
- Password: `admin123`
**Change the admin password immediately after first login.** For persistent deployments, use the systemd services in `systemd/`. See the full manual for setup instructions.
### 6. Run database migrations
After the initial setup, apply the feature migrations in order:
```bash
cd backend
node migrations/add_weekly_reports_table.js
node migrations/add_knowledge_base_table.js
node migrations/add_archer_tickets_table.js
```
---
## Configuration
The application is configured via `.env` files. These files are gitignored and must be created manually per environment.
### Backend: `backend/.env`
```
PORT=3001
API_HOST=localhost
CORS_ORIGINS=http://YOUR_IP:3000
SESSION_SECRET=change-this-to-a-random-secret
NODE_ENV=development
# Optional: NVD API key for higher rate limits
# Register at https://nvd.nist.gov/developers/request-an-api-key
NVD_API_KEY=your-key-here
```
### Frontend: `frontend/.env`
```
REACT_APP_API_BASE=http://YOUR_IP:3001/api
REACT_APP_API_HOST=http://YOUR_IP:3001
```
Replace `YOUR_IP` with the machine's IP address or `localhost` for local development.
**Important:** React caches environment variables at build/start time. After changing `frontend/.env`, you must fully restart the frontend process. A page refresh alone is not sufficient.
---
## Running the Application
### Using the helper scripts (recommended)
From the project root:
```bash
./start-servers.sh # Starts backend and frontend in the background
./stop-servers.sh # Stops all servers
```
The start script saves PIDs to `backend.pid` and `frontend.pid`. Logs are written to `backend/backend.log` and `frontend/frontend.log`.
### Running manually
```bash
# Terminal 1 - backend
cd backend
node server.js
# Terminal 2 - frontend
cd frontend
npm start
```
### Default ports
- Frontend: http://localhost:3000
- Backend API: http://localhost:3001
---
## Features ## Features
### Authentication and User Roles | Feature | Description |
|---------|-------------|
| **CVE Management** | Track CVEs across multiple vendors with document storage and NVD auto-fill |
| **Reporting** | Ivanti host finding triage with donut charts, inline editing, advanced filtering, CSV/XLSX export |
| **Ivanti Queue** | Personal staging list for batch FP, Archer, CARD, and Granite workflows |
| **FP Workflow** | Submit false positive workflows directly to Ivanti API with attachments |
| **Compliance** | Weekly AEO xlsx upload with diff preview, drift detection, per-team metric health cards |
| **Archive Tracking** | Automatic detection of disappeared/returned findings with BU reassignment classification |
| **Findings Trend** | Historical open vs closed chart with archive activity sparkline and shift reason tooltips |
| **Jira Integration** | Create, sync, and track Jira Data Center tickets linked to CVE/vendor pairs |
| **Archer Tickets** | Track risk acceptance exceptions (EXC numbers) linked to findings |
| **CARD API** | Granite/CARD asset lookup integration for network device workflows |
| **Knowledge Base** | Internal document library with inline PDF/Markdown viewing |
| **Access Control** | Four user groups (Admin, Standard_User, Leadership, Read_Only) with full audit trail |
All routes require authentication. Three roles are supported: ## Project Structure
| Role | Permissions |
|---|---|
| `viewer` | Read-only access to CVEs, documents, weekly reports, knowledge base, Archer tickets |
| `editor` | All viewer permissions plus: create/update CVEs, upload documents, upload weekly reports, manage knowledge base articles, manage Archer tickets |
| `admin` | All editor permissions plus: delete documents, delete weekly reports, manage users, view audit logs |
Sessions expire after 24 hours. Session tokens are stored in `httpOnly` cookies.
### CVE Management
- Add CVEs with full metadata: CVE ID, vendor, severity (Critical/High/Medium/Low), description, published date, and status (Open/In Progress/Addressed/Resolved)
- The same CVE ID can be tracked across multiple vendors independently
- Filter the CVE list by search term, vendor, severity, and status
- Edit any field on an existing CVE entry; file paths are updated automatically when CVE ID or vendor changes
- Delete a single vendor entry or all vendor entries for a CVE ID
- Paginated list view to prevent performance issues with large datasets
- Quick Check: look up a CVE ID and see all vendors tracking it with their current status
### NVD Integration
- Auto-fill CVE description, severity, and published date from the NIST NVD API 2.0 when adding a new CVE
- Bulk NVD Sync: fetch updated metadata for all CVEs in the database in one operation (editor/admin)
- CVSS severity mapping cascades: v3.1 preferred, then v3.0, then v2.0
- NVD API key support via `NVD_API_KEY` environment variable for higher rate limits
### Document Management
Documents are attached to a CVE/vendor pair and stored on disk under `backend/uploads/<CVE-ID>/<vendor>/`.
Supported document types: `advisory`, `email`, `screenshot`, `patch`, `other`
Allowed file extensions: PDF, images (PNG, JPG, GIF, BMP, TIFF), Office documents (DOC, DOCX, XLS, XLSX, PPT, PPTX), text files (TXT, MD, CSV, LOG), email files (MSG, EML), and others (RTF, HTML, XML, JSON, YAML, ODF variants).
File size limit: 10 MB per upload.
### Weekly Reports
Editors and admins can upload weekly vulnerability reports as `.xlsx` files. The report is processed by a Python script (`backend/scripts/split_cve_report.py`) that:
1. Reads the `Vulnerabilities` sheet
2. Splits rows where multiple CVE IDs are comma-separated in the `CVE ID` column into individual rows
3. Saves the processed file alongside the original
Both the original and processed files can be downloaded from the weekly reports list. Only the most recently uploaded report is marked as current. Admins can delete old report records and their associated files.
### Archer Risk Acceptance Tickets
Track Archer exception tickets (EXC numbers) linked to specific CVE/vendor pairs.
- EXC number format: `EXC-NNNNN`
- Statuses: `Draft`, `Open`, `Under Review`, `Accepted`
- Optional Archer URL field for deep-linking to the Archer record
- Filter tickets by CVE ID, vendor, or status
- EXC numbers are unique across the system
### Knowledge Base
A document library for internal reference material such as policies, runbooks, and vendor advisories.
- Upload documents with a title, optional description, and category
- View documents inline in the browser (PDFs render in an iframe; markdown files are rendered as HTML)
- Download any document
- Filter and browse by category
- Editors and admins can upload and delete; all authenticated users can view
Allowed file types: PDF, Markdown, TXT, Office documents, HTML, JSON, YAML, and images.
### User Management (Admin)
Admins can create, update, and delete user accounts from the UI. Supported operations:
- Create users with a role assignment
- Change username, email, password, role, or active status
- Deactivating a user immediately invalidates all their active sessions
- Admins cannot demote themselves or deactivate their own account
### Audit Log (Admin)
Every state-changing action is recorded with the user identity, IP address, action type, target entity, and a before/after details payload. Admins can view the audit log with filtering by user, action type, entity type, and date range. Results are paginated.
---
## API Reference
All endpoints are prefixed with `/api`. All endpoints except `/api/auth/login` and `/api/auth/logout` require a valid session cookie.
### Auth
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | `/api/auth/login` | Public | Log in, receive session cookie |
| POST | `/api/auth/logout` | Public | Invalidate session |
| GET | `/api/auth/me` | Session | Get current user info |
| POST | `/api/auth/cleanup-sessions` | Session | Delete expired sessions |
### CVEs
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/cves` | viewer+ | List CVEs with optional filters: `search`, `vendor`, `severity`, `status` |
| POST | `/api/cves` | editor+ | Create a new CVE entry |
| PUT | `/api/cves/:id` | editor+ | Update a CVE entry by row ID |
| PATCH | `/api/cves/:cveId/status` | editor+ | Update status for all vendor rows matching a CVE ID |
| DELETE | `/api/cves/:id` | editor+ | Delete a single CVE vendor entry |
| DELETE | `/api/cves/by-cve-id/:cveId` | editor+ | Delete all vendor entries for a CVE ID |
| GET | `/api/cves/check/:cveId` | viewer+ | Quick check: does this CVE exist and what is its status? |
| GET | `/api/cves/distinct-ids` | viewer+ | List all distinct CVE IDs (used by NVD sync) |
| GET | `/api/cves/:cveId/vendors` | viewer+ | List all vendor entries for a specific CVE ID |
### Documents
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/cves/:cveId/documents` | viewer+ | List documents for a CVE, optionally filtered by `?vendor=` |
| POST | `/api/cves/:cveId/documents` | editor+ | Upload a document for a CVE/vendor pair |
| DELETE | `/api/documents/:id` | admin | Delete a document and its file from disk |
### NVD
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/nvd/lookup/:cveId` | viewer+ | Look up a single CVE in the NVD API |
| POST | `/api/cves/nvd-sync` | editor+ | Bulk update CVE metadata from NVD |
### Weekly Reports
| Method | Path | Role | Description |
|---|---|---|---|
| POST | `/api/weekly-reports/upload` | editor+ | Upload and process a `.xlsx` vulnerability report |
| GET | `/api/weekly-reports` | viewer+ | List all uploaded reports |
| GET | `/api/weekly-reports/:id/download/:type` | viewer+ | Download `original` or `processed` file |
| DELETE | `/api/weekly-reports/:id` | admin | Delete a report record and its files |
### Knowledge Base
| Method | Path | Role | Description |
|---|---|---|---|
| POST | `/api/knowledge-base/upload` | editor+ | Upload a new knowledge base document |
| GET | `/api/knowledge-base` | viewer+ | List all articles |
| GET | `/api/knowledge-base/:id` | viewer+ | Get article metadata |
| GET | `/api/knowledge-base/:id/content` | viewer+ | Get file content for inline display |
| GET | `/api/knowledge-base/:id/download` | viewer+ | Download the file |
| DELETE | `/api/knowledge-base/:id` | editor+ | Delete article and file |
### Archer Tickets
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/archer-tickets` | viewer+ | List tickets, optional filters: `cve_id`, `vendor`, `status` |
| POST | `/api/archer-tickets` | editor+ | Create a new Archer ticket |
| PUT | `/api/archer-tickets/:id` | editor+ | Update an Archer ticket |
| DELETE | `/api/archer-tickets/:id` | editor+ | Delete an Archer ticket |
### Users (Admin only)
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/users` | admin | List all users |
| GET | `/api/users/:id` | admin | Get a single user |
| POST | `/api/users` | admin | Create a user |
| PATCH | `/api/users/:id` | admin | Update a user |
| DELETE | `/api/users/:id` | admin | Delete a user |
### Audit Logs (Admin only)
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/audit-logs` | admin | Paginated audit log with filters |
| GET | `/api/audit-logs/actions` | admin | List distinct action types |
### Utility
| Method | Path | Role | Description |
|---|---|---|---|
| GET | `/api/vendors` | viewer+ | List all distinct vendor names |
| GET | `/api/stats` | viewer+ | Dashboard statistics (total CVEs, critical count, addressed count, document count) |
---
## Architecture
``` ```
cve-dashboard/ cve-dashboard/
├── start-servers.sh # Start backend + frontend in background
├── stop-servers.sh # Stop all servers
├── backend/ ├── backend/
│ ├── server.js # Express app, CVE/document endpoints, middleware │ ├── server.js # Express API server
│ ├── setup.js # One-time DB initialization and default admin creation │ ├── setup.js # Database initialization (run once)
│ ├── cve_database.db # SQLite database (gitignored) │ ├── routes/ # API route handlers
│ ├── uploads/ # File storage (gitignored) │ ├── helpers/ # API clients (Ivanti, Jira, Atlas, CARD)
│ ├── <CVE-ID>/ │ ├── middleware/ # Auth middleware
└── <vendor>/ # CVE documents stored here ├── migrations/ # Schema migrations (for existing deployments)
│ ├── weekly_reports/ # Uploaded vulnerability reports └── scripts/ # Compliance parser, data import utilities
│ │ ├── knowledge_base/ # Knowledge base documents ├── frontend/
│ └── temp/ # Temporary upload staging directory ├── src/
│ ├── routes/ │ ├── App.js # Main app with routing
│ │ ├── auth.js # Login, logout, session check │ │ ├── components/ # React components
│ │ ── users.js # User CRUD (admin) │ │ ── contexts/ # Auth context
│ ├── auditLog.js # Audit log viewer (admin) └── public/
│ │ ├── nvdLookup.js # NVD API proxy ├── docs/
│ ├── weeklyReports.js # Weekly report upload and management │ ├── api/ # API specs (Ivanti, Atlas, Jira)
│ ├── knowledgeBase.js # Knowledge base document management │ ├── design/ # Design system, workflow diagrams
│ └── archerTickets.js # Archer EXC ticket CRUD ├── guides/ # User guides, full reference manual
│ ├── middleware/ │ ├── security/ # Security audits and remediation plans
│ └── auth.js # requireAuth and requireRole middleware ├── testing/ # Test plans and scripts
── helpers/ ── troubleshooting/ # Investigation scripts and reports
├── auditLog.js # logAudit helper ├── systemd/ # systemd service files
│ │ └── excelProcessor.js # Calls Python script for report processing ├── start-servers.sh
│ ├── migrations/ └── stop-servers.sh
│ │ ├── add_weekly_reports_table.js
│ │ ├── add_knowledge_base_table.js
│ │ └── add_archer_tickets_table.js
│ └── scripts/
│ ├── split_cve_report.py # Python: splits multi-CVE rows in Excel reports
│ └── requirements.txt # pandas, openpyxl
└── frontend/
└── src/
├── App.js # Main application, CVE list, filters, modals
├── App.css # Global styles
├── contexts/
│ └── AuthContext.js # Auth state provider
└── components/
├── LoginForm.js # Login page
├── UserMenu.js # User dropdown in header
├── UserManagement.js # Admin user management panel
├── AuditLog.js # Admin audit log viewer
├── NvdSyncModal.js # Bulk NVD sync dialog
├── WeeklyReportModal.js # Weekly report upload dialog
├── KnowledgeBaseModal.js # Knowledge base upload/list
└── KnowledgeBaseViewer.js # Inline document viewer
``` ```
--- ## Tech Stack
## Database Schema | Layer | Technology |
|-------|------------|
| Backend | Node.js 18+, Express 5, SQLite3 |
| Frontend | React 19, Recharts, Lucide React |
| Auth | bcryptjs, cookie-based sessions, express-rate-limit |
| Compliance | Python 3, pandas, openpyxl |
### Core tables ## Documentation
**`cves`** - One row per CVE/vendor pair. `UNIQUE(cve_id, vendor)`. - **[Full Reference Manual](docs/guides/full-reference-manual.md)** — comprehensive feature documentation, API reference, database schema, security model, and configuration details
- **[Migration Guide](backend/migrations/README.md)** — schema migration scripts for upgrading existing deployments
- **[Design System](docs/design/design-system.md)** — UI component patterns and color system
- **[Ivanti API Reference](docs/api/ivanti-api-reference.md)** — Ivanti/RiskSense API integration details
- **[Jira API Use Cases](docs/api/jira-api-use-cases.md)** — Jira Data Center API compliance summary
**`documents`** - Files attached to a CVE/vendor pair. Foreign key to `cves(cve_id)`. ## License
**`required_documents`** - Vendor-specific document requirements (advisory, screenshot, etc.). Internal use only — Charter Communications / NTS-AEO.
**`users`** - Accounts with roles: `admin`, `editor`, `viewer`.
**`sessions`** - Active sessions. Expire after 24 hours.
**`audit_logs`** - Append-only log of all state-changing actions.
### Feature tables (added by migrations)
**`weekly_reports`** - Metadata for uploaded vulnerability reports. Tracks original and processed file paths, row counts, uploader, and a `is_current` flag.
**`knowledge_base`** - Document library entries with title, slug, category, description, and file metadata.
**`archer_tickets`** - Archer EXC exception tickets linked to CVE/vendor pairs. `UNIQUE(exc_number)`.
### View
**`cve_document_status`** - Aggregates document counts per CVE/vendor and derives a `compliance_status` (`Complete` when an advisory is present, otherwise `Missing Required Docs`).
--- ---
## Security Model *Designed and built by Jordan Ramos (jordan.ramos@spectrum.com)*
### File upload security
- Extension allowlist enforced by Multer; executables (`.exe`, `.js`, `.sh`, `.py`, `.bat`, etc.) are blocked
- MIME type prefix validation in addition to extension checking
- 10 MB per-file size limit
- Filenames are sanitized: path separators, `..` sequences, null bytes, and non-alphanumeric characters are removed
### Path traversal prevention
- `sanitizePathSegment()` strips `/`, `\`, `..`, and null bytes from any value used in `path.join()`
- `isPathWithinUploads()` verifies resolved paths stay within the uploads root before any file operation
### Input validation
- CVE ID must match `/^CVE-\d{4}-\d{4,}$/`
- Severity must be one of: `Critical`, `High`, `Medium`, `Low`
- Status must be one of: `Open`, `Addressed`, `In Progress`, `Resolved`
- Archer EXC numbers must match `/^EXC-\d+$/`
- All database operations use prepared statements
### Error handling
- 500 responses never leak internal error messages to the client
- Full errors are logged server-side only
- Descriptive 400/409 responses are safe because they contain only validation messages written by the application
### Security headers
Applied to all responses:
- `X-Content-Type-Options: nosniff`
- `X-Frame-Options: SAMEORIGIN`
- `X-XSS-Protection: 1; mode=block`
- `Referrer-Policy: strict-origin-when-cross-origin`
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`
### Session cookies
`httpOnly: true`, `sameSite: lax`, `secure: true` in production.
---
## Migrations
Migrations are standalone Node.js scripts that alter the database directly. Run them in the order listed. They use `CREATE TABLE IF NOT EXISTS`, so they are safe to run again if needed.
```bash
cd backend
node migrations/add_weekly_reports_table.js
node migrations/add_knowledge_base_table.js
node migrations/add_archer_tickets_table.js
```
For an existing deployment upgrading from an earlier schema, also check the legacy migration scripts in `backend/`:
- `migrate_multivendor.js` - Adds multi-vendor support to an older single-vendor schema
- `migrate-audit-log.js` - Adds the audit_logs table to pre-auth deployments
- `migrate-to-1.1.js` - General 1.0 to 1.1 schema update

View File

@@ -1,211 +0,0 @@
# Weekly Vulnerability Report Upload Feature
## Overview
A new feature has been added to the CVE Dashboard that allows users to upload their weekly vulnerability reports in Excel format (.xlsx) and automatically process them to split multiple CVE IDs into separate rows for easier filtering and analysis.
## What Was Implemented
### Backend Changes
1. **Database Migration** (`backend/migrations/add_weekly_reports_table.js`)
- Created `weekly_reports` table to store report metadata
- Tracks upload date, file paths, row counts, and which report is current
- Indexed for fast queries
2. **Excel Processor** (`backend/helpers/excelProcessor.js`)
- Executes Python script via Node.js child_process
- Parses row counts from Python output
- Handles errors, timeouts (30 seconds), and validation
3. **API Routes** (`backend/routes/weeklyReports.js`)
- `POST /api/weekly-reports/upload` - Upload and process Excel file
- `GET /api/weekly-reports` - List all reports
- `GET /api/weekly-reports/:id/download/:type` - Download original or processed file
- `DELETE /api/weekly-reports/:id` - Delete report (admin only)
4. **Python Script** (`backend/scripts/split_cve_report.py`)
- Moved from ~/Documents to backend/scripts
- Splits comma-separated CVE IDs into separate rows
- Duplicates device/IP data for each CVE
### Frontend Changes
1. **Weekly Report Modal** (`frontend/src/components/WeeklyReportModal.js`)
- Phase-based UI: idle → uploading → processing → success
- File upload with .xlsx validation
- Display existing reports with current report indicator (★)
- Download buttons for both original and processed files
2. **App.js Integration**
- Added "Weekly Report" button next to NVD Sync button
- State management for modal visibility
- Modal rendering
## How to Use
### Starting the Application
1. **Backend:**
```bash
cd backend
node server.js
```
2. **Frontend:**
```bash
cd frontend
npm start
```
### Using the Feature
1. **Access the Feature**
- Login as an editor or admin user
- Look for the "Weekly Report" button in the top header (next to "NVD Sync")
2. **Upload a Report**
- Click the "Weekly Report" button
- Click "Choose File" and select your .xlsx file
- Click "Upload & Process"
- Wait for processing to complete (usually 5-10 seconds)
3. **Download Processed Report**
- After upload succeeds, you'll see row counts (e.g., "45 → 67 rows")
- Click "Download Processed" to get the split version
- The current week's report is marked with a ★ star icon
4. **Access Previous Reports**
- All previous reports are listed below the upload section
- Click the download icons to get original or processed versions
- Reports are labeled as "This week's report", "Last week's report", or by date
### What the Processing Does
**Before Processing:**
| HOSTNAME | IP | CVE ID |
|----------|------------|---------------------------|
| server01 | 10.0.0.1 | CVE-2024-1234, CVE-2024-5678 |
**After Processing:**
| HOSTNAME | IP | CVE ID |
|----------|------------|---------------------------|
| server01 | 10.0.0.1 | CVE-2024-1234 |
| server01 | 10.0.0.1 | CVE-2024-5678 |
Each CVE now has its own row, making it easy to:
- Sort by CVE ID
- Filter for specific CVEs
- Research CVEs one by one per device
## File Locations
### New Files Created
```
backend/
scripts/
split_cve_report.py # Python script for CVE splitting
requirements.txt # Python dependencies
routes/
weeklyReports.js # API endpoints
helpers/
excelProcessor.js # Python integration
migrations/
add_weekly_reports_table.js # Database migration
uploads/
weekly_reports/ # Uploaded and processed files
frontend/
src/
components/
WeeklyReportModal.js # Upload modal UI
```
### Modified Files
```
backend/
server.js # Added route mounting
frontend/
src/
App.js # Added button and modal
```
## Security & Permissions
- **Upload**: Requires editor or admin role
- **Download**: Any authenticated user
- **Delete**: Admin only
- **File Validation**: Only .xlsx files accepted, 10MB limit
- **Audit Logging**: All uploads, downloads, and deletions are logged
## Troubleshooting
### Backend Issues
**Python not found:**
```bash
# Install Python 3
sudo apt-get install python3
```
**Missing dependencies:**
```bash
# Install pandas and openpyxl
pip3 install pandas openpyxl
```
**Port already in use:**
```bash
# Find and kill process using port 3001
lsof -i :3001
kill -9 <PID>
```
### Frontend Issues
**Button not visible:**
- Make sure you're logged in as editor or admin
- Viewer role cannot upload reports
**Upload fails:**
- Check file is .xlsx format (not .xls or .csv)
- Ensure file has "Vulnerabilities" sheet with "CVE ID" column
- Check file size is under 10MB
**Processing timeout:**
- Large files (10,000+ rows) may timeout
- Try reducing file size or increase timeout in `excelProcessor.js`
## Testing Checklist
- [x] Backend starts without errors
- [x] Frontend compiles successfully
- [x] Database migration completed
- [x] Python dependencies installed
- [ ] Upload .xlsx file (manual test in browser)
- [ ] Verify processed file has split CVEs (manual test)
- [ ] Download original and processed files (manual test)
- [ ] Verify current report marked with star (manual test)
- [ ] Test as viewer - button should be hidden (manual test)
## Future Enhancements
Possible improvements:
- Progress bar during Python processing
- Email notifications when processing completes
- Scheduled automatic uploads
- Report comparison (diff between weeks)
- Export to other formats (CSV, JSON)
- Bulk delete old reports
- Report validation before upload
## Support
For issues or questions:
1. Check the troubleshooting section above
2. Review audit logs for error details
3. Check browser console for frontend errors
4. Review backend server logs for API errors

View File

@@ -251,14 +251,14 @@
"updated": 1, "updated": 1,
"link": null, "link": null,
"locked": false, "locked": false,
"text": "Backend API (Express.js)\nPort: 3001\n\nRoutes:\n• /api/auth - Authentication (login/logout)\n• /api/users - User management\n• /api/cves - CVE operations\n• /api/documents - Document upload/download\n• /api/audit-log - Audit logging\n• /api/nvd-lookup - NVD integration\n• /api/weekly-reports - Weekly reports", "text": "Backend API (Express.js)\nPort: 3001\n\nRoutes:\n• /api/auth - Authentication (login/logout)\n• /api/users - User management\n• /api/cves - CVE operations\n• /api/documents - Document upload/download\n• /api/audit-log - Audit logging\n• /api/nvd-lookup - NVD integration",
"fontSize": 14, "fontSize": 14,
"fontFamily": 1, "fontFamily": 1,
"textAlign": "left", "textAlign": "left",
"verticalAlign": "middle", "verticalAlign": "middle",
"baseline": 163, "baseline": 163,
"containerId": "backend-box", "containerId": "backend-box",
"originalText": "Backend API (Express.js)\nPort: 3001\n\nRoutes:\n• /api/auth - Authentication (login/logout)\n• /api/users - User management\n• /api/cves - CVE operations\n• /api/documents - Document upload/download\n• /api/audit-log - Audit logging\n• /api/nvd-lookup - NVD integration\n• /api/weekly-reports - Weekly reports" "originalText": "Backend API (Express.js)\nPort: 3001\n\nRoutes:\n• /api/auth - Authentication (login/logout)\n• /api/users - User management\n• /api/cves - CVE operations\n• /api/documents - Document upload/download\n• /api/audit-log - Audit logging\n• /api/nvd-lookup - NVD integration"
}, },
{ {
"id": "db-box", "id": "db-box",
@@ -820,14 +820,14 @@
"updated": 1, "updated": 1,
"link": null, "link": null,
"locked": false, "locked": false,
"text": "Key Features:\n• Quick CVE status check\n• Multi-vendor support\n• Document management\n• Compliance tracking\n• Search & filter\n• Weekly report uploads\n• Audit logging", "text": "Key Features:\n• Quick CVE status check\n• Multi-vendor support\n• Document management\n• Compliance tracking\n• Search & filter\n• Audit logging",
"fontSize": 12, "fontSize": 12,
"fontFamily": 1, "fontFamily": 1,
"textAlign": "left", "textAlign": "left",
"verticalAlign": "top", "verticalAlign": "top",
"baseline": 113, "baseline": 113,
"containerId": null, "containerId": null,
"originalText": "Key Features:\n• Quick CVE status check\n• Multi-vendor support\n• Document management\n• Compliance tracking\n• Search & filter\n• Weekly report uploads\n• Audit logging" "originalText": "Key Features:\n• Quick CVE status check\n• Multi-vendor support\n• Document management\n• Compliance tracking\n• Search & filter\n• Audit logging"
} }
], ],
"appState": { "appState": {

View File

@@ -3,6 +3,10 @@ PORT=3001
API_HOST=localhost API_HOST=localhost
CORS_ORIGINS=http://localhost:3000 CORS_ORIGINS=http://localhost:3000
# Session secret — REQUIRED. Server will not start without this.
# Generate with: openssl rand -base64 32
SESSION_SECRET=
# NVD API Key (optional - increases rate limit from 5 to 50 requests per 30s) # NVD API Key (optional - increases rate limit from 5 to 50 requests per 30s)
# Request one at https://nvd.nist.gov/developers/request-an-api-key # Request one at https://nvd.nist.gov/developers/request-an-api-key
NVD_API_KEY= NVD_API_KEY=
@@ -13,5 +17,56 @@ IVANTI_API_KEY=
IVANTI_CLIENT_ID=1550 IVANTI_CLIENT_ID=1550
IVANTI_FIRST_NAME= IVANTI_FIRST_NAME=
IVANTI_LAST_NAME= IVANTI_LAST_NAME=
# Comma-separated list of BU values to sync from Ivanti.
# Broadening this pulls findings for additional BUs into the local cache.
# Users see only their assigned teams' findings (filtered at query time).
# Default if unset: NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM
IVANTI_BU_FILTER=NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM
# Set to true if behind Charter's SSL inspection proxy (replicates Python verify=False) # Set to true if behind Charter's SSL inspection proxy (replicates Python verify=False)
IVANTI_SKIP_TLS=false IVANTI_SKIP_TLS=false
# Atlas InfoSec API (atlas-infosec.caas.charterlab.com)
# Service account credentials for Basic Auth — used to sync and manage action plans
ATLAS_API_URL=
ATLAS_API_USER=
ATLAS_API_PASS=
# Set to true if behind Charter's SSL inspection proxy (disables TLS cert verification)
ATLAS_SKIP_TLS=false
# Jira Data Center REST API
# VPN or Charter Network connection required for all Jira instances.
# Service accounts use Basic Auth (JIRA_API_USER + JIRA_API_TOKEN).
# PATs require ATLSUP approval and naming convention: Function - Team - ATLSUP-XXXXX
# Rate limits: 1440 requests/day, burst of 60/minute.
JIRA_BASE_URL=
JIRA_AUTH_METHOD=basic
# Basic Auth — service account credentials
JIRA_API_USER=
JIRA_API_TOKEN=
# PAT Auth — set JIRA_AUTH_METHOD=pat to use
JIRA_PAT=
# Default project key and issue type for creating issues from the dashboard
JIRA_PROJECT_KEY=
JIRA_ISSUE_TYPE=Task
# Set to true if behind Charter's SSL inspection proxy
JIRA_SKIP_TLS=false
# CARD Asset Ownership API (card.charter.com / card.caas.stage.charterlab.com)
# OAuth Bearer token auth — service account must be onboarded with the CARD team.
# Tokens are acquired automatically via Basic Auth and cached for 1 hour.
CARD_API_URL=
CARD_API_USER=
CARD_API_PASS=
# Set to true if behind Charter's SSL inspection proxy
CARD_SKIP_TLS=false
# PostgreSQL Database (Docker container steam-postgres)
# If set, the backend uses Postgres instead of SQLite.
# Format: postgresql://user:password@host:port/database
DATABASE_URL=postgresql://steam:<password>@localhost:5433/cve_dashboard
# GitLab Feedback Integration (bug reports and feature requests from the dashboard)
# PAT needs 'api' scope. Project ID is the numeric ID from GitLab project settings.
GITLAB_URL=http://steam-gitlab.charterlab.com
GITLAB_PROJECT_ID=
GITLAB_PAT=

View File

@@ -0,0 +1,48 @@
/**
* Property-Based Test: Password Change Round-Trip
*
* Feature: user-profile, Property 3: Password change round-trip
*
* For any valid current password and any new password of 8+ characters,
* after a successful change, bcrypt.compare(newPassword, storedHash) returns true.
*
* Validates: Requirements 2.2, 2.7
*/
const fc = require('fast-check');
const bcrypt = require('bcryptjs');
// bcrypt cost factor — production uses 10, but we use 4 (the minimum) here
// to keep 100 iterations feasible within test timeouts. The round-trip property
// holds regardless of cost factor.
const BCRYPT_COST = 4;
describe('Feature: user-profile, Property 3: Password change round-trip', () => {
it('after a password change, bcrypt.compare(newPassword, newHash) returns true', async () => {
await fc.assert(
fc.asyncProperty(
// Current password: any non-empty string (length >= 1)
fc.string({ minLength: 1, maxLength: 72 }),
// New password: any string of length >= 8 (bcrypt max input is 72 bytes)
fc.string({ minLength: 8, maxLength: 72 }),
async (currentPassword, newPassword) => {
// Step 1: Hash the current password (simulates existing stored hash)
const currentHash = await bcrypt.hash(currentPassword, BCRYPT_COST);
// Step 2: Verify the current password against the stored hash
// (simulates the bcrypt.compare check in the change-password route)
const currentPasswordValid = await bcrypt.compare(currentPassword, currentHash);
expect(currentPasswordValid).toBe(true);
// Step 3: Hash the new password (simulates bcrypt.hash(newPassword, 10) in the route)
const newHash = await bcrypt.hash(newPassword, BCRYPT_COST);
// Step 4: Verify the new password matches the new hash (round-trip property)
const newPasswordValid = await bcrypt.compare(newPassword, newHash);
expect(newPasswordValid).toBe(true);
}
),
{ numRuns: 100 }
);
}, 120000); // 2-minute timeout for 100 bcrypt iterations
});

View File

@@ -0,0 +1,84 @@
/**
* Property-Based Test: Profile API Returns Complete User Data Matching Database
*
* Feature: user-profile, Property 2: Profile API returns complete user data matching database
*
* For any active user record, the profile route's mapping logic produces a
* response object with all 6 required fields (id, username, email, group,
* created_at, last_login) and each value matches the corresponding column
* in the users table. The `group` field maps from the `user_group` column.
*
* Validates: Requirements 4.1
*/
const fc = require('fast-check');
/**
* Simulates the exact mapping logic from GET /api/auth/profile in routes/auth.js:
*
* res.json({
* id: user.id,
* username: user.username,
* email: user.email,
* group: user.user_group,
* created_at: user.created_at,
* last_login: user.last_login
* });
*/
function mapUserRowToProfileResponse(user) {
return {
id: user.id,
username: user.username,
email: user.email,
group: user.user_group,
created_at: user.created_at,
last_login: user.last_login
};
}
describe('Feature: user-profile, Property 2: Profile API returns complete user data matching database', () => {
it('profile response contains all 6 required fields matching the database row', () => {
fc.assert(
fc.property(
// Generate arbitrary user rows matching the users table schema
fc.record({
id: fc.integer({ min: 1, max: 1000000 }),
username: fc.string({ minLength: 1, maxLength: 50 }),
email: fc.string({ minLength: 3, maxLength: 255 }),
user_group: fc.constantFrom('Admin', 'Standard_User', 'Read_Only'),
created_at: fc.integer({ min: 1577836800000, max: 1924991999000 })
.map(ts => new Date(ts).toISOString().replace('T', ' ').slice(0, 19)),
last_login: fc.oneof(
fc.integer({ min: 1577836800000, max: 1924991999000 })
.map(ts => new Date(ts).toISOString().replace('T', ' ').slice(0, 19)),
fc.constant(null)
),
is_active: fc.constant(1)
}),
(userRow) => {
const response = mapUserRowToProfileResponse(userRow);
// Assert all 6 required fields are present
expect(response).toHaveProperty('id');
expect(response).toHaveProperty('username');
expect(response).toHaveProperty('email');
expect(response).toHaveProperty('group');
expect(response).toHaveProperty('created_at');
expect(response).toHaveProperty('last_login');
// Assert each value matches the corresponding database column
expect(response.id).toBe(userRow.id);
expect(response.username).toBe(userRow.username);
expect(response.email).toBe(userRow.email);
expect(response.group).toBe(userRow.user_group); // group maps from user_group
expect(response.created_at).toBe(userRow.created_at);
expect(response.last_login).toBe(userRow.last_login);
// Assert exactly 6 keys — no extra fields leaked
expect(Object.keys(response)).toHaveLength(6);
}
),
{ numRuns: 100 }
);
});
});

View File

@@ -0,0 +1,39 @@
/**
* Property-Based Test: Short Passwords Are Rejected (Server-Side)
*
* Feature: user-profile, Property 6 (server-side): Short passwords are rejected
*
* For any string of length 0 to 7, the server-side validation logic
* (newPassword.length < 8) correctly identifies them as too short,
* meaning the password change would return 400 and the stored hash
* would remain unchanged.
*
* Validates: Requirements 2.5, 5.4
*/
const fc = require('fast-check');
describe('Feature: user-profile, Property 6 (server-side): Short passwords are rejected', () => {
it('any string of length 07 is rejected by the server-side length validation', () => {
fc.assert(
fc.property(
// Generate arbitrary strings of length 0 to 7
fc.string({ minLength: 0, maxLength: 7 }),
(shortPassword) => {
// This is the exact validation check from POST /api/auth/change-password:
// if (newPassword.length < 8) return res.status(400).json({ error: '...' })
const wouldBeRejected = shortPassword.length < 8;
// Every generated string must be rejected by the validation
expect(wouldBeRejected).toBe(true);
// The stored hash remains unchanged because the route returns
// early before reaching the bcrypt.hash / UPDATE query.
// This is a structural guarantee — the early return prevents
// any mutation of the password_hash column.
}
),
{ numRuns: 100 }
);
});
});

View File

@@ -0,0 +1,53 @@
/**
* Property-Based Test: Incorrect Current Password Is Always Rejected
*
* Feature: user-profile, Property 4: Incorrect current password is always rejected
*
* For any password string that does not match the user's current password,
* the endpoint returns 401 and the stored hash remains unchanged.
*
* Validates: Requirements 2.3
*/
const fc = require('fast-check');
const bcrypt = require('bcryptjs');
// bcrypt cost factor — production uses 10, but we use 4 (the minimum) here
// to keep 100 iterations feasible within test timeouts. The rejection property
// holds regardless of cost factor.
const BCRYPT_COST = 4;
describe('Feature: user-profile, Property 4: Incorrect current password is always rejected', () => {
it('bcrypt.compare rejects any wrong password and the stored hash remains unchanged', async () => {
await fc.assert(
fc.asyncProperty(
// Current password: any non-empty string (bcrypt max input is 72 bytes)
fc.string({ minLength: 1, maxLength: 72 }),
// Wrong password: any non-empty string (will be filtered to differ from current)
fc.string({ minLength: 1, maxLength: 72 }),
async (currentPassword, wrongPassword) => {
// Ensure the wrong password is always different from the current password
fc.pre(wrongPassword !== currentPassword);
// Step 1: Hash the current password (simulates existing stored hash)
const currentHash = await bcrypt.hash(currentPassword, BCRYPT_COST);
// Capture the hash before the failed attempt
const hashBefore = currentHash;
// Step 2: Attempt to verify the wrong password against the stored hash
// (simulates the bcrypt.compare check in the change-password route)
const isValid = await bcrypt.compare(wrongPassword, currentHash);
// The wrong password must always be rejected
expect(isValid).toBe(false);
// Step 3: The stored hash remains unchanged after the failed attempt
// (no mutation should occur on rejection)
expect(currentHash).toBe(hashBefore);
}
),
{ numRuns: 100 }
);
}, 120000); // 2-minute timeout for 100 bcrypt iterations
});

View File

@@ -0,0 +1,108 @@
/**
* Property-Based Test: JQL Window Invariant
*
* Feature: jira-api-compliance-cleanup, Property 1: JQL window is always 72 hours in bulk sync
*
* For any non-empty array of valid-looking issue keys passed to searchIssuesByKeys(),
* the generated JQL string SHALL contain the substring `updated >= -72h` and
* SHALL contain the substring `project =`.
*
* Validates: Requirements 2.1, 2.3
*/
const fc = require('fast-check');
// Capture the JQL that flows through the HTTP layer.
let capturedJql = null;
// Mock https to intercept the request URL (which contains the JQL) and return
// a fake 200 response. This prevents real network calls while letting the
// real searchIssuesByKeys → searchIssues → jiraGet → jiraRequest chain execute.
jest.mock('https', () => ({
request: jest.fn((options, callback) => {
const fullPath = options.path || '';
const jqlMatch = fullPath.match(/[?&]jql=([^&]*)/);
if (jqlMatch) {
capturedJql = decodeURIComponent(jqlMatch[1]);
}
const mockResponse = {
statusCode: 200,
on: jest.fn((event, handler) => {
if (event === 'data') {
handler(JSON.stringify({ total: 0, issues: [] }));
}
if (event === 'end') {
handler();
}
}),
};
// Use setImmediate so the callback fires on the same tick after promises
// resolve, but still asynchronously as Node's http expects.
setImmediate(() => callback(mockResponse));
return {
on: jest.fn(),
write: jest.fn(),
end: jest.fn(),
destroy: jest.fn(),
};
}),
}));
// Set required env vars before requiring the module so the module-level
// constants pick them up.
process.env.JIRA_PROJECT_KEY = 'TESTPROJ';
process.env.JIRA_BASE_URL = 'https://jira.example.com';
process.env.JIRA_API_USER = 'testuser';
process.env.JIRA_API_TOKEN = 'testtoken';
const jiraApi = require('../helpers/jiraApi');
describe('Feature: jira-api-compliance-cleanup, Property 1: JQL window is always 72h in bulk sync', () => {
// Use fake timers so the rate-limiter's inter-request delays (12 seconds)
// resolve instantly. We preserve setImmediate so the https mock callback
// still fires asynchronously as expected.
beforeAll(() => {
jest.useFakeTimers({ doNotFake: ['setImmediate', 'nextTick'] });
});
afterAll(() => {
jest.useRealTimers();
});
beforeEach(() => {
capturedJql = null;
});
// Generator: produces a valid Jira issue key like "AB-1", "PROJ-42", etc.
const issueKeyArb = fc.tuple(
fc.stringMatching(/^[A-Z]{2,10}$/),
fc.integer({ min: 1, max: 99999 })
).map(([prefix, num]) => `${prefix}-${num}`);
// Generator: non-empty array of issue keys (1 to 50 keys)
const issueKeysArb = fc.array(issueKeyArb, { minLength: 1, maxLength: 50 });
it('searchIssuesByKeys() always generates JQL containing `updated >= -72h` and `project =`', async () => {
await fc.assert(
fc.asyncProperty(issueKeysArb, async (issueKeys) => {
capturedJql = null;
// Start the call — it will hit waitForDelay which uses setTimeout
const promise = jiraApi.searchIssuesByKeys(issueKeys);
// Advance fake timers to resolve any pending setTimeout from the
// rate limiter's waitForDelay function.
jest.advanceTimersByTime(5000);
await promise;
expect(capturedJql).not.toBeNull();
expect(capturedJql).toContain('updated >= -72h');
expect(capturedJql).toContain('project =');
}),
{ numRuns: 100 }
);
}, 60000);
});

View File

@@ -0,0 +1,146 @@
/**
* Example-Based Tests: Route Removal and Remaining Routes
*
* Feature: jira-api-compliance-cleanup
*
* Property 2: Search route is absent from router (Example)
* After the route removal, a POST request to /api/jira/search SHALL return HTTP 404.
* Validates: Requirements 1.1, 1.2
*
* Property 3: Existing routes remain functional after search route removal (Example)
* The routes GET /lookup/:issueKey, POST /sync-all, POST /:id/sync, and
* POST /create-in-jira SHALL continue to respond with non-404 status codes.
* Validates: Requirements 1.3, 1.4, 1.5, 1.6
*/
const http = require('http');
const express = require('express');
// Mock the auth middleware so routes don't require real sessions/cookies.
jest.mock('../middleware/auth', () => ({
requireAuth: () => (req, res, next) => {
req.user = { id: 1, username: 'test', group: 'Admin' };
next();
},
requireGroup: () => (req, res, next) => next(),
}));
// Mock the audit log helper to be a no-op.
jest.mock('../helpers/auditLog', () => jest.fn());
// Mock the jiraApi helper — mark it as not configured so routes return 503
// (which is fine; we only care that they are NOT 404).
jest.mock('../helpers/jiraApi', () => ({
isConfigured: false,
getRateLimitStatus: jest.fn(() => ({
burst: { remaining: 60, limit: 60 },
daily: { remaining: 1440, limit: 1440 },
})),
}));
const createJiraTicketsRouter = require('../routes/jiraTickets');
// Minimal db mock — callback-style methods that return empty results.
function createMockDb() {
return {
get: jest.fn((_sql, _params, cb) => cb(null, null)),
all: jest.fn((_sql, _params, cb) => cb(null, [])),
run: jest.fn(function (_sql, _params, cb) {
if (typeof cb === 'function') cb.call({ lastID: 1, changes: 0 }, null);
}),
};
}
/**
* Helper: send an HTTP request to the test server and return { statusCode }.
*/
function request(server, method, path, body) {
return new Promise((resolve, reject) => {
const addr = server.address();
const options = {
hostname: '127.0.0.1',
port: addr.port,
path,
method,
headers: { 'Content-Type': 'application/json' },
};
const req = http.request(options, (res) => {
// Consume the response body so the socket closes cleanly.
const chunks = [];
res.on('data', (chunk) => chunks.push(chunk));
res.on('end', () => {
resolve({ statusCode: res.statusCode });
});
});
req.on('error', reject);
if (body) {
req.write(JSON.stringify(body));
}
req.end();
});
}
describe('Feature: jira-api-compliance-cleanup — route removal tests', () => {
let app;
let server;
beforeAll((done) => {
const db = createMockDb();
app = express();
app.use(express.json());
app.use('/api/jira-tickets', createJiraTicketsRouter(db));
// Listen on a random available port.
server = app.listen(0, '127.0.0.1', done);
});
afterAll((done) => {
server.close(done);
});
// ---------------------------------------------------------------------------
// Property 2: POST /api/jira-tickets/search returns 404
// Validates: Requirements 1.1, 1.2
// ---------------------------------------------------------------------------
describe('Property 2: Search route is absent', () => {
it('POST /api/jira-tickets/search returns HTTP 404', async () => {
const res = await request(server, 'POST', '/api/jira-tickets/search', {
jql: 'project = TEST',
});
expect(res.statusCode).toBe(404);
});
});
// ---------------------------------------------------------------------------
// Property 3: Remaining routes respond with non-404 status codes
// Validates: Requirements 1.3, 1.4, 1.5, 1.6
// ---------------------------------------------------------------------------
describe('Property 3: Existing routes remain functional', () => {
it('GET /api/jira-tickets/lookup/:issueKey returns non-404', async () => {
const res = await request(server, 'GET', '/api/jira-tickets/lookup/TEST-1');
expect(res.statusCode).not.toBe(404);
});
it('POST /api/jira-tickets/sync-all returns non-404', async () => {
const res = await request(server, 'POST', '/api/jira-tickets/sync-all');
expect(res.statusCode).not.toBe(404);
});
it('POST /api/jira-tickets/:id/sync returns non-404', async () => {
const res = await request(server, 'POST', '/api/jira-tickets/1/sync');
expect(res.statusCode).not.toBe(404);
});
it('POST /api/jira-tickets/create-in-jira returns non-404', async () => {
const res = await request(server, 'POST', '/api/jira-tickets/create-in-jira', {
cve_id: 'CVE-2024-12345',
vendor: 'TestVendor',
summary: 'Test summary',
});
expect(res.statusCode).not.toBe(404);
});
});
});

Binary file not shown.

Binary file not shown.

478
backend/db-schema.sql Normal file
View File

@@ -0,0 +1,478 @@
-- =============================================================================
-- CVE Dashboard — Complete PostgreSQL Schema (v1.0.0)
-- =============================================================================
-- Translates the full SQLite schema (setup.js) to PostgreSQL 16.
-- Designed for idempotent execution: safe to run multiple times via psql or
-- pool.query() without errors or duplicate data.
--
-- Usage:
-- psql -h localhost -p 5433 -U steam -d cve_dashboard -f backend/db-schema.sql
-- OR
-- const schema = fs.readFileSync('backend/db-schema.sql', 'utf8');
-- await pool.query(schema);
-- =============================================================================
-- =============================================================================
-- Core CVE tracking tables
-- =============================================================================
CREATE TABLE IF NOT EXISTS cves (
id SERIAL PRIMARY KEY,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
severity VARCHAR(20) NOT NULL,
description TEXT,
published_date DATE,
status VARCHAR(50) DEFAULT 'Open',
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
created_by INTEGER,
UNIQUE(cve_id, vendor)
);
CREATE INDEX IF NOT EXISTS idx_cve_id ON cves(cve_id);
CREATE INDEX IF NOT EXISTS idx_vendor ON cves(vendor);
CREATE INDEX IF NOT EXISTS idx_severity ON cves(severity);
CREATE INDEX IF NOT EXISTS idx_status ON cves(status);
CREATE TABLE IF NOT EXISTS documents (
id SERIAL PRIMARY KEY,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
name VARCHAR(255) NOT NULL,
type VARCHAR(50) NOT NULL,
file_path VARCHAR(500) NOT NULL,
file_size VARCHAR(20),
mime_type VARCHAR(100),
uploaded_at TIMESTAMPTZ DEFAULT NOW(),
notes TEXT
);
CREATE INDEX IF NOT EXISTS idx_doc_cve_id ON documents(cve_id);
CREATE INDEX IF NOT EXISTS idx_doc_vendor ON documents(vendor);
CREATE INDEX IF NOT EXISTS idx_doc_type ON documents(type);
CREATE TABLE IF NOT EXISTS required_documents (
id SERIAL PRIMARY KEY,
vendor VARCHAR(100) NOT NULL,
document_type VARCHAR(50) NOT NULL,
is_mandatory BOOLEAN DEFAULT TRUE,
description TEXT,
UNIQUE(vendor, document_type)
);
-- =============================================================================
-- Authentication and session management
-- =============================================================================
CREATE TABLE IF NOT EXISTS users (
id SERIAL PRIMARY KEY,
username VARCHAR(50) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
role VARCHAR(20) NOT NULL DEFAULT 'viewer' CHECK (role IN ('admin', 'editor', 'viewer')),
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMPTZ DEFAULT NOW(),
last_login TIMESTAMPTZ,
user_group VARCHAR(20) NOT NULL DEFAULT 'Read_Only'
CHECK (user_group IN ('Admin', 'Standard_User', 'Leadership', 'Read_Only')),
bu_teams TEXT NOT NULL DEFAULT ''
);
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
CREATE INDEX IF NOT EXISTS idx_users_user_group ON users(user_group);
CREATE TABLE IF NOT EXISTS sessions (
id SERIAL PRIMARY KEY,
session_id VARCHAR(255) UNIQUE NOT NULL,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
expires_at TIMESTAMPTZ NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_sessions_session_id ON sessions(session_id);
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions(user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_expires ON sessions(expires_at);
-- =============================================================================
-- Audit logging
-- =============================================================================
CREATE TABLE IF NOT EXISTS audit_logs (
id SERIAL PRIMARY KEY,
user_id INTEGER,
username VARCHAR(50) NOT NULL,
action VARCHAR(50) NOT NULL,
entity_type VARCHAR(50) NOT NULL,
entity_id VARCHAR(100),
details TEXT,
ip_address VARCHAR(45),
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_audit_user_id ON audit_logs(user_id);
CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_logs(action);
CREATE INDEX IF NOT EXISTS idx_audit_entity_type ON audit_logs(entity_type);
CREATE INDEX IF NOT EXISTS idx_audit_created_at ON audit_logs(created_at);
-- =============================================================================
-- Jira integration
-- =============================================================================
CREATE TABLE IF NOT EXISTS jira_tickets (
id SERIAL PRIMARY KEY,
cve_id TEXT NOT NULL,
vendor TEXT NOT NULL,
ticket_key TEXT NOT NULL,
url TEXT,
summary TEXT,
status TEXT DEFAULT 'Open' CHECK (status IN ('Open', 'In Progress', 'Closed')),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_jira_tickets_cve ON jira_tickets(cve_id, vendor);
CREATE INDEX IF NOT EXISTS idx_jira_tickets_status ON jira_tickets(status);
-- =============================================================================
-- Archer integration
-- =============================================================================
CREATE TABLE IF NOT EXISTS archer_tickets (
id SERIAL PRIMARY KEY,
exc_number TEXT NOT NULL UNIQUE,
archer_url TEXT,
status TEXT DEFAULT 'Draft' CHECK (status IN ('Draft', 'Open', 'Under Review', 'Accepted')),
cve_id TEXT NOT NULL,
vendor TEXT NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_archer_tickets_cve ON archer_tickets(cve_id, vendor);
CREATE INDEX IF NOT EXISTS idx_archer_tickets_status ON archer_tickets(status);
CREATE INDEX IF NOT EXISTS idx_archer_tickets_exc ON archer_tickets(exc_number);
-- =============================================================================
-- Knowledge base
-- =============================================================================
CREATE TABLE IF NOT EXISTS knowledge_base (
id SERIAL PRIMARY KEY,
title VARCHAR(255) NOT NULL,
slug VARCHAR(255) UNIQUE NOT NULL,
description TEXT,
category VARCHAR(100),
file_path VARCHAR(500),
file_name VARCHAR(255),
file_type VARCHAR(50),
file_size INTEGER,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
created_by INTEGER REFERENCES users(id)
);
CREATE INDEX IF NOT EXISTS idx_knowledge_base_slug ON knowledge_base(slug);
CREATE INDEX IF NOT EXISTS idx_knowledge_base_category ON knowledge_base(category);
CREATE INDEX IF NOT EXISTS idx_knowledge_base_created_at ON knowledge_base(created_at DESC);
-- =============================================================================
-- Ivanti findings — individual rows (replaces findings_json blob)
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_findings (
id TEXT PRIMARY KEY,
host_id INTEGER,
title TEXT NOT NULL DEFAULT '',
severity NUMERIC(4,2) NOT NULL DEFAULT 0,
vrr_group TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
ip_address TEXT NOT NULL DEFAULT '',
dns TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT '',
sla_status TEXT NOT NULL DEFAULT '',
due_date DATE,
last_found_on DATE,
bu_ownership TEXT NOT NULL DEFAULT '',
cves TEXT[] DEFAULT '{}',
workflow_id TEXT,
workflow_state TEXT,
workflow_type TEXT,
state TEXT NOT NULL DEFAULT 'open' CHECK (state IN ('open', 'closed')),
note TEXT NOT NULL DEFAULT '',
override_host_name TEXT,
override_dns TEXT,
synced_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_findings_state ON ivanti_findings(state);
CREATE INDEX IF NOT EXISTS idx_findings_bu ON ivanti_findings(bu_ownership);
CREATE INDEX IF NOT EXISTS idx_findings_severity ON ivanti_findings(severity);
CREATE INDEX IF NOT EXISTS idx_findings_state_bu ON ivanti_findings(state, bu_ownership);
-- =============================================================================
-- Ivanti sync state (single-row pattern — replaces ivanti_findings_cache metadata)
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_sync_state (
id INTEGER PRIMARY KEY DEFAULT 1 CHECK (id = 1),
total INTEGER DEFAULT 0,
workflows_json TEXT DEFAULT '[]',
synced_at TIMESTAMPTZ,
sync_status TEXT DEFAULT 'never',
error_message TEXT
);
-- =============================================================================
-- Ivanti counts cache (single-row pattern for FP workflow counts)
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_counts_cache (
id INTEGER PRIMARY KEY DEFAULT 1 CHECK (id = 1),
open_count INTEGER DEFAULT 0,
closed_count INTEGER DEFAULT 0,
synced_at TIMESTAMPTZ,
fp_workflow_counts_json TEXT DEFAULT '{}',
fp_id_counts_json TEXT DEFAULT '{}'
);
-- =============================================================================
-- Ivanti counts history
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_counts_history (
id SERIAL PRIMARY KEY,
open_count INTEGER NOT NULL,
closed_count INTEGER NOT NULL,
recorded_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS ivanti_counts_history_by_bu (
id SERIAL PRIMARY KEY,
bu_ownership TEXT NOT NULL,
state TEXT NOT NULL CHECK (state IN ('open', 'closed')),
count INTEGER NOT NULL DEFAULT 0,
recorded_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_counts_history_bu ON ivanti_counts_history_by_bu(bu_ownership);
CREATE INDEX IF NOT EXISTS idx_counts_history_bu_date ON ivanti_counts_history_by_bu(recorded_at);
-- =============================================================================
-- Ivanti FP (False Positive) submissions
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_fp_submissions (
id SERIAL PRIMARY KEY,
user_id INTEGER NOT NULL,
username TEXT NOT NULL,
ivanti_workflow_batch_id INTEGER,
ivanti_generated_id TEXT,
ivanti_workflow_batch_uuid TEXT,
workflow_name TEXT NOT NULL,
reason TEXT NOT NULL,
description TEXT,
expiration_date TEXT NOT NULL,
scope_override TEXT NOT NULL DEFAULT 'Authorized',
finding_ids_json TEXT NOT NULL,
queue_item_ids_json TEXT NOT NULL,
attachment_count INTEGER DEFAULT 0,
attachment_results_json TEXT,
status TEXT NOT NULL DEFAULT 'success' CHECK (status IN ('success', 'partial', 'failed')),
lifecycle_status TEXT NOT NULL DEFAULT 'submitted'
CHECK (lifecycle_status IN ('submitted', 'approved', 'rejected', 'rework', 'resubmitted')),
error_message TEXT,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NULL
);
CREATE INDEX IF NOT EXISTS idx_fp_submissions_user ON ivanti_fp_submissions(user_id);
CREATE INDEX IF NOT EXISTS idx_fp_submissions_ivanti_id ON ivanti_fp_submissions(ivanti_generated_id);
CREATE TABLE IF NOT EXISTS ivanti_fp_submission_history (
id SERIAL PRIMARY KEY,
submission_id INTEGER NOT NULL REFERENCES ivanti_fp_submissions(id) ON DELETE CASCADE,
user_id INTEGER NOT NULL,
username TEXT NOT NULL,
change_type TEXT NOT NULL CHECK (change_type IN (
'created', 'fields_updated', 'findings_added',
'attachments_added', 'status_changed'
)),
change_details_json TEXT,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_fp_history_submission ON ivanti_fp_submission_history(submission_id);
-- =============================================================================
-- Ivanti todo queue (FP, Archer, CARD, GRANITE workflows)
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_todo_queue (
id SERIAL PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
finding_id TEXT NOT NULL,
finding_title TEXT,
cves_json TEXT,
ip_address TEXT,
hostname TEXT,
vendor TEXT NOT NULL,
workflow_type TEXT NOT NULL CHECK (workflow_type IN ('FP', 'Archer', 'CARD', 'GRANITE')),
status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'complete')),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_todo_queue_user ON ivanti_todo_queue(user_id, status);
-- =============================================================================
-- Ivanti archive detection and anomaly tracking
-- =============================================================================
CREATE TABLE IF NOT EXISTS ivanti_finding_archives (
id SERIAL PRIMARY KEY,
finding_id TEXT NOT NULL UNIQUE,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
ip_address TEXT NOT NULL DEFAULT '',
current_state TEXT NOT NULL CHECK (current_state IN ('ARCHIVED', 'RETURNED', 'CLOSED', 'CLOSED_GONE')),
last_severity NUMERIC(4,2) NOT NULL DEFAULT 0,
first_archived_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_transition_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_archive_finding_id ON ivanti_finding_archives(finding_id);
CREATE INDEX IF NOT EXISTS idx_archive_current_state ON ivanti_finding_archives(current_state);
CREATE TABLE IF NOT EXISTS ivanti_archive_transitions (
id SERIAL PRIMARY KEY,
archive_id INTEGER NOT NULL REFERENCES ivanti_finding_archives(id),
from_state TEXT NOT NULL,
to_state TEXT NOT NULL,
severity_at_transition NUMERIC(4,2) NOT NULL DEFAULT 0,
reason TEXT NOT NULL DEFAULT '',
transitioned_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_transition_archive_id ON ivanti_archive_transitions(archive_id);
CREATE TABLE IF NOT EXISTS ivanti_sync_anomaly_log (
id SERIAL PRIMARY KEY,
sync_timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(),
open_count_delta INTEGER NOT NULL DEFAULT 0,
closed_count_delta INTEGER NOT NULL DEFAULT 0,
newly_archived_count INTEGER NOT NULL DEFAULT 0,
returned_count INTEGER NOT NULL DEFAULT 0,
classification_json TEXT NOT NULL DEFAULT '{}',
return_classification_json TEXT NOT NULL DEFAULT '{}',
is_significant BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_anomaly_sync_timestamp ON ivanti_sync_anomaly_log(sync_timestamp);
CREATE TABLE IF NOT EXISTS ivanti_finding_bu_history (
id SERIAL PRIMARY KEY,
finding_id TEXT NOT NULL,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
previous_bu TEXT NOT NULL,
new_bu TEXT NOT NULL,
detected_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_bu_history_finding_id ON ivanti_finding_bu_history(finding_id);
CREATE INDEX IF NOT EXISTS idx_bu_history_detected_at ON ivanti_finding_bu_history(detected_at);
-- =============================================================================
-- Atlas action plans cache
-- =============================================================================
CREATE TABLE IF NOT EXISTS atlas_action_plans_cache (
id SERIAL PRIMARY KEY,
host_id INTEGER NOT NULL UNIQUE,
has_action_plan BOOLEAN NOT NULL DEFAULT FALSE,
plan_count INTEGER NOT NULL DEFAULT 0,
plans_json TEXT NOT NULL DEFAULT '[]',
synced_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_atlas_cache_host_id ON atlas_action_plans_cache(host_id);
-- =============================================================================
-- Compliance (NTS AEO) tracking
-- =============================================================================
CREATE TABLE IF NOT EXISTS compliance_uploads (
id SERIAL PRIMARY KEY,
filename TEXT NOT NULL,
report_date TEXT,
uploaded_by INTEGER REFERENCES users(id) ON DELETE SET NULL,
uploaded_at TIMESTAMPTZ DEFAULT NOW(),
new_count INTEGER DEFAULT 0,
resolved_count INTEGER DEFAULT 0,
recurring_count INTEGER DEFAULT 0,
summary_json TEXT
);
CREATE TABLE IF NOT EXISTS compliance_items (
id SERIAL PRIMARY KEY,
upload_id INTEGER NOT NULL REFERENCES compliance_uploads(id) ON DELETE CASCADE,
hostname TEXT NOT NULL,
ip_address TEXT,
device_type TEXT,
team TEXT,
metric_id TEXT NOT NULL,
metric_desc TEXT,
category TEXT,
extra_json TEXT,
status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'resolved')),
first_seen_upload_id INTEGER REFERENCES compliance_uploads(id) ON DELETE SET NULL,
resolved_upload_id INTEGER REFERENCES compliance_uploads(id) ON DELETE SET NULL,
seen_count INTEGER DEFAULT 1,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_compliance_items_upload ON compliance_items(upload_id);
CREATE INDEX IF NOT EXISTS idx_compliance_items_identity ON compliance_items(hostname, metric_id);
CREATE INDEX IF NOT EXISTS idx_compliance_items_team_status ON compliance_items(team, status);
CREATE TABLE IF NOT EXISTS compliance_notes (
id SERIAL PRIMARY KEY,
hostname TEXT NOT NULL,
metric_id TEXT NOT NULL,
note TEXT NOT NULL,
group_id TEXT,
created_by INTEGER REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_compliance_notes_identity ON compliance_notes(hostname, metric_id);
CREATE INDEX IF NOT EXISTS idx_compliance_notes_group ON compliance_notes(group_id);
-- =============================================================================
-- Seed data
-- =============================================================================
-- Required documents (idempotent via unique constraint on vendor + document_type)
INSERT INTO required_documents (vendor, document_type, is_mandatory, description) VALUES
('Microsoft', 'advisory', TRUE, 'Official Microsoft Security Advisory'),
('Microsoft', 'screenshot', FALSE, 'Proof of patch application'),
('Cisco', 'advisory', TRUE, 'Cisco Security Advisory'),
('Oracle', 'advisory', TRUE, 'Oracle Security Alert'),
('VMware', 'advisory', TRUE, 'VMware Security Advisory'),
('Adobe', 'advisory', TRUE, 'Adobe Security Bulletin')
ON CONFLICT (vendor, document_type) DO NOTHING;
-- Ivanti sync state — ensure single row exists
INSERT INTO ivanti_sync_state (id, total, workflows_json, sync_status)
VALUES (1, 0, '[]', 'never')
ON CONFLICT (id) DO NOTHING;
-- Ivanti counts cache — ensure single row exists
INSERT INTO ivanti_counts_cache (id, open_count, closed_count, fp_workflow_counts_json, fp_id_counts_json)
VALUES (1, 0, 0, '{}', '{}')
ON CONFLICT (id) DO NOTHING;

46
backend/db.js Normal file
View File

@@ -0,0 +1,46 @@
// PostgreSQL Connection Pool
// All route files import this module instead of receiving a sqlite3 `db` parameter.
// Configured via DATABASE_URL environment variable.
// Ensure dotenv is loaded before accessing env vars
require('dotenv').config({ path: require('path').join(__dirname, '.env') });
const { Pool } = require('pg');
if (!process.env.DATABASE_URL) {
console.error('[DB] FATAL: DATABASE_URL environment variable is not set.');
console.error('[DB] Expected format: postgresql://user:password@host:port/database');
process.exit(1);
}
const pool = new Pool({
connectionString: process.env.DATABASE_URL,
max: 10, // Maximum connections in pool
idleTimeoutMillis: 30000, // Close idle connections after 30s
connectionTimeoutMillis: 5000, // Fail if connection takes >5s
});
// Log unexpected pool errors (connection drops, etc.)
pool.on('error', (err) => {
console.error('[DB Pool] Unexpected error on idle client:', err.message);
});
// Track active connections and warn when approaching exhaustion
let _activeCount = 0;
pool.on('acquire', () => {
_activeCount++;
if (_activeCount >= 8) {
console.warn(`[DB Pool] WARNING: ${_activeCount}/10 connections active — approaching exhaustion`);
}
});
pool.on('release', () => { _activeCount--; });
// Health check — verify connection on startup
pool.query('SELECT NOW()')
.then(() => console.log('[DB Pool] Connected to PostgreSQL'))
.catch((err) => {
console.error('[DB Pool] Failed to connect:', err.message);
console.error('[DB Pool] Check DATABASE_URL and ensure Postgres is running on port 5433');
});
module.exports = pool;

104
backend/helpers/atlasApi.js Normal file
View File

@@ -0,0 +1,104 @@
// Shared Atlas InfoSec API helpers
// Centralizes HTTP calls so the atlas router uses a single implementation.
// Follows the same promise-based pattern as ivantiApi.js.
const https = require('https');
const http = require('http');
// ---------------------------------------------------------------------------
// Configuration — read from process.env at module load
// ---------------------------------------------------------------------------
const ATLAS_API_URL = process.env.ATLAS_API_URL || '';
const ATLAS_API_USER = process.env.ATLAS_API_USER || '';
const ATLAS_API_PASS = process.env.ATLAS_API_PASS || '';
const ATLAS_SKIP_TLS = process.env.ATLAS_SKIP_TLS === 'true';
const requiredVars = ['ATLAS_API_URL', 'ATLAS_API_USER', 'ATLAS_API_PASS'];
const missingVars = requiredVars.filter((v) => !process.env[v]);
if (missingVars.length > 0) {
console.warn(`[atlas-api] WARNING: Missing required environment variables: ${missingVars.join(', ')}. Atlas API calls will fail.`);
}
const isConfigured = missingVars.length === 0;
// ---------------------------------------------------------------------------
// Generic request — supports GET, PUT, PATCH, POST
// ---------------------------------------------------------------------------
function atlasRequest(method, urlPath, body, options) {
const timeout = (options && options.timeout) || 15000;
const authString = Buffer.from(ATLAS_API_USER + ':' + ATLAS_API_PASS).toString('base64');
const fullUrl = new URL(ATLAS_API_URL + urlPath);
const isHttps = fullUrl.protocol === 'https:';
const transport = isHttps ? https : http;
const headers = {
'accept': 'application/json',
'authorization': 'Basic ' + authString
};
let bodyStr = null;
if (body !== null && body !== undefined) {
bodyStr = JSON.stringify(body);
headers['content-type'] = 'application/json';
headers['content-length'] = Buffer.byteLength(bodyStr);
}
return new Promise((resolve, reject) => {
const reqOptions = {
hostname: fullUrl.hostname,
port: fullUrl.port || (isHttps ? 443 : 80),
path: fullUrl.pathname + fullUrl.search,
method: method,
headers: headers,
timeout: timeout
};
if (isHttps) {
reqOptions.rejectUnauthorized = !ATLAS_SKIP_TLS;
}
const req = transport.request(reqOptions, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => resolve({ status: res.statusCode, body: data }));
});
req.on('timeout', () => req.destroy(new Error(method + ' ' + urlPath + ' timed out')));
req.on('error', (err) => {
reject(new Error(method + ' ' + urlPath + ' failed: ' + err.message));
});
if (bodyStr) {
req.write(bodyStr);
}
req.end();
});
}
// ---------------------------------------------------------------------------
// Convenience wrappers
// ---------------------------------------------------------------------------
function atlasGet(urlPath, options) {
return atlasRequest('GET', urlPath, null, options);
}
function atlasPut(urlPath, body, options) {
return atlasRequest('PUT', urlPath, body, options);
}
function atlasPatch(urlPath, body, options) {
return atlasRequest('PATCH', urlPath, body, options);
}
function atlasPost(urlPath, body, options) {
return atlasRequest('POST', urlPath, body, options);
}
module.exports = {
isConfigured,
atlasRequest,
atlasGet,
atlasPut,
atlasPatch,
atlasPost
};

View File

@@ -1,21 +1,19 @@
// Audit Log Helper // Audit Log Helper
// Fire-and-forget insert - never blocks the response // Fire-and-forget insert - never blocks the response
const pool = require('../db');
function logAudit(db, { userId, username, action, entityType, entityId, details, ipAddress }) { function logAudit({ userId, username, action, entityType, entityId, details, ipAddress }) {
const detailsStr = details && typeof details === 'object' const detailsStr = details && typeof details === 'object'
? JSON.stringify(details) ? JSON.stringify(details)
: details || null; : details || null;
db.run( pool.query(
`INSERT INTO audit_logs (user_id, username, action, entity_type, entity_id, details, ip_address) `INSERT INTO audit_logs (user_id, username, action, entity_type, entity_id, details, ip_address)
VALUES (?, ?, ?, ?, ?, ?, ?)`, VALUES ($1, $2, $3, $4, $5, $6, $7)`,
[userId || null, username || 'unknown', action, entityType, entityId || null, detailsStr, ipAddress || null], [userId || null, username || 'unknown', action, entityType, entityId || null, detailsStr, ipAddress || null]
(err) => { ).catch((err) => {
if (err) { console.error('Audit log error:', err.message);
console.error('Audit log error:', err.message); });
}
}
);
} }
module.exports = logAudit; module.exports = logAudit;

305
backend/helpers/cardApi.js Normal file
View File

@@ -0,0 +1,305 @@
// Shared CARD API helpers
// Centralizes HTTP calls for the CARD asset ownership API.
// Follows the same promise-based pattern as atlasApi.js, with the addition
// of OAuth Bearer token management (auto-acquire, cache, refresh, 401 retry).
//
// CARD API versioning:
// - Read endpoints (GET): /api/v1/...
// - Mutation endpoints (POST): /api/v2/...
const https = require('https');
const http = require('http');
// ---------------------------------------------------------------------------
// Configuration — read from process.env at module load
// ---------------------------------------------------------------------------
const CARD_API_URL = process.env.CARD_API_URL || '';
const CARD_API_USER = process.env.CARD_API_USER || '';
const CARD_API_PASS = process.env.CARD_API_PASS || '';
const CARD_SKIP_TLS = process.env.CARD_SKIP_TLS === 'true';
const requiredVars = ['CARD_API_URL', 'CARD_API_USER', 'CARD_API_PASS'];
const missingVars = requiredVars.filter((v) => !process.env[v]);
if (missingVars.length > 0) {
console.warn(`[card-api] WARNING: Missing required environment variables: ${missingVars.join(', ')}. CARD API calls will fail.`);
}
const isConfigured = missingVars.length === 0;
// ---------------------------------------------------------------------------
// Token Manager — OAuth Bearer token with 1-hour TTL
// ---------------------------------------------------------------------------
let cachedToken = null; // { token: string, expiresAt: number (epoch ms) }
function tokenIsValid() {
if (!cachedToken) return false;
// Refresh if within 60 seconds of expiry
return cachedToken.expiresAt - Date.now() > 60_000;
}
function invalidateToken() {
cachedToken = null;
}
/**
* Acquire a new Bearer token from CARD /api/v1/auth/get_token using Basic Auth.
* Caches the token in memory with a 1-hour TTL.
*/
function acquireToken(timeout) {
const authString = Buffer.from(CARD_API_USER + ':' + CARD_API_PASS).toString('base64');
const fullUrl = new URL(CARD_API_URL + '/api/v1/auth/get_token');
const isHttps = fullUrl.protocol === 'https:';
const transport = isHttps ? https : http;
return new Promise((resolve, reject) => {
const reqOptions = {
hostname: fullUrl.hostname,
port: fullUrl.port || (isHttps ? 443 : 80),
path: fullUrl.pathname + fullUrl.search,
method: 'POST',
headers: {
'accept': 'application/json',
'authorization': 'Basic ' + authString,
'content-length': '0',
},
timeout: timeout || 15000,
};
if (isHttps) {
reqOptions.rejectUnauthorized = !CARD_SKIP_TLS;
}
const req = transport.request(reqOptions, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => {
if (res.statusCode < 200 || res.statusCode >= 300) {
return reject(new Error(
`[card-api] Token acquisition failed with HTTP ${res.statusCode}: ${data.substring(0, 500)}`
));
}
// The CARD API returns the token as a JSON string or object.
// Try to parse; fall back to raw body as the token string.
let token;
try {
const parsed = JSON.parse(data);
token = typeof parsed === 'string' ? parsed
: parsed.token || parsed.access_token || data.trim();
} catch (_) {
// Response may be a plain token string (unquoted)
token = data.trim();
}
if (!token) {
return reject(new Error('[card-api] Token parse failure: empty token in response body.'));
}
cachedToken = {
token,
expiresAt: Date.now() + 60 * 60 * 1000, // 1-hour TTL
};
resolve(cachedToken.token);
});
});
req.on('timeout', () => req.destroy(new Error('GET /api/v1/auth/get_token timed out')));
req.on('error', (err) => {
reject(new Error(`[card-api] GET /api/v1/auth/get_token failed: ${err.message}`));
});
req.end();
});
}
/**
* Ensure we have a valid Bearer token, acquiring or refreshing as needed.
*/
async function ensureToken(timeout) {
if (tokenIsValid()) return cachedToken.token;
return acquireToken(timeout);
}
// ---------------------------------------------------------------------------
// Generic request — supports GET and POST with Bearer auth + 401 retry
// ---------------------------------------------------------------------------
async function cardRequest(method, urlPath, body, options) {
const timeout = (options && options.timeout) || 15000;
const skipAuth = (options && options.skipAuth) || false;
async function doRequest(bearerToken) {
const fullUrl = new URL(CARD_API_URL + urlPath);
const isHttps = fullUrl.protocol === 'https:';
const transport = isHttps ? https : http;
const headers = { 'accept': 'application/json' };
if (bearerToken) {
headers['authorization'] = 'Bearer ' + bearerToken;
}
let bodyStr = null;
if (body !== null && body !== undefined) {
bodyStr = JSON.stringify(body);
headers['content-type'] = 'application/json';
headers['content-length'] = Buffer.byteLength(bodyStr);
}
return new Promise((resolve, reject) => {
const reqOptions = {
hostname: fullUrl.hostname,
port: fullUrl.port || (isHttps ? 443 : 80),
path: fullUrl.pathname + fullUrl.search,
method,
headers,
timeout,
};
if (isHttps) {
reqOptions.rejectUnauthorized = !CARD_SKIP_TLS;
}
const req = transport.request(reqOptions, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => resolve({ status: res.statusCode, body: data }));
});
req.on('timeout', () => req.destroy(new Error(`${method} ${urlPath} timed out`)));
req.on('error', (err) => {
reject(new Error(`[card-api] ${method} ${urlPath} failed: ${err.message}`));
});
if (bodyStr) req.write(bodyStr);
req.end();
});
}
// Skip auth for the token endpoint itself
if (skipAuth) {
return doRequest(null);
}
// Normal flow: ensure token → request → retry once on 401
let token = await ensureToken(timeout);
let result = await doRequest(token);
if (result.status === 401) {
// Invalidate and retry exactly once
invalidateToken();
token = await ensureToken(timeout);
result = await doRequest(token);
}
return result;
}
// ---------------------------------------------------------------------------
// Convenience wrappers
// ---------------------------------------------------------------------------
function cardGet(urlPath, options) {
return cardRequest('GET', urlPath, null, options);
}
function cardPost(urlPath, body, options) {
return cardRequest('POST', urlPath, body, options);
}
// ---------------------------------------------------------------------------
// High-level helpers used by the UAT test and routes
// ---------------------------------------------------------------------------
/**
* Test connection by acquiring a token. Returns { ok, token } or { ok, error }.
*/
async function testConnection() {
try {
const token = await acquireToken();
return { ok: true, token: token.substring(0, 12) + '...' };
} catch (err) {
return { ok: false, error: err.message };
}
}
/**
* GET /api/v1/teams — list all CARD teams.
*/
async function getTeams() {
const res = await cardGet('/api/v1/teams');
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* GET /api/v1/team/{teamName}/assets — list assets for a team.
*/
async function getTeamAssets(teamName, { disposition, page, pageSize } = {}) {
const params = new URLSearchParams();
if (disposition) params.set('disposition', disposition);
if (page) params.set('page', String(page));
params.set('page_size', String(pageSize || 50));
const qs = params.toString();
const res = await cardGet(`/api/v1/team/${encodeURIComponent(teamName)}/assets${qs ? '?' + qs : ''}`);
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* GET /api/v1/owner/{assetId} — get owner record including update_token.
*/
async function getOwner(assetId) {
const res = await cardGet(`/api/v1/owner/${encodeURIComponent(assetId)}`);
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* POST /api/v2/owner/{assetId}/confirm — confirm asset to a team.
*/
async function confirmAsset(assetId, teamName, updateToken, comment) {
const params = new URLSearchParams({ update_token: updateToken });
if (comment) params.set('comment', comment);
const res = await cardPost(
`/api/v2/owner/${encodeURIComponent(assetId)}/confirm?${params.toString()}`,
{ name: teamName }
);
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* POST /api/v2/owner/{assetId}/decline — decline asset from a team.
*/
async function declineAsset(assetId, teamName, updateToken, comment) {
const params = new URLSearchParams({ update_token: updateToken });
if (comment) params.set('comment', comment);
const res = await cardPost(
`/api/v2/owner/${encodeURIComponent(assetId)}/decline?${params.toString()}`,
{ name: teamName }
);
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* POST /api/v2/owner/{assetId}/{fromTeam}/redirect — redirect asset between teams.
*/
async function redirectAsset(assetId, fromTeam, toTeam, updateToken) {
const params = new URLSearchParams({ update_token: updateToken });
const res = await cardPost(
`/api/v2/owner/${encodeURIComponent(assetId)}/${encodeURIComponent(fromTeam)}/redirect?${params.toString()}`,
{ name: toTeam }
);
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
module.exports = {
isConfigured,
missingVars,
cardRequest,
cardGet,
cardPost,
testConnection,
getTeams,
getTeamAssets,
getOwner,
confirmAsset,
declineAsset,
redirectAsset,
invalidateToken,
};

View File

@@ -0,0 +1,332 @@
// Drift Checker — compares xlsx schema against parser config to detect structural drift
// Returns categorised findings: breaking, silent_miss, cosmetic
const fs = require('fs');
const path = require('path');
/**
* Load and validate the compliance parser configuration file.
* @param {string} configPath — absolute or relative path to compliance_config.json
* @returns {object} parsed config with metric_categories, core_cols, skip_sheets
* @throws {Error} descriptive error if file missing, invalid JSON, or missing required keys
*/
function loadConfig(configPath) {
let raw;
try {
raw = fs.readFileSync(configPath, 'utf8');
} catch (err) {
if (err.code === 'ENOENT') {
throw new Error(`Configuration file not found: ${configPath}`);
}
throw new Error(`Failed to read configuration file: ${err.message}`);
}
let config;
try {
config = JSON.parse(raw);
} catch (err) {
throw new Error(`Configuration file contains invalid JSON: ${err.message}`);
}
if (!config.metric_categories || typeof config.metric_categories !== 'object' || Array.isArray(config.metric_categories)) {
throw new Error('Configuration file is missing required key "metric_categories" (must be an object)');
}
if (!Array.isArray(config.core_cols)) {
throw new Error('Configuration file is missing required key "core_cols" (must be an array)');
}
if (!Array.isArray(config.skip_sheets)) {
throw new Error('Configuration file is missing required key "skip_sheets" (must be an array)');
}
return config;
}
/**
* Compare an xlsx schema against the parser config and produce a drift report.
* @param {object} schema — output of extract_xlsx_schema.py: { sheets: [{ name, columns, metric_values? }] }
* @param {object} config — parsed compliance_config.json: { metric_categories, core_cols, skip_sheets }
* @returns {{ breaking: Array, silent_miss: Array, cosmetic: Array }}
*/
function compareSchemaToDrift(schema, config) {
const breaking = [];
const silent_miss = [];
const cosmetic = [];
const metricCategoryKeys = new Set(Object.keys(config.metric_categories));
const coreCols = new Set(config.core_cols);
const skipSheets = new Set(config.skip_sheets);
// Build lookup of xlsx sheet names and find the Summary sheet
const xlsxSheetNames = new Set();
let summarySheet = null;
for (const sheet of schema.sheets) {
xlsxSheetNames.add(sheet.name);
if (sheet.name === 'Summary') {
summarySheet = sheet;
}
}
// Identify detail sheets: present in xlsx AND not in skip_sheets
const detailSheets = schema.sheets.filter(s => !skipSheets.has(s.name));
// Build set of metric values from the Summary sheet (used by multiple rules)
const summaryMetrics = new Set(
(summarySheet && Array.isArray(summarySheet.metric_values)) ? summarySheet.metric_values : []
);
// --- Breaking rules ---
// Missing core column: a detail sheet is missing a column from core_cols.
// Collect per-column stats first, then classify: if a column is missing from
// ALL detail sheets it's breaking. If missing from only some (e.g. 5.8.1 uses
// CMDB columns), it's cosmetic — the parser handles it via extra_json.
const coreColMissingMap = {}; // col -> [sheet names missing it]
for (const sheet of detailSheets) {
const sheetCols = new Set(sheet.columns || []);
for (const coreCol of config.core_cols) {
if (!sheetCols.has(coreCol)) {
if (!coreColMissingMap[coreCol]) coreColMissingMap[coreCol] = [];
coreColMissingMap[coreCol].push(sheet.name);
}
}
}
for (const coreCol of Object.keys(coreColMissingMap)) {
const missingSheets = coreColMissingMap[coreCol];
if (detailSheets.length > 0 && missingSheets.length >= detailSheets.length) {
// Missing from ALL detail sheets — genuinely breaking
breaking.push({
severity: 'breaking',
message: `Core column "${coreCol}" is missing from all ${detailSheets.length} detail sheet(s)`,
value: coreCol,
sheet: null
});
} else {
// Missing from some sheets — structural difference, not drift
cosmetic.push({
severity: 'cosmetic',
message: `Core column "${coreCol}" is missing from ${missingSheets.length} of ${detailSheets.length} detail sheet(s): ${missingSheets.join(', ')}`,
value: coreCol,
sheet: null
});
}
}
// Missing detail sheet: a sheet in metric_categories (not in skip_sheets) is absent from xlsx.
// If the metric still appears in the Summary's metric_values, it's tracked but has zero
// violations this week — downgrade to cosmetic instead of breaking.
for (const metricKey of metricCategoryKeys) {
if (!skipSheets.has(metricKey) && !xlsxSheetNames.has(metricKey)) {
if (summaryMetrics.has(metricKey)) {
cosmetic.push({
severity: 'cosmetic',
message: `Metric "${metricKey}" has no detail sheet this week — still tracked in Summary (zero violations)`,
value: metricKey,
sheet: null
});
} else {
breaking.push({
severity: 'breaking',
message: `Expected detail sheet "${metricKey}" (metric category) is missing from the workbook`,
value: metricKey,
sheet: null
});
}
}
}
// --- Silent-miss rules ---
// Unknown metric value: a metric value in Summary is not a key in metric_categories
if (summarySheet && Array.isArray(summarySheet.metric_values)) {
for (const metricVal of summarySheet.metric_values) {
if (!metricCategoryKeys.has(metricVal)) {
silent_miss.push({
severity: 'silent_miss',
message: `Unknown metric "${metricVal}" in Summary — not in metric_categories`,
value: metricVal,
sheet: 'Summary'
});
}
}
}
// Unknown sheet: an xlsx sheet not in skip_sheets and not in metric_categories
for (const sheet of schema.sheets) {
if (!skipSheets.has(sheet.name) && !metricCategoryKeys.has(sheet.name)) {
silent_miss.push({
severity: 'silent_miss',
message: `Unknown sheet "${sheet.name}" — not in skip_sheets or metric_categories`,
value: sheet.name,
sheet: sheet.name
});
}
}
// --- Cosmetic rules ---
// New column in detail sheet: a detail sheet has columns not in core_cols
for (const sheet of detailSheets) {
for (const col of (sheet.columns || [])) {
if (!coreCols.has(col)) {
cosmetic.push({
severity: 'cosmetic',
message: `New column "${col}" in sheet "${sheet.name}" — will be captured in extra_json`,
value: col,
sheet: sheet.name
});
}
}
}
// Stale metric category: a key in metric_categories not in Summary metric values
for (const metricKey of metricCategoryKeys) {
if (!summaryMetrics.has(metricKey)) {
cosmetic.push({
severity: 'cosmetic',
message: `Stale metric category "${metricKey}" — not found in Summary sheet metric values`,
value: metricKey,
sheet: null
});
}
}
return { breaking, silent_miss, cosmetic };
}
/**
* Reconcile the parser config to resolve breaking drift findings.
*
* Breaking — "missing detail sheet":
* A metric_categories key has no matching xlsx sheet. But if the metric
* still appears in the Summary sheet's metric_values, it's a legitimate
* tracked metric that simply doesn't have violations this week — keep it.
* Only remove metrics absent from BOTH the xlsx sheets AND the Summary.
*
* Breaking — "missing core column":
* A core_cols entry is absent from one or more detail sheets. Only remove
* if the column is missing from ALL detail sheets (some sheets like 5.8.1
* have a completely different column structure and shouldn't cause removal).
*
* Silent-miss — "unknown metric":
* A metric value in the Summary is not in metric_categories. Add it as 'Other'.
*
* Silent-miss — "unknown sheet":
* Left as a warning. Auto-adding unknown sheets creates a reconcile loop.
*
* @param {string} configPath — path to compliance_config.json
* @param {object} driftReport — the drift report from compareSchemaToDrift()
* @param {object} [schema] — optional xlsx schema (with sheets[].name and Summary metric_values)
* @returns {{ changes: Array<{ action: string, key: string, value: string }>, config: object }}
*/
function reconcileConfig(configPath, driftReport, schema) {
const config = loadConfig(configPath);
const changes = [];
// Build a set of metric values from the Summary sheet (if schema provided)
const summaryMetrics = new Set();
if (schema && Array.isArray(schema.sheets)) {
const summarySheet = schema.sheets.find(function(s) { return s.name === 'Summary'; });
if (summarySheet && Array.isArray(summarySheet.metric_values)) {
summarySheet.metric_values.forEach(function(v) { summaryMetrics.add(v); });
}
}
// Build a set of xlsx sheet names (if schema provided)
const xlsxSheetNames = new Set();
if (schema && Array.isArray(schema.sheets)) {
schema.sheets.forEach(function(s) { xlsxSheetNames.add(s.name); });
}
// Count how many detail sheets exist in the xlsx (excluding skip_sheets)
const skipSheets = new Set(config.skip_sheets);
const detailSheetCount = schema
? schema.sheets.filter(function(s) { return !skipSheets.has(s.name); }).length
: 0;
// --- Resolve breaking findings ---
for (const finding of (driftReport.breaking || [])) {
// Missing detail sheet: remove from metric_categories ONLY if the metric
// is also absent from the Summary's metric_values. If it's in the Summary,
// it's still a tracked metric — the sheet just has zero violations this week.
if (finding.message.includes('is missing from the workbook') && finding.value in config.metric_categories) {
if (summaryMetrics.has(finding.value)) {
// Metric is in the Summary — keep it, just note it's sheet-less this week
changes.push({
action: 'kept',
key: 'metric_categories',
value: finding.value,
detail: `Kept metric "${finding.value}" — no detail sheet this week but still tracked in Summary`
});
} else {
const oldCategory = config.metric_categories[finding.value];
delete config.metric_categories[finding.value];
changes.push({
action: 'removed',
key: 'metric_categories',
value: finding.value,
detail: `Removed stale metric category "${finding.value}" (was "${oldCategory}") — absent from both workbook sheets and Summary`
});
}
}
// Missing core column: only remove if the column is missing from ALL detail sheets.
// Some sheets (e.g. 5.8.1 with CMDB columns) have a completely different structure
// and shouldn't cause removal of columns that exist in most other sheets.
if (finding.message.includes('is missing core column') && config.core_cols.includes(finding.value)) {
if (!changes.some(function(c) { return c.key === 'core_cols' && c.value === finding.value; })) {
const missingFromCount = (driftReport.breaking || []).filter(
function(f) { return f.message.includes('is missing core column') && f.value === finding.value; }
).length;
if (detailSheetCount > 0 && missingFromCount >= detailSheetCount) {
// Missing from ALL detail sheets — safe to remove
config.core_cols = config.core_cols.filter(function(c) { return c !== finding.value; });
changes.push({
action: 'removed',
key: 'core_cols',
value: finding.value,
detail: `Removed core column "${finding.value}" — missing from all ${detailSheetCount} detail sheet(s)`
});
} else {
// Missing from some sheets but present in others — keep it
changes.push({
action: 'kept',
key: 'core_cols',
value: finding.value,
detail: `Kept core column "${finding.value}" — missing from ${missingFromCount} of ${detailSheetCount} detail sheet(s)`
});
}
}
}
}
// --- Resolve silent-miss findings ---
for (const finding of (driftReport.silent_miss || [])) {
// Unknown metric in Summary: add to metric_categories as 'Other'
if (finding.message.includes('not in metric_categories') && !(finding.value in config.metric_categories)) {
config.metric_categories[finding.value] = 'Other';
changes.push({
action: 'added',
key: 'metric_categories',
value: finding.value,
detail: `Added new metric "${finding.value}" to metric_categories as "Other"`
});
}
// Unknown sheet: left as a warning — auto-adding creates a reconcile loop.
}
// Only write if there were actual config mutations (not just 'kept' entries)
const hasMutations = changes.some(function(c) { return c.action !== 'kept'; });
if (hasMutations) {
fs.writeFileSync(configPath, JSON.stringify(config, null, 2) + '\n', 'utf8');
}
return { changes, config };
}
module.exports = { compareSchemaToDrift, loadConfig, reconcileConfig };

View File

@@ -1,93 +0,0 @@
const { spawn } = require('child_process');
const path = require('path');
const fs = require('fs');
/**
* Process vulnerability report Excel file by splitting CVE IDs into separate rows
* @param {string} inputPath - Path to original Excel file
* @param {string} outputPath - Path for processed Excel file
* @returns {Promise<{original_rows: number, processed_rows: number, output_path: string}>}
*/
function processVulnerabilityReport(inputPath, outputPath) {
return new Promise((resolve, reject) => {
const scriptPath = path.join(__dirname, '..', 'scripts', 'split_cve_report.py');
// Verify script exists
if (!fs.existsSync(scriptPath)) {
return reject(new Error(`Python script not found: ${scriptPath}`));
}
// Verify input file exists
if (!fs.existsSync(inputPath)) {
return reject(new Error(`Input file not found: ${inputPath}`));
}
const python = spawn('python3', [scriptPath, inputPath, outputPath]);
let stdout = '';
let stderr = '';
let timedOut = false;
// 30 second timeout
const timeout = setTimeout(() => {
timedOut = true;
python.kill();
reject(new Error('Processing timed out. File may be too large or corrupted.'));
}, 30000);
python.stdout.on('data', (data) => {
stdout += data.toString();
});
python.stderr.on('data', (data) => {
stderr += data.toString();
});
python.on('close', (code) => {
clearTimeout(timeout);
if (timedOut) return;
if (code !== 0) {
// Parse Python error messages
if (stderr.includes('Sheet') && stderr.includes('not found')) {
return reject(new Error('Invalid Excel file. Expected "Vulnerabilities" sheet with "CVE ID" column.'));
}
if (stderr.includes('pandas') || stderr.includes('openpyxl')) {
return reject(new Error('Python dependencies missing. Run: pip3 install pandas openpyxl'));
}
return reject(new Error(`Python script failed: ${stderr || 'Unknown error'}`));
}
// Parse output for row counts
const originalMatch = stdout.match(/Original rows:\s*(\d+)/);
const newMatch = stdout.match(/New rows:\s*(\d+)/);
if (!originalMatch || !newMatch) {
return reject(new Error('Failed to parse row counts from Python output'));
}
// Verify output file was created
if (!fs.existsSync(outputPath)) {
return reject(new Error('Processed file was not created'));
}
resolve({
original_rows: parseInt(originalMatch[1]),
processed_rows: parseInt(newMatch[1]),
output_path: outputPath
});
});
python.on('error', (err) => {
clearTimeout(timeout);
if (err.code === 'ENOENT') {
reject(new Error('Python 3 is required but not found. Please install Python.'));
} else {
reject(err);
}
});
});
}
module.exports = { processVulnerabilityReport };

View File

@@ -0,0 +1,154 @@
// Shared Ivanti / RiskSense API helpers
// Centralizes HTTP calls so ivantiWorkflows.js, ivantiFindings.js, and
// ivantiFpWorkflow.js all use the same implementation.
const https = require('https');
const IVANTI_URL_BASE = 'https://platform4.risksense.com/api/v1';
// ---------------------------------------------------------------------------
// JSON POST — used for search, workflow creation, etc.
// ---------------------------------------------------------------------------
function ivantiPost(urlPath, body, apiKey, skipTls) {
const bodyStr = JSON.stringify(body);
const fullUrl = new URL(IVANTI_URL_BASE + urlPath);
return new Promise((resolve, reject) => {
const options = {
hostname: fullUrl.hostname,
path: fullUrl.pathname + fullUrl.search,
method: 'POST',
headers: {
'accept': '*/*',
'content-type': 'application/json',
'x-api-key': apiKey,
'x-http-client-type': 'browser',
'content-length': Buffer.byteLength(bodyStr)
},
rejectUnauthorized: !skipTls,
timeout: 15000
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => resolve({ status: res.statusCode, body: data }));
});
req.on('timeout', () => req.destroy(new Error('Request timed out')));
req.on('error', reject);
req.write(bodyStr);
req.end();
});
}
// ---------------------------------------------------------------------------
// Multipart POST — used for file attachment uploads.
// Constructs multipart/form-data manually using Node's https module.
// ---------------------------------------------------------------------------
function ivantiMultipartPost(urlPath, fileBuffer, fileName, apiKey, skipTls) {
const boundary = '----IvantiUpload' + Date.now().toString(36) + Math.random().toString(36).slice(2);
const fullUrl = new URL(IVANTI_URL_BASE + urlPath);
// Build multipart body
const preamble = Buffer.from(
`--${boundary}\r\n` +
`Content-Disposition: form-data; name="file"; filename="${fileName}"\r\n` +
`Content-Type: application/octet-stream\r\n\r\n`
);
const epilogue = Buffer.from(`\r\n--${boundary}--\r\n`);
const bodyBuffer = Buffer.concat([preamble, fileBuffer, epilogue]);
return new Promise((resolve, reject) => {
const options = {
hostname: fullUrl.hostname,
path: fullUrl.pathname + fullUrl.search,
method: 'POST',
headers: {
'accept': '*/*',
'content-type': `multipart/form-data; boundary=${boundary}`,
'x-api-key': apiKey,
'x-http-client-type': 'browser',
'content-length': bodyBuffer.length
},
rejectUnauthorized: !skipTls,
timeout: 30000
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => resolve({ status: res.statusCode, body: data }));
});
req.on('timeout', () => req.destroy(new Error('Request timed out')));
req.on('error', reject);
req.write(bodyBuffer);
req.end();
});
}
// ---------------------------------------------------------------------------
// Multipart form POST — used for endpoints that accept mixed form fields + files.
// fields: array of { name, value } for text form fields
// files: array of { name, buffer, filename } for file uploads
// ---------------------------------------------------------------------------
function ivantiFormPost(urlPath, fields, files, apiKey, skipTls) {
const boundary = '----IvantiForm' + Date.now().toString(36) + Math.random().toString(36).slice(2);
const fullUrl = new URL(IVANTI_URL_BASE + urlPath);
const parts = [];
// Text fields
for (const { name, value } of fields) {
parts.push(Buffer.from(
`--${boundary}\r\n` +
`Content-Disposition: form-data; name="${name}"\r\n\r\n` +
`${value}\r\n`
));
}
// File fields
for (const { name, buffer, filename, contentType } of files) {
parts.push(Buffer.from(
`--${boundary}\r\n` +
`Content-Disposition: form-data; name="${name}"; filename="${filename}"\r\n` +
`Content-Type: ${contentType || 'application/octet-stream'}\r\n\r\n`
));
parts.push(buffer);
parts.push(Buffer.from('\r\n'));
}
parts.push(Buffer.from(`--${boundary}--\r\n`));
const bodyBuffer = Buffer.concat(parts);
return new Promise((resolve, reject) => {
const options = {
hostname: fullUrl.hostname,
path: fullUrl.pathname + fullUrl.search,
method: 'POST',
headers: {
'accept': '*/*',
'content-type': `multipart/form-data; boundary=${boundary}`,
'x-api-key': apiKey,
'x-http-client-type': 'browser',
'content-length': bodyBuffer.length
},
rejectUnauthorized: !skipTls,
timeout: 60000
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => resolve({ status: res.statusCode, body: data }));
});
req.on('timeout', () => req.destroy(new Error('Request timed out')));
req.on('error', reject);
req.write(bodyBuffer);
req.end();
});
}
module.exports = { IVANTI_URL_BASE, ivantiPost, ivantiMultipartPost, ivantiFormPost };

453
backend/helpers/jiraApi.js Normal file
View File

@@ -0,0 +1,453 @@
// Shared Jira Data Center REST API helpers
// Centralizes HTTP calls for Jira issue operations.
// Follows the same promise-based pattern as atlasApi.js and ivantiApi.js.
//
// =========================================================================
// Charter Jira REST API Compliance
// =========================================================================
// Authentication:
// - Service accounts use Basic Auth (required for shared integrations).
// - PATs require ATLSUP approval and naming convention:
// Function - Team - Approved ATLSUP ticket
// - SSO must NOT be used for REST API integrations.
//
// Rate limiting (Charter-posted):
// - 1 440 requests/day max
// - Burst cap of 60 requests/minute (accumulates 1 req/idle minute)
// - 429 response when limits are hit server-side
//
// Automation delays (Charter requirement):
// - 1 second delay between GET requests
// - 2 second delay between PUT, POST, or DELETE requests
//
// Forbidden patterns:
// - /rest/api/2/field — must specify fields explicitly in every call
// - /rest/api/2/issue/bulk — bulk updates are not allowed
// - Single-issue GET loops — use bulk JQL search instead
//
// Required patterns:
// - All GET requests MUST include a ?fields= parameter
// - JQL MUST include at least one of: project+updated, assignee+updated,
// status+updated
// - JQL should use &updated>=-Xh to only fetch changed issues
// - maxResults=1000 for search queries
// - Issues must be updated one at a time (no bulk PUT)
// =========================================================================
const https = require('https');
const http = require('http');
// ---------------------------------------------------------------------------
// Configuration — read from process.env at module load
// ---------------------------------------------------------------------------
const JIRA_BASE_URL = process.env.JIRA_BASE_URL || '';
const JIRA_AUTH_METHOD = (process.env.JIRA_AUTH_METHOD || 'basic').toLowerCase();
const JIRA_API_USER = process.env.JIRA_API_USER || '';
const JIRA_API_TOKEN = process.env.JIRA_API_TOKEN || '';
const JIRA_PAT = process.env.JIRA_PAT || '';
const JIRA_SKIP_TLS = process.env.JIRA_SKIP_TLS === 'true';
const JIRA_PROJECT_KEY = process.env.JIRA_PROJECT_KEY || '';
const JIRA_ISSUE_TYPE = process.env.JIRA_ISSUE_TYPE || 'Task';
const requiredVars = JIRA_AUTH_METHOD === 'pat'
? ['JIRA_BASE_URL', 'JIRA_PAT']
: ['JIRA_BASE_URL', 'JIRA_API_USER', 'JIRA_API_TOKEN'];
const missingVars = requiredVars.filter((v) => !process.env[v]);
if (missingVars.length > 0) {
console.warn(`[jira-api] WARNING: Missing required environment variables: ${missingVars.join(', ')}. Jira API calls will fail.`);
}
const isConfigured = missingVars.length === 0;
// ---------------------------------------------------------------------------
// Default fields — every GET must specify fields explicitly.
// /rest/api/2/field is forbidden; we define the field list here.
// ---------------------------------------------------------------------------
const DEFAULT_FIELDS = [
'summary', 'status', 'assignee', 'created', 'updated',
'priority', 'issuetype', 'project', 'resolution'
];
// ---------------------------------------------------------------------------
// Rate limiter — enforces Charter's posted limits
// 1 440 events/day, burst of 60 events/minute
// ---------------------------------------------------------------------------
const DAILY_LIMIT = 1440;
const BURST_LIMIT = 60;
const MINUTE_MS = 60 * 1000;
const DAY_MS = 24 * 60 * 60 * 1000;
let dailyLog = [];
let minuteLog = [];
function pruneLog(log, windowMs) {
const cutoff = Date.now() - windowMs;
while (log.length > 0 && log[0] < cutoff) {
log.shift();
}
}
function checkRateLimit() {
pruneLog(dailyLog, DAY_MS);
pruneLog(minuteLog, MINUTE_MS);
if (dailyLog.length >= DAILY_LIMIT) {
return { allowed: false, reason: `Daily Jira API limit reached (${DAILY_LIMIT}/day). Resets at midnight.` };
}
if (minuteLog.length >= BURST_LIMIT) {
return { allowed: false, reason: `Burst Jira API limit reached (${BURST_LIMIT}/min). Wait and retry.` };
}
return { allowed: true };
}
function recordRequest() {
const now = Date.now();
dailyLog.push(now);
minuteLog.push(now);
}
/**
* Return current rate limit usage for diagnostics.
*/
function getRateLimitStatus() {
pruneLog(dailyLog, DAY_MS);
pruneLog(minuteLog, MINUTE_MS);
return {
daily: { used: dailyLog.length, limit: DAILY_LIMIT, remaining: DAILY_LIMIT - dailyLog.length },
burst: { used: minuteLog.length, limit: BURST_LIMIT, remaining: BURST_LIMIT - minuteLog.length }
};
}
// ---------------------------------------------------------------------------
// Inter-request delay — Charter automation requirements
// 1s between GETs, 2s between PUT/POST/DELETE
// ---------------------------------------------------------------------------
const GET_DELAY_MS = 1000;
const WRITE_DELAY_MS = 2000;
let lastRequestTime = 0;
let lastRequestMethod = '';
/**
* Wait the required delay before issuing the next request.
* GET → 1s, PUT/POST/DELETE → 2s since the previous request.
*/
function waitForDelay(method) {
const now = Date.now();
const requiredDelay = (lastRequestMethod === 'GET') ? GET_DELAY_MS
: (lastRequestMethod !== '') ? WRITE_DELAY_MS : 0;
const elapsed = now - lastRequestTime;
const remaining = requiredDelay - elapsed;
if (remaining > 0) {
return new Promise(resolve => setTimeout(resolve, remaining));
}
return Promise.resolve();
}
// ---------------------------------------------------------------------------
// Blocked endpoint guard
// ---------------------------------------------------------------------------
const BLOCKED_PATHS = [
'/rest/api/2/field', // Must specify fields in call, not query field list
'/rest/api/2/issue/bulk', // Bulk updates are not allowed
];
function isBlockedPath(urlPath) {
return BLOCKED_PATHS.some(blocked => urlPath.startsWith(blocked));
}
// ---------------------------------------------------------------------------
// Generic request — supports GET, POST, PUT, DELETE
// Enforces rate limits, inter-request delays, and blocked-path guards.
// ---------------------------------------------------------------------------
async function jiraRequest(method, urlPath, body, options) {
// Block forbidden endpoints
if (isBlockedPath(urlPath)) {
return Promise.reject(new Error(`Blocked: ${urlPath} is not allowed per Charter Jira API policy.`));
}
const limit = checkRateLimit();
if (!limit.allowed) {
return Promise.reject(new Error(limit.reason));
}
// Enforce inter-request delay
await waitForDelay(method);
const timeout = (options && options.timeout) || 15000;
const fullUrl = new URL(JIRA_BASE_URL + urlPath);
const isHttps = fullUrl.protocol === 'https:';
const transport = isHttps ? https : http;
const headers = {
'accept': 'application/json'
};
// Auth header
if (JIRA_AUTH_METHOD === 'pat') {
headers['authorization'] = 'Bearer ' + JIRA_PAT;
} else {
const authString = Buffer.from(JIRA_API_USER + ':' + JIRA_API_TOKEN).toString('base64');
headers['authorization'] = 'Basic ' + authString;
}
let bodyStr = null;
if (body !== null && body !== undefined) {
bodyStr = JSON.stringify(body);
headers['content-type'] = 'application/json';
headers['content-length'] = Buffer.byteLength(bodyStr);
}
recordRequest();
lastRequestTime = Date.now();
lastRequestMethod = method;
return new Promise((resolve, reject) => {
const reqOptions = {
hostname: fullUrl.hostname,
port: fullUrl.port || (isHttps ? 443 : 80),
path: fullUrl.pathname + fullUrl.search,
method: method,
headers: headers,
timeout: timeout
};
if (isHttps) {
reqOptions.rejectUnauthorized = !JIRA_SKIP_TLS;
}
const req = transport.request(reqOptions, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => {
if (res.statusCode === 429) {
resolve({ status: 429, body: data, rateLimited: true });
} else {
resolve({ status: res.statusCode, body: data });
}
});
});
req.on('timeout', () => req.destroy(new Error(method + ' ' + urlPath + ' timed out')));
req.on('error', (err) => {
reject(new Error(method + ' ' + urlPath + ' failed: ' + err.message));
});
if (bodyStr) {
req.write(bodyStr);
}
req.end();
});
}
// ---------------------------------------------------------------------------
// Convenience wrappers
// ---------------------------------------------------------------------------
function jiraGet(urlPath, options) {
return jiraRequest('GET', urlPath, null, options);
}
function jiraPost(urlPath, body, options) {
return jiraRequest('POST', urlPath, body, options);
}
function jiraPut(urlPath, body, options) {
return jiraRequest('PUT', urlPath, body, options);
}
function jiraDelete(urlPath, options) {
return jiraRequest('DELETE', urlPath, null, options);
}
// ---------------------------------------------------------------------------
// High-level Jira operations — all comply with Charter requirements
// ---------------------------------------------------------------------------
/**
* Fetch a single issue by key using a GET with explicit ?fields= parameter.
* Charter requires all GETs to specify fields — /rest/api/2/field is forbidden.
*
* NOTE: For syncing multiple tickets, prefer searchIssuesByKeys() which uses
* a single bulk JQL search instead of one GET per issue.
*
* @param {string} issueKey - e.g. "VULN-123"
* @param {string[]} [fields] - Jira field names to return
*/
async function getIssue(issueKey, fields) {
const jql = `key = "${issueKey}" AND project = ${JIRA_PROJECT_KEY}`;
const result = await searchIssues(jql, { fields: fields || DEFAULT_FIELDS, maxResults: 1, startAt: 0 });
if (result.ok && result.data.issues && result.data.issues.length > 0) {
return { ok: true, data: result.data.issues[0] };
}
if (result.ok && (!result.data.issues || result.data.issues.length === 0)) {
return { ok: false, status: 404, body: 'Issue not found' };
}
return result;
}
/**
* Bulk-fetch issues by their keys using a single JQL search.
* This is the Charter-compliant way to sync multiple tickets — avoids
* querying one issue at a time.
*
* @param {string[]} issueKeys - Array of Jira issue keys
* @param {object} [opts] - { fields, maxResults }
*/
async function searchIssuesByKeys(issueKeys, opts) {
if (!issueKeys || issueKeys.length === 0) {
return { ok: true, data: { total: 0, issues: [] } };
}
// Build JQL: key in (KEY-1, KEY-2, ...) — Charter requires project+updated
// or similar, but key-based search is inherently scoped. We add updated
// clause for compliance.
const keyList = issueKeys.map(k => `"${k}"`).join(', ');
const jql = `key in (${keyList}) AND updated >= -72h AND project = ${JIRA_PROJECT_KEY}`;
const fields = (opts && opts.fields) || DEFAULT_FIELDS;
const maxResults = Math.min((opts && opts.maxResults) || 1000, 1000);
return searchIssues(jql, { fields, maxResults, startAt: 0 });
}
/**
* Search issues via JQL (POST to /rest/api/2/search).
* Charter requirements enforced:
* - fields array is always specified (never omitted)
* - maxResults capped at 1000
*
* The caller is responsible for including an &updated clause in the JQL
* for recurring/scheduled queries.
*
* @param {string} jql - JQL query string
* @param {object} [opts] - { startAt, maxResults, fields }
*/
async function searchIssues(jql, opts) {
const startAt = (opts && opts.startAt) || 0;
const maxResults = Math.min((opts && opts.maxResults) || 1000, 1000);
const fields = (opts && opts.fields) || DEFAULT_FIELDS;
const fieldList = encodeURIComponent(fields.join(','));
const encodedJql = encodeURIComponent(jql);
const queryString = `?jql=${encodedJql}&fields=${fieldList}&maxResults=${maxResults}&startAt=${startAt}`;
const res = await jiraGet('/rest/api/2/search' + queryString);
if (res.status === 200) {
return { ok: true, data: JSON.parse(res.body) };
}
return { ok: false, status: res.status, body: res.body, rateLimited: res.rateLimited };
}
/**
* Create a new Jira issue (POST, subject to 2s delay).
* @param {object} fields - Jira issue fields object
*/
async function createIssue(fields) {
const res = await jiraPost('/rest/api/2/issue', { fields });
if (res.status === 201) {
return { ok: true, data: JSON.parse(res.body) };
}
return { ok: false, status: res.status, body: res.body, rateLimited: res.rateLimited };
}
/**
* Update a single Jira issue (PUT, subject to 2s delay).
* Charter forbids bulk updates — issues must be updated one at a time.
* @param {string} issueKey
* @param {object} fields - Fields to update
*/
async function updateIssue(issueKey, fields) {
const res = await jiraPut(
`/rest/api/2/issue/${encodeURIComponent(issueKey)}`,
{ fields }
);
// Jira returns 204 on successful update
if (res.status === 204) {
return { ok: true };
}
return { ok: false, status: res.status, body: res.body, rateLimited: res.rateLimited };
}
/**
* Add a comment to an existing issue (POST, subject to 2s delay).
*/
async function addComment(issueKey, commentBody) {
const res = await jiraPost(
`/rest/api/2/issue/${encodeURIComponent(issueKey)}/comment`,
{ body: commentBody }
);
if (res.status === 201) {
return { ok: true, data: JSON.parse(res.body) };
}
return { ok: false, status: res.status, body: res.body, rateLimited: res.rateLimited };
}
/**
* Transition an issue to a new status (POST, subject to 2s delay).
* @param {string} issueKey
* @param {string} transitionId
*/
async function transitionIssue(issueKey, transitionId) {
const res = await jiraPost(
`/rest/api/2/issue/${encodeURIComponent(issueKey)}/transitions`,
{ transition: { id: transitionId } }
);
if (res.status === 204) {
return { ok: true };
}
return { ok: false, status: res.status, body: res.body, rateLimited: res.rateLimited };
}
/**
* Get available transitions for an issue.
* Uses GET with explicit fields parameter (transitions endpoint returns
* transitions by default, but we include the query param for compliance).
*/
async function getTransitions(issueKey) {
const res = await jiraGet(
`/rest/api/2/issue/${encodeURIComponent(issueKey)}/transitions`
);
if (res.status === 200) {
return { ok: true, data: JSON.parse(res.body) };
}
return { ok: false, status: res.status, body: res.body, rateLimited: res.rateLimited };
}
/**
* Test connectivity — calls /rest/api/2/myself to verify credentials.
* This is a lightweight GET that returns the authenticated user.
*/
async function testConnection() {
try {
const res = await jiraGet('/rest/api/2/myself');
if (res.status === 200) {
const user = JSON.parse(res.body);
return { ok: true, user: { name: user.name, displayName: user.displayName, emailAddress: user.emailAddress } };
}
return { ok: false, status: res.status, body: res.body };
} catch (err) {
return { ok: false, error: err.message };
}
}
module.exports = {
isConfigured,
jiraRequest,
jiraGet,
jiraPost,
jiraPut,
jiraDelete,
getIssue,
searchIssuesByKeys,
searchIssues,
createIssue,
updateIssue,
addComment,
transitionIssue,
getTransitions,
testConnection,
getRateLimitStatus,
DEFAULT_FIELDS,
JIRA_PROJECT_KEY,
JIRA_ISSUE_TYPE
};

26
backend/helpers/teams.js Normal file
View File

@@ -0,0 +1,26 @@
// Shared BU team constants and validation
// Used by user management routes, auth middleware, and frontend-facing endpoints.
const KNOWN_TEAMS = ['STEAM', 'ACCESS-ENG', 'ACCESS-OPS', 'INTELDEV'];
/**
* Parse and validate a comma-separated teams string.
* @param {string} teamsString - Comma-separated team identifiers (e.g. 'STEAM,ACCESS-ENG')
* @returns {{ valid: boolean, teams: string[], invalid: string[] }}
*/
function validateTeams(teamsString) {
if (!teamsString || typeof teamsString !== 'string' || teamsString.trim() === '') {
return { valid: true, teams: [], invalid: [] };
}
const teams = teamsString.split(',').map(t => t.trim()).filter(Boolean);
const invalid = teams.filter(t => !KNOWN_TEAMS.includes(t));
return {
valid: invalid.length === 0,
teams,
invalid
};
}
module.exports = { KNOWN_TEAMS, validateTeams };

View File

@@ -1,7 +1,8 @@
// Authentication Middleware // Authentication Middleware
const pool = require('../db');
// Require authenticated user // Require authenticated user — no parameters needed, pool is imported directly
function requireAuth(db) { function requireAuth() {
return async (req, res, next) => { return async (req, res, next) => {
const sessionId = req.cookies?.session_id; const sessionId = req.cookies?.session_id;
@@ -10,19 +11,15 @@ function requireAuth(db) {
} }
try { try {
const session = await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.get( `SELECT s.*, u.id as user_id, u.username, u.email, u.role, u.user_group, u.bu_teams, u.is_active
`SELECT s.*, u.id as user_id, u.username, u.email, u.role, u.is_active FROM sessions s
FROM sessions s JOIN users u ON s.user_id = u.id
JOIN users u ON s.user_id = u.id WHERE s.session_id = $1 AND s.expires_at > NOW()`,
WHERE s.session_id = ? AND s.expires_at > datetime('now')`, [sessionId]
[sessionId], );
(err, row) => {
if (err) reject(err); const session = rows[0];
else resolve(row);
}
);
});
if (!session) { if (!session) {
return res.status(401).json({ error: 'Session expired or invalid' }); return res.status(401).json({ error: 'Session expired or invalid' });
@@ -37,7 +34,9 @@ function requireAuth(db) {
id: session.user_id, id: session.user_id,
username: session.username, username: session.username,
email: session.email, email: session.email,
role: session.role role: session.role,
group: session.user_group,
teams: session.bu_teams ? session.bu_teams.split(',').filter(Boolean) : []
}; };
next(); next();
@@ -48,18 +47,18 @@ function requireAuth(db) {
}; };
} }
// Require specific role(s) // Require specific group(s)
function requireRole(...allowedRoles) { function requireGroup(...allowedGroups) {
return (req, res, next) => { return (req, res, next) => {
if (!req.user) { if (!req.user) {
return res.status(401).json({ error: 'Authentication required' }); return res.status(401).json({ error: 'Authentication required' });
} }
if (!allowedRoles.includes(req.user.role)) { if (!allowedGroups.includes(req.user.group)) {
return res.status(403).json({ return res.status(403).json({
error: 'Insufficient permissions', error: 'Insufficient permissions',
required: allowedRoles, required: allowedGroups,
current: req.user.role current: req.user.group
}); });
} }
@@ -67,4 +66,4 @@ function requireRole(...allowedRoles) {
}; };
} }
module.exports = { requireAuth, requireRole }; module.exports = { requireAuth, requireGroup };

View File

@@ -1,96 +0,0 @@
#!/usr/bin/env node
// Migration script: Add audit_logs table
// Run: node migrate-audit-log.js
const sqlite3 = require('sqlite3').verbose();
const fs = require('fs');
const DB_FILE = './cve_database.db';
const BACKUP_FILE = `./cve_database_backup_${Date.now()}.db`;
function run(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function(err) {
if (err) reject(err);
else resolve(this);
});
});
}
function get(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.get(sql, params, (err, row) => {
if (err) reject(err);
else resolve(row);
});
});
}
async function migrate() {
console.log('╔════════════════════════════════════════════════════════╗');
console.log('║ CVE Database Migration: Add Audit Logs ║');
console.log('╚════════════════════════════════════════════════════════╝\n');
if (!fs.existsSync(DB_FILE)) {
console.log('❌ Database not found. Run setup.js for fresh install.');
process.exit(1);
}
// Backup database
console.log('📦 Creating backup...');
fs.copyFileSync(DB_FILE, BACKUP_FILE);
console.log(` ✓ Backup saved to: ${BACKUP_FILE}\n`);
const db = new sqlite3.Database(DB_FILE);
try {
// Check if table already exists
const exists = await get(db,
"SELECT name FROM sqlite_master WHERE type='table' AND name='audit_logs'"
);
if (exists) {
console.log('⏭️ audit_logs table already exists, nothing to do.');
} else {
console.log('1⃣ Creating audit_logs table...');
await run(db, `
CREATE TABLE audit_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER,
username VARCHAR(50) NOT NULL,
action VARCHAR(50) NOT NULL,
entity_type VARCHAR(50) NOT NULL,
entity_id VARCHAR(100),
details TEXT,
ip_address VARCHAR(45),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
`);
console.log(' ✓ Table created');
console.log('2⃣ Creating indexes...');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_audit_user_id ON audit_logs(user_id)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_logs(action)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_audit_entity_type ON audit_logs(entity_type)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_audit_created_at ON audit_logs(created_at)');
console.log(' ✓ Indexes created');
}
console.log('\n╔════════════════════════════════════════════════════════╗');
console.log('║ MIGRATION COMPLETE! ║');
console.log('╚════════════════════════════════════════════════════════╝');
console.log('\n📋 Summary:');
console.log(' ✓ audit_logs table ready');
console.log(`\n💾 Backup saved: ${BACKUP_FILE}`);
console.log('\n🚀 Restart your server to apply changes.\n');
} catch (error) {
console.error('\n❌ Migration failed:', error.message);
console.log(`\n🔄 To restore from backup: cp ${BACKUP_FILE} ${DB_FILE}`);
process.exit(1);
} finally {
db.close();
}
}
migrate();

View File

@@ -1,289 +0,0 @@
#!/usr/bin/env node
// Migration script: v1.0.0 -> v1.1.0
// Adds: users, sessions tables, multi-vendor support, vendor column in documents
// Run: node migrate-to-1.1.js
const sqlite3 = require('sqlite3').verbose();
const bcrypt = require('bcryptjs');
const fs = require('fs');
const path = require('path');
const DB_FILE = './cve_database.db';
const BACKUP_FILE = `./cve_database_backup_${Date.now()}.db`;
async function migrate() {
console.log('╔════════════════════════════════════════════════════════╗');
console.log('║ CVE Database Migration: v1.0.0 → v1.1.0 ║');
console.log('╚════════════════════════════════════════════════════════╝\n');
// Check if database exists
if (!fs.existsSync(DB_FILE)) {
console.log('❌ Database not found. Run setup.js for fresh install.');
process.exit(1);
}
// Backup database
console.log('📦 Creating backup...');
fs.copyFileSync(DB_FILE, BACKUP_FILE);
console.log(` ✓ Backup saved to: ${BACKUP_FILE}\n`);
const db = new sqlite3.Database(DB_FILE);
try {
// Run migrations in sequence
await addUsersTable(db);
await addSessionsTable(db);
await addVendorToDocuments(db);
await updateCvesConstraint(db);
await createDefaultAdmin(db);
await updateView(db);
console.log('\n╔════════════════════════════════════════════════════════╗');
console.log('║ MIGRATION COMPLETE! ║');
console.log('╚════════════════════════════════════════════════════════╝');
console.log('\n📋 Summary:');
console.log(' ✓ Users table added');
console.log(' ✓ Sessions table added');
console.log(' ✓ Vendor column added to documents');
console.log(' ✓ Multi-vendor constraint applied to cves');
console.log(' ✓ Default admin user created (admin/admin123)');
console.log(`\n💾 Backup saved: ${BACKUP_FILE}`);
console.log('\n🚀 Restart your server to apply changes.\n');
} catch (error) {
console.error('\n❌ Migration failed:', error.message);
console.log(`\n🔄 To restore from backup: cp ${BACKUP_FILE} ${DB_FILE}`);
process.exit(1);
} finally {
db.close();
}
}
function run(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function(err) {
if (err) reject(err);
else resolve(this);
});
});
}
function get(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.get(sql, params, (err, row) => {
if (err) reject(err);
else resolve(row);
});
});
}
function all(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows);
});
});
}
async function addUsersTable(db) {
console.log('1⃣ Adding users table...');
const exists = await get(db,
"SELECT name FROM sqlite_master WHERE type='table' AND name='users'"
);
if (exists) {
console.log(' ⏭️ Users table already exists, skipping');
return;
}
await run(db, `
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username VARCHAR(50) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
role VARCHAR(20) NOT NULL DEFAULT 'viewer',
is_active BOOLEAN DEFAULT 1,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
last_login TIMESTAMP,
CHECK (role IN ('admin', 'editor', 'viewer'))
)
`);
await run(db, 'CREATE INDEX IF NOT EXISTS idx_users_username ON users(username)');
console.log(' ✓ Users table created');
}
async function addSessionsTable(db) {
console.log('2⃣ Adding sessions table...');
const exists = await get(db,
"SELECT name FROM sqlite_master WHERE type='table' AND name='sessions'"
);
if (exists) {
console.log(' ⏭️ Sessions table already exists, skipping');
return;
}
await run(db, `
CREATE TABLE sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id VARCHAR(255) UNIQUE NOT NULL,
user_id INTEGER NOT NULL,
expires_at TIMESTAMP NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
)
`);
await run(db, 'CREATE INDEX IF NOT EXISTS idx_sessions_session_id ON sessions(session_id)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions(user_id)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_sessions_expires ON sessions(expires_at)');
console.log(' ✓ Sessions table created');
}
async function addVendorToDocuments(db) {
console.log('3⃣ Adding vendor column to documents...');
// Check if vendor column exists
const columns = await all(db, "PRAGMA table_info(documents)");
const hasVendor = columns.some(col => col.name === 'vendor');
if (hasVendor) {
console.log(' ⏭️ Vendor column already exists, skipping');
return;
}
// Add vendor column
await run(db, "ALTER TABLE documents ADD COLUMN vendor VARCHAR(100)");
// Populate vendor from the cves table based on cve_id
await run(db, `
UPDATE documents
SET vendor = (
SELECT c.vendor
FROM cves c
WHERE c.cve_id = documents.cve_id
LIMIT 1
)
WHERE vendor IS NULL
`);
// Set default for any remaining nulls
await run(db, "UPDATE documents SET vendor = 'Unknown' WHERE vendor IS NULL");
await run(db, 'CREATE INDEX IF NOT EXISTS idx_doc_vendor ON documents(vendor)');
console.log(' ✓ Vendor column added and populated');
}
async function updateCvesConstraint(db) {
console.log('4⃣ Updating CVEs table for multi-vendor support...');
// Check current schema
const tableInfo = await get(db,
"SELECT sql FROM sqlite_master WHERE type='table' AND name='cves'"
);
if (tableInfo.sql.includes('UNIQUE(cve_id, vendor)')) {
console.log(' ⏭️ Multi-vendor constraint already exists, skipping');
return;
}
// SQLite doesn't support ALTER CONSTRAINT, so we need to rebuild the table
console.log(' 📋 Rebuilding table with new constraint...');
// Create new table with correct schema
await run(db, `
CREATE TABLE cves_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
severity VARCHAR(20) NOT NULL,
description TEXT,
published_date DATE,
status VARCHAR(50) DEFAULT 'Open',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(cve_id, vendor)
)
`);
// Copy data
await run(db, `
INSERT INTO cves_new (id, cve_id, vendor, severity, description, published_date, status, created_at, updated_at)
SELECT id, cve_id, vendor, severity, description, published_date, status, created_at, updated_at
FROM cves
`);
// Drop old table
await run(db, 'DROP TABLE cves');
// Rename new table
await run(db, 'ALTER TABLE cves_new RENAME TO cves');
// Recreate indexes
await run(db, 'CREATE INDEX IF NOT EXISTS idx_cve_id ON cves(cve_id)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_vendor ON cves(vendor)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_severity ON cves(severity)');
await run(db, 'CREATE INDEX IF NOT EXISTS idx_status ON cves(status)');
console.log(' ✓ Multi-vendor constraint applied');
}
async function createDefaultAdmin(db) {
console.log('5⃣ Creating default admin user...');
const exists = await get(db, "SELECT id FROM users WHERE username = 'admin'");
if (exists) {
console.log(' ⏭️ Admin user already exists, skipping');
return;
}
const passwordHash = await bcrypt.hash('admin123', 10);
await run(db, `
INSERT INTO users (username, email, password_hash, role, is_active)
VALUES (?, ?, ?, ?, ?)
`, ['admin', 'admin@localhost', passwordHash, 'admin', 1]);
console.log(' ✓ Admin user created (admin/admin123)');
}
async function updateView(db) {
console.log('6⃣ Updating document status view...');
// Drop old view if exists
await run(db, 'DROP VIEW IF EXISTS cve_document_status');
// Create updated view with multi-vendor support
await run(db, `
CREATE VIEW cve_document_status AS
SELECT
c.id as record_id,
c.cve_id,
c.vendor,
c.severity,
c.status,
COUNT(DISTINCT d.id) as total_documents,
COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) as advisory_count,
COUNT(DISTINCT CASE WHEN d.type = 'email' THEN d.id END) as email_count,
COUNT(DISTINCT CASE WHEN d.type = 'screenshot' THEN d.id END) as screenshot_count,
CASE
WHEN COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) > 0
THEN 'Complete'
ELSE 'Missing Required Docs'
END as compliance_status
FROM cves c
LEFT JOIN documents d ON c.cve_id = d.cve_id AND c.vendor = d.vendor
GROUP BY c.id, c.cve_id, c.vendor, c.severity, c.status
`);
console.log(' ✓ View updated');
}
// Run migration
migrate();

View File

@@ -1,39 +0,0 @@
// Migration: Add jira_tickets table
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting JIRA tickets migration...');
db.serialize(() => {
// Create jira_tickets table
db.run(`
CREATE TABLE IF NOT EXISTS jira_tickets (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id TEXT NOT NULL,
vendor TEXT NOT NULL,
ticket_key TEXT NOT NULL,
url TEXT,
summary TEXT,
status TEXT DEFAULT 'Open' CHECK(status IN ('Open', 'In Progress', 'Closed')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (cve_id, vendor) REFERENCES cves(cve_id, vendor) ON DELETE CASCADE
)
`, (err) => {
if (err) console.error('Error creating table:', err);
else console.log('✓ jira_tickets table created');
});
// Create indexes
db.run('CREATE INDEX IF NOT EXISTS idx_jira_tickets_cve ON jira_tickets(cve_id, vendor)');
db.run('CREATE INDEX IF NOT EXISTS idx_jira_tickets_status ON jira_tickets(status)');
console.log('✓ Indexes created');
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -1,128 +0,0 @@
const sqlite3 = require('sqlite3').verbose();
const db = new sqlite3.Database('./cve_database.db');
console.log('🔄 Starting database migration for multi-vendor support...\n');
db.serialize(() => {
// Backup existing data
console.log('📦 Creating backup tables...');
db.run(`CREATE TABLE IF NOT EXISTS cves_backup AS SELECT * FROM cves`, (err) => {
if (err) console.error('Backup error:', err);
else console.log('✓ CVEs backed up');
});
db.run(`CREATE TABLE IF NOT EXISTS documents_backup AS SELECT * FROM documents`, (err) => {
if (err) console.error('Backup error:', err);
else console.log('✓ Documents backed up');
});
// Drop old table
console.log('\n🗑 Dropping old cves table...');
db.run(`DROP TABLE IF EXISTS cves`, (err) => {
if (err) {
console.error('Drop error:', err);
return;
}
console.log('✓ Old table dropped');
// Create new table with UNIQUE(cve_id, vendor) instead of UNIQUE(cve_id)
console.log('\n🏗 Creating new cves table with multi-vendor support...');
db.run(`
CREATE TABLE cves (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
severity VARCHAR(20) NOT NULL,
description TEXT,
published_date DATE,
status VARCHAR(50) DEFAULT 'Open',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(cve_id, vendor)
)
`, (err) => {
if (err) {
console.error('Create error:', err);
return;
}
console.log('✓ New table created with UNIQUE(cve_id, vendor)');
// Restore data
console.log('\n📥 Restoring data...');
db.run(`INSERT INTO cves SELECT * FROM cves_backup`, (err) => {
if (err) {
console.error('Restore error:', err);
return;
}
console.log('✓ Data restored');
// Recreate indexes
console.log('\n🔍 Creating indexes...');
db.run(`CREATE INDEX idx_cve_id ON cves(cve_id)`, () => {
console.log('✓ Index: idx_cve_id');
});
db.run(`CREATE INDEX idx_vendor ON cves(vendor)`, () => {
console.log('✓ Index: idx_vendor');
});
db.run(`CREATE INDEX idx_severity ON cves(severity)`, () => {
console.log('✓ Index: idx_severity');
});
db.run(`CREATE INDEX idx_status ON cves(status)`, () => {
console.log('✓ Index: idx_status');
});
// Update view
console.log('\n👁 Updating cve_document_status view...');
db.run(`DROP VIEW IF EXISTS cve_document_status`, (err) => {
if (err) console.error('Drop view error:', err);
db.run(`
CREATE VIEW cve_document_status AS
SELECT
c.id as record_id,
c.cve_id,
c.vendor,
c.severity,
c.status,
COUNT(DISTINCT d.id) as total_documents,
COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) as advisory_count,
COUNT(DISTINCT CASE WHEN d.type = 'email' THEN d.id END) as email_count,
COUNT(DISTINCT CASE WHEN d.type = 'screenshot' THEN d.id END) as screenshot_count,
CASE
WHEN COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) > 0
THEN 'Complete'
ELSE 'Missing Required Docs'
END as compliance_status
FROM cves c
LEFT JOIN documents d ON c.cve_id = d.cve_id AND c.vendor = d.vendor
GROUP BY c.id, c.cve_id, c.vendor, c.severity, c.status
`, (err) => {
if (err) {
console.error('Create view error:', err);
} else {
console.log('✓ View recreated');
}
console.log('\n✅ Migration complete!');
console.log('\n📊 Summary:');
db.get('SELECT COUNT(*) as count FROM cves', (err, row) => {
if (!err) console.log(` Total CVE entries: ${row.count}`);
db.get('SELECT COUNT(DISTINCT cve_id) as count FROM cves', (err, row) => {
if (!err) console.log(` Unique CVE IDs: ${row.count}`);
console.log('\n💡 Next steps:');
console.log(' 1. Restart backend: pkill -f "node server.js" && node server.js &');
console.log(' 2. Replace frontend/src/App.js with multi-vendor version');
console.log(' 3. Test by adding same CVE with multiple vendors\n');
db.close();
});
});
});
});
});
});
});
});

View File

@@ -0,0 +1,41 @@
# Database Migrations
These migration scripts were used to evolve the database schema during development. **They are NOT needed for fresh deployments**`setup.js` contains the complete v1.0.0 schema.
These are retained for reference and for upgrading existing deployments that were set up before v1.0.0.
## Schema Migrations (run in order for existing deployments)
| Script | Purpose |
|--------|---------|
| `add_ivanti_sync_table.js` | Creates `ivanti_sync_state` table for tracking Ivanti sync status |
| `add_ivanti_findings_tables.js` | Creates `ivanti_findings_cache`, `ivanti_finding_notes`, `ivanti_counts_cache`, `ivanti_finding_overrides` tables |
| `add_ivanti_counts_history_table.js` | Creates `ivanti_counts_history` table for trend chart data |
| `add_ivanti_todo_queue_table.js` | Creates `ivanti_todo_queue` table for FP/Archer workflow queuing |
| `add_todo_queue_hostname.js` | Adds `hostname` column to `ivanti_todo_queue` |
| `add_todo_queue_ip_address.js` | Adds `ip_address` column to `ivanti_todo_queue` |
| `add_fp_submissions_table.js` | Creates `ivanti_fp_submissions` table for false positive workflow tracking |
| `add_fp_submission_editing.js` | Adds `lifecycle_status`, `ivanti_workflow_batch_uuid`, `updated_at` columns and `ivanti_fp_submission_history` table |
| `add_knowledge_base_table.js` | Creates `knowledge_base` table for KB article storage |
| `add_user_groups.js` | Adds `user_group` column to `users` table with validation triggers |
| `add_created_by_columns.js` | Adds `created_by` column to `compliance_notes` and `knowledge_base` tables |
| `add_compliance_tables.js` | Creates `compliance_uploads`, `compliance_items`, `compliance_notes` tables |
| `add_compliance_notes_group_id.js` | Adds `group_id` column to `compliance_notes` for multi-metric note grouping |
| `add_archer_tickets_table.js` | Creates `archer_tickets` table for Archer exception tracking |
| `add_archer_tickets_timestamps.js` | Adds `created_at` and `updated_at` columns to `archer_tickets` |
| `add_jira_sync_columns.js` | Adds Jira sync-related columns to `jira_tickets` |
| `add_card_workflow_type.js` | Adds `CARD` to `workflow_type` CHECK constraint on `ivanti_todo_queue` |
| `add_granite_workflow_type.js` | Adds `GRANITE` to `workflow_type` CHECK constraint on `ivanti_todo_queue` |
| `add_finding_archive_tables.js` | Creates `ivanti_finding_archives` and `ivanti_archive_transitions` tables |
| `add_closed_gone_state.js` | Adds `CLOSED_GONE` to `current_state` CHECK constraint on `ivanti_finding_archives` |
| `add_sync_anomaly_tables.js` | Creates `ivanti_sync_anomaly_log` and `ivanti_finding_bu_history` tables |
| `add_atlas_action_plans_cache.js` | Creates `atlas_action_plans_cache` table for Atlas API caching |
| `add_return_classification.js` | Adds `return_classification_json` column to `ivanti_sync_anomaly_log` |
## Data Migrations (one-time backfills)
| Script | Purpose |
|--------|---------|
| `backfill_anomaly_log.js` | Synthesizes anomaly log entries from existing archive transitions for historical chart data |
| `backfill_return_classification.js` | Populates `return_classification_json` for existing anomaly rows with returned findings. Supports `--force` flag to re-run. |
| `reclassify_bu_roundtrips.js` | Reclassifies archive transitions that were BU reassignment round-trips (archived then returned within 14 days) from the default `severity_score_drift` to `bu_reassignment` |

View File

@@ -0,0 +1,56 @@
// Migration: Add created_at / updated_at columns to archer_tickets
//
// SQLite does not support ALTER TABLE ADD COLUMN IF NOT EXISTS, so we check
// PRAGMA table_info first and only add the column when it is absent.
//
// Run on any instance where archer_tickets was created before these columns
// were added to the schema (symptoms: every /api/archer-tickets call → 500).
//
// Usage: node backend/migrations/add_archer_tickets_timestamps.js
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting archer_tickets timestamp migration...');
db.all('PRAGMA table_info(archer_tickets)', [], (err, columns) => {
if (err) {
console.error('Error reading table info:', err);
return db.close();
}
const names = columns.map(c => c.name);
db.serialize(() => {
if (!names.includes('created_at')) {
db.run(
`ALTER TABLE archer_tickets ADD COLUMN created_at DATETIME DEFAULT CURRENT_TIMESTAMP`,
(err) => {
if (err) console.error('Error adding created_at:', err);
else console.log('✓ created_at column added');
}
);
} else {
console.log('✓ created_at already exists — skipping');
}
if (!names.includes('updated_at')) {
db.run(
`ALTER TABLE archer_tickets ADD COLUMN updated_at DATETIME DEFAULT CURRENT_TIMESTAMP`,
(err) => {
if (err) console.error('Error adding updated_at:', err);
else console.log('✓ updated_at column added');
}
);
} else {
console.log('✓ updated_at already exists — skipping');
}
});
db.close(() => {
console.log('Migration complete. Restart the backend server.');
});
});

View File

@@ -0,0 +1,37 @@
// Migration: Add atlas_action_plans_cache table
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting Atlas action plans cache migration...');
db.serialize(() => {
// Cache table — one row per host, holding cached Atlas action plan status
db.run(`
CREATE TABLE IF NOT EXISTS atlas_action_plans_cache (
id INTEGER PRIMARY KEY AUTOINCREMENT,
host_id INTEGER NOT NULL UNIQUE,
has_action_plan INTEGER NOT NULL DEFAULT 0,
plan_count INTEGER NOT NULL DEFAULT 0,
plans_json TEXT NOT NULL DEFAULT '[]',
synced_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
)
`, (err) => {
if (err) console.error('Error creating atlas_action_plans_cache table:', err);
else console.log('✓ atlas_action_plans_cache table created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_atlas_cache_host_id
ON atlas_action_plans_cache(host_id)
`, (err) => {
if (err) console.error('Error creating host_id index:', err);
else console.log('✓ idx_atlas_cache_host_id index created');
});
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,79 @@
// Migration: Add CARD to workflow_type CHECK constraint on ivanti_todo_queue
// SQLite cannot ALTER a CHECK constraint, so this recreates the table.
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_card_workflow_type migration...');
db.serialize(() => {
db.run('PRAGMA foreign_keys = OFF', (err) => {
if (err) console.error('PRAGMA error:', err);
});
db.run('BEGIN TRANSACTION', (err) => {
if (err) { console.error('BEGIN error:', err); return; }
});
db.run(`
CREATE TABLE ivanti_todo_queue_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
finding_id TEXT NOT NULL,
finding_title TEXT,
cves_json TEXT,
ip_address TEXT,
vendor TEXT NOT NULL,
workflow_type TEXT NOT NULL CHECK(workflow_type IN ('FP', 'Archer', 'CARD')),
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'complete')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
)
`, (err) => {
if (err) console.error('Error creating new table:', err);
else console.log('✓ ivanti_todo_queue_new created');
});
db.run(
'INSERT INTO ivanti_todo_queue_new SELECT * FROM ivanti_todo_queue',
(err) => {
if (err) console.error('Error copying data:', err);
else console.log('✓ Data copied');
}
);
db.run('DROP TABLE ivanti_todo_queue', (err) => {
if (err) console.error('Error dropping old table:', err);
else console.log('✓ Old table dropped');
});
db.run(
'ALTER TABLE ivanti_todo_queue_new RENAME TO ivanti_todo_queue',
(err) => {
if (err) console.error('Error renaming table:', err);
else console.log('✓ Table renamed');
}
);
db.run(
'CREATE INDEX IF NOT EXISTS idx_todo_queue_user ON ivanti_todo_queue(user_id, status)',
(err) => {
if (err) console.error('Error creating index:', err);
else console.log('✓ Index recreated');
}
);
db.run('COMMIT', (err) => {
if (err) console.error('COMMIT error:', err);
else console.log('✓ Transaction committed');
});
db.run('PRAGMA foreign_keys = ON', () => {});
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,130 @@
// Migration: Add CLOSED_GONE state to ivanti_finding_archives
//
// The archive table tracks findings that disappear from the Open findings set.
// Previously it only tracked: ARCHIVED → RETURNED → CLOSED.
//
// This migration adds a CLOSED_GONE state for findings that were confirmed
// in the Ivanti Closed set but then disappeared from it on a subsequent sync.
// This closes a visibility gap where findings could vanish from the Closed API
// results (e.g., due to VRR rescore below the severity threshold) without
// being tracked.
//
// SQLite does not support ALTER TABLE to modify CHECK constraints, so this
// migration recreates the table with the expanded constraint.
//
// Safe to re-run — uses IF NOT EXISTS and checks for existing data.
//
// Usage: node backend/migrations/add_closed_gone_state.js
const path = require('path');
const sqlite3 = require('sqlite3').verbose();
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting CLOSED_GONE state migration...');
function run(sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function (err) {
if (err) reject(err);
else resolve(this);
});
});
}
function all(sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
async function migrate() {
// Check if the table already has the CLOSED_GONE state
const tableInfo = await all("SELECT sql FROM sqlite_master WHERE name='ivanti_finding_archives'");
if (tableInfo.length > 0 && tableInfo[0].sql.includes('CLOSED_GONE')) {
console.log('✓ ivanti_finding_archives already has CLOSED_GONE state — skipping');
return;
}
if (tableInfo.length === 0) {
// Table doesn't exist yet — create it fresh with the new constraint
await run(`
CREATE TABLE ivanti_finding_archives (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL UNIQUE,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
ip_address TEXT NOT NULL DEFAULT '',
current_state TEXT NOT NULL CHECK(current_state IN ('ARCHIVED','RETURNED','CLOSED','CLOSED_GONE')),
last_severity REAL NOT NULL DEFAULT 0,
first_archived_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_transition_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`);
console.log('✓ Created ivanti_finding_archives with CLOSED_GONE state');
return;
}
// Table exists but needs the constraint updated — recreate with data migration
console.log(' Recreating table with expanded CHECK constraint...');
await run('BEGIN TRANSACTION');
try {
// 1. Rename existing table
await run('ALTER TABLE ivanti_finding_archives RENAME TO ivanti_finding_archives_old');
// 2. Create new table with expanded constraint
await run(`
CREATE TABLE ivanti_finding_archives (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL UNIQUE,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
ip_address TEXT NOT NULL DEFAULT '',
current_state TEXT NOT NULL CHECK(current_state IN ('ARCHIVED','RETURNED','CLOSED','CLOSED_GONE')),
last_severity REAL NOT NULL DEFAULT 0,
first_archived_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_transition_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`);
// 3. Copy data
await run(`
INSERT INTO ivanti_finding_archives
(id, finding_id, finding_title, host_name, ip_address, current_state,
last_severity, first_archived_at, last_transition_at, created_at)
SELECT id, finding_id, finding_title, host_name, ip_address, current_state,
last_severity, first_archived_at, last_transition_at, created_at
FROM ivanti_finding_archives_old
`);
// 4. Recreate indexes
await run('CREATE INDEX IF NOT EXISTS idx_archive_finding_id ON ivanti_finding_archives(finding_id)');
await run('CREATE INDEX IF NOT EXISTS idx_archive_current_state ON ivanti_finding_archives(current_state)');
// 5. Drop old table
await run('DROP TABLE ivanti_finding_archives_old');
await run('COMMIT');
console.log('✓ ivanti_finding_archives updated with CLOSED_GONE state');
} catch (err) {
await run('ROLLBACK').catch(() => {});
throw err;
}
}
migrate()
.then(() => {
console.log('Migration complete.');
db.close();
})
.catch((err) => {
console.error('Migration failed:', err);
db.close();
process.exit(1);
});

View File

@@ -0,0 +1,29 @@
// Migration: Add group_id column to compliance_notes table
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_compliance_notes_group_id migration...');
db.serialize(() => {
db.run(`ALTER TABLE compliance_notes ADD COLUMN group_id TEXT`, (err) => {
if (err) console.error('Error adding group_id column:', err);
else console.log('✓ group_id column added to compliance_notes');
});
db.run(`CREATE INDEX IF NOT EXISTS idx_compliance_notes_group ON compliance_notes(group_id)`, (err) => {
if (err) console.error('Error creating group_id index:', err);
else console.log('✓ idx_compliance_notes_group created');
});
db.run(`UPDATE compliance_notes SET group_id = 'legacy-' || id WHERE group_id IS NULL`, (err) => {
if (err) console.error('Error backfilling group_id:', err);
else console.log('✓ Existing rows backfilled with legacy group_id');
});
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,108 @@
// Migration: Add compliance_uploads, compliance_items, compliance_notes tables
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_compliance_tables migration...');
db.serialize(() => {
// Each xlsx upload — one row per file ingested
db.run(`
CREATE TABLE IF NOT EXISTS compliance_uploads (
id INTEGER PRIMARY KEY AUTOINCREMENT,
filename TEXT NOT NULL,
report_date TEXT,
uploaded_by INTEGER,
uploaded_at DATETIME DEFAULT CURRENT_TIMESTAMP,
new_count INTEGER DEFAULT 0,
resolved_count INTEGER DEFAULT 0,
recurring_count INTEGER DEFAULT 0,
FOREIGN KEY (uploaded_by) REFERENCES users(id) ON DELETE SET NULL
)
`, (err) => {
if (err) console.error('Error creating compliance_uploads:', err);
else console.log('✓ compliance_uploads created');
});
// One row per non-compliant asset per metric per upload.
// hostname + metric_id is the stable identity key used to link history and notes.
db.run(`
CREATE TABLE IF NOT EXISTS compliance_items (
id INTEGER PRIMARY KEY AUTOINCREMENT,
upload_id INTEGER NOT NULL,
hostname TEXT NOT NULL,
ip_address TEXT,
device_type TEXT,
team TEXT,
metric_id TEXT NOT NULL,
metric_desc TEXT,
category TEXT,
extra_json TEXT,
status TEXT NOT NULL DEFAULT 'active' CHECK(status IN ('active', 'resolved')),
first_seen_upload_id INTEGER,
resolved_upload_id INTEGER,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (upload_id) REFERENCES compliance_uploads(id) ON DELETE CASCADE,
FOREIGN KEY (first_seen_upload_id) REFERENCES compliance_uploads(id) ON DELETE SET NULL,
FOREIGN KEY (resolved_upload_id) REFERENCES compliance_uploads(id) ON DELETE SET NULL
)
`, (err) => {
if (err) console.error('Error creating compliance_items:', err);
else console.log('✓ compliance_items created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_compliance_items_upload
ON compliance_items(upload_id)
`, (err) => {
if (err) console.error('Error creating upload index:', err);
else console.log('✓ idx_compliance_items_upload created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_compliance_items_identity
ON compliance_items(hostname, metric_id)
`, (err) => {
if (err) console.error('Error creating identity index:', err);
else console.log('✓ idx_compliance_items_identity created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_compliance_items_team_status
ON compliance_items(team, status)
`, (err) => {
if (err) console.error('Error creating team/status index:', err);
else console.log('✓ idx_compliance_items_team_status created');
});
// Notes keyed on (hostname, metric_id) — persists across uploads.
// Each note is its own row so history is preserved.
db.run(`
CREATE TABLE IF NOT EXISTS compliance_notes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
hostname TEXT NOT NULL,
metric_id TEXT NOT NULL,
note TEXT NOT NULL,
created_by INTEGER,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (created_by) REFERENCES users(id) ON DELETE SET NULL
)
`, (err) => {
if (err) console.error('Error creating compliance_notes:', err);
else console.log('✓ compliance_notes created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_compliance_notes_identity
ON compliance_notes(hostname, metric_id)
`, (err) => {
if (err) console.error('Error creating notes identity index:', err);
else console.log('✓ idx_compliance_notes_identity created');
});
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,76 @@
// Migration: Add created_by column to cves, archer_tickets, and jira_tickets tables
// Stores the user ID of the creator for ownership-based delete checks.
// Idempotent — safe to run multiple times.
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
/**
* Run the migration against the given database instance.
* Exported for testing with in-memory databases.
* @param {sqlite3.Database} db
* @returns {Promise<void>}
*/
function runMigration(db) {
return new Promise((resolve, reject) => {
const tables = ['cves', 'archer_tickets', 'jira_tickets'];
let completed = 0;
db.serialize(() => {
tables.forEach((table) => {
db.all(`PRAGMA table_info(${table})`, (err, columns) => {
if (err) {
// Table may not exist yet — skip gracefully
console.log(`⚠ Could not inspect ${table}: ${err.message} — skipping`);
completed++;
if (completed === tables.length) resolve();
return;
}
const hasCreatedBy = columns.some(col => col.name === 'created_by');
if (hasCreatedBy) {
console.log(`${table}.created_by already exists — skipping`);
completed++;
if (completed === tables.length) resolve();
return;
}
db.run(
`ALTER TABLE ${table} ADD COLUMN created_by INTEGER REFERENCES users(id)`,
(err) => {
if (err) {
reject(err);
return;
}
console.log(`✓ Added created_by column to ${table}`);
completed++;
if (completed === tables.length) resolve();
}
);
});
});
});
});
}
// Run directly if executed as a script
if (require.main === module) {
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_created_by_columns migration...');
runMigration(db)
.then(() => {
console.log('Migration complete!');
db.close(() => {
console.log('Database connection closed.');
});
})
.catch((err) => {
console.error('Migration failed:', err);
db.close();
process.exit(1);
});
}
module.exports = { runMigration };

View File

@@ -0,0 +1,75 @@
// Migration: Add ivanti_finding_archives and ivanti_archive_transitions tables
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting finding archive tables migration...');
db.serialize(() => {
// Archive records — one row per finding that has entered the archive lifecycle
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_finding_archives (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL UNIQUE,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
ip_address TEXT NOT NULL DEFAULT '',
current_state TEXT NOT NULL CHECK(current_state IN ('ARCHIVED', 'RETURNED', 'CLOSED')),
last_severity REAL NOT NULL DEFAULT 0,
first_archived_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_transition_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`, (err) => {
if (err) console.error('Error creating ivanti_finding_archives table:', err);
else console.log('✓ ivanti_finding_archives table created');
});
// Transition history — one row per state change on an archive record
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_archive_transitions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
archive_id INTEGER NOT NULL,
from_state TEXT NOT NULL,
to_state TEXT NOT NULL,
severity_at_transition REAL NOT NULL DEFAULT 0,
reason TEXT NOT NULL DEFAULT '',
transitioned_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (archive_id) REFERENCES ivanti_finding_archives(id)
)
`, (err) => {
if (err) console.error('Error creating ivanti_archive_transitions table:', err);
else console.log('✓ ivanti_archive_transitions table created');
});
// Indexes for query performance
db.run(`
CREATE INDEX IF NOT EXISTS idx_archive_finding_id
ON ivanti_finding_archives(finding_id)
`, (err) => {
if (err) console.error('Error creating idx_archive_finding_id:', err);
else console.log('✓ idx_archive_finding_id index created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_archive_current_state
ON ivanti_finding_archives(current_state)
`, (err) => {
if (err) console.error('Error creating idx_archive_current_state:', err);
else console.log('✓ idx_archive_current_state index created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_transition_archive_id
ON ivanti_archive_transitions(archive_id)
`, (err) => {
if (err) console.error('Error creating idx_transition_archive_id:', err);
else console.log('✓ idx_transition_archive_id index created');
});
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,94 @@
// Migration: Add FP submission editing support (lifecycle status, batch UUID, history table)
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting FP submission editing migration...');
db.serialize(() => {
// Add lifecycle_status column to ivanti_fp_submissions
// Wrapped in try/catch style via callback — SQLite throws if column already exists
db.run(
`ALTER TABLE ivanti_fp_submissions ADD COLUMN lifecycle_status TEXT NOT NULL DEFAULT 'submitted' CHECK(lifecycle_status IN ('submitted', 'approved', 'rejected', 'rework', 'resubmitted'))`,
(err) => {
if (err) {
if (err.message.includes('duplicate column')) {
console.log('✓ lifecycle_status column already exists');
} else {
console.error('Error adding lifecycle_status column:', err.message);
}
} else {
console.log('✓ lifecycle_status column added');
}
}
);
// Add ivanti_workflow_batch_uuid column
db.run(
`ALTER TABLE ivanti_fp_submissions ADD COLUMN ivanti_workflow_batch_uuid TEXT`,
(err) => {
if (err) {
if (err.message.includes('duplicate column')) {
console.log('✓ ivanti_workflow_batch_uuid column already exists');
} else {
console.error('Error adding ivanti_workflow_batch_uuid column:', err.message);
}
} else {
console.log('✓ ivanti_workflow_batch_uuid column added');
}
}
);
// Add updated_at column (SQLite requires constant defaults for ALTER TABLE, so default to NULL)
db.run(
`ALTER TABLE ivanti_fp_submissions ADD COLUMN updated_at DATETIME DEFAULT NULL`,
(err) => {
if (err) {
if (err.message.includes('duplicate column')) {
console.log('✓ updated_at column already exists');
} else {
console.error('Error adding updated_at column:', err.message);
}
} else {
console.log('✓ updated_at column added');
}
}
);
// Create submission history table
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_fp_submission_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
submission_id INTEGER NOT NULL,
user_id INTEGER NOT NULL,
username TEXT NOT NULL,
change_type TEXT NOT NULL CHECK(change_type IN (
'created', 'fields_updated', 'findings_added',
'attachments_added', 'status_changed'
)),
change_details_json TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (submission_id) REFERENCES ivanti_fp_submissions(id) ON DELETE CASCADE
)
`, (err) => {
if (err) console.error('Error creating history table:', err.message);
else console.log('✓ ivanti_fp_submission_history table created');
});
// Create index on submission_id for history lookups
db.run(
`CREATE INDEX IF NOT EXISTS idx_fp_history_submission ON ivanti_fp_submission_history(submission_id)`,
(err) => {
if (err) console.error('Error creating history index:', err.message);
else console.log('✓ idx_fp_history_submission index created');
}
);
console.log('✓ Migration statements queued');
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,57 @@
// Migration: Add ivanti_fp_submissions table
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting ivanti_fp_submissions migration...');
db.serialize(() => {
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_fp_submissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
username TEXT NOT NULL,
ivanti_workflow_batch_id INTEGER,
ivanti_generated_id TEXT,
workflow_name TEXT NOT NULL,
reason TEXT NOT NULL,
description TEXT,
expiration_date TEXT NOT NULL,
scope_override TEXT NOT NULL DEFAULT 'Authorized',
finding_ids_json TEXT NOT NULL,
queue_item_ids_json TEXT NOT NULL,
attachment_count INTEGER DEFAULT 0,
attachment_results_json TEXT,
status TEXT NOT NULL DEFAULT 'success' CHECK(status IN ('success', 'partial', 'failed')),
error_message TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`, (err) => {
if (err) console.error('Error creating table:', err);
else console.log('✓ ivanti_fp_submissions table created');
});
db.run(
'CREATE INDEX IF NOT EXISTS idx_fp_submissions_user ON ivanti_fp_submissions(user_id)',
(err) => {
if (err) console.error('Error creating index:', err);
else console.log('✓ user_id index created');
}
);
db.run(
'CREATE INDEX IF NOT EXISTS idx_fp_submissions_ivanti_id ON ivanti_fp_submissions(ivanti_generated_id)',
(err) => {
if (err) console.error('Error creating index:', err);
else console.log('✓ ivanti_generated_id index created');
}
);
console.log('✓ Migration statements queued');
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,80 @@
// Migration: Add GRANITE to workflow_type CHECK constraint on ivanti_todo_queue
// SQLite cannot ALTER a CHECK constraint, so this recreates the table.
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_granite_workflow_type migration...');
db.serialize(() => {
db.run('PRAGMA foreign_keys = OFF', (err) => {
if (err) console.error('PRAGMA error:', err);
});
db.run('BEGIN TRANSACTION', (err) => {
if (err) { console.error('BEGIN error:', err); return; }
});
db.run(`
CREATE TABLE ivanti_todo_queue_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
finding_id TEXT NOT NULL,
finding_title TEXT,
cves_json TEXT,
ip_address TEXT,
hostname TEXT,
vendor TEXT NOT NULL,
workflow_type TEXT NOT NULL CHECK(workflow_type IN ('FP', 'Archer', 'CARD', 'GRANITE')),
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'complete')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
)
`, (err) => {
if (err) console.error('Error creating new table:', err);
else console.log('✓ ivanti_todo_queue_new created');
});
db.run(
'INSERT INTO ivanti_todo_queue_new SELECT id, user_id, finding_id, finding_title, cves_json, ip_address, hostname, vendor, workflow_type, status, created_at, updated_at FROM ivanti_todo_queue',
(err) => {
if (err) console.error('Error copying data:', err);
else console.log('✓ Data copied');
}
);
db.run('DROP TABLE ivanti_todo_queue', (err) => {
if (err) console.error('Error dropping old table:', err);
else console.log('✓ Old table dropped');
});
db.run(
'ALTER TABLE ivanti_todo_queue_new RENAME TO ivanti_todo_queue',
(err) => {
if (err) console.error('Error renaming table:', err);
else console.log('✓ Table renamed');
}
);
db.run(
'CREATE INDEX IF NOT EXISTS idx_todo_queue_user ON ivanti_todo_queue(user_id, status)',
(err) => {
if (err) console.error('Error creating index:', err);
else console.log('✓ Index recreated');
}
);
db.run('COMMIT', (err) => {
if (err) console.error('COMMIT error:', err);
else console.log('✓ Transaction committed');
});
db.run('PRAGMA foreign_keys = ON', () => {}); // FIXME: Callback does not handle the error parameter (should be `(err) => { if (err) ... }`)
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,41 @@
// Migration: Add ivanti_counts_history table
//
// Stores a snapshot of open/closed Ivanti finding counts on every sync.
// Unlike ivanti_counts_cache (single-row, always overwritten), this table
// accumulates all snapshots so time-series charts can be built from it.
//
// The GET /api/ivanti/findings/counts/history endpoint aggregates these rows
// to the last snapshot per calendar day using a ROW_NUMBER window function.
//
// NOTE: This table is also created automatically at server startup via
// CREATE TABLE IF NOT EXISTS in initTables() (ivantiFindings.js).
// This script is provided for manual setup on fresh installs and for
// documentation consistency with other migration files.
//
// Usage: node backend/migrations/add_ivanti_counts_history_table.js
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting ivanti_counts_history migration...');
db.serialize(() => {
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_counts_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
open_count INTEGER NOT NULL,
closed_count INTEGER NOT NULL,
recorded_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`, (err) => {
if (err) console.error('Error creating ivanti_counts_history table:', err);
else console.log('✓ ivanti_counts_history table created (or already exists)');
});
});
db.close(() => {
console.log('Migration complete.');
});

View File

@@ -0,0 +1,58 @@
// Migration: Add ivanti_findings_cache and ivanti_finding_notes tables
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting Ivanti findings tables migration...');
db.serialize(() => {
// Cache table — single row holding the latest sync result
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_findings_cache (
id INTEGER PRIMARY KEY CHECK (id = 1),
total INTEGER DEFAULT 0,
findings_json TEXT DEFAULT '[]',
synced_at DATETIME,
sync_status TEXT DEFAULT 'never',
error_message TEXT
)
`, (err) => {
if (err) console.error('Error creating findings cache table:', err);
else console.log('✓ ivanti_findings_cache table created');
});
db.run(`
INSERT OR IGNORE INTO ivanti_findings_cache (id, total, findings_json, sync_status)
VALUES (1, 0, '[]', 'never')
`, (err) => {
if (err) console.error('Error seeding findings cache row:', err);
else console.log('✓ ivanti_findings_cache row seeded');
});
// Notes table — one row per finding, persists across cache refreshes
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_finding_notes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL UNIQUE,
note TEXT NOT NULL DEFAULT '',
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`, (err) => {
if (err) console.error('Error creating finding notes table:', err);
else console.log('✓ ivanti_finding_notes table created');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_finding_notes_finding_id
ON ivanti_finding_notes(finding_id)
`, (err) => {
if (err) console.error('Error creating notes index:', err);
else console.log('✓ finding_id index created');
});
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,43 @@
// Migration: Add ivanti_todo_queue table
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting ivanti_todo_queue migration...');
db.serialize(() => {
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_todo_queue (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
finding_id TEXT NOT NULL,
finding_title TEXT,
cves_json TEXT,
vendor TEXT NOT NULL,
workflow_type TEXT NOT NULL CHECK(workflow_type IN ('FP', 'Archer')),
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'complete')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
)
`, (err) => {
if (err) console.error('Error creating table:', err);
else console.log('✓ ivanti_todo_queue table created');
});
db.run(
'CREATE INDEX IF NOT EXISTS idx_todo_queue_user ON ivanti_todo_queue(user_id, status)',
(err) => {
if (err) console.error('Error creating index:', err);
else console.log('✓ User+status index created');
}
);
console.log('✓ Migration statements queued');
});
db.close(() => {
console.log('Migration complete!');
});

View File

@@ -0,0 +1,63 @@
// Migration: Add Jira API sync columns to jira_tickets table
// Adds jira_id, jira_status, and last_synced_at columns to support
// live synchronization with Jira Data Center REST API.
// Idempotent — safe to run multiple times.
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting Jira sync columns migration...');
const newColumns = [
{ name: 'jira_id', sql: 'ALTER TABLE jira_tickets ADD COLUMN jira_id TEXT' },
{ name: 'jira_status', sql: 'ALTER TABLE jira_tickets ADD COLUMN jira_status TEXT' },
{ name: 'last_synced_at', sql: 'ALTER TABLE jira_tickets ADD COLUMN last_synced_at DATETIME' }
];
db.all('PRAGMA table_info(jira_tickets)', (err, columns) => {
if (err) {
console.error('Could not inspect jira_tickets:', err.message);
console.log('Run migrate_jira_tickets.js first to create the table.');
db.close();
return;
}
const existingNames = new Set(columns.map(c => c.name));
let pending = 0;
db.serialize(() => {
newColumns.forEach(({ name, sql }) => {
if (existingNames.has(name)) {
console.log(`✓ jira_tickets.${name} already exists — skipping`);
} else {
pending++;
db.run(sql, (runErr) => {
if (runErr) {
console.error(`✗ Failed to add ${name}:`, runErr.message);
} else {
console.log(`✓ Added jira_tickets.${name}`);
}
pending--;
if (pending === 0) finish();
});
}
});
// Create index on jira_id for lookups
db.run('CREATE INDEX IF NOT EXISTS idx_jira_tickets_jira_id ON jira_tickets(jira_id)', (idxErr) => {
if (idxErr) console.error('Index error:', idxErr.message);
else console.log('✓ jira_id index created');
});
if (pending === 0) finish();
});
});
function finish() {
db.close(() => {
console.log('Migration complete!');
});
}

View File

@@ -0,0 +1,57 @@
// Migration: Add return_classification_json column to ivanti_sync_anomaly_log
//
// Stores the classification breakdown for returned findings (e.g., how many
// returned due to BU reassignment back to team, severity re-escalation, etc.)
//
// Safe to re-run — uses ALTER TABLE with IF NOT EXISTS pattern.
//
// Usage: node backend/migrations/add_return_classification.js
const path = require('path');
const sqlite3 = require('sqlite3').verbose();
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting return classification migration...');
function run(sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function (err) {
if (err) reject(err);
else resolve(this);
});
});
}
function all(sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
async function migrate() {
// Check if column already exists
const columns = await all(`PRAGMA table_info(ivanti_sync_anomaly_log)`);
const hasColumn = columns.some(c => c.name === 'return_classification_json');
if (!hasColumn) {
await run(`ALTER TABLE ivanti_sync_anomaly_log ADD COLUMN return_classification_json TEXT NOT NULL DEFAULT '{}'`);
console.log('✓ Added return_classification_json column to ivanti_sync_anomaly_log');
} else {
console.log('✓ return_classification_json column already exists — skipping');
}
}
migrate()
.then(() => {
console.log('Migration complete.');
db.close();
})
.catch((err) => {
console.error('Migration failed:', err);
db.close();
process.exit(1);
});

View File

@@ -0,0 +1,90 @@
// Migration: Add sync anomaly detection and BU drift monitoring tables
//
// Creates two new tables:
// - ivanti_sync_anomaly_log — stores one row per sync cycle with the
// anomaly summary breakdown (count deltas, classification, significance).
// - ivanti_finding_bu_history — records BU change events detected on
// individual findings across syncs.
//
// Safe to re-run — uses CREATE TABLE IF NOT EXISTS and CREATE INDEX IF NOT EXISTS.
//
// Usage: node backend/migrations/add_sync_anomaly_tables.js
const path = require('path');
const sqlite3 = require('sqlite3').verbose();
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting sync anomaly tables migration...');
function run(sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function (err) {
if (err) reject(err);
else resolve(this);
});
});
}
function all(sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
async function migrate() {
// 1. Create ivanti_sync_anomaly_log table
await run(`
CREATE TABLE IF NOT EXISTS ivanti_sync_anomaly_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sync_timestamp DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
open_count_delta INTEGER NOT NULL DEFAULT 0,
closed_count_delta INTEGER NOT NULL DEFAULT 0,
newly_archived_count INTEGER NOT NULL DEFAULT 0,
returned_count INTEGER NOT NULL DEFAULT 0,
classification_json TEXT NOT NULL DEFAULT '{}',
is_significant INTEGER NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`);
console.log('✓ ivanti_sync_anomaly_log table ready');
// 2. Create ivanti_finding_bu_history table
await run(`
CREATE TABLE IF NOT EXISTS ivanti_finding_bu_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
previous_bu TEXT NOT NULL,
new_bu TEXT NOT NULL,
detected_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`);
console.log('✓ ivanti_finding_bu_history table ready');
// 3. Create indexes
await run('CREATE INDEX IF NOT EXISTS idx_anomaly_sync_timestamp ON ivanti_sync_anomaly_log(sync_timestamp)');
console.log('✓ idx_anomaly_sync_timestamp index ready');
await run('CREATE INDEX IF NOT EXISTS idx_bu_history_finding_id ON ivanti_finding_bu_history(finding_id)');
console.log('✓ idx_bu_history_finding_id index ready');
await run('CREATE INDEX IF NOT EXISTS idx_bu_history_detected_at ON ivanti_finding_bu_history(detected_at)');
console.log('✓ idx_bu_history_detected_at index ready');
}
migrate()
.then(() => {
console.log('Migration complete.');
db.close();
})
.catch((err) => {
console.error('Migration failed:', err);
db.close();
process.exit(1);
});

View File

@@ -0,0 +1,25 @@
// Migration: Add hostname column to ivanti_todo_queue
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_todo_queue_hostname migration...');
db.run(
'ALTER TABLE ivanti_todo_queue ADD COLUMN hostname TEXT',
(err) => {
if (err) {
// Column may already exist if migration was run before
if (err.message.includes('duplicate column name')) {
console.log('✓ hostname column already exists, skipping');
} else {
console.error('Error adding column:', err);
}
} else {
console.log('✓ hostname column added');
}
db.close(() => console.log('Migration complete!'));
}
);

View File

@@ -0,0 +1,25 @@
// Migration: Add ip_address column to ivanti_todo_queue
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_todo_queue_ip_address migration...');
db.run(
'ALTER TABLE ivanti_todo_queue ADD COLUMN ip_address TEXT',
(err) => {
if (err) {
// Column may already exist if migration was run before
if (err.message.includes('duplicate column name')) {
console.log('✓ ip_address column already exists, skipping');
} else {
console.error('Error adding column:', err);
}
} else {
console.log('✓ ip_address column added');
}
db.close(() => console.log('Migration complete!'));
}
);

View File

@@ -0,0 +1,68 @@
// Migration: Add bu_teams column to users table
// Stores comma-separated BU team identifiers per user (e.g. 'STEAM,ACCESS-ENG')
// Existing users get empty string (admin must assign teams post-migration)
// Idempotent — safe to run multiple times
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const DB_FILE = path.join(__dirname, '..', 'cve_database.db');
/**
* Run the migration against the given database instance.
* Exported for testing with in-memory databases.
* @param {sqlite3.Database} db
* @returns {Promise<void>}
*/
function runMigration(db) {
return new Promise((resolve, reject) => {
db.serialize(() => {
// Check if bu_teams column already exists
db.all("PRAGMA table_info(users)", (err, columns) => {
if (err) {
reject(err);
return;
}
const hasBuTeams = columns.some(col => col.name === 'bu_teams');
if (hasBuTeams) {
console.log('✓ bu_teams column already exists — skipping migration');
resolve();
return;
}
console.log('Adding bu_teams column to users table...');
db.run(
`ALTER TABLE users ADD COLUMN bu_teams TEXT NOT NULL DEFAULT ''`,
(err) => {
if (err) {
reject(err);
return;
}
console.log('✓ Added bu_teams column (default: empty string)');
console.log(' Note: Admin must assign teams to existing users via user management UI');
resolve();
}
);
});
});
});
}
// Run directly if executed as a script
if (require.main === module) {
const db = new sqlite3.Database(DB_FILE);
runMigration(db)
.then(() => {
console.log('Migration complete.');
db.close();
})
.catch((err) => {
console.error('Migration failed:', err.message);
db.close();
process.exit(1);
});
}
module.exports = { runMigration };

View File

@@ -0,0 +1,146 @@
// Migration: Add user_group column to users table and map legacy roles
// Mapping: admin→Admin, editor→Standard_User, viewer→Read_Only
// NULL/unrecognized roles default to Read_Only
// Idempotent — safe to run multiple times
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
/**
* Run the migration against the given database instance.
* Exported for testing with in-memory databases.
* @param {sqlite3.Database} db
* @returns {Promise<void>}
*/
function runMigration(db) {
return new Promise((resolve, reject) => {
db.serialize(() => {
// Check if user_group column already exists
db.all("PRAGMA table_info(users)", (err, columns) => {
if (err) {
reject(err);
return;
}
const hasUserGroup = columns.some(col => col.name === 'user_group');
if (hasUserGroup) {
console.log('✓ user_group column already exists — skipping migration');
resolve();
return;
}
console.log('Adding user_group column to users table...');
// SQLite doesn't support ADD COLUMN with CHECK inline in all versions,
// so we add the column first, map values, then recreate with constraint.
// However, SQLite also doesn't support ALTER TABLE ADD CONSTRAINT.
// Strategy: add column, map values, create index.
// The CHECK constraint is enforced via table rebuild.
db.run(
`ALTER TABLE users ADD COLUMN user_group VARCHAR(20) NOT NULL DEFAULT 'Read_Only'`,
(err) => {
if (err) {
reject(err);
return;
}
console.log('✓ Added user_group column');
// Map existing roles to groups
db.run(
`UPDATE users SET user_group = 'Admin' WHERE role = 'admin'`,
function(err) {
if (err) { reject(err); return; }
console.log(` ✓ Mapped ${this.changes} admin(s) → Admin`);
db.run(
`UPDATE users SET user_group = 'Standard_User' WHERE role = 'editor'`,
function(err) {
if (err) { reject(err); return; }
console.log(` ✓ Mapped ${this.changes} editor(s) → Standard_User`);
db.run(
`UPDATE users SET user_group = 'Read_Only' WHERE role = 'viewer'`,
function(err) {
if (err) { reject(err); return; }
console.log(` ✓ Mapped ${this.changes} viewer(s) → Read_Only`);
// Map NULL or unrecognized roles to Read_Only
db.run(
`UPDATE users SET user_group = 'Read_Only' WHERE user_group = 'Read_Only' AND role NOT IN ('admin', 'editor', 'viewer')`,
function(err) {
if (err) { reject(err); return; }
console.log(` ✓ Mapped ${this.changes} unrecognized role(s) → Read_Only`);
// Create index on user_group
db.run(
`CREATE INDEX IF NOT EXISTS idx_users_user_group ON users(user_group)`,
(err) => {
if (err) { reject(err); return; }
console.log('✓ Created idx_users_user_group index');
// Add CHECK constraint via trigger (SQLite can't ALTER TABLE ADD CONSTRAINT)
db.run(
`CREATE TRIGGER IF NOT EXISTS check_user_group_insert
BEFORE INSERT ON users
FOR EACH ROW
WHEN NEW.user_group NOT IN ('Admin', 'Standard_User', 'Leadership', 'Read_Only')
BEGIN
SELECT RAISE(ABORT, 'Invalid user_group value. Must be Admin, Standard_User, Leadership, or Read_Only');
END`,
(err) => {
if (err) { reject(err); return; }
db.run(
`CREATE TRIGGER IF NOT EXISTS check_user_group_update
BEFORE UPDATE OF user_group ON users
FOR EACH ROW
WHEN NEW.user_group NOT IN ('Admin', 'Standard_User', 'Leadership', 'Read_Only')
BEGIN
SELECT RAISE(ABORT, 'Invalid user_group value. Must be Admin, Standard_User, Leadership, or Read_Only');
END`,
(err) => {
if (err) { reject(err); return; }
console.log('✓ Created user_group validation triggers');
console.log('Migration complete!');
resolve();
}
);
}
);
}
);
}
);
}
);
}
);
}
);
}
);
});
});
});
}
// Run directly if executed as a script
if (require.main === module) {
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Starting add_user_groups migration...');
runMigration(db)
.then(() => {
db.close(() => {
console.log('Database connection closed.');
});
})
.catch((err) => {
console.error('Migration failed:', err);
db.close();
process.exit(1);
});
}
module.exports = { runMigration };

View File

@@ -1,59 +0,0 @@
// Migration: Add weekly_reports table for vulnerability report uploads
const sqlite3 = require('sqlite3').verbose();
const path = require('path');
const dbPath = path.join(__dirname, '..', 'cve_database.db');
const db = new sqlite3.Database(dbPath);
console.log('Running migration: add_weekly_reports_table');
db.serialize(() => {
db.run(`
CREATE TABLE IF NOT EXISTS weekly_reports (
id INTEGER PRIMARY KEY AUTOINCREMENT,
upload_date DATE NOT NULL,
week_label VARCHAR(50),
original_filename VARCHAR(255),
processed_filename VARCHAR(255),
original_file_path VARCHAR(500),
processed_file_path VARCHAR(500),
row_count_original INTEGER,
row_count_processed INTEGER,
uploaded_by INTEGER,
uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
is_current BOOLEAN DEFAULT 0,
FOREIGN KEY (uploaded_by) REFERENCES users(id)
)
`, (err) => {
if (err) {
console.error('Error creating weekly_reports table:', err);
process.exit(1);
}
console.log('✓ Created weekly_reports table');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_weekly_reports_date
ON weekly_reports(upload_date DESC)
`, (err) => {
if (err) {
console.error('Error creating date index:', err);
process.exit(1);
}
console.log('✓ Created index on upload_date');
});
db.run(`
CREATE INDEX IF NOT EXISTS idx_weekly_reports_current
ON weekly_reports(is_current)
`, (err) => {
if (err) {
console.error('Error creating current index:', err);
process.exit(1);
}
console.log('✓ Created index on is_current');
console.log('\nMigration completed successfully!');
db.close();
});
});

View File

@@ -0,0 +1,160 @@
#!/usr/bin/env node
// backfill_anomaly_log.js — One-time backfill of ivanti_sync_anomaly_log
//
// Synthesizes anomaly log entries from existing ivanti_archive_transitions
// and ivanti_counts_history data so the archive activity sparkline on the
// Findings Trend chart has historical data to display.
//
// Safe to run multiple times — checks for existing rows before inserting.
//
// Usage: node backend/migrations/backfill_anomaly_log.js
const path = require('path');
const sqlite3 = require('sqlite3').verbose();
const DB_PATH = path.join(__dirname, '..', 'cve_database.db');
function dbAll(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
function dbGet(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.get(sql, params, (err, row) => {
if (err) reject(err);
else resolve(row);
});
});
}
function dbRun(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function (err) {
if (err) reject(err);
else resolve(this);
});
});
}
async function main() {
const db = new sqlite3.Database(DB_PATH);
// Check if anomaly log already has data
const existing = await dbGet(db, 'SELECT COUNT(*) as cnt FROM ivanti_sync_anomaly_log');
if (existing.cnt > 0) {
console.log(`ivanti_sync_anomaly_log already has ${existing.cnt} rows — skipping backfill.`);
console.log('To force re-run, delete existing rows first:');
console.log(' sqlite3 backend/cve_database.db "DELETE FROM ivanti_sync_anomaly_log;"');
db.close();
return;
}
// Get archive transitions grouped by date
const transitions = await dbAll(db,
`SELECT DATE(transitioned_at) as date,
to_state,
reason,
COUNT(*) as cnt
FROM ivanti_archive_transitions
GROUP BY date, to_state, reason
ORDER BY date`
);
// Get counts history (last snapshot per day) for delta computation
const countsRows = await dbAll(db,
`SELECT date, open_count, closed_count FROM (
SELECT DATE(recorded_at) AS date,
open_count, closed_count,
ROW_NUMBER() OVER (
PARTITION BY DATE(recorded_at)
ORDER BY recorded_at DESC
) AS rn
FROM ivanti_counts_history
) WHERE rn = 1
ORDER BY date ASC`
);
// Build a map of date -> { open_count, closed_count }
const countsMap = {};
for (const row of countsRows) {
countsMap[row.date] = { open: row.open_count, closed: row.closed_count };
}
// Build per-date anomaly summaries from transitions
const dateMap = {};
for (const t of transitions) {
if (!dateMap[t.date]) {
dateMap[t.date] = { archived: 0, returned: 0, classification: {} };
}
const entry = dateMap[t.date];
if (t.to_state === 'ARCHIVED') {
entry.archived += t.cnt;
// All pre-feature transitions have reason 'severity_score_drift'
// but from the investigation we know the 04/24 batch was mostly
// BU reassignment. We can't retroactively classify without the
// Ivanti API, so we label them as 'unclassified' (pre-feature).
entry.classification.unclassified = (entry.classification.unclassified || 0) + t.cnt;
} else if (t.to_state === 'RETURNED') {
entry.returned += t.cnt;
}
// CLOSED transitions are not archive events — they're findings
// confirmed in the closed set, so we don't count them as archived.
}
// Compute deltas and insert rows
const dates = Object.keys(dateMap).sort();
let inserted = 0;
for (const date of dates) {
const entry = dateMap[date];
const counts = countsMap[date];
// Find the previous day's counts for delta computation
const dateIdx = countsRows.findIndex(r => r.date === date);
let openDelta = 0;
let closedDelta = 0;
if (counts && dateIdx > 0) {
const prev = countsRows[dateIdx - 1];
openDelta = counts.open - prev.open_count;
closedDelta = counts.closed - prev.closed_count;
}
const isSignificant = entry.archived > 5 ? 1 : 0;
const classificationJson = JSON.stringify(entry.classification);
await dbRun(db,
`INSERT INTO ivanti_sync_anomaly_log
(sync_timestamp, open_count_delta, closed_count_delta,
newly_archived_count, returned_count, classification_json, is_significant)
VALUES (?, ?, ?, ?, ?, ?, ?)`,
[
`${date}T23:59:00`,
openDelta,
closedDelta,
entry.archived,
entry.returned,
classificationJson,
isSignificant,
]
);
inserted++;
const sigLabel = isSignificant ? ' [SIGNIFICANT]' : '';
console.log(` ${date}: ${entry.archived} archived, ${entry.returned} returned, delta open=${openDelta} closed=${closedDelta}${sigLabel}`);
}
console.log(`\nBackfill complete: ${inserted} anomaly log entries created.`);
db.close();
}
main().catch(err => {
console.error('Fatal error:', err);
process.exit(1);
});

View File

@@ -0,0 +1,165 @@
#!/usr/bin/env node
// backfill_return_classification.js
//
// Retroactively populates return_classification_json for existing anomaly log
// rows that have returned_count > 0 but an empty return classification.
//
// For each such row, looks at archive transitions that went ARCHIVED → RETURNED
// on that date, then finds the *prior* archive reason (the most recent
// transition to ARCHIVED for that same archive record) to determine why the
// finding originally left — which tells us why it came back.
//
// Safe to run multiple times — only updates rows with empty classification.
//
// Usage: node backend/migrations/backfill_return_classification.js
const path = require('path');
const sqlite3 = require('sqlite3').verbose();
const DB_PATH = path.join(__dirname, '..', 'cve_database.db');
function dbAll(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
function dbGet(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.get(sql, params, (err, row) => {
if (err) reject(err);
else resolve(row);
});
});
}
function dbRun(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function (err) {
if (err) reject(err);
else resolve(this);
});
});
}
async function main() {
const db = new sqlite3.Database(DB_PATH);
// Find anomaly log rows that have returned findings but no return classification
const rows = await dbAll(db,
`SELECT id, sync_timestamp, returned_count, return_classification_json
FROM ivanti_sync_anomaly_log
WHERE returned_count > 0
ORDER BY sync_timestamp ASC`
);
if (rows.length === 0) {
console.log('No anomaly log rows with returned findings found — nothing to backfill.');
db.close();
return;
}
const force = process.argv.includes('--force');
let updated = 0;
let skipped = 0;
for (const row of rows) {
// Skip if already has a non-empty classification (unless --force)
if (!force) {
let existing = {};
try { existing = JSON.parse(row.return_classification_json || '{}'); } catch (_) {}
const hasData = Object.values(existing).some(v => v > 0);
if (hasData) {
skipped++;
continue;
}
}
// Find the date of this anomaly row
const date = row.sync_timestamp.split('T')[0].split(' ')[0];
// Find all ARCHIVED → RETURNED transitions on this date
const returnTransitions = await dbAll(db,
`SELECT archive_id
FROM ivanti_archive_transitions
WHERE to_state = 'RETURNED'
AND DATE(transitioned_at) = ?`,
[date]
);
if (returnTransitions.length === 0) {
// No transitions found for this date — try a wider window (±1 day)
// since sync_timestamp and transitioned_at might not align exactly
const wider = await dbAll(db,
`SELECT archive_id
FROM ivanti_archive_transitions
WHERE to_state = 'RETURNED'
AND DATE(transitioned_at) BETWEEN DATE(?, '-1 day') AND DATE(?, '+1 day')`,
[date, date]
);
if (wider.length === 0) {
console.log(` ${date}: ${row.returned_count} returned but no matching transitions found — skipping`);
continue;
}
returnTransitions.push(...wider);
}
// For each returned finding, look up the prior archive reason
const classification = { bu_reassignment: 0, severity_drift: 0, closed_on_platform: 0, decommissioned: 0 };
const seen = new Set();
for (const rt of returnTransitions) {
if (seen.has(rt.archive_id)) continue;
seen.add(rt.archive_id);
// Find the most recent ARCHIVED transition *before* this return
// (the reason it was archived before it came back)
const archiveTransition = await dbGet(db,
`SELECT reason FROM ivanti_archive_transitions
WHERE archive_id = ? AND to_state = 'ARCHIVED'
AND transitioned_at <= (
SELECT transitioned_at FROM ivanti_archive_transitions
WHERE archive_id = ? AND to_state = 'RETURNED'
AND DATE(transitioned_at) BETWEEN DATE(?, '-1 day') AND DATE(?, '+1 day')
ORDER BY transitioned_at DESC LIMIT 1
)
ORDER BY transitioned_at DESC LIMIT 1`,
[rt.archive_id, rt.archive_id, date, date]
);
if (archiveTransition && archiveTransition.reason) {
const reasonKey = archiveTransition.reason.split(':')[0];
if (reasonKey in classification) {
classification[reasonKey]++;
}
}
}
const classificationJson = JSON.stringify(classification);
await dbRun(db,
`UPDATE ivanti_sync_anomaly_log
SET return_classification_json = ?
WHERE id = ?`,
[classificationJson, row.id]
);
const parts = Object.entries(classification)
.filter(([, v]) => v > 0)
.map(([k, v]) => `${v} ${k}`);
const breakdown = parts.length > 0 ? parts.join(', ') : 'unclassified';
console.log(` ${date}: ${row.returned_count} returned — ${breakdown}`);
updated++;
}
console.log(`\nBackfill complete: ${updated} rows updated, ${skipped} already had data.`);
db.close();
}
main().catch(err => {
console.error('Fatal error:', err);
process.exit(1);
});

View File

@@ -0,0 +1,102 @@
#!/usr/bin/env node
// reclassify_bu_roundtrips.js
//
// Reclassifies archive transitions that were part of a BU reassignment
// round-trip. These are findings that were archived (disappeared from sync)
// and then returned within a short window — indicating they were temporarily
// reassigned to a different BU and then reassigned back.
//
// The original drift checker couldn't classify these correctly because by the
// time it queried Ivanti, the findings had already been reassigned back to
// the expected BUs.
//
// After running this, re-run backfill_return_classification.js to update
// the anomaly log with the corrected reasons.
//
// Usage: node backend/migrations/reclassify_bu_roundtrips.js
const path = require('path');
const sqlite3 = require('sqlite3').verbose();
const DB_PATH = path.join(__dirname, '..', 'cve_database.db');
// Findings that were archived and returned within this many days are
// considered BU reassignment round-trips
const ROUNDTRIP_WINDOW_DAYS = 14;
function dbAll(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
function dbRun(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.run(sql, params, function (err) {
if (err) reject(err);
else resolve(this);
});
});
}
async function main() {
const db = new sqlite3.Database(DB_PATH);
// Find archive transitions where the finding was archived and then returned
// within the roundtrip window, and the archive reason is still the default
// severity_score_drift placeholder
const roundtrips = await dbAll(db, `
SELECT
t_arch.id AS archive_transition_id,
t_arch.archive_id,
a.finding_id,
a.finding_title,
t_arch.reason AS current_reason,
DATE(t_arch.transitioned_at) AS archived_date,
DATE(t_ret.transitioned_at) AS returned_date,
JULIANDAY(t_ret.transitioned_at) - JULIANDAY(t_arch.transitioned_at) AS days_between
FROM ivanti_archive_transitions t_arch
JOIN ivanti_finding_archives a ON a.id = t_arch.archive_id
JOIN ivanti_archive_transitions t_ret
ON t_ret.archive_id = t_arch.archive_id
AND t_ret.to_state = 'RETURNED'
AND t_ret.transitioned_at > t_arch.transitioned_at
WHERE t_arch.to_state = 'ARCHIVED'
AND t_arch.reason = 'severity_score_drift'
AND (JULIANDAY(t_ret.transitioned_at) - JULIANDAY(t_arch.transitioned_at)) BETWEEN 0 AND ?
ORDER BY t_arch.transitioned_at DESC
`, [ROUNDTRIP_WINDOW_DAYS]);
if (roundtrips.length === 0) {
console.log('No BU reassignment round-trips found to reclassify.');
db.close();
return;
}
console.log(`Found ${roundtrips.length} archive transitions to reclassify as bu_reassignment:\n`);
let updated = 0;
for (const rt of roundtrips) {
console.log(` Finding ${rt.finding_id}: archived ${rt.archived_date}, returned ${rt.returned_date} (${Math.round(rt.days_between)}d) — ${rt.current_reason} → bu_reassignment`);
await dbRun(db,
`UPDATE ivanti_archive_transitions SET reason = 'bu_reassignment' WHERE id = ?`,
[rt.archive_transition_id]
);
updated++;
}
console.log(`\nReclassified ${updated} transitions.`);
console.log('\nNow run the return classification backfill to update anomaly log rows:');
console.log(' node backend/migrations/backfill_return_classification.js');
db.close();
}
main().catch(err => {
console.error('Fatal error:', err);
process.exit(1);
});

View File

@@ -1,6 +1,7 @@
// routes/archerTickets.js // routes/archerTickets.js
const express = require('express'); const express = require('express');
const { requireAuth, requireRole } = require('../middleware/auth'); const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog'); const logAudit = require('../helpers/auditLog');
// Validation helpers // Validation helpers
@@ -13,42 +14,43 @@ function isValidVendor(vendor) {
return typeof vendor === 'string' && vendor.trim().length > 0 && vendor.length <= 200; return typeof vendor === 'string' && vendor.trim().length > 0 && vendor.length <= 200;
} }
function createArcherTicketsRouter(db) { function createArcherTicketsRouter() {
const router = express.Router(); const router = express.Router();
// Get all Archer tickets (with optional filters) // Get all Archer tickets (with optional filters)
router.get('/', requireAuth(db), (req, res) => { router.get('/', requireAuth(), async (req, res) => {
const { cve_id, vendor, status } = req.query; const { cve_id, vendor, status } = req.query;
let query = 'SELECT * FROM archer_tickets WHERE 1=1'; let query = 'SELECT * FROM archer_tickets WHERE 1=1';
const params = []; const params = [];
let paramIndex = 1;
if (cve_id) { if (cve_id) {
query += ' AND cve_id = ?'; query += ` AND cve_id = $${paramIndex++}`;
params.push(cve_id); params.push(cve_id);
} }
if (vendor) { if (vendor) {
query += ' AND vendor = ?'; query += ` AND vendor = $${paramIndex++}`;
params.push(vendor); params.push(vendor);
} }
if (status) { if (status) {
query += ' AND status = ?'; query += ` AND status = $${paramIndex++}`;
params.push(status); params.push(status);
} }
query += ' ORDER BY created_at DESC'; query += ' ORDER BY created_at DESC';
db.all(query, params, (err, rows) => { try {
if (err) { const { rows } = await pool.query(query, params);
console.error('Error fetching Archer tickets:', err);
return res.status(500).json({ error: 'Internal server error.' });
}
res.json(rows); res.json(rows);
}); } catch (err) {
console.error('Error fetching Archer tickets:', err);
res.status(500).json({ error: 'Internal server error.' });
}
}); });
// Create Archer ticket // Create Archer ticket
router.post('/', requireAuth(db), requireRole('editor', 'admin'), (req, res) => { router.post('/', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { exc_number, archer_url, status, cve_id, vendor } = req.body; const { exc_number, archer_url, status, cve_id, vendor } = req.body;
// Validation // Validation
@@ -73,38 +75,38 @@ function createArcherTicketsRouter(db) {
const validatedStatus = status || 'Draft'; const validatedStatus = status || 'Draft';
db.run( try {
`INSERT INTO archer_tickets (exc_number, archer_url, status, cve_id, vendor) const { rows } = await pool.query(
VALUES (?, ?, ?, ?, ?)`, `INSERT INTO archer_tickets (exc_number, archer_url, status, cve_id, vendor, created_by)
[exc_number.trim(), archer_url || null, validatedStatus, cve_id, vendor], VALUES ($1, $2, $3, $4, $5, $6)
function(err) { RETURNING id`,
if (err) { [exc_number.trim(), archer_url || null, validatedStatus, cve_id, vendor, req.user.id]
console.error('Error creating Archer ticket:', err); );
if (err.message.includes('UNIQUE constraint failed')) {
return res.status(409).json({ error: 'An Archer ticket with this EXC number already exists.' });
}
return res.status(500).json({ error: 'Internal server error.' });
}
logAudit(db, { logAudit({
userId: req.user.id, userId: req.user.id,
action: 'CREATE_ARCHER_TICKET', action: 'CREATE_ARCHER_TICKET',
targetType: 'archer_ticket', entityType: 'archer_ticket',
targetId: this.lastID, entityId: String(rows[0].id),
details: { exc_number, archer_url, status: validatedStatus, cve_id, vendor }, details: { exc_number, archer_url, status: validatedStatus, cve_id, vendor },
ipAddress: req.ip ipAddress: req.ip
}); });
res.status(201).json({ res.status(201).json({
id: this.lastID, id: rows[0].id,
message: 'Archer ticket created successfully' message: 'Archer ticket created successfully'
}); });
} catch (err) {
console.error('Error creating Archer ticket:', err);
if (err.code === '23505') {
return res.status(409).json({ error: 'An Archer ticket with this EXC number already exists.' });
} }
); res.status(500).json({ error: 'Internal server error.' });
}
}); });
// Update Archer ticket // Update Archer ticket
router.put('/:id', requireAuth(db), requireRole('editor', 'admin'), (req, res) => { router.put('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params; const { id } = req.params;
const { exc_number, archer_url, status } = req.body; const { exc_number, archer_url, status } = req.body;
@@ -124,29 +126,27 @@ function createArcherTicketsRouter(db) {
return res.status(400).json({ error: 'Invalid status. Must be Draft, Open, Under Review, or Accepted.' }); return res.status(400).json({ error: 'Invalid status. Must be Draft, Open, Under Review, or Accepted.' });
} }
// Get existing ticket try {
db.get('SELECT * FROM archer_tickets WHERE id = ?', [id], (err, existing) => { const { rows } = await pool.query('SELECT * FROM archer_tickets WHERE id = $1', [id]);
if (err) { const existing = rows[0];
console.error(err);
return res.status(500).json({ error: 'Internal server error.' });
}
if (!existing) { if (!existing) {
return res.status(404).json({ error: 'Archer ticket not found.' }); return res.status(404).json({ error: 'Archer ticket not found.' });
} }
const updates = []; const updates = [];
const params = []; const params = [];
let paramIndex = 1;
if (exc_number !== undefined) { if (exc_number !== undefined) {
updates.push('exc_number = ?'); updates.push(`exc_number = $${paramIndex++}`);
params.push(exc_number.trim()); params.push(exc_number.trim());
} }
if (archer_url !== undefined) { if (archer_url !== undefined) {
updates.push('archer_url = ?'); updates.push(`archer_url = $${paramIndex++}`);
params.push(archer_url || null); params.push(archer_url || null);
} }
if (status !== undefined) { if (status !== undefined) {
updates.push('status = ?'); updates.push(`status = $${paramIndex++}`);
params.push(status); params.push(status);
} }
@@ -154,67 +154,113 @@ function createArcherTicketsRouter(db) {
return res.status(400).json({ error: 'No fields to update.' }); return res.status(400).json({ error: 'No fields to update.' });
} }
updates.push('updated_at = CURRENT_TIMESTAMP'); updates.push('updated_at = NOW()');
params.push(id); params.push(id);
db.run( const result = await pool.query(
`UPDATE archer_tickets SET ${updates.join(', ')} WHERE id = ?`, `UPDATE archer_tickets SET ${updates.join(', ')} WHERE id = $${paramIndex}`,
params, params
function(err) {
if (err) {
console.error(err);
if (err.message.includes('UNIQUE constraint failed')) {
return res.status(409).json({ error: 'An Archer ticket with this EXC number already exists.' });
}
return res.status(500).json({ error: 'Internal server error.' });
}
logAudit(db, {
userId: req.user.id,
action: 'UPDATE_ARCHER_TICKET',
targetType: 'archer_ticket',
targetId: id,
details: { before: existing, changes: req.body },
ipAddress: req.ip
});
res.json({ message: 'Archer ticket updated successfully', changes: this.changes });
}
); );
});
logAudit({
userId: req.user.id,
action: 'UPDATE_ARCHER_TICKET',
entityType: 'archer_ticket',
entityId: String(id),
details: { before: existing, changes: req.body },
ipAddress: req.ip
});
res.json({ message: 'Archer ticket updated successfully', changes: result.rowCount });
} catch (err) {
console.error(err);
if (err.code === '23505') {
return res.status(409).json({ error: 'An Archer ticket with this EXC number already exists.' });
}
res.status(500).json({ error: 'Internal server error.' });
}
}); });
// Delete Archer ticket // Delete Archer ticket
router.delete('/:id', requireAuth(db), requireRole('editor', 'admin'), (req, res) => { router.delete('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params; const { id } = req.params;
db.get('SELECT * FROM archer_tickets WHERE id = ?', [id], (err, ticket) => { try {
if (err) { const { rows } = await pool.query('SELECT * FROM archer_tickets WHERE id = $1', [id]);
console.error(err); const ticket = rows[0];
return res.status(500).json({ error: 'Internal server error.' });
}
if (!ticket) { if (!ticket) {
return res.status(404).json({ error: 'Archer ticket not found.' }); return res.status(404).json({ error: 'Archer ticket not found.' });
} }
db.run('DELETE FROM archer_tickets WHERE id = ?', [id], function(err) { // Admin bypasses all delete restrictions
if (err) { if (req.user.group === 'Admin') {
console.error(err); return performArcherDelete();
return res.status(500).json({ error: 'Internal server error.' }); }
}
logAudit(db, { // Standard_User: ownership check
if (ticket.created_by && ticket.created_by !== req.user.id) {
return res.status(403).json({ error: 'You can only delete resources you created' });
}
// Standard_User: compliance linkage check
const excNumber = ticket.exc_number;
try {
const { rows: compLinks } = await pool.query(
`SELECT ci.id, ci.extra_json
FROM compliance_items ci
JOIN compliance_uploads cu ON ci.upload_id = cu.id
WHERE ci.status = 'active' AND ci.extra_json ILIKE $1`,
[`%${excNumber}%`]
);
const isLinked = (compLinks || []).some(cl => {
const json = cl.extra_json || '';
return json.includes(excNumber);
});
if (isLinked) {
return res.status(403).json({ error: 'Cannot delete ticket linked to compliance report. Contact an admin.' });
}
} catch (compErr) {
if (!compErr.message.includes('does not exist')) throw compErr;
}
return performArcherDelete();
async function performArcherDelete() {
await pool.query('DELETE FROM archer_tickets WHERE id = $1', [id]);
logAudit({
userId: req.user.id, userId: req.user.id,
action: 'DELETE_ARCHER_TICKET', action: 'DELETE_ARCHER_TICKET',
targetType: 'archer_ticket', entityType: 'archer_ticket',
targetId: id, entityId: String(id),
details: { deleted: ticket }, details: { deleted: ticket },
ipAddress: req.ip ipAddress: req.ip
}); });
res.json({ message: 'Archer ticket deleted successfully' }); res.json({ message: 'Archer ticket deleted successfully' });
}); }
}); } catch (err) {
console.error(err);
res.status(500).json({ error: 'Internal server error.' });
}
});
// GET /status-trend — ticket counts grouped by creation date + status
router.get('/status-trend', requireAuth(), async (req, res) => {
try {
const { rows } = await pool.query(
`SELECT DATE(created_at) AS date, status, COUNT(*) AS count
FROM archer_tickets
GROUP BY DATE(created_at), status
ORDER BY date ASC`
);
res.json({ statusTrend: rows });
} catch (err) {
console.error('Error fetching Archer status trend:', err);
res.status(500).json({ error: 'Internal server error.' });
}
}); });
return router; return router;

476
backend/routes/atlas.js Normal file
View File

@@ -0,0 +1,476 @@
// Atlas InfoSec Action Plans Routes
// Proxies CRUD operations to the Atlas API and maintains a local cache
// for fast badge rendering on the ReportingPage.
const express = require('express');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog');
const { isConfigured, atlasGet, atlasPut, atlasPatch, atlasPost } = require('../helpers/atlasApi');
const fs = require('fs');
const path = require('path');
const VALID_PLAN_TYPES = ['decommission', 'remediation', 'false_positive', 'risk_acceptance', 'scan_exclusion'];
const DATE_PATTERN = /^\d{4}-\d{2}-\d{2}$/;
// Diagnostic log helper
function syncLog(msg) {
const line = `${new Date().toISOString()} ${msg}\n`;
try { fs.appendFileSync(path.join(__dirname, '..', 'atlas-sync-debug.log'), line); } catch (_) { /* ignore */ }
console.log(msg);
}
// ---------------------------------------------------------------------------
// Pure aggregation function — exported for testability
// ---------------------------------------------------------------------------
function aggregateAtlasMetrics(rows) {
const result = {
totalHosts: rows.length,
hostsWithPlans: 0,
hostsWithoutPlans: 0,
plansByType: {},
plansByStatus: {},
totalPlans: 0
};
for (const row of rows) {
if (row.has_action_plan === true || row.has_action_plan === 1) {
result.hostsWithPlans++;
} else {
result.hostsWithoutPlans++;
}
let plans;
try {
plans = JSON.parse(row.plans_json);
} catch (e) {
continue;
}
if (!Array.isArray(plans)) continue;
for (const plan of plans) {
result.totalPlans++;
if (plan.plan_type) {
result.plansByType[plan.plan_type] = (result.plansByType[plan.plan_type] || 0) + 1;
}
if (plan.status) {
result.plansByStatus[plan.status] = (result.plansByStatus[plan.status] || 0) + 1;
}
}
}
return result;
}
// ---------------------------------------------------------------------------
// Router factory
// ---------------------------------------------------------------------------
function createAtlasRouter() {
const router = express.Router();
// GET /metrics
router.get('/metrics', requireAuth(), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
try {
const { rows } = await pool.query(
`SELECT has_action_plan, plans_json FROM atlas_action_plans_cache`
);
const metrics = aggregateAtlasMetrics(rows);
res.json(metrics);
} catch (err) {
console.error('[Atlas] Error fetching metrics:', err.message);
res.status(500).json({ error: 'Failed to fetch Atlas metrics.' });
}
});
// GET /status
router.get('/status', requireAuth(), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
try {
const { rows } = await pool.query(
`SELECT host_id, has_action_plan, plan_count, plans_json, synced_at FROM atlas_action_plans_cache`
);
res.json(rows);
} catch (err) {
console.error('[Atlas] Error fetching status:', err.message);
res.status(500).json({ error: 'Failed to fetch Atlas status.' });
}
});
// POST /sync
router.post('/sync', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
try {
// Read Ivanti findings and extract unique non-null hostIds
const { rows: findingsRows } = await pool.query(
`SELECT DISTINCT host_id FROM ivanti_findings WHERE host_id IS NOT NULL AND host_id > 0`
);
const hostIds = findingsRows.map(r => r.host_id);
if (hostIds.length === 0) {
return res.json({ synced: 0, withPlans: 0, failed: 0 });
}
let synced = 0;
let withPlans = 0;
let failed = 0;
const BATCH_SIZE = 5;
for (let i = 0; i < hostIds.length; i += BATCH_SIZE) {
const batch = hostIds.slice(i, i + BATCH_SIZE);
const results = await Promise.allSettled(
batch.map(async (hostId) => {
const result = await atlasGet('/hosts/' + hostId + '/action-plans');
return { hostId, result };
})
);
for (const settled of results) {
if (settled.status === 'rejected') {
failed++;
console.warn('[Atlas Sync] Request failed for host:', settled.reason?.message || settled.reason);
continue;
}
const { hostId, result } = settled.value;
if (result.status >= 200 && result.status < 300) {
let allPlans = [];
let activePlans = [];
try {
const parsed = JSON.parse(result.body);
if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {
activePlans = Array.isArray(parsed.active) ? parsed.active : [];
const inactive = Array.isArray(parsed.inactive) ? parsed.inactive : [];
allPlans = [...activePlans, ...inactive];
} else if (Array.isArray(parsed)) {
allPlans = parsed;
activePlans = parsed;
}
} catch (e) {
allPlans = [];
activePlans = [];
}
const planCount = activePlans.length;
const hasActionPlan = planCount > 0;
try {
if (!hasActionPlan) {
const { rows: existingRows } = await pool.query(
`SELECT has_action_plan, plans_json, synced_at FROM atlas_action_plans_cache WHERE host_id = $1`,
[hostId]
);
const existing = existingRows[0];
if (existing && existing.has_action_plan === true) {
let existingPlans = [];
try { existingPlans = JSON.parse(existing.plans_json || '[]'); } catch (_) {}
const hasBulkStub = existingPlans.some(p => p.source === 'bulk-create');
if (hasBulkStub) {
const ageMs = Date.now() - new Date(existing.synced_at).getTime();
const TEN_MINUTES = 10 * 60 * 1000;
if (ageMs < TEN_MINUTES) {
synced++;
withPlans++;
continue;
}
}
}
}
await pool.query(
`INSERT INTO atlas_action_plans_cache (host_id, has_action_plan, plan_count, plans_json, synced_at)
VALUES ($1, $2, $3, $4, NOW())
ON CONFLICT(host_id) DO UPDATE SET
has_action_plan = EXCLUDED.has_action_plan,
plan_count = EXCLUDED.plan_count,
plans_json = EXCLUDED.plans_json,
synced_at = EXCLUDED.synced_at`,
[hostId, hasActionPlan, planCount, JSON.stringify(allPlans)]
);
} catch (dbErr) {
console.error('[Atlas Sync] DB upsert failed for host', hostId, ':', dbErr.message);
}
synced++;
if (hasActionPlan) withPlans++;
} else {
failed++;
console.warn(`[Atlas Sync] Non-2xx response for host ${hostId}: status ${result.status}`);
}
}
}
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'ATLAS_SYNC',
entityType: 'atlas_action_plans',
entityId: null,
details: { synced, withPlans, failed, totalHosts: hostIds.length },
ipAddress: req.ip
});
res.json({ synced, withPlans, failed });
} catch (err) {
console.error('[Atlas Sync] Unexpected error:', err.message);
res.status(500).json({ error: 'Atlas sync failed: ' + err.message });
}
});
// GET /hosts/:hostId/action-plans
router.get('/hosts/:hostId/action-plans', requireAuth(), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
const hostId = parseInt(req.params.hostId, 10);
if (!Number.isInteger(hostId) || hostId <= 0) {
return res.status(400).json({ error: 'hostId must be a positive integer' });
}
try {
const result = await atlasGet('/hosts/' + hostId + '/action-plans');
if (result.status >= 200 && result.status < 300) {
let body;
try { body = JSON.parse(result.body); } catch (e) { body = result.body; }
res.status(result.status).json(body);
} else {
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (e) { errorBody = { error: result.body }; }
res.status(result.status).json(errorBody);
}
} catch (err) {
console.error('[Atlas] GET action-plans failed for host', hostId, ':', err.message);
res.status(502).json({ error: 'Failed to reach Atlas API: ' + err.message });
}
});
// PUT /hosts/:hostId/action-plans
router.put('/hosts/:hostId/action-plans', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
const hostId = parseInt(req.params.hostId, 10);
if (!Number.isInteger(hostId) || hostId <= 0) {
return res.status(400).json({ error: 'hostId must be a positive integer' });
}
const { plan_type, commit_date } = req.body || {};
if (!plan_type || !VALID_PLAN_TYPES.includes(plan_type)) {
return res.status(400).json({ error: 'plan_type must be one of: ' + VALID_PLAN_TYPES.join(', ') });
}
if (!commit_date || !DATE_PATTERN.test(commit_date)) {
return res.status(400).json({ error: 'commit_date must be a valid YYYY-MM-DD date string' });
}
try {
const result = await atlasPut('/hosts/' + hostId + '/action-plans', req.body);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'ATLAS_CREATE_PLAN',
entityType: 'atlas_action_plan',
entityId: String(hostId),
details: { hostId, plan_type, commit_date },
ipAddress: req.ip
});
if (result.status >= 200 && result.status < 300) {
let body;
try { body = JSON.parse(result.body); } catch (e) { body = result.body; }
res.status(result.status).json(body);
} else {
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (e) { errorBody = { error: result.body }; }
res.status(result.status).json(errorBody);
}
} catch (err) {
console.error('[Atlas] PUT action-plans failed for host', hostId, ':', err.message);
res.status(502).json({ error: 'Failed to reach Atlas API: ' + err.message });
}
});
// PATCH /hosts/:hostId/action-plans
router.patch('/hosts/:hostId/action-plans', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
const hostId = parseInt(req.params.hostId, 10);
if (!Number.isInteger(hostId) || hostId <= 0) {
return res.status(400).json({ error: 'hostId must be a positive integer' });
}
const { action_plan_id, updates } = req.body || {};
if (!action_plan_id || typeof action_plan_id !== 'string' || action_plan_id.trim() === '') {
return res.status(400).json({ error: 'action_plan_id is required and must be a non-empty string' });
}
if (!updates || typeof updates !== 'object' || Array.isArray(updates)) {
return res.status(400).json({ error: 'updates is required and must be an object' });
}
try {
const result = await atlasPatch('/hosts/' + hostId + '/action-plans', req.body);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'ATLAS_UPDATE_PLAN',
entityType: 'atlas_action_plan',
entityId: String(hostId),
details: { hostId, action_plan_id },
ipAddress: req.ip
});
if (result.status >= 200 && result.status < 300) {
let body;
try { body = JSON.parse(result.body); } catch (e) { body = result.body; }
res.status(result.status).json(body);
} else {
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (e) { errorBody = { error: result.body }; }
res.status(result.status).json(errorBody);
}
} catch (err) {
console.error('[Atlas] PATCH action-plans failed for host', hostId, ':', err.message);
res.status(502).json({ error: 'Failed to reach Atlas API: ' + err.message });
}
});
// POST /hosts/bulk-action-plans
router.post('/hosts/bulk-action-plans', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
const { host_ids, plan_type, commit_date } = req.body || {};
if (!Array.isArray(host_ids) || host_ids.length === 0) {
return res.status(400).json({ error: 'host_ids must be a non-empty array of positive integers' });
}
for (const id of host_ids) {
if (!Number.isInteger(id) || id <= 0) {
return res.status(400).json({ error: 'host_ids must be a non-empty array of positive integers' });
}
}
if (!plan_type || !VALID_PLAN_TYPES.includes(plan_type)) {
return res.status(400).json({ error: 'plan_type must be one of: ' + VALID_PLAN_TYPES.join(', ') });
}
if (!commit_date || !DATE_PATTERN.test(commit_date)) {
return res.status(400).json({ error: 'commit_date must be a valid YYYY-MM-DD date string' });
}
try {
const result = await atlasPost('/hosts/create-bulk-action-plans', req.body);
if (result.status >= 200 && result.status < 300) {
let body;
try { body = JSON.parse(result.body); } catch (e) { body = result.body; }
// Optimistically update local cache
for (const hid of host_ids) {
try {
const { rows: existingRows } = await pool.query(
`SELECT plan_count, plans_json FROM atlas_action_plans_cache WHERE host_id = $1`,
[hid]
);
const existing = existingRows[0];
let existingPlans = [];
if (existing && existing.plans_json) {
try { existingPlans = JSON.parse(existing.plans_json); } catch (_) {}
}
const stubPlan = { plan_type, commit_date, source: 'bulk-create', created_at: new Date().toISOString() };
const updatedPlans = [...existingPlans, stubPlan];
const newCount = updatedPlans.length;
await pool.query(
`INSERT INTO atlas_action_plans_cache (host_id, has_action_plan, plan_count, plans_json, synced_at)
VALUES ($1, true, $2, $3, NOW())
ON CONFLICT(host_id) DO UPDATE SET
has_action_plan = true,
plan_count = EXCLUDED.plan_count,
plans_json = EXCLUDED.plans_json,
synced_at = EXCLUDED.synced_at`,
[hid, newCount, JSON.stringify(updatedPlans)]
);
} catch (cacheErr) {
console.error('[Atlas] Cache update failed for host', hid, ':', cacheErr.message);
}
}
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'ATLAS_BULK_CREATE_PLANS',
entityType: 'atlas_action_plan',
entityId: null,
details: { host_ids, plan_type, commit_date, count: host_ids.length },
ipAddress: req.ip
});
res.status(result.status).json(body);
} else {
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (e) { errorBody = { error: result.body }; }
res.status(result.status).json(errorBody);
}
} catch (err) {
console.error('[Atlas] POST bulk-action-plans failed:', err.message);
res.status(502).json({ error: 'Failed to reach Atlas API: ' + err.message });
}
});
// POST /hosts/vulnerabilities
router.post('/hosts/vulnerabilities', requireAuth(), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'Atlas API is not configured. Check ATLAS_API_URL, ATLAS_API_USER, and ATLAS_API_PASS environment variables.' });
}
const { host_ids } = req.body || {};
if (!Array.isArray(host_ids) || host_ids.length === 0) {
return res.status(400).json({ error: 'host_ids must be a non-empty array of positive integers' });
}
for (const id of host_ids) {
if (!Number.isInteger(id) || id <= 0) {
return res.status(400).json({ error: 'host_ids must be a non-empty array of positive integers' });
}
}
try {
const result = await atlasPost('/ivanti-vulnerabilities-by-host', { host_ids }, { timeout: 30000 });
if (result.status >= 200 && result.status < 300) {
let body;
try { body = JSON.parse(result.body); } catch (e) { body = result.body; }
res.status(result.status).json(body);
} else {
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (e) { errorBody = { error: result.body }; }
res.status(result.status).json(errorBody);
}
} catch (err) {
console.error('[Atlas] POST hosts/vulnerabilities failed:', err.message);
res.status(502).json({ error: 'Failed to reach Atlas API: ' + err.message });
}
});
return router;
}
module.exports = createAtlasRouter;
module.exports.aggregateAtlasMetrics = aggregateAtlasMetrics;

View File

@@ -1,11 +1,13 @@
// Audit Log Routes (Admin only) // Audit Log Routes (Admin only)
const express = require('express'); const express = require('express');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
function createAuditLogRouter(db, requireAuth, requireRole) { function createAuditLogRouter() {
const router = express.Router(); const router = express.Router();
// All routes require admin role // All routes require Admin group
router.use(requireAuth(db), requireRole('admin')); router.use(requireAuth(), requireGroup('Admin'));
// Get paginated audit logs with filters // Get paginated audit logs with filters
router.get('/', async (req, res) => { router.get('/', async (req, res) => {
@@ -24,25 +26,26 @@ function createAuditLogRouter(db, requireAuth, requireRole) {
let where = []; let where = [];
let params = []; let params = [];
let paramIndex = 1;
if (user) { if (user) {
where.push('username LIKE ?'); where.push(`username ILIKE $${paramIndex++}`);
params.push(`%${user}%`); params.push(`%${user}%`);
} }
if (action) { if (action) {
where.push('action = ?'); where.push(`action = $${paramIndex++}`);
params.push(action); params.push(action);
} }
if (entityType) { if (entityType) {
where.push('entity_type = ?'); where.push(`entity_type = $${paramIndex++}`);
params.push(entityType); params.push(entityType);
} }
if (startDate) { if (startDate) {
where.push('created_at >= ?'); where.push(`created_at >= $${paramIndex++}`);
params.push(startDate); params.push(startDate);
} }
if (endDate) { if (endDate) {
where.push('created_at <= ?'); where.push(`created_at <= $${paramIndex++}`);
params.push(endDate + ' 23:59:59'); params.push(endDate + ' 23:59:59');
} }
@@ -50,36 +53,25 @@ function createAuditLogRouter(db, requireAuth, requireRole) {
try { try {
// Get total count // Get total count
const countRow = await new Promise((resolve, reject) => { const countResult = await pool.query(
db.get( `SELECT COUNT(*) as total FROM audit_logs ${whereClause}`,
`SELECT COUNT(*) as total FROM audit_logs ${whereClause}`, params
params, );
(err, row) => { const total = parseInt(countResult.rows[0].total);
if (err) reject(err);
else resolve(row);
}
);
});
// Get paginated results // Get paginated results
const rows = await new Promise((resolve, reject) => { const dataResult = await pool.query(
db.all( `SELECT * FROM audit_logs ${whereClause} ORDER BY created_at DESC LIMIT $${paramIndex++} OFFSET $${paramIndex++}`,
`SELECT * FROM audit_logs ${whereClause} ORDER BY created_at DESC LIMIT ? OFFSET ?`, [...params, pageSize, offset]
[...params, pageSize, offset], );
(err, rows) => {
if (err) reject(err);
else resolve(rows);
}
);
});
res.json({ res.json({
logs: rows, logs: dataResult.rows,
pagination: { pagination: {
page: parseInt(page), page: parseInt(page),
limit: pageSize, limit: pageSize,
total: countRow.total, total: total,
totalPages: Math.ceil(countRow.total / pageSize) totalPages: Math.ceil(total / pageSize)
} }
}); });
} catch (err) { } catch (err) {
@@ -91,16 +83,9 @@ function createAuditLogRouter(db, requireAuth, requireRole) {
// Get distinct action types for filter dropdown // Get distinct action types for filter dropdown
router.get('/actions', async (req, res) => { router.get('/actions', async (req, res) => {
try { try {
const rows = await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.all( 'SELECT DISTINCT action FROM audit_logs ORDER BY action'
'SELECT DISTINCT action FROM audit_logs ORDER BY action', );
(err, rows) => {
if (err) reject(err);
else resolve(rows);
}
);
});
res.json(rows.map(r => r.action)); res.json(rows.map(r => r.action));
} catch (err) { } catch (err) {
console.error('Audit log actions error:', err); console.error('Audit log actions error:', err);

View File

@@ -2,12 +2,36 @@
const express = require('express'); const express = require('express');
const bcrypt = require('bcryptjs'); const bcrypt = require('bcryptjs');
const crypto = require('crypto'); const crypto = require('crypto');
const rateLimit = require('express-rate-limit');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
function createAuthRouter(db, logAudit) { const loginLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 20, // 20 attempts per window
standardHeaders: true,
legacyHeaders: false,
message: { error: 'Too many login attempts. Please try again in 15 minutes.' }
});
function createAuthRouter(logAudit) {
const router = express.Router(); const router = express.Router();
// Login /**
router.post('/login', async (req, res) => { * POST /api/auth/login
*
* Authenticates a user with username and password, creates a session,
* and sets an httpOnly session cookie. Rate-limited to 20 attempts per 15 minutes.
*
* @body {string} username - The user's login username
* @body {string} password - The user's password
* @returns {object} 200 - { message: 'Login successful', user: { id, username, email, group } }
* @returns {object} 400 - { error: 'Username and password are required' }
* @returns {object} 401 - { error: 'Invalid username or password' } | { error: 'Account is disabled' }
* @returns {object} 429 - { error: 'Too many login attempts. Please try again in 15 minutes.' }
* @returns {object} 500 - { error: 'Login failed' }
*/
router.post('/login', loginLimiter, async (req, res) => {
const { username, password } = req.body; const { username, password } = req.body;
if (!username || !password) { if (!username || !password) {
@@ -16,19 +40,14 @@ function createAuthRouter(db, logAudit) {
try { try {
// Find user // Find user
const user = await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.get( 'SELECT * FROM users WHERE username = $1',
'SELECT * FROM users WHERE username = ?', [username]
[username], );
(err, row) => { const user = rows[0];
if (err) reject(err);
else resolve(row);
}
);
});
if (!user) { if (!user) {
logAudit(db, { logAudit({
userId: null, userId: null,
username: username, username: username,
action: 'login_failed', action: 'login_failed',
@@ -41,7 +60,7 @@ function createAuthRouter(db, logAudit) {
} }
if (!user.is_active) { if (!user.is_active) {
logAudit(db, { logAudit({
userId: user.id, userId: user.id,
username: username, username: username,
action: 'login_failed', action: 'login_failed',
@@ -56,7 +75,7 @@ function createAuthRouter(db, logAudit) {
// Verify password // Verify password
const validPassword = await bcrypt.compare(password, user.password_hash); const validPassword = await bcrypt.compare(password, user.password_hash);
if (!validPassword) { if (!validPassword) {
logAudit(db, { logAudit({
userId: user.id, userId: user.id,
username: username, username: username,
action: 'login_failed', action: 'login_failed',
@@ -73,28 +92,16 @@ function createAuthRouter(db, logAudit) {
const expiresAt = new Date(Date.now() + 24 * 60 * 60 * 1000); // 24 hours const expiresAt = new Date(Date.now() + 24 * 60 * 60 * 1000); // 24 hours
// Create session // Create session
await new Promise((resolve, reject) => { await pool.query(
db.run( 'INSERT INTO sessions (session_id, user_id, expires_at) VALUES ($1, $2, $3)',
'INSERT INTO sessions (session_id, user_id, expires_at) VALUES (?, ?, ?)', [sessionId, user.id, expiresAt.toISOString()]
[sessionId, user.id, expiresAt.toISOString()], );
(err) => {
if (err) reject(err);
else resolve();
}
);
});
// Update last login // Update last login
await new Promise((resolve, reject) => { await pool.query(
db.run( 'UPDATE users SET last_login = NOW() WHERE id = $1',
'UPDATE users SET last_login = CURRENT_TIMESTAMP WHERE id = ?', [user.id]
[user.id], );
(err) => {
if (err) reject(err);
else resolve();
}
);
});
// Set cookie // Set cookie
res.cookie('session_id', sessionId, { res.cookie('session_id', sessionId, {
@@ -104,13 +111,13 @@ function createAuthRouter(db, logAudit) {
maxAge: 24 * 60 * 60 * 1000 // 24 hours maxAge: 24 * 60 * 60 * 1000 // 24 hours
}); });
logAudit(db, { logAudit({
userId: user.id, userId: user.id,
username: user.username, username: user.username,
action: 'login', action: 'login',
entityType: 'auth', entityType: 'auth',
entityId: null, entityId: null,
details: { role: user.role }, details: { group: user.user_group },
ipAddress: req.ip ipAddress: req.ip
}); });
@@ -120,7 +127,8 @@ function createAuthRouter(db, logAudit) {
id: user.id, id: user.id,
username: user.username, username: user.username,
email: user.email, email: user.email,
role: user.role group: user.user_group,
teams: user.bu_teams ? user.bu_teams.split(',').filter(Boolean) : []
} }
}); });
} catch (err) { } catch (err) {
@@ -129,33 +137,44 @@ function createAuthRouter(db, logAudit) {
} }
}); });
// Logout /**
* POST /api/auth/logout
*
* Ends the current user session by deleting it from the database
* and clearing the session cookie.
*
* @returns {object} 200 - { message: 'Logged out successfully' }
*/
router.post('/logout', async (req, res) => { router.post('/logout', async (req, res) => {
const sessionId = req.cookies?.session_id; const sessionId = req.cookies?.session_id;
if (sessionId) { if (sessionId) {
// Look up user before deleting session // Look up user before deleting session
const session = await new Promise((resolve) => { let session = null;
db.get( try {
const { rows } = await pool.query(
`SELECT u.id as user_id, u.username FROM sessions s `SELECT u.id as user_id, u.username FROM sessions s
JOIN users u ON s.user_id = u.id JOIN users u ON s.user_id = u.id
WHERE s.session_id = ?`, WHERE s.session_id = $1`,
[sessionId], [sessionId]
(err, row) => resolve(row || null)
); );
}); session = rows[0] || null;
} catch (err) {
// Non-critical — proceed with logout
}
// Delete session from database // Delete session from database
await new Promise((resolve) => { try {
db.run( await pool.query(
'DELETE FROM sessions WHERE session_id = ?', 'DELETE FROM sessions WHERE session_id = $1',
[sessionId], [sessionId]
() => resolve()
); );
}); } catch (err) {
// Non-critical — proceed with logout
}
if (session) { if (session) {
logAudit(db, { logAudit({
userId: session.user_id, userId: session.user_id,
username: session.username, username: session.username,
action: 'logout', action: 'logout',
@@ -172,7 +191,16 @@ function createAuthRouter(db, logAudit) {
res.json({ message: 'Logged out successfully' }); res.json({ message: 'Logged out successfully' });
}); });
// Get current user /**
* GET /api/auth/me
*
* Returns the currently authenticated user based on the session cookie.
* Clears the cookie and returns 401 if the session is expired or the account is disabled.
*
* @returns {object} 200 - { user: { id, username, email, group } }
* @returns {object} 401 - { error: 'Not authenticated' } | { error: 'Session expired' } | { error: 'Account is disabled' }
* @returns {object} 500 - { error: 'Failed to get user' }
*/
router.get('/me', async (req, res) => { router.get('/me', async (req, res) => {
const sessionId = req.cookies?.session_id; const sessionId = req.cookies?.session_id;
@@ -181,19 +209,15 @@ function createAuthRouter(db, logAudit) {
} }
try { try {
const session = await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.get( `SELECT s.*, u.id as user_id, u.username, u.email, u.user_group, u.bu_teams, u.is_active
`SELECT s.*, u.id as user_id, u.username, u.email, u.role, u.is_active FROM sessions s
FROM sessions s JOIN users u ON s.user_id = u.id
JOIN users u ON s.user_id = u.id WHERE s.session_id = $1 AND s.expires_at > NOW()`,
WHERE s.session_id = ? AND s.expires_at > datetime('now')`, [sessionId]
[sessionId], );
(err, row) => {
if (err) reject(err); const session = rows[0];
else resolve(row);
}
);
});
if (!session) { if (!session) {
res.clearCookie('session_id'); res.clearCookie('session_id');
@@ -210,7 +234,8 @@ function createAuthRouter(db, logAudit) {
id: session.user_id, id: session.user_id,
username: session.username, username: session.username,
email: session.email, email: session.email,
role: session.role group: session.user_group,
teams: session.bu_teams ? session.bu_teams.split(',').filter(Boolean) : []
} }
}); });
} catch (err) { } catch (err) {
@@ -219,23 +244,136 @@ function createAuthRouter(db, logAudit) {
} }
}); });
// Clean up expired sessions (admin only) /**
router.post('/cleanup-sessions', async (req, res) => { * GET /api/auth/profile
// Basic auth check - require a valid session to call this *
const sessionId = req.cookies?.session_id; * Returns the full profile for the currently authenticated user.
if (!sessionId) { * Queries the database for up-to-date account details including
return res.status(401).json({ error: 'Authentication required' }); * creation date and last login timestamp.
} *
* @returns {object} 200 - { id, username, email, group, created_at, last_login }
* @returns {object} 401 - { error: 'Account is disabled' } (clears session cookie)
* @returns {object} 500 - { error: 'Failed to fetch profile' }
*/
router.get('/profile', requireAuth(), async (req, res) => {
try { try {
await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.run( 'SELECT id, username, email, user_group, created_at, last_login, is_active FROM users WHERE id = $1',
"DELETE FROM sessions WHERE expires_at < datetime('now')", [req.user.id]
(err) => { );
if (err) reject(err);
else resolve(); const user = rows[0];
}
); if (!user || !user.is_active) {
res.clearCookie('session_id');
return res.status(401).json({ error: 'Account is disabled' });
}
res.json({
id: user.id,
username: user.username,
email: user.email,
group: user.user_group,
created_at: user.created_at,
last_login: user.last_login
}); });
} catch (err) {
console.error('Profile fetch error:', err);
res.status(500).json({ error: 'Failed to fetch profile' });
}
});
// Rate limiter for password change — 5 attempts per 15-minute window, keyed by session cookie
const passwordChangeLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 5,
standardHeaders: true,
legacyHeaders: false,
keyGenerator: (req) => req.cookies?.session_id || req.ip,
message: { error: 'Too many password change attempts. Please try again later.' }
});
/**
* POST /api/auth/change-password
*
* Allows the authenticated user to change their own password.
* Rate-limited to 5 attempts per 15-minute window per session.
*
* @body {string} currentPassword - The user's current password
* @body {string} newPassword - The desired new password (min 8 characters)
* @returns {object} 200 - { message: 'Password changed successfully' }
* @returns {object} 400 - { error: 'Current password and new password are required' } | { error: 'New password must be at least 8 characters' }
* @returns {object} 401 - { error: 'Account is disabled' } | { error: 'Current password is incorrect' }
* @returns {object} 429 - { error: 'Too many password change attempts. Please try again later.' }
* @returns {object} 500 - { error: 'Failed to change password' }
*/
router.post('/change-password', requireAuth(), passwordChangeLimiter, async (req, res) => {
const { currentPassword, newPassword } = req.body;
if (!currentPassword || !newPassword) {
return res.status(400).json({ error: 'Current password and new password are required' });
}
if (newPassword.length < 8) {
return res.status(400).json({ error: 'New password must be at least 8 characters' });
}
try {
// Fetch user's password hash and active status
const { rows } = await pool.query(
'SELECT password_hash, is_active FROM users WHERE id = $1',
[req.user.id]
);
const user = rows[0];
if (!user || !user.is_active) {
return res.status(401).json({ error: 'Account is disabled' });
}
// Verify current password
const validPassword = await bcrypt.compare(currentPassword, user.password_hash);
if (!validPassword) {
return res.status(401).json({ error: 'Current password is incorrect' });
}
// Hash new password and update
const newHash = await bcrypt.hash(newPassword, 10);
await pool.query(
'UPDATE users SET password_hash = $1 WHERE id = $2',
[newHash, req.user.id]
);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'password_change',
entityType: 'auth',
entityId: null,
details: null,
ipAddress: req.ip
});
res.json({ message: 'Password changed successfully' });
} catch (err) {
console.error('Password change error:', err);
res.status(500).json({ error: 'Failed to change password' });
}
});
/**
* POST /api/auth/cleanup-sessions
*
* Deletes all expired sessions from the database. Requires Admin group.
*
* @returns {object} 200 - { message: 'Expired sessions cleaned up' }
* @returns {object} 401 - { error: 'Authentication required' }
* @returns {object} 403 - { error: 'Insufficient permissions', required: ['Admin'], current: '...' }
* @returns {object} 500 - { error: 'Cleanup failed' }
*/
router.post('/cleanup-sessions', requireAuth(), requireGroup('Admin'), async (req, res) => {
try {
await pool.query("DELETE FROM sessions WHERE expires_at < NOW()");
res.json({ message: 'Expired sessions cleaned up' }); res.json({ message: 'Expired sessions cleaned up' });
} catch (err) { } catch (err) {
console.error('Session cleanup error:', err); console.error('Session cleanup error:', err);

393
backend/routes/cardApi.js Normal file
View File

@@ -0,0 +1,393 @@
// CARD Asset Ownership API Routes
// Proxies CARD operations (confirm, decline, redirect, search) and orchestrates
// the two-step update_token flow for mutations.
const express = require('express');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog');
const {
isConfigured,
missingVars,
getTeams,
getTeamAssets,
getOwner,
confirmAsset,
declineAsset,
redirectAsset,
} = require('../helpers/cardApi');
// ---------------------------------------------------------------------------
// Error classification — maps CARD API / token errors to client responses
// ---------------------------------------------------------------------------
function handleCardError(err, res) {
const msg = err.message || String(err);
console.error('[card-api]', msg);
if (msg.includes('Token acquisition failed')) {
if (msg.includes('HTTP 401')) {
return res.status(401).json({ error: 'CARD authorization failed. Check service account credentials.' });
}
if (msg.includes('HTTP 403')) {
return res.status(403).json({ error: 'CARD access denied. The service account may not be onboarded with the CARD team.' });
}
if (msg.includes('HTTP 525')) {
return res.status(502).json({ error: 'CARD LDAP error. The service account may not be provisioned correctly.' });
}
}
if (msg.includes('401')) {
return res.status(401).json({ error: 'CARD token expired or invalid. The request has been retried once automatically.' });
}
if (msg.includes('403')) {
return res.status(403).json({ error: 'Insufficient CARD permissions for this operation.' });
}
return res.status(502).json({ error: 'CARD API request failed.', details: msg });
}
// ---------------------------------------------------------------------------
// Router factory
// ---------------------------------------------------------------------------
function createCardApiRouter() {
const router = express.Router();
// GET /status
router.get('/status', requireAuth(), (req, res) => {
if (!isConfigured) {
return res.status(503).json({ configured: false, error: 'CARD API is not configured.', missingVars });
}
res.json({ configured: true });
});
// GET /teams
router.get('/teams', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'CARD API is not configured.', missingVars });
}
try {
const result = await getTeams();
if (result.ok) {
let body;
try { body = JSON.parse(result.body); } catch (_) { body = result.body; }
const teams = Array.isArray(body) ? body : (body && body.teams) || [];
return res.json(teams);
}
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (_) { errorBody = { error: result.body }; }
return res.status(result.status).json(errorBody);
} catch (err) {
return handleCardError(err, res);
}
});
// GET /teams/:teamName/assets
router.get('/teams/:teamName/assets', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'CARD API is not configured.', missingVars });
}
const { teamName } = req.params;
const { disposition, page, page_size } = req.query;
if (!disposition) {
return res.status(400).json({ error: 'disposition query parameter is required.' });
}
try {
const result = await getTeamAssets(teamName, {
disposition,
page: page ? parseInt(page, 10) : undefined,
pageSize: page_size ? parseInt(page_size, 10) : 50,
});
if (result.ok) {
let body;
try { body = JSON.parse(result.body); } catch (_) { body = result.body; }
let resultCount = 0;
if (body && typeof body === 'object' && typeof body.total === 'number') {
resultCount = body.total;
} else if (body && Array.isArray(body.assets)) {
resultCount = body.assets.length;
}
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'card_search',
entityType: 'card_asset',
entityId: teamName,
details: { disposition, resultCount },
ipAddress: req.ip,
});
return res.json(body);
}
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (_) { errorBody = { error: result.body }; }
return res.status(result.status).json(errorBody);
} catch (err) {
return handleCardError(err, res);
}
});
// GET /owner/:assetId
router.get('/owner/:assetId', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'CARD API is not configured.', missingVars });
}
const { assetId } = req.params;
try {
const result = await getOwner(assetId);
if (result.ok) {
let body;
try { body = JSON.parse(result.body); } catch (_) { body = result.body; }
return res.json(body);
}
let errorBody;
try { errorBody = JSON.parse(result.body); } catch (_) { errorBody = { error: result.body }; }
return res.status(result.status).json(errorBody);
} catch (err) {
return handleCardError(err, res);
}
});
// POST /queue/:queueItemId/confirm
router.post('/queue/:queueItemId/confirm', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'CARD API is not configured.', missingVars });
}
const { queueItemId } = req.params;
const { teamName, assetId, comment } = req.body;
if (!teamName || typeof teamName !== 'string' || !teamName.trim()) {
return res.status(400).json({ error: 'teamName is required.' });
}
if (!assetId || typeof assetId !== 'string' || !assetId.trim()) {
return res.status(400).json({ error: 'assetId is required.' });
}
try {
const { rows } = await pool.query(
'SELECT * FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2 AND workflow_type = $3',
[queueItemId, req.user.id, 'CARD']
);
const item = rows[0];
if (!item) {
return res.status(404).json({ error: 'Queue item not found.' });
}
if (item.status !== 'pending') {
return res.status(400).json({ error: 'Only pending queue items can be executed.' });
}
const ownerResult = await getOwner(assetId);
if (!ownerResult.ok) {
const errMsg = `Failed to fetch owner record: HTTP ${ownerResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'confirm', assetId, error: errMsg, cardStatus: ownerResult.status }, ipAddress: req.ip });
let errorBody;
try { errorBody = JSON.parse(ownerResult.body); } catch (_) { errorBody = { error: ownerResult.body }; }
return res.status(ownerResult.status).json(errorBody);
}
let ownerData;
try { ownerData = JSON.parse(ownerResult.body); } catch (_) { ownerData = {}; }
const updateToken = ownerData.owner && ownerData.owner.update_token;
if (!updateToken) {
const errMsg = 'update_token not found in owner record.';
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'confirm', assetId, error: errMsg, cardStatus: null }, ipAddress: req.ip });
return res.status(502).json({ error: 'CARD API request failed.', details: errMsg });
}
const confirmResult = await confirmAsset(assetId, teamName.trim(), updateToken, comment || '');
if (confirmResult.ok) {
await pool.query(
"UPDATE ivanti_todo_queue SET status = 'complete', updated_at = NOW() WHERE id = $1",
[queueItemId]
);
let cardResponse;
try { cardResponse = JSON.parse(confirmResult.body); } catch (_) { cardResponse = confirmResult.body; }
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_confirm', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { assetId, teamName: teamName.trim(), comment: comment || '', cardStatus: confirmResult.status }, ipAddress: req.ip });
return res.json({ success: true, cardResponse });
}
const errMsg = `Confirm failed: HTTP ${confirmResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'confirm', assetId, error: errMsg, cardStatus: confirmResult.status }, ipAddress: req.ip });
let errorBody;
try { errorBody = JSON.parse(confirmResult.body); } catch (_) { errorBody = { error: confirmResult.body }; }
return res.status(confirmResult.status).json(errorBody);
} catch (err) {
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'confirm', assetId, error: err.message, cardStatus: null }, ipAddress: req.ip });
return handleCardError(err, res);
}
});
// POST /queue/:queueItemId/decline
router.post('/queue/:queueItemId/decline', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'CARD API is not configured.', missingVars });
}
const { queueItemId } = req.params;
const { teamName, assetId, comment } = req.body;
if (!teamName || typeof teamName !== 'string' || !teamName.trim()) {
return res.status(400).json({ error: 'teamName is required.' });
}
if (!assetId || typeof assetId !== 'string' || !assetId.trim()) {
return res.status(400).json({ error: 'assetId is required.' });
}
try {
const { rows } = await pool.query(
'SELECT * FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2 AND workflow_type = $3',
[queueItemId, req.user.id, 'CARD']
);
const item = rows[0];
if (!item) {
return res.status(404).json({ error: 'Queue item not found.' });
}
if (item.status !== 'pending') {
return res.status(400).json({ error: 'Only pending queue items can be executed.' });
}
const ownerResult = await getOwner(assetId);
if (!ownerResult.ok) {
const errMsg = `Failed to fetch owner record: HTTP ${ownerResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'decline', assetId, error: errMsg, cardStatus: ownerResult.status }, ipAddress: req.ip });
let errorBody;
try { errorBody = JSON.parse(ownerResult.body); } catch (_) { errorBody = { error: ownerResult.body }; }
return res.status(ownerResult.status).json(errorBody);
}
let ownerData;
try { ownerData = JSON.parse(ownerResult.body); } catch (_) { ownerData = {}; }
const updateToken = ownerData.owner && ownerData.owner.update_token;
if (!updateToken) {
const errMsg = 'update_token not found in owner record.';
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'decline', assetId, error: errMsg, cardStatus: null }, ipAddress: req.ip });
return res.status(502).json({ error: 'CARD API request failed.', details: errMsg });
}
const declineResult = await declineAsset(assetId, teamName.trim(), updateToken, comment || '');
if (declineResult.ok) {
await pool.query(
"UPDATE ivanti_todo_queue SET status = 'complete', updated_at = NOW() WHERE id = $1",
[queueItemId]
);
let cardResponse;
try { cardResponse = JSON.parse(declineResult.body); } catch (_) { cardResponse = declineResult.body; }
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_decline', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { assetId, teamName: teamName.trim(), comment: comment || '', cardStatus: declineResult.status }, ipAddress: req.ip });
return res.json({ success: true, cardResponse });
}
const errMsg = `Decline failed: HTTP ${declineResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'decline', assetId, error: errMsg, cardStatus: declineResult.status }, ipAddress: req.ip });
let errorBody;
try { errorBody = JSON.parse(declineResult.body); } catch (_) { errorBody = { error: declineResult.body }; }
return res.status(declineResult.status).json(errorBody);
} catch (err) {
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'decline', assetId, error: err.message, cardStatus: null }, ipAddress: req.ip });
return handleCardError(err, res);
}
});
// POST /queue/:queueItemId/redirect
router.post('/queue/:queueItemId/redirect', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!isConfigured) {
return res.status(503).json({ error: 'CARD API is not configured.', missingVars });
}
const { queueItemId } = req.params;
const { fromTeam, toTeam, assetId } = req.body;
if (!fromTeam || typeof fromTeam !== 'string' || !fromTeam.trim()) {
return res.status(400).json({ error: 'fromTeam is required.' });
}
if (!toTeam || typeof toTeam !== 'string' || !toTeam.trim()) {
return res.status(400).json({ error: 'toTeam is required.' });
}
if (!assetId || typeof assetId !== 'string' || !assetId.trim()) {
return res.status(400).json({ error: 'assetId is required.' });
}
try {
const { rows } = await pool.query(
'SELECT * FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2 AND workflow_type = $3',
[queueItemId, req.user.id, 'CARD']
);
const item = rows[0];
if (!item) {
return res.status(404).json({ error: 'Queue item not found.' });
}
if (item.status !== 'pending') {
return res.status(400).json({ error: 'Only pending queue items can be executed.' });
}
const ownerResult = await getOwner(assetId);
if (!ownerResult.ok) {
const errMsg = `Failed to fetch owner record: HTTP ${ownerResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'redirect', assetId, error: errMsg, cardStatus: ownerResult.status }, ipAddress: req.ip });
let errorBody;
try { errorBody = JSON.parse(ownerResult.body); } catch (_) { errorBody = { error: ownerResult.body }; }
return res.status(ownerResult.status).json(errorBody);
}
let ownerData;
try { ownerData = JSON.parse(ownerResult.body); } catch (_) { ownerData = {}; }
const updateToken = ownerData.owner && ownerData.owner.update_token;
if (!updateToken) {
const errMsg = 'update_token not found in owner record.';
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'redirect', assetId, error: errMsg, cardStatus: null }, ipAddress: req.ip });
return res.status(502).json({ error: 'CARD API request failed.', details: errMsg });
}
const redirectResult = await redirectAsset(assetId, fromTeam.trim(), toTeam.trim(), updateToken);
if (redirectResult.ok) {
await pool.query(
"UPDATE ivanti_todo_queue SET status = 'complete', updated_at = NOW() WHERE id = $1",
[queueItemId]
);
let cardResponse;
try { cardResponse = JSON.parse(redirectResult.body); } catch (_) { cardResponse = redirectResult.body; }
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_redirect', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { assetId, fromTeam: fromTeam.trim(), toTeam: toTeam.trim(), cardStatus: redirectResult.status }, ipAddress: req.ip });
return res.json({ success: true, cardResponse });
}
const errMsg = `Redirect failed: HTTP ${redirectResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'redirect', assetId, error: errMsg, cardStatus: redirectResult.status }, ipAddress: req.ip });
let errorBody;
try { errorBody = JSON.parse(redirectResult.body); } catch (_) { errorBody = { error: redirectResult.body }; }
return res.status(redirectResult.status).json(errorBody);
} catch (err) {
logAudit({ userId: req.user.id, username: req.user.username, action: 'card_action_failed', entityType: 'ivanti_todo_queue', entityId: String(queueItemId), details: { actionType: 'redirect', assetId, error: err.message, cardStatus: null }, ipAddress: req.ip });
return handleCardError(err, res);
}
});
return router;
}
module.exports = createCardApiRouter;

View File

@@ -0,0 +1,675 @@
// Compliance Routes — AEO metric tracking
// Handles xlsx upload/parse, non-compliant item history, and notes.
const express = require('express');
const path = require('path');
const fs = require('fs');
const crypto = require('crypto');
const { spawn } = require('child_process');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const { loadConfig, compareSchemaToDrift, reconcileConfig } = require('../helpers/driftChecker');
const logAudit = require('../helpers/auditLog');
const PARSER_SCRIPT = path.join(__dirname, '../scripts/parse_compliance_xlsx.py');
const SCHEMA_SCRIPT = path.join(__dirname, '../scripts/extract_xlsx_schema.py');
const CONFIG_PATH = path.join(__dirname, '..', 'scripts', 'compliance_config.json');
const PYTHON_BIN = process.env.PYTHON_BIN || 'python3';
const TEMP_DIR = path.join(process.cwd(), 'uploads', 'temp');
const ALLOWED_TEAMS = new Set(['STEAM', 'ACCESS-ENG', 'ACCESS-OPS', 'INTELDEV']);
// ---------------------------------------------------------------------------
// Run Python parser, return parsed object
// ---------------------------------------------------------------------------
function parseXlsx(filePath) {
return new Promise((resolve, reject) => {
const py = spawn(PYTHON_BIN, [PARSER_SCRIPT, filePath]);
let out = '';
let err = '';
py.stdout.on('data', d => { out += d; });
py.stderr.on('data', d => { err += d; });
py.on('close', code => {
if (code !== 0) return reject(new Error(err || `Parser exited with code ${code}`));
try { resolve(JSON.parse(out)); }
catch (e) { reject(new Error('Parser returned invalid JSON')); }
});
py.on('error', reject);
});
}
function extractXlsxSchema(filePath) {
return new Promise((resolve, reject) => {
const py = spawn(PYTHON_BIN, [SCHEMA_SCRIPT, filePath]);
let out = '';
let err = '';
py.stdout.on('data', d => { out += d; });
py.stderr.on('data', d => { err += d; });
py.on('close', code => {
if (code !== 0) return reject(new Error(err || `Schema extractor exited with code ${code}`));
try { resolve(JSON.parse(out)); }
catch (e) { reject(new Error('Schema extractor returned invalid JSON')); }
});
py.on('error', reject);
});
}
function isSafeTempPath(filePath) {
const resolved = path.resolve(filePath);
return resolved.startsWith(TEMP_DIR + path.sep) && path.extname(resolved) === '.json';
}
// ---------------------------------------------------------------------------
// Compute diff: new / recurring / resolved
// ---------------------------------------------------------------------------
async function computeDiff(incomingItems) {
const { rows: activeRows } = await pool.query(
`SELECT hostname, metric_id FROM compliance_items WHERE status = 'active'`
);
const activeKeys = new Set(activeRows.map(r => `${r.hostname}|||${r.metric_id}`));
const newKeys = new Set(incomingItems.map(i => `${i.hostname}|||${i.metric_id}`));
let newCount = 0, recurringCount = 0, resolvedCount = 0;
for (const k of newKeys) { if (activeKeys.has(k)) recurringCount++; else newCount++; }
for (const k of activeKeys) { if (!newKeys.has(k)) resolvedCount++; }
return { newCount, recurringCount, resolvedCount };
}
// ---------------------------------------------------------------------------
// Write a parsed upload to the DB (within a transaction)
// ---------------------------------------------------------------------------
async function persistUpload({ items, summary, reportDate, filename, userId }) {
const { rows: activeRows } = await pool.query(
`SELECT id, hostname, metric_id, seen_count, first_seen_upload_id FROM compliance_items WHERE status = 'active'`
);
const activeMap = {};
activeRows.forEach(r => { activeMap[`${r.hostname}|||${r.metric_id}`] = r; });
const newKeys = new Set(items.map(i => `${i.hostname}|||${i.metric_id}`));
const client = await pool.connect();
try {
await client.query('BEGIN');
// 1. Insert the upload record
const uploadResult = await client.query(
`INSERT INTO compliance_uploads (filename, report_date, uploaded_by, uploaded_at, summary_json)
VALUES ($1, $2, $3, NOW(), $4)
RETURNING id`,
[filename, reportDate || null, userId || null, JSON.stringify(summary)]
);
const uploadId = uploadResult.rows[0].id;
let newCount = 0, recurringCount = 0, resolvedCount = 0;
// 2. Upsert each incoming non-compliant item
for (const item of items) {
const key = `${item.hostname}|||${item.metric_id}`;
const existing = activeMap[key];
const extraStr = JSON.stringify(item.extra_json || {});
if (existing) {
await client.query(
`UPDATE compliance_items
SET upload_id = $1, seen_count = $2, ip_address = $3, device_type = $4, extra_json = $5
WHERE id = $6`,
[uploadId, existing.seen_count + 1, item.ip_address, item.device_type, extraStr, existing.id]
);
recurringCount++;
} else {
await client.query(
`INSERT INTO compliance_items
(upload_id, hostname, ip_address, device_type, team, metric_id, metric_desc,
category, extra_json, status, first_seen_upload_id, seen_count)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, 'active', $10, 1)`,
[uploadId, item.hostname, item.ip_address, item.device_type, item.team,
item.metric_id, item.metric_desc, item.category, extraStr, uploadId]
);
newCount++;
}
}
// 3. Mark items not present in this upload as resolved
for (const [key, row] of Object.entries(activeMap)) {
if (!newKeys.has(key)) {
await client.query(
`UPDATE compliance_items SET status = 'resolved', resolved_upload_id = $1 WHERE id = $2`,
[uploadId, row.id]
);
resolvedCount++;
}
}
// 4. Update upload with final counts
await client.query(
`UPDATE compliance_uploads SET new_count = $1, resolved_count = $2, recurring_count = $3 WHERE id = $4`,
[newCount, resolvedCount, recurringCount, uploadId]
);
await client.query('COMMIT');
return { uploadId, newCount, recurringCount, resolvedCount };
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
}
// ---------------------------------------------------------------------------
// Group flat compliance_items rows into per-device objects
// ---------------------------------------------------------------------------
function groupByHostname(rows, noteHostnames) {
const deviceMap = {};
for (const row of rows) {
if (!deviceMap[row.hostname]) {
deviceMap[row.hostname] = {
hostname: row.hostname, ip_address: row.ip_address || '', device_type: row.device_type || '',
team: row.team || '', status: row.status, failing_metrics: [],
seen_count: row.seen_count || 1, first_seen: row.first_seen || null,
last_seen: row.last_seen || null, resolved_on: row.resolved_on || null,
has_notes: noteHostnames.has(row.hostname),
};
}
const dev = deviceMap[row.hostname];
dev.failing_metrics.push({ metric_id: row.metric_id, metric_desc: row.metric_desc || '', category: row.category || '' });
if ((row.seen_count || 1) > dev.seen_count) dev.seen_count = row.seen_count;
if (row.first_seen && (!dev.first_seen || row.first_seen < dev.first_seen)) dev.first_seen = row.first_seen;
if (row.last_seen && (!dev.last_seen || row.last_seen > dev.last_seen)) dev.last_seen = row.last_seen;
}
return Object.values(deviceMap);
}
// ---------------------------------------------------------------------------
// Pure helpers
// ---------------------------------------------------------------------------
const BUCKET_ORDER = ['1 cycle', '23 cycles', '46 cycles', '7+ cycles'];
function bucketAgingItems(items) {
const teams = ['STEAM', 'ACCESS-ENG', 'ACCESS-OPS', 'INTELDEV'];
const buckets = {};
for (const b of BUCKET_ORDER) {
buckets[b] = { bucket: b, total: 0 };
for (const t of teams) buckets[b][t] = 0;
}
for (const item of items) {
const sc = item.seen_count;
let label;
if (sc === 1) label = '1 cycle';
else if (sc >= 2 && sc <= 3) label = '23 cycles';
else if (sc >= 4 && sc <= 6) label = '46 cycles';
else label = '7+ cycles';
buckets[label].total += 1;
if (item.team in buckets[label]) buckets[label][item.team] += 1;
}
return BUCKET_ORDER.map(b => buckets[b]);
}
function computeWaterfall(uploads) {
let start = 0;
return uploads.map((row) => {
const end = start + row.new_count + row.recurring_count - row.resolved_count;
const entry = { date: row.report_date, start, new_count: row.new_count, recurring_count: row.recurring_count, resolved_count: row.resolved_count, end };
start = end;
return entry;
});
}
// ---------------------------------------------------------------------------
// Router factory
// ---------------------------------------------------------------------------
function createComplianceRouter(upload) {
const router = express.Router();
// All compliance routes require authentication
router.use(requireAuth());
// POST /preview
router.post('/preview', requireGroup('Admin', 'Standard_User'), (req, res) => {
upload.single('file')(req, res, async (uploadErr) => {
if (uploadErr) return res.status(400).json({ error: uploadErr.message });
if (!req.file) return res.status(400).json({ error: 'No file uploaded' });
if (path.extname(req.file.originalname).toLowerCase() !== '.xlsx') {
fs.unlink(req.file.path, () => {});
return res.status(400).json({ error: 'File must be an .xlsx spreadsheet' });
}
try {
let drift = null, drift_error = null;
let config;
try { config = loadConfig(CONFIG_PATH); } catch (configErr) {
fs.unlink(req.file.path, () => {});
return res.status(500).json({ error: 'Configuration file could not be loaded: ' + configErr.message });
}
let xlsxSchema = null;
try {
xlsxSchema = await extractXlsxSchema(req.file.path);
if (xlsxSchema.error) throw new Error(xlsxSchema.error);
drift = compareSchemaToDrift(xlsxSchema, config);
} catch (driftErr) {
drift = null;
drift_error = driftErr.message || 'Drift check failed';
}
const parsed = await parseXlsx(req.file.path);
if (parsed.error) {
fs.unlink(req.file.path, () => {});
return res.status(422).json({ error: parsed.error });
}
const diff = await computeDiff(parsed.items);
if (!fs.existsSync(TEMP_DIR)) fs.mkdirSync(TEMP_DIR, { recursive: true });
const tempFilename = `compliance_preview_${Date.now()}_${Math.random().toString(36).slice(2)}.json`;
const tempFilePath = path.join(TEMP_DIR, tempFilename);
fs.writeFileSync(tempFilePath, JSON.stringify({
items: parsed.items, summary: parsed.summary,
report_date: parsed.report_date,
filename: req.file.originalname.replace(/[^\w.\-() ]/g, '_'),
}));
fs.unlink(req.file.path, () => {});
res.json({
drift, drift_error, schema: xlsxSchema,
diff: { new_count: diff.newCount, recurring_count: diff.recurringCount, resolved_count: diff.resolvedCount },
tempFile: tempFilePath, filename: req.file.originalname,
report_date: parsed.report_date, total_items: parsed.total,
});
} catch (err) {
fs.unlink(req.file.path, () => {});
console.error('[Compliance] Preview error:', err.message);
res.status(500).json({ error: 'Failed to parse file: ' + err.message });
}
});
});
// POST /reconcile-config
router.post('/reconcile-config', requireGroup('Admin'), async (req, res) => {
const { drift, schema } = req.body;
if (!drift || typeof drift !== 'object') return res.status(400).json({ error: 'drift report is required in request body' });
const hasFindings = (drift.breaking && drift.breaking.length > 0) || (drift.silent_miss && drift.silent_miss.length > 0);
if (!hasFindings) return res.status(400).json({ error: 'No breaking or silent-miss findings to reconcile' });
try {
const { changes } = reconcileConfig(CONFIG_PATH, drift, schema || null);
if (changes.length === 0) return res.json({ changes: [], message: 'No changes needed' });
for (const change of changes) {
logAudit({ userId: req.user.id, username: req.user.username, action: 'compliance_config_reconcile', entityType: 'compliance_config', entityId: change.value, details: { action: change.action, key: change.key, detail: change.detail }, ipAddress: req.ip });
}
res.json({ changes, message: `Reconciled ${changes.length} config change(s)` });
} catch (err) {
console.error('[Compliance] Reconcile config error:', err.message);
res.status(500).json({ error: 'Failed to reconcile config: ' + err.message });
}
});
// POST /commit
router.post('/commit', requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { tempFile, filename, report_date } = req.body;
if (!tempFile || typeof tempFile !== 'string') return res.status(400).json({ error: 'tempFile is required' });
if (!isSafeTempPath(tempFile)) return res.status(400).json({ error: 'Invalid tempFile path' });
if (!fs.existsSync(tempFile)) return res.status(400).json({ error: 'Preview session expired — please upload again' });
let parsed;
try { parsed = JSON.parse(fs.readFileSync(tempFile, 'utf8')); }
catch { return res.status(400).json({ error: 'Could not read preview data — please upload again' }); }
try {
const result = await persistUpload({
items: parsed.items, summary: parsed.summary,
reportDate: report_date || parsed.report_date,
filename: filename || parsed.filename,
userId: req.user?.id || null,
});
fs.unlink(tempFile, () => {});
const { rows } = await pool.query(
`SELECT id, filename, report_date, uploaded_at, new_count, resolved_count, recurring_count
FROM compliance_uploads WHERE id = $1`, [result.uploadId]
);
res.json({ upload: rows[0] });
} catch (err) {
console.error('[Compliance] Commit error:', err.message);
res.status(500).json({ error: 'Failed to commit upload: ' + err.message });
}
});
// GET /uploads
router.get('/uploads', async (req, res) => {
try {
const { rows } = await pool.query(
`SELECT id, filename, report_date, uploaded_at, new_count, resolved_count, recurring_count
FROM compliance_uploads ORDER BY id DESC`
);
res.json({ uploads: rows });
} catch (err) {
console.error('[Compliance] GET /uploads error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// POST /rollback/:uploadId
router.post('/rollback/:uploadId', requireGroup('Admin'), async (req, res) => {
const uploadId = parseInt(req.params.uploadId, 10);
if (isNaN(uploadId)) return res.status(400).json({ error: 'Invalid upload ID' });
try {
const { rows: uploadRows } = await pool.query(
`SELECT id, filename, report_date, new_count, resolved_count, recurring_count FROM compliance_uploads WHERE id = $1`, [uploadId]
);
const upload = uploadRows[0];
if (!upload) return res.status(404).json({ error: 'Upload not found' });
const { rows: latestRows } = await pool.query(`SELECT id FROM compliance_uploads ORDER BY id DESC LIMIT 1`);
if (latestRows[0].id !== uploadId) {
return res.status(400).json({ error: 'Only the most recent upload can be rolled back', latest_upload_id: latestRows[0].id });
}
const { rows: prevRows } = await pool.query(`SELECT id FROM compliance_uploads WHERE id < $1 ORDER BY id DESC LIMIT 1`, [uploadId]);
const previousUpload = prevRows[0];
const client = await pool.connect();
try {
await client.query('BEGIN');
const deleteNew = await client.query(
`DELETE FROM compliance_items WHERE first_seen_upload_id = $1 AND upload_id = $1`, [uploadId]
);
const reactivate = await client.query(
`UPDATE compliance_items SET status = 'active', resolved_upload_id = NULL WHERE resolved_upload_id = $1`, [uploadId]
);
if (previousUpload) {
await client.query(
`UPDATE compliance_items SET upload_id = $1, seen_count = GREATEST(seen_count - 1, 1) WHERE upload_id = $2 AND first_seen_upload_id != $2`,
[previousUpload.id, uploadId]
);
}
await client.query(`DELETE FROM compliance_uploads WHERE id = $1`, [uploadId]);
await client.query('COMMIT');
logAudit({ userId: req.user.id, username: req.user.username, action: 'compliance_upload_rollback', entityType: 'compliance_upload', entityId: String(uploadId), details: { filename: upload.filename, report_date: upload.report_date, items_deleted: deleteNew.rowCount, items_reactivated: reactivate.rowCount }, ipAddress: req.ip });
res.json({ message: `Rolled back upload "${upload.filename}"`, rolled_back: { upload_id: uploadId, filename: upload.filename, report_date: upload.report_date, items_deleted: deleteNew.rowCount, items_reactivated: reactivate.rowCount } });
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
} catch (err) {
console.error('[Compliance] Rollback error:', err.message);
res.status(500).json({ error: 'Failed to rollback upload: ' + err.message });
}
});
// GET /summary
router.get('/summary', async (req, res) => {
const team = req.query.team;
if (team && !ALLOWED_TEAMS.has(team)) return res.status(400).json({ error: 'Invalid team' });
try {
const { rows: latestRows } = await pool.query(
`SELECT id, summary_json, report_date, uploaded_at FROM compliance_uploads ORDER BY id DESC LIMIT 1`
);
const latestUpload = latestRows[0];
if (!latestUpload || !latestUpload.summary_json) return res.json({ entries: [], overall_scores: {}, upload: null });
let summary;
try { summary = JSON.parse(latestUpload.summary_json); } catch { return res.json({ entries: [], overall_scores: {}, upload: null }); }
let entries = summary.entries || [];
if (team) entries = entries.filter(e => e.team === team);
res.json({ entries, overall_scores: summary.overall_scores || {}, upload: { id: latestUpload.id, report_date: latestUpload.report_date, uploaded_at: latestUpload.uploaded_at } });
} catch (err) {
console.error('[Compliance] GET /summary error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// GET /items
router.get('/items', async (req, res) => {
const { team, status = 'active' } = req.query;
if (!team) return res.status(400).json({ error: 'team is required' });
if (!ALLOWED_TEAMS.has(team)) return res.status(400).json({ error: 'Invalid team' });
if (!['active', 'resolved'].includes(status)) return res.status(400).json({ error: 'Invalid status' });
try {
const { rows } = await pool.query(
`SELECT ci.hostname, ci.ip_address, ci.device_type, ci.team, ci.metric_id, ci.metric_desc, ci.category, ci.status, ci.seen_count,
fu.report_date AS first_seen, lu.report_date AS last_seen, ru.report_date AS resolved_on
FROM compliance_items ci
LEFT JOIN compliance_uploads fu ON ci.first_seen_upload_id = fu.id
LEFT JOIN compliance_uploads lu ON ci.upload_id = lu.id
LEFT JOIN compliance_uploads ru ON ci.resolved_upload_id = ru.id
WHERE ci.team = $1 AND ci.status = $2
ORDER BY ci.hostname, ci.metric_id`,
[team, status]
);
const { rows: noteRows } = await pool.query(`SELECT DISTINCT hostname FROM compliance_notes`);
const noteHostnames = new Set(noteRows.map(r => r.hostname));
const devices = groupByHostname(rows, noteHostnames);
res.json({ devices, team, status });
} catch (err) {
console.error('[Compliance] GET /items error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// GET /items/:hostname
router.get('/items/:hostname', async (req, res) => {
const hostname = req.params.hostname;
if (!hostname || hostname.length > 300) return res.status(400).json({ error: 'Invalid hostname' });
try {
const { rows: metricRows } = await pool.query(
`SELECT ci.metric_id, ci.metric_desc, ci.category, ci.status, ci.ip_address, ci.device_type, ci.team, ci.seen_count, ci.extra_json,
fu.report_date AS first_seen, fu.uploaded_at AS first_seen_at, lu.report_date AS last_seen, lu.uploaded_at AS last_seen_at, ru.report_date AS resolved_on
FROM compliance_items ci
LEFT JOIN compliance_uploads fu ON ci.first_seen_upload_id = fu.id
LEFT JOIN compliance_uploads lu ON ci.upload_id = lu.id
LEFT JOIN compliance_uploads ru ON ci.resolved_upload_id = ru.id
WHERE ci.hostname = $1
ORDER BY ci.status DESC, ci.metric_id`, [hostname]
);
if (metricRows.length === 0) return res.status(404).json({ error: 'Device not found' });
const metrics = metricRows.map(r => ({ ...r, extra: (() => { try { return JSON.parse(r.extra_json || '{}'); } catch { return {}; } })(), extra_json: undefined }));
const { rows: notes } = await pool.query(
`SELECT cn.id, cn.metric_id, cn.note, cn.group_id, cn.created_at, u.username AS created_by
FROM compliance_notes cn LEFT JOIN users u ON cn.created_by = u.id
WHERE cn.hostname = $1 ORDER BY cn.created_at DESC`, [hostname]
);
const identity = metricRows.find(r => r.status === 'active') || metricRows[0];
res.json({ hostname, ip_address: identity.ip_address || '', device_type: identity.device_type || '', team: identity.team || '', metrics, notes });
} catch (err) {
console.error('[Compliance] GET /items/:hostname error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// POST /notes
router.post('/notes', requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { hostname, metric_id, metric_ids, note } = req.body;
if (!hostname || typeof hostname !== 'string' || hostname.length > 300 || !/^[a-zA-Z0-9._-]+$/.test(hostname)) return res.status(400).json({ error: 'Invalid hostname format' });
let resolvedIds;
if (metric_ids !== undefined) {
if (!Array.isArray(metric_ids)) return res.status(400).json({ error: 'metric_ids must be an array' });
resolvedIds = metric_ids;
} else if (metric_id !== undefined && metric_id !== null && metric_id !== '') {
if (typeof metric_id !== 'string' || metric_id.length > 50) return res.status(400).json({ error: 'Invalid metric_id' });
resolvedIds = [metric_id];
} else {
return res.status(400).json({ error: 'metric_id or metric_ids is required' });
}
if (resolvedIds.length === 0) return res.status(400).json({ error: 'At least one metric ID is required' });
for (let i = 0; i < resolvedIds.length; i++) {
const mid = resolvedIds[i];
if (!mid || typeof mid !== 'string' || mid.length === 0 || mid.length > 50) return res.status(400).json({ error: `Invalid metric_id at index ${i}` });
}
const noteText = String(note || '').trim().slice(0, 1000);
if (!noteText) return res.status(400).json({ error: 'Note cannot be empty' });
const groupId = crypto.randomUUID();
const userId = req.user?.id || null;
const client = await pool.connect();
try {
await client.query('BEGIN');
const insertedIds = [];
for (const mid of resolvedIds) {
const { rows } = await client.query(
`INSERT INTO compliance_notes (hostname, metric_id, note, group_id, created_by, created_at) VALUES ($1, $2, $3, $4, $5, NOW()) RETURNING id`,
[hostname, mid, noteText, groupId, userId]
);
insertedIds.push(rows[0].id);
}
await client.query('COMMIT');
const { rows: notes } = await pool.query(
`SELECT cn.id, cn.hostname, cn.metric_id, cn.note, cn.group_id, cn.created_at, u.username AS created_by
FROM compliance_notes cn LEFT JOIN users u ON cn.created_by = u.id
WHERE cn.id = ANY($1) ORDER BY cn.id ASC`, [insertedIds]
);
res.status(201).json({ notes });
} catch (err) {
await client.query('ROLLBACK');
console.error('[Compliance] POST /notes error:', err.message);
res.status(500).json({ error: 'Failed to save note' });
} finally {
client.release();
}
});
// GET /notes/:hostname/:metricId
router.get('/notes/:hostname/:metricId', async (req, res) => {
const { hostname, metricId } = req.params;
if (!hostname || hostname.length > 300) return res.status(400).json({ error: 'Invalid hostname' });
if (!metricId || metricId.length > 50) return res.status(400).json({ error: 'Invalid metricId' });
try {
const { rows: notes } = await pool.query(
`SELECT cn.id, cn.note, cn.created_at, u.username AS created_by
FROM compliance_notes cn LEFT JOIN users u ON cn.created_by = u.id
WHERE cn.hostname = $1 AND cn.metric_id = $2 ORDER BY cn.created_at DESC`, [hostname, metricId]
);
res.json({ notes });
} catch (err) {
console.error('[Compliance] GET /notes error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// DELETE /notes/:id
router.delete('/notes/:id', requireGroup('Admin', 'Standard_User'), async (req, res) => {
const noteId = parseInt(req.params.id, 10);
if (isNaN(noteId)) return res.status(400).json({ error: 'Invalid note ID' });
const deleteGroup = req.query.group === 'true';
try {
const { rows } = await pool.query(`SELECT id, hostname, metric_id, note, group_id, created_by FROM compliance_notes WHERE id = $1`, [noteId]);
const noteRow = rows[0];
if (!noteRow) return res.status(404).json({ error: 'Note not found' });
const isAuthor = req.user && String(req.user.id) === String(noteRow.created_by);
const isAdminUser = req.user && req.user.group === 'Admin';
if (!isAuthor && !isAdminUser) return res.status(403).json({ error: 'You can only delete your own notes' });
let deleted = 0;
if (deleteGroup && noteRow.group_id) {
const result = await pool.query(`DELETE FROM compliance_notes WHERE group_id = $1`, [noteRow.group_id]);
deleted = result.rowCount;
} else {
const result = await pool.query(`DELETE FROM compliance_notes WHERE id = $1`, [noteId]);
deleted = result.rowCount;
}
logAudit({ userId: req.user.id, username: req.user.username, action: 'compliance_note_delete', entityType: 'compliance_note', entityId: String(noteId), details: JSON.stringify({ hostname: noteRow.hostname, group_id: noteRow.group_id, deleted_count: deleted }), ipAddress: req.ip });
res.json({ deleted });
} catch (err) {
console.error('[Compliance] DELETE /notes error:', err.message);
res.status(500).json({ error: 'Failed to delete note' });
}
});
// GET /trends
router.get('/trends', async (req, res) => {
try {
const { rows: uploads } = await pool.query(
`SELECT id, report_date, COALESCE(new_count, 0) AS new_count, COALESCE(recurring_count, 0) AS recurring_count, COALESCE(resolved_count, 0) AS resolved_count, COALESCE(new_count, 0) + COALESCE(recurring_count, 0) AS total_active FROM compliance_uploads ORDER BY report_date ASC`
);
if (uploads.length === 0) return res.json({ trends: [] });
const { rows: teamRows } = await pool.query(
`SELECT ci.upload_id, ci.team, COUNT(ci.id)::int AS count FROM compliance_items ci WHERE ci.team IS NOT NULL GROUP BY ci.upload_id, ci.team`
);
const teamMap = {};
teamRows.forEach(r => { if (!teamMap[r.upload_id]) teamMap[r.upload_id] = {}; teamMap[r.upload_id][r.team] = r.count; });
const trends = uploads.map(u => ({
report_date: u.report_date, new_count: u.new_count, recurring_count: u.recurring_count, resolved_count: u.resolved_count, total_active: u.total_active,
STEAM: teamMap[u.id]?.STEAM || 0, 'ACCESS-ENG': teamMap[u.id]?.['ACCESS-ENG'] || 0, 'ACCESS-OPS': teamMap[u.id]?.['ACCESS-OPS'] || 0, INTELDEV: teamMap[u.id]?.INTELDEV || 0,
}));
res.json({ trends });
} catch (err) {
console.error('[Compliance] GET /trends error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// GET /mttr
router.get('/mttr', async (req, res) => {
try {
const { rows } = await pool.query(`SELECT COALESCE(seen_count, 1) AS seen_count, team FROM compliance_items WHERE status = 'active'`);
if (rows.length === 0) return res.json({ aging: [] });
const aging = bucketAgingItems(rows);
res.json({ aging });
} catch (err) {
console.error('[Compliance] GET /mttr error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// GET /top-recurring
router.get('/top-recurring', async (req, res) => {
try {
const { rows } = await pool.query(
`SELECT id, report_date, COALESCE(new_count, 0) AS new_count, COALESCE(recurring_count, 0) AS recurring_count, COALESCE(resolved_count, 0) AS resolved_count FROM compliance_uploads ORDER BY report_date ASC`
);
const waterfall = computeWaterfall(rows);
res.json({ waterfall });
} catch (err) {
console.error('[Compliance] GET /top-recurring error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
// GET /category-trend
router.get('/category-trend', async (req, res) => {
try {
const { rows } = await pool.query(
`SELECT cu.report_date, COALESCE(ci.category, 'Unknown') AS category, COUNT(ci.id)::int AS count
FROM compliance_uploads cu JOIN compliance_items ci ON ci.upload_id = cu.id
GROUP BY cu.id, cu.report_date, category ORDER BY cu.report_date ASC`
);
res.json({ categoryTrend: rows });
} catch (err) {
console.error('[Compliance] GET /category-trend error:', err.message);
res.status(500).json({ error: 'Database error' });
}
});
return router;
}
module.exports = { createComplianceRouter, bucketAgingItems, computeWaterfall };

111
backend/routes/feedback.js Normal file
View File

@@ -0,0 +1,111 @@
// Feedback route — proxies bug reports and feature requests to GitLab Issues API
// Keeps the GitLab PAT server-side so it's never exposed to the browser.
const express = require('express');
const https = require('https');
const http = require('http');
const { requireAuth } = require('../middleware/auth');
function createFeedbackRouter() {
const router = express.Router();
const GITLAB_URL = process.env.GITLAB_URL || '';
const GITLAB_PROJECT_ID = process.env.GITLAB_PROJECT_ID || '';
const GITLAB_PAT = process.env.GITLAB_PAT || '';
router.post('/', requireAuth(), async (req, res) => {
if (!GITLAB_URL || !GITLAB_PROJECT_ID || !GITLAB_PAT) {
return res.status(503).json({ error: 'Feedback integration not configured' });
}
const { type, title, description, page } = req.body;
if (!type || !title || !description) {
return res.status(400).json({ error: 'type, title, and description are required' });
}
if (!['bug', 'feature'].includes(type)) {
return res.status(400).json({ error: 'type must be "bug" or "feature"' });
}
const labels = type === 'bug' ? 'bug' : 'enhancement';
const prefix = type === 'bug' ? '🐛 Bug' : '✨ Feature Request';
const username = req.user?.username || 'unknown';
const body = [
`**Submitted by:** ${username}`,
page ? `**Page:** ${page}` : null,
`**Type:** ${prefix}`,
'',
'---',
'',
description,
].filter(Boolean).join('\n');
const postData = JSON.stringify({
title: `[${prefix}] ${title}`,
description: body,
labels,
});
const apiUrl = `${GITLAB_URL.replace(/\/$/, '')}/api/v4/projects/${encodeURIComponent(GITLAB_PROJECT_ID)}/issues`;
try {
const result = await new Promise((resolve, reject) => {
const parsed = new URL(apiUrl);
const transport = parsed.protocol === 'https:' ? https : http;
const reqOpts = {
method: 'POST',
hostname: parsed.hostname,
port: parsed.port,
path: parsed.pathname + parsed.search,
headers: {
'Content-Type': 'application/json',
'PRIVATE-TOKEN': GITLAB_PAT,
'Content-Length': Buffer.byteLength(postData),
},
rejectAuthorized: false,
};
const apiReq = transport.request(reqOpts, (apiRes) => {
let data = '';
apiRes.on('data', chunk => data += chunk);
apiRes.on('end', () => {
try {
resolve({ status: apiRes.statusCode, body: JSON.parse(data) });
} catch {
resolve({ status: apiRes.statusCode, body: data });
}
});
});
apiReq.on('error', reject);
apiReq.write(postData);
apiReq.end();
});
if (result.status === 201) {
console.log(`[Feedback] Issue #${result.body.iid} created by ${username}: ${title}`);
res.json({
success: true,
issue: {
id: result.body.iid,
url: result.body.web_url,
title: result.body.title,
},
});
} else {
console.error(`[Feedback] GitLab API returned ${result.status}:`, result.body);
res.status(502).json({ error: 'GitLab API error', details: result.body });
}
} catch (err) {
console.error('[Feedback] Request failed:', err.message);
res.status(502).json({ error: 'Failed to connect to GitLab' });
}
});
return router;
}
module.exports = createFeedbackRouter;

View File

@@ -0,0 +1,149 @@
// Ivanti Archive Routes — list, stats, and transition history for archived findings
const express = require('express');
const pool = require('../db');
const { requireAuth } = require('../middleware/auth');
const VALID_STATES = ['ACTIVE', 'ARCHIVED', 'RETURNED', 'CLOSED'];
/**
* Find the most severe active finding related to an archived finding.
*/
function findRelatedActive(archive, activeFindings) {
const archiveTitle = (archive.finding_title || '').toLowerCase();
const matches = activeFindings.filter(f => {
if (f.hostName !== archive.host_name) return false;
if (f.id === archive.finding_id) return false;
const activeTitle = (f.title || '').toLowerCase();
if (!archiveTitle.includes(activeTitle) && !activeTitle.includes(archiveTitle)) return false;
return true;
});
if (matches.length === 0) return null;
const best = matches.reduce((a, b) => (b.severity > a.severity ? b : a));
return { id: best.id, title: best.title, severity: best.severity };
}
function createIvantiArchiveRouter() {
const router = express.Router();
// All routes require authentication
router.use(requireAuth());
// GET / — List archive records with optional state filtering
router.get('/', async (req, res) => {
const { state } = req.query;
if (state && !VALID_STATES.includes(state)) {
return res.status(400).json({
error: 'Invalid state parameter. Valid values: ACTIVE, ARCHIVED, RETURNED, CLOSED'
});
}
try {
let query = 'SELECT * FROM ivanti_finding_archives';
const params = [];
let paramIndex = 1;
if (state) {
query += ` WHERE current_state = $${paramIndex++}`;
params.push(state);
}
query += ' ORDER BY last_transition_at DESC';
const { rows: archives } = await pool.query(query, params);
// Fetch active findings for related-finding enrichment
// In the new schema, active findings are in ivanti_findings table
let activeFindings = [];
try {
const { rows: findingsRows } = await pool.query(
`SELECT id, title, host_name AS "hostName", severity FROM ivanti_findings WHERE state = 'open'`
);
activeFindings = findingsRows;
} catch (cacheErr) {
console.warn('Failed to load findings for related-active matching:', cacheErr);
}
// Enrich each archive record with related active finding info
const enrichedArchives = archives.map(archive => ({
...archive,
related_active: findRelatedActive(archive, activeFindings)
}));
res.json({ archives: enrichedArchives, total: enrichedArchives.length });
} catch (err) {
console.error('Archive list error:', err);
res.status(500).json({ error: 'Failed to fetch archive records' });
}
});
// GET /stats — Summary counts by lifecycle state
router.get('/stats', async (req, res) => {
try {
const { rows } = await pool.query(
`SELECT current_state, COUNT(*) as count
FROM ivanti_finding_archives
GROUP BY current_state`
);
const stats = { ACTIVE: 0, ARCHIVED: 0, RETURNED: 0, CLOSED: 0 };
for (const row of rows) {
if (stats.hasOwnProperty(row.current_state)) {
stats[row.current_state] = parseInt(row.count);
}
}
// ACTIVE = total live findings count
const countResult = await pool.query(
`SELECT COUNT(*) as total FROM ivanti_findings WHERE state = 'open'`
);
stats.ACTIVE = parseInt(countResult.rows[0].total) || 0;
const total = stats.ACTIVE + stats.ARCHIVED + stats.RETURNED + stats.CLOSED;
res.json({ ...stats, total });
} catch (err) {
console.error('Archive stats error:', err);
res.status(500).json({ error: 'Failed to fetch archive stats' });
}
});
// GET /:findingId/history — Transition history for a specific archived finding
router.get('/:findingId/history', async (req, res) => {
const { findingId } = req.params;
try {
const { rows: archiveRows } = await pool.query(
'SELECT id FROM ivanti_finding_archives WHERE finding_id = $1',
[findingId]
);
const archive = archiveRows[0];
if (!archive) {
return res.json({ finding_id: findingId, transitions: [] });
}
const { rows: transitions } = await pool.query(
`SELECT * FROM ivanti_archive_transitions
WHERE archive_id = $1
ORDER BY transitioned_at DESC`,
[archive.id]
);
res.json({ finding_id: findingId, transitions });
} catch (err) {
console.error('Archive history error:', err);
res.status(500).json({ error: 'Failed to fetch transition history' });
}
});
return router;
}
module.exports = createIvantiArchiveRouter;

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,789 @@
// routes/ivantiFpWorkflow.js
const express = require('express');
const multer = require('multer');
const path = require('path');
const fs = require('fs');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const { ivantiFormPost, ivantiPost } = require('../helpers/ivantiApi');
const logAudit = require('../helpers/auditLog');
// ---------------------------------------------------------------------------
// Pure helpers (exported for testing)
// ---------------------------------------------------------------------------
const ALLOWED_EXTENSIONS = new Set([
'.pdf', '.png', '.jpg', '.jpeg', '.gif',
'.doc', '.docx', '.xlsx', '.csv', '.txt', '.zip'
]);
function isAllowedFileExtension(filename) {
if (!filename || typeof filename !== 'string') return false;
const ext = path.extname(filename).toLowerCase();
return ALLOWED_EXTENSIONS.has(ext);
}
function validateFpWorkflowForm(body) {
const errors = {};
if (!body.name || typeof body.name !== 'string' || body.name.trim().length === 0) {
errors.name = 'Workflow name is required.';
} else if (body.name.trim().length > 255) {
errors.name = 'Workflow name must be 255 characters or fewer.';
}
if (!body.reason || typeof body.reason !== 'string' || body.reason.trim().length === 0) {
errors.reason = 'Reason is required.';
}
if (body.description !== undefined && body.description !== null && body.description !== '') {
if (typeof body.description !== 'string') errors.description = 'Description must be a string.';
else if (body.description.length > 2000) errors.description = 'Description must be 2000 characters or fewer.';
}
if (!body.expirationDate || typeof body.expirationDate !== 'string' || body.expirationDate.trim().length === 0) {
errors.expirationDate = 'Expiration date is required.';
} else {
const parsed = new Date(body.expirationDate);
if (isNaN(parsed.getTime())) {
errors.expirationDate = 'Expiration date must be a valid date.';
} else {
const today = new Date(); today.setHours(0, 0, 0, 0);
const expDay = new Date(parsed); expDay.setHours(0, 0, 0, 0);
if (expDay <= today) errors.expirationDate = 'Expiration date must be in the future.';
else { const maxDate = new Date(today); maxDate.setDate(maxDate.getDate() + 120); if (expDay > maxDate) errors.expirationDate = 'Expiration date cannot be more than 120 days from today.'; }
}
}
return errors;
}
function buildSubjectFilterRequest(findingIds) {
return JSON.stringify({ subject: 'hostFinding', filterRequest: { filters: [{ field: 'id', exclusive: false, operator: 'IN', value: findingIds.map(id => String(id)).join(',') }] } });
}
function buildIvantiFormFields(formData, findingIds) {
const scopeMap = { 'Authorized': 'AUTHORIZED', 'None': 'NONE', 'Automated': 'AUTOMATED' };
return [
{ name: 'name', value: formData.name },
{ name: 'reason', value: formData.reason },
{ name: 'description', value: formData.description || '' },
{ name: 'expirationDate', value: formData.expirationDate },
{ name: 'overrideControl', value: scopeMap[formData.scopeOverride] || 'AUTHORIZED' },
{ name: 'subjectFilterRequest', value: buildSubjectFilterRequest(findingIds) },
{ name: 'isEmptyWorkflow', value: findingIds.length === 0 ? 'true' : 'false' }
];
}
const LIFECYCLE_STATUSES = new Set(['submitted', 'approved', 'rejected', 'rework', 'resubmitted']);
function validateLifecycleTransition(currentStatus, newStatus) {
if (currentStatus === 'approved') return { valid: false, error: 'This submission is finalized and cannot be edited.' };
if (!LIFECYCLE_STATUSES.has(newStatus)) return { valid: false, error: 'Invalid lifecycle status.' };
return { valid: true };
}
function mergeFindings(existingJson, newIds) {
const existing = JSON.parse(existingJson || '[]');
const merged = [...new Set([...existing, ...newIds])];
return JSON.stringify(merged);
}
function buildSubmissionHistoryEntry(changeType, details, userId, username) {
return { user_id: userId, username: username, change_type: changeType, change_details_json: JSON.stringify(details), created_at: new Date().toISOString() };
}
// ---------------------------------------------------------------------------
// Resolve workflow batch UUID
// ---------------------------------------------------------------------------
async function resolveWorkflowBatchUuid(submission, apiKey, clientId, skipTls) {
if (submission.ivanti_workflow_batch_uuid) return submission.ivanti_workflow_batch_uuid;
const searchUrl = `/client/${encodeURIComponent(clientId)}/workflowBatch/search`;
const workflowName = submission.workflow_name || '';
const searchBody = { filters: workflowName ? [{ field: 'name', exclusive: false, operator: 'EXACT', value: workflowName }] : [], projection: 'internal', sort: [{ field: 'created', direction: 'DESC' }], page: 0, size: 10 };
let result;
try { result = await ivantiPost(searchUrl, searchBody, apiKey, skipTls); } catch (e) { return null; }
if (result.status !== 200) return null;
let uuid = null;
try {
const data = JSON.parse(result.body);
let batches = data._embedded?.workflowBatches || data._embedded?.workflowBatch || data.content || data.data || (Array.isArray(data) ? data : []);
const batchId = String(submission.ivanti_workflow_batch_id);
const batch = batches.find(b => String(b.id) === batchId) || batches[0];
if (batch) uuid = batch.uuid || batch.workflowBatchUuid || batch.batchUuid || batch.groupUuid || batch.group_uuid || batch.workflow_batch_uuid || batch.uid || null;
} catch (e) { return null; }
if (uuid && submission.id) {
pool.query(`UPDATE ivanti_fp_submissions SET ivanti_workflow_batch_uuid = $1 WHERE id = $2`, [uuid, submission.id]).catch(() => {});
}
return uuid;
}
// ---------------------------------------------------------------------------
// Multer configuration
// ---------------------------------------------------------------------------
const uploadStorage = multer.memoryStorage();
const fpUpload = multer({ storage: uploadStorage, limits: { fileSize: 10 * 1024 * 1024 }, fileFilter: (req, file, cb) => { if (isAllowedFileExtension(file.originalname)) cb(null, true); else cb(new Error(`File type not allowed: ${path.extname(file.originalname)}`)); } }).array('attachments', 10);
// ---------------------------------------------------------------------------
// Router factory
// ---------------------------------------------------------------------------
function createIvantiFpWorkflowRouter() {
const router = express.Router();
// GET /documents/search
router.get('/documents/search', requireAuth(), async (req, res) => {
const q = (req.query.q || '').trim();
try {
let rows;
if (q) {
const like = `%${q}%`;
const result = await pool.query(
`SELECT id, cve_id, vendor, name, type, file_size, mime_type, uploaded_at FROM documents WHERE name ILIKE $1 OR cve_id ILIKE $2 OR vendor ILIKE $3 ORDER BY uploaded_at DESC LIMIT 50`,
[like, like, like]
);
rows = result.rows;
} else {
const result = await pool.query(`SELECT id, cve_id, vendor, name, type, file_size, mime_type, uploaded_at FROM documents ORDER BY uploaded_at DESC LIMIT 50`);
rows = result.rows;
}
res.json(rows || []);
} catch (err) {
console.error('Error searching documents:', err);
res.status(500).json({ error: 'Database error.' });
}
});
// POST / — Create FP workflow
router.post('/', requireAuth(), requireGroup('Admin', 'Standard_User'), (req, res) => {
fpUpload(req, res, (multerErr) => {
if (multerErr) {
if (multerErr.code === 'LIMIT_FILE_SIZE') return res.status(400).json({ error: 'File exceeds the 10 MB size limit.' });
return res.status(400).json({ error: multerErr.message });
}
let findingIds, queueItemIds;
try { findingIds = JSON.parse(req.body.findingIds || '[]'); queueItemIds = JSON.parse(req.body.queueItemIds || '[]'); }
catch (e) { return res.status(400).json({ error: 'findingIds and queueItemIds must be valid JSON arrays.' }); }
if (!Array.isArray(findingIds) || findingIds.length === 0) return res.status(400).json({ error: 'At least one finding ID is required.' });
if (!Array.isArray(queueItemIds) || queueItemIds.length === 0) return res.status(400).json({ error: 'At least one queue item ID is required.' });
let libraryDocIds;
try { libraryDocIds = JSON.parse(req.body.libraryDocIds || '[]'); } catch (e) { return res.status(400).json({ error: 'libraryDocIds must be a valid JSON array.' }); }
if (!Array.isArray(libraryDocIds)) return res.status(400).json({ error: 'libraryDocIds must be a valid JSON array.' });
for (const id of libraryDocIds) { if (!Number.isInteger(id) || id <= 0) return res.status(400).json({ error: `Invalid library document ID: ${id}.` }); }
const validationErrors = validateFpWorkflowForm(req.body);
if (Object.keys(validationErrors).length > 0) return res.status(400).json({ success: false, errors: validationErrors });
const files = req.files || [];
for (const file of files) { if (!isAllowedFileExtension(file.originalname)) return res.status(400).json({ error: `File type not allowed: ${file.originalname}` }); }
(async () => {
// Verify queue items
const { rows: queueRows } = await pool.query(
`SELECT id, workflow_type, status, user_id FROM ivanti_todo_queue WHERE id = ANY($1)`, [queueItemIds]
);
if (!queueRows || queueRows.length !== queueItemIds.length) return res.status(400).json({ error: 'One or more queue items not found.' });
for (const row of queueRows) {
if (row.user_id !== req.user.id) return res.status(403).json({ error: 'You can only submit your own queue items.' });
if (row.workflow_type !== 'FP') return res.status(400).json({ error: `Queue item ${row.id} is not an FP workflow type.` });
if (row.status !== 'pending') return res.status(400).json({ error: `Queue item ${row.id} is not in pending status.` });
}
const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const skipTls = process.env.IVANTI_SKIP_TLS === 'true';
if (!apiKey) return res.status(500).json({ success: false, error: 'Ivanti API key is not configured.', step: 'create_workflow' });
const formFields = buildIvantiFormFields(req.body, findingIds);
// Look up library documents
let libraryDocs = [];
const libraryAttachmentResults = [];
if (libraryDocIds.length > 0) {
const { rows: docRows } = await pool.query(`SELECT id, name, file_path, file_size, mime_type FROM documents WHERE id = ANY($1)`, [libraryDocIds]);
libraryDocs = docRows;
const foundIds = new Set(libraryDocs.map(d => d.id));
const missingIds = libraryDocIds.filter(id => !foundIds.has(id));
if (missingIds.length > 0) return res.status(400).json({ error: `Library document IDs not found: ${missingIds.join(', ')}` });
}
const libraryFormFiles = [];
for (const doc of libraryDocs) {
try { const buffer = fs.readFileSync(doc.file_path); libraryFormFiles.push({ name: 'files', buffer, filename: doc.name }); libraryAttachmentResults.push({ filename: doc.name, success: true, source: 'library', documentId: doc.id }); }
catch (readErr) { libraryAttachmentResults.push({ success: false, error: 'File not found on disk', source: 'library', documentId: doc.id, filename: doc.name }); }
}
const localFormFiles = files.map(f => ({ name: 'files', buffer: f.buffer, filename: f.originalname }));
const formFiles = [...localFormFiles, ...libraryFormFiles];
const createUrl = `/client/${encodeURIComponent(clientId)}/workflowBatch/falsePositive/request`;
let createResult;
try { createResult = await ivantiFormPost(createUrl, formFields, formFiles, apiKey, skipTls); }
catch (networkErr) {
logAudit({ userId: req.user.id, username: req.user.username, action: 'ivanti_fp_workflow_failed', entityType: 'ivanti_workflow', details: { error: networkErr.message, findingIds }, ipAddress: req.ip });
return res.status(502).json({ success: false, error: 'Failed to connect to Ivanti API.', step: 'create_workflow', details: networkErr.message });
}
if (createResult.status !== 200 && createResult.status !== 201 && createResult.status !== 202) {
const errorMap = { 401: 'Ivanti API key is invalid or missing.', 419: 'API key lacks workflow creation permissions.', 429: 'Ivanti API rate limit reached.' };
const errorMsg = errorMap[createResult.status] || `Workflow creation failed: ${createResult.status}`;
logAudit({ userId: req.user.id, username: req.user.username, action: 'ivanti_fp_workflow_failed', entityType: 'ivanti_workflow', details: { error: errorMsg, status: createResult.status, findingIds }, ipAddress: req.ip });
return res.status(createResult.status === 429 ? 429 : 502).json({ success: false, error: errorMsg, step: 'create_workflow' });
}
let workflowBatchId;
try { const createData = JSON.parse(createResult.body); workflowBatchId = createData.id; }
catch (parseErr) { return res.status(502).json({ success: false, error: 'Failed to parse Ivanti API response.', step: 'create_workflow' }); }
let workflowBatchUuid = null;
try { workflowBatchUuid = await resolveWorkflowBatchUuid({ id: null, ivanti_workflow_batch_id: workflowBatchId, ivanti_workflow_batch_uuid: null, workflow_name: req.body.name }, apiKey, clientId, skipTls); } catch (e) {}
const localAttachmentResults = files.map(f => ({ filename: f.originalname, success: true, source: 'local' }));
const allAttachmentResults = [...localAttachmentResults, ...libraryAttachmentResults];
const totalAttachmentCount = files.length + libraryDocIds.length;
try {
await pool.query(
`INSERT INTO ivanti_fp_submissions (user_id, username, ivanti_workflow_batch_id, ivanti_workflow_batch_uuid, ivanti_generated_id, workflow_name, reason, description, expiration_date, scope_override, finding_ids_json, queue_item_ids_json, attachment_count, attachment_results_json, status, error_message)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16)`,
[req.user.id, req.user.username, workflowBatchId, workflowBatchUuid, null, req.body.name, req.body.reason, req.body.description || null, req.body.expirationDate, req.body.scopeOverride || 'Authorized', JSON.stringify(findingIds), JSON.stringify(queueItemIds), totalAttachmentCount, JSON.stringify(allAttachmentResults), 'success', null]
);
} catch (dbErr) { console.error('Failed to insert submission record:', dbErr); }
logAudit({ userId: req.user.id, username: req.user.username, action: 'ivanti_fp_workflow_created', entityType: 'ivanti_workflow', entityId: String(workflowBatchId), details: { workflowName: req.body.name, findingIds, attachmentCount: totalAttachmentCount, status: 'success' }, ipAddress: req.ip });
let queueItemsUpdated = 0;
try {
const qResult = await pool.query(`UPDATE ivanti_todo_queue SET status='complete', updated_at=NOW() WHERE id = ANY($1) AND user_id=$2`, [queueItemIds, req.user.id]);
queueItemsUpdated = qResult.rowCount;
} catch (queueErr) { console.error('Failed to update queue items:', queueErr); }
res.json({ success: true, workflowBatchId, queueItemsUpdated, status: 'success' });
})().catch((unexpectedErr) => { console.error('Unexpected error in FP workflow submission:', unexpectedErr); res.status(500).json({ success: false, error: 'Internal server error.' }); });
});
});
// GET /submissions
router.get('/submissions', requireAuth(), async (req, res) => {
try {
const { rows: submissions } = await pool.query(`SELECT * FROM ivanti_fp_submissions WHERE user_id = $1 ORDER BY created_at DESC`, [req.user.id]);
if (submissions.length > 0) {
const submissionIds = submissions.map(s => s.id);
const { rows: historyRows } = await pool.query(`SELECT * FROM ivanti_fp_submission_history WHERE submission_id = ANY($1) ORDER BY created_at ASC`, [submissionIds]);
const historyMap = {};
for (const row of historyRows) { if (!historyMap[row.submission_id]) historyMap[row.submission_id] = []; historyMap[row.submission_id].push(row); }
for (const sub of submissions) { sub.history = historyMap[sub.id] || []; }
const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const skipTls = process.env.IVANTI_SKIP_TLS === 'true';
if (apiKey) {
try {
for (const sub of submissions) {
if (!sub.workflow_name) continue;
const searchUrl = `/client/${encodeURIComponent(clientId)}/workflowBatch/search`;
const searchBody = { filters: [{ field: 'name', exclusive: false, operator: 'EXACT', value: sub.workflow_name }], projection: 'internal', sort: [{ field: 'created', direction: 'DESC' }], page: 0, size: 1 };
const result = await ivantiPost(searchUrl, searchBody, apiKey, skipTls);
if (result.status === 200) {
const data = JSON.parse(result.body);
const batches = data._embedded?.workflowBatches || data._embedded?.workflowBatch || [];
const batch = batches[0];
if (batch) { sub.ivanti_rework_note = batch.reworkNote || null; sub.ivanti_approval_note = batch.approvalNote || null; sub.ivanti_current_state_notes = batch.currentStateUserNotes || null; sub.ivanti_previous_state_notes = batch.previousStateUserNotes || null; sub.ivanti_current_state = batch.currentState || null; }
}
}
} catch (e) { console.error('Error enriching submissions with Ivanti notes:', e.message); }
}
}
res.json(submissions);
} catch (err) { console.error('Error fetching FP submissions:', err); res.status(500).json({ error: 'Internal server error.' }); }
});
// PUT /submissions/:id — Edit FP workflow fields
router.put('/submissions/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), (req, res) => {
(async () => {
const submissionId = req.params.id;
// 1. Fetch submission and verify ownership
const { rows: subRows } = await pool.query(
`SELECT * FROM ivanti_fp_submissions WHERE id = $1`, [submissionId]
);
const submission = subRows[0];
if (!submission) {
return res.status(404).json({ error: 'Submission not found.' });
}
if (submission.user_id !== req.user.id) {
return res.status(403).json({ error: 'You can only edit your own submissions.' });
}
// 2. Lifecycle guard
if (submission.lifecycle_status === 'approved') {
return res.status(400).json({ error: 'This submission is finalized and cannot be edited.' });
}
// 3. Validate body
const validationErrors = validateFpWorkflowForm(req.body);
if (Object.keys(validationErrors).length > 0) {
return res.status(400).json({ success: false, errors: validationErrors });
}
// 4. Proxy to Ivanti
const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const skipTls = process.env.IVANTI_SKIP_TLS === 'true';
if (!apiKey) {
return res.status(500).json({ success: false, error: 'Ivanti API key is not configured.' });
}
const scopeMap = { 'Authorized': 'AUTHORIZED', 'None': 'NONE', 'Automated': 'AUTOMATED' };
const updateUrl = `/client/${encodeURIComponent(clientId)}/workflowBatch/falsePositive/update`;
const updateBody = {
id: submission.ivanti_workflow_batch_id,
name: req.body.name,
reason: req.body.reason,
description: req.body.description || '',
expirationDate: req.body.expirationDate,
overrideControl: scopeMap[req.body.scopeOverride] || 'AUTHORIZED'
};
let ivantiResult;
try {
ivantiResult = await ivantiPost(updateUrl, updateBody, apiKey, skipTls);
} catch (networkErr) {
logAudit({
userId: req.user.id, username: req.user.username,
action: 'ivanti_fp_submission_edit_failed', entityType: 'ivanti_workflow',
details: { error: networkErr.message, submissionId },
ipAddress: req.ip
});
return res.status(502).json({ success: false, error: 'Failed to connect to Ivanti API.', details: networkErr.message });
}
if (ivantiResult.status !== 200 && ivantiResult.status !== 201 && ivantiResult.status !== 202) {
const errorMap = {
401: 'Ivanti API key is invalid or missing. Contact your administrator.',
419: 'API key lacks permissions for this operation.',
429: 'Ivanti API rate limit reached. Please try again in a few minutes.'
};
const errorMsg = ivantiResult.status >= 500
? 'Ivanti API is temporarily unavailable. Please try again later.'
: (errorMap[ivantiResult.status] || `Operation failed: ${ivantiResult.status}`);
return res.status(ivantiResult.status === 429 ? 429 : 502).json({ success: false, error: errorMsg });
}
// 5. Determine new lifecycle_status
let newLifecycleStatus = submission.lifecycle_status;
if (submission.lifecycle_status === 'rejected' || submission.lifecycle_status === 'rework') {
newLifecycleStatus = 'resubmitted';
}
// 6. Update local record
try {
await pool.query(
`UPDATE ivanti_fp_submissions
SET workflow_name = $1, reason = $2, description = $3, expiration_date = $4, scope_override = $5, lifecycle_status = $6, updated_at = NOW()
WHERE id = $7`,
[req.body.name, req.body.reason, req.body.description || null, req.body.expirationDate, req.body.scopeOverride || 'Authorized', newLifecycleStatus, submissionId]
);
} catch (dbErr) {
console.error('Failed to update submission record:', dbErr);
return res.status(500).json({ success: false, error: 'Failed to update local record.' });
}
// 7. Insert history row
const historyEntry = buildSubmissionHistoryEntry('fields_updated', {
changed: {
name: { from: submission.workflow_name, to: req.body.name },
reason: { from: submission.reason, to: req.body.reason },
description: { from: submission.description, to: req.body.description || '' },
expirationDate: { from: submission.expiration_date, to: req.body.expirationDate },
scopeOverride: { from: submission.scope_override, to: req.body.scopeOverride || 'Authorized' }
}
}, req.user.id, req.user.username);
try {
await pool.query(
`INSERT INTO ivanti_fp_submission_history (submission_id, user_id, username, change_type, change_details_json, created_at)
VALUES ($1, $2, $3, $4, $5, NOW())`,
[submissionId, historyEntry.user_id, historyEntry.username, historyEntry.change_type, historyEntry.change_details_json]
);
} catch (histErr) {
console.error('Failed to insert history row:', histErr);
}
// 8. Audit log
logAudit({
userId: req.user.id, username: req.user.username,
action: 'ivanti_fp_submission_edited', entityType: 'ivanti_workflow',
entityId: String(submission.ivanti_workflow_batch_id),
details: { submissionId, workflowName: req.body.name },
ipAddress: req.ip
});
// 9. Return updated record
const { rows: updatedRows } = await pool.query(
`SELECT * FROM ivanti_fp_submissions WHERE id = $1`, [submissionId]
);
res.json({ success: true, submission: updatedRows[0] });
})().catch((unexpectedErr) => {
console.error('Unexpected error in PUT /submissions/:id:', unexpectedErr);
res.status(500).json({ success: false, error: 'Internal server error.' });
});
});
// POST /submissions/:id/findings — Map additional findings to existing workflow
router.post('/submissions/:id/findings', requireAuth(), requireGroup('Admin', 'Standard_User'), (req, res) => {
(async () => {
const submissionId = req.params.id;
const { findingIds, queueItemIds } = req.body;
if (!Array.isArray(findingIds) || findingIds.length === 0) {
return res.status(400).json({ error: 'At least one finding ID is required.' });
}
if (!Array.isArray(queueItemIds) || queueItemIds.length === 0) {
return res.status(400).json({ error: 'At least one queue item ID is required.' });
}
// 1. Fetch submission and verify ownership
const { rows: subRows } = await pool.query(
`SELECT * FROM ivanti_fp_submissions WHERE id = $1`, [submissionId]
);
const submission = subRows[0];
if (!submission) {
return res.status(404).json({ error: 'Submission not found.' });
}
if (submission.user_id !== req.user.id) {
return res.status(403).json({ error: 'You can only edit your own submissions.' });
}
// 2. Lifecycle guard
if (submission.lifecycle_status === 'approved') {
return res.status(400).json({ error: 'This submission is finalized and cannot be edited.' });
}
// 3. Verify queue items belong to user, are FP type, and pending
const { rows: queueRows } = await pool.query(
`SELECT id, workflow_type, status, user_id FROM ivanti_todo_queue WHERE id = ANY($1)`,
[queueItemIds]
);
if (queueRows.length !== queueItemIds.length) {
return res.status(400).json({ error: 'One or more queue items not found.' });
}
for (const row of queueRows) {
if (row.user_id !== req.user.id) {
return res.status(403).json({ error: 'You can only submit your own queue items.' });
}
if (row.workflow_type !== 'FP') {
return res.status(400).json({ error: `Queue item ${row.id} is not an FP workflow type.` });
}
if (row.status !== 'pending') {
return res.status(400).json({ error: `Queue item ${row.id} is not in pending status.` });
}
}
// 4. Proxy to Ivanti map endpoint
const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const skipTls = process.env.IVANTI_SKIP_TLS === 'true';
if (!apiKey) {
return res.status(500).json({ success: false, error: 'Ivanti API key is not configured.' });
}
const mapUuid = await resolveWorkflowBatchUuid(submission, apiKey, clientId, skipTls);
if (!mapUuid) {
return res.status(400).json({ success: false, error: 'Could not resolve workflow batch UUID. The workflow may not exist in Ivanti.' });
}
const mapUrl = `/client/${encodeURIComponent(clientId)}/workflowBatch/falsePositive/${encodeURIComponent(mapUuid)}/map`;
const mappedIds = [];
const failedIds = [];
for (const fid of findingIds) {
const mapBody = {
subject: 'hostFinding',
filterRequest: {
filters: [{ field: 'id', exclusive: false, operator: 'EXACT', value: String(fid) }]
}
};
try {
const result = await ivantiPost(mapUrl, mapBody, apiKey, skipTls);
if (result.status === 200 || result.status === 201 || result.status === 202) {
mappedIds.push(fid);
} else {
failedIds.push({ id: fid, status: result.status });
}
} catch (err) {
failedIds.push({ id: fid, error: err.message });
}
}
if (mappedIds.length === 0) {
return res.status(502).json({ success: false, error: 'Failed to map any findings to the workflow.' });
}
// 5. Merge only successfully mapped finding IDs
const mergedJson = mergeFindings(submission.finding_ids_json, mappedIds);
try {
await pool.query(
`UPDATE ivanti_fp_submissions SET finding_ids_json = $1, updated_at = NOW() WHERE id = $2`,
[mergedJson, submissionId]
);
} catch (dbErr) {
console.error('Failed to update finding_ids_json:', dbErr);
}
// 6. Mark only successfully mapped queue items as complete
let queueItemsUpdated = 0;
const mappedSet = new Set(mappedIds.map(String));
const successQueueIds = queueItemIds.filter((qid, idx) => {
const queueItem = queueRows.find(r => r.id === qid);
return queueItem && mappedSet.has(String(findingIds[idx]));
});
if (successQueueIds.length > 0) {
try {
const qResult = await pool.query(
`UPDATE ivanti_todo_queue SET status='complete', updated_at=NOW() WHERE id = ANY($1) AND user_id=$2`,
[successQueueIds, req.user.id]
);
queueItemsUpdated = qResult.rowCount;
} catch (queueErr) {
console.error('Failed to update queue items:', queueErr);
}
}
// 7. Insert history row
const historyEntry = buildSubmissionHistoryEntry('findings_added', {
addedFindingIds: mappedIds,
failedFindingIds: failedIds.map(f => f.id || f),
queueItemIds: successQueueIds
}, req.user.id, req.user.username);
try {
await pool.query(
`INSERT INTO ivanti_fp_submission_history (submission_id, user_id, username, change_type, change_details_json, created_at)
VALUES ($1, $2, $3, $4, $5, NOW())`,
[submissionId, historyEntry.user_id, historyEntry.username, historyEntry.change_type, historyEntry.change_details_json]
);
} catch (histErr) {
console.error('Failed to insert history row:', histErr);
}
// 8. Audit log
logAudit({
userId: req.user.id, username: req.user.username,
action: 'ivanti_fp_findings_added', entityType: 'ivanti_workflow',
entityId: String(submission.ivanti_workflow_batch_id),
details: { submissionId, addedFindingIds: findingIds, queueItemsUpdated },
ipAddress: req.ip
});
res.json({ success: true, addedFindings: mappedIds, failedFindings: failedIds, queueItemsUpdated });
})().catch((unexpectedErr) => {
console.error('Unexpected error in POST /submissions/:id/findings:', unexpectedErr);
res.status(500).json({ success: false, error: 'Internal server error.' });
});
});
// POST /submissions/:id/attachments — Upload additional attachments
router.post('/submissions/:id/attachments', requireAuth(), requireGroup('Admin', 'Standard_User'), (req, res) => {
fpUpload(req, res, (multerErr) => {
if (multerErr) {
if (multerErr.code === 'LIMIT_FILE_SIZE') {
return res.status(400).json({ error: 'File exceeds the 10 MB size limit.' });
}
return res.status(400).json({ error: multerErr.message });
}
const files = req.files || [];
let libraryDocIds;
try { libraryDocIds = JSON.parse(req.body.libraryDocIds || '[]'); }
catch (e) { return res.status(400).json({ error: 'libraryDocIds must be a valid JSON array.' }); }
if (!Array.isArray(libraryDocIds)) return res.status(400).json({ error: 'libraryDocIds must be a valid JSON array.' });
for (const id of libraryDocIds) { if (!Number.isInteger(id) || id <= 0) return res.status(400).json({ error: `Invalid library document ID: ${id}.` }); }
if (files.length === 0 && libraryDocIds.length === 0) {
return res.status(400).json({ error: 'At least one file or library document is required.' });
}
for (const file of files) {
if (!isAllowedFileExtension(file.originalname)) {
return res.status(400).json({ error: `File type not allowed: ${file.originalname}` });
}
}
(async () => {
const submissionId = req.params.id;
const { rows: subRows } = await pool.query(
`SELECT * FROM ivanti_fp_submissions WHERE id = $1`, [submissionId]
);
const submission = subRows[0];
if (!submission) return res.status(404).json({ error: 'Submission not found.' });
if (submission.user_id !== req.user.id) return res.status(403).json({ error: 'You can only edit your own submissions.' });
if (submission.lifecycle_status === 'approved') return res.status(400).json({ error: 'This submission is finalized and cannot be edited.' });
// Look up library documents
let libraryDocs = [];
if (libraryDocIds.length > 0) {
const { rows: docRows } = await pool.query(
`SELECT id, name, file_path, file_size, mime_type FROM documents WHERE id = ANY($1)`, [libraryDocIds]
);
libraryDocs = docRows;
const foundIds = new Set(libraryDocs.map(d => d.id));
const missingIds = libraryDocIds.filter(id => !foundIds.has(id));
if (missingIds.length > 0) return res.status(400).json({ error: `Library document IDs not found: ${missingIds.join(', ')}` });
}
const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const skipTls = process.env.IVANTI_SKIP_TLS === 'true';
if (!apiKey) return res.status(500).json({ success: false, error: 'Ivanti API key is not configured.' });
const attachUuid = await resolveWorkflowBatchUuid(submission, apiKey, clientId, skipTls);
if (!attachUuid) return res.status(400).json({ success: false, error: 'Could not resolve workflow batch UUID.' });
const attachUrl = `/client/${encodeURIComponent(clientId)}/workflowBatch/falsePositive/${encodeURIComponent(attachUuid)}/attach`;
const attachmentResults = [];
// Upload local files
for (const f of files) {
try {
const formFiles = [{ name: 'file', buffer: f.buffer, filename: f.originalname, contentType: f.mimetype || 'application/octet-stream' }];
const result = await ivantiFormPost(attachUrl, [], formFiles, apiKey, skipTls);
const success = result.status === 200 || result.status === 201 || result.status === 202;
attachmentResults.push({ filename: f.originalname, success, source: 'local', ...(success ? {} : { error: `Upload failed: ${result.status}` }) });
} catch (uploadErr) {
attachmentResults.push({ filename: f.originalname, success: false, source: 'local', error: uploadErr.message });
}
}
// Upload library files
for (const doc of libraryDocs) {
let buffer;
try { buffer = fs.readFileSync(doc.file_path); }
catch (readErr) { attachmentResults.push({ success: false, error: 'File not found on disk', source: 'library', documentId: doc.id, filename: doc.name }); continue; }
try {
const formFiles = [{ name: 'file', buffer, filename: doc.name, contentType: doc.mime_type || 'application/octet-stream' }];
const result = await ivantiFormPost(attachUrl, [], formFiles, apiKey, skipTls);
const success = result.status === 200 || result.status === 201 || result.status === 202;
attachmentResults.push({ filename: doc.name, success, source: 'library', documentId: doc.id, ...(success ? {} : { error: `Upload failed: ${result.status}` }) });
} catch (uploadErr) {
attachmentResults.push({ filename: doc.name, success: false, source: 'library', documentId: doc.id, error: uploadErr.message });
}
}
// Update attachment_count and attachment_results_json
const existingResults = JSON.parse(submission.attachment_results_json || '[]');
const allResults = [...existingResults, ...attachmentResults];
const successCount = attachmentResults.filter(r => r.success).length;
const newAttachmentCount = (submission.attachment_count || 0) + successCount;
try {
await pool.query(
`UPDATE ivanti_fp_submissions SET attachment_count = $1, attachment_results_json = $2, updated_at = NOW() WHERE id = $3`,
[newAttachmentCount, JSON.stringify(allResults), submissionId]
);
} catch (dbErr) { console.error('Failed to update attachment records:', dbErr); }
// Insert history row
const historyEntry = buildSubmissionHistoryEntry('attachments_added', { files: attachmentResults }, req.user.id, req.user.username);
try {
await pool.query(
`INSERT INTO ivanti_fp_submission_history (submission_id, user_id, username, change_type, change_details_json, created_at)
VALUES ($1, $2, $3, $4, $5, NOW())`,
[submissionId, historyEntry.user_id, historyEntry.username, historyEntry.change_type, historyEntry.change_details_json]
);
} catch (histErr) { console.error('Failed to insert history row:', histErr); }
logAudit({
userId: req.user.id, username: req.user.username,
action: 'ivanti_fp_attachments_added', entityType: 'ivanti_workflow',
entityId: String(submission.ivanti_workflow_batch_id),
details: { submissionId, attachmentResults, libraryDocCount: libraryDocIds.length },
ipAddress: req.ip
});
const allSucceeded = attachmentResults.every(r => r.success);
res.json({ success: true, attachmentResults, status: allSucceeded ? 'success' : 'partial' });
})().catch((unexpectedErr) => {
console.error('Unexpected error in POST /submissions/:id/attachments:', unexpectedErr);
res.status(500).json({ success: false, error: 'Internal server error.' });
});
});
});
// PATCH /submissions/:id/status — Update lifecycle status
router.patch('/submissions/:id/status', requireAuth(), requireGroup('Admin', 'Standard_User'), (req, res) => {
(async () => {
const submissionId = req.params.id;
const newStatus = req.body.lifecycle_status;
const { rows: subRows } = await pool.query(
`SELECT * FROM ivanti_fp_submissions WHERE id = $1`, [submissionId]
);
const submission = subRows[0];
if (!submission) return res.status(404).json({ error: 'Submission not found.' });
if (submission.user_id !== req.user.id) return res.status(403).json({ error: 'You can only edit your own submissions.' });
const transition = validateLifecycleTransition(submission.lifecycle_status, newStatus);
if (!transition.valid) return res.status(400).json({ error: transition.error });
const previousStatus = submission.lifecycle_status;
try {
await pool.query(
`UPDATE ivanti_fp_submissions SET lifecycle_status = $1, updated_at = NOW() WHERE id = $2`,
[newStatus, submissionId]
);
} catch (dbErr) {
console.error('Failed to update lifecycle status:', dbErr);
return res.status(500).json({ success: false, error: 'Failed to update status.' });
}
const historyEntry = buildSubmissionHistoryEntry('status_changed', { from: previousStatus, to: newStatus }, req.user.id, req.user.username);
try {
await pool.query(
`INSERT INTO ivanti_fp_submission_history (submission_id, user_id, username, change_type, change_details_json, created_at)
VALUES ($1, $2, $3, $4, $5, NOW())`,
[submissionId, historyEntry.user_id, historyEntry.username, historyEntry.change_type, historyEntry.change_details_json]
);
} catch (histErr) { console.error('Failed to insert history row:', histErr); }
logAudit({
userId: req.user.id, username: req.user.username,
action: 'ivanti_fp_status_changed', entityType: 'ivanti_workflow',
entityId: String(submission.ivanti_workflow_batch_id),
details: { submissionId, from: previousStatus, to: newStatus },
ipAddress: req.ip
});
res.json({ success: true, previousStatus, newStatus });
})().catch((unexpectedErr) => {
console.error('Unexpected error in PATCH /submissions/:id/status:', unexpectedErr);
res.status(500).json({ success: false, error: 'Internal server error.' });
});
});
return router;
}
module.exports = createIvantiFpWorkflowRouter;
module.exports.validateFpWorkflowForm = validateFpWorkflowForm;
module.exports.buildIvantiFormFields = buildIvantiFormFields;
module.exports.buildSubjectFilterRequest = buildSubjectFilterRequest;
module.exports.isAllowedFileExtension = isAllowedFileExtension;
module.exports.validateLifecycleTransition = validateLifecycleTransition;
module.exports.mergeFindings = mergeFindings;
module.exports.buildSubmissionHistoryEntry = buildSubmissionHistoryEntry;

View File

@@ -0,0 +1,352 @@
// routes/ivantiTodoQueue.js
const express = require('express');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog');
const VALID_WORKFLOW_TYPES = ['FP', 'Archer', 'CARD', 'GRANITE'];
const VALID_STATUSES = ['pending', 'complete'];
function isValidVendor(vendor) {
if (typeof vendor !== 'string') return false;
const trimmed = vendor.trim();
return trimmed.length > 0 && trimmed.length <= 200;
}
function createIvantiTodoQueueRouter() {
const router = express.Router();
// GET /api/ivanti/todo-queue
router.get('/', requireAuth(), async (req, res) => {
try {
const { rows } = await pool.query(
`SELECT q.*
FROM ivanti_todo_queue q
WHERE q.user_id = $1
ORDER BY q.vendor ASC, q.created_at ASC`,
[req.user.id]
);
const parsed = rows.map((r) => ({
...r,
cves: r.cves_json ? JSON.parse(r.cves_json) : [],
}));
res.json(parsed);
} catch (err) {
console.error('Error fetching todo queue:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
// POST /api/ivanti/todo-queue/batch
router.post('/batch', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { findings, workflow_type, vendor } = req.body;
if (!Array.isArray(findings) || findings.length < 1 || findings.length > 200) {
return res.status(400).json({ error: 'findings array must contain 1-200 items.' });
}
for (let i = 0; i < findings.length; i++) {
const f = findings[i];
if (!f || typeof f.finding_id !== 'string' || f.finding_id.trim().length === 0) {
return res.status(400).json({ error: 'Each finding must have a non-empty finding_id string.' });
}
}
if (!VALID_WORKFLOW_TYPES.includes(workflow_type)) {
return res.status(400).json({ error: 'workflow_type must be FP, Archer, CARD, or GRANITE.' });
}
if (!['CARD', 'GRANITE'].includes(workflow_type)) {
if (!isValidVendor(vendor)) {
return res.status(400).json({ error: 'vendor is required for FP and Archer workflows.' });
}
}
if (vendor !== undefined && vendor !== '' && typeof vendor === 'string' && vendor.trim().length > 200) {
return res.status(400).json({ error: 'vendor must be under 200 chars.' });
}
const vendorVal = ['CARD', 'GRANITE'].includes(workflow_type) ? '' : vendor.trim();
const userId = req.user.id;
const client = await pool.connect();
try {
await client.query('BEGIN');
const insertedIds = [];
for (const f of findings) {
const findingId = f.finding_id.trim();
const title = f.finding_title && typeof f.finding_title === 'string'
? f.finding_title.slice(0, 500) : null;
const cvesJson = Array.isArray(f.cves) ? JSON.stringify(f.cves) : null;
const ipVal = f.ip_address && typeof f.ip_address === 'string'
? f.ip_address.trim().slice(0, 64) : null;
const hostVal = f.hostname && typeof f.hostname === 'string'
? f.hostname.trim().slice(0, 255) : null;
const { rows } = await client.query(
`INSERT INTO ivanti_todo_queue
(user_id, finding_id, finding_title, cves_json, ip_address, hostname, vendor, workflow_type)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING id`,
[userId, findingId, title, cvesJson, ipVal, hostVal, vendorVal, workflow_type]
);
insertedIds.push(rows[0].id);
}
await client.query('COMMIT');
// Fetch all inserted rows
const { rows: fetchedRows } = await pool.query(
`SELECT * FROM ivanti_todo_queue WHERE id = ANY($1)`,
[insertedIds]
);
const items = fetchedRows.map((r) => ({
...r,
cves: r.cves_json ? JSON.parse(r.cves_json) : [],
}));
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'batch_add_to_queue',
entityType: 'ivanti_todo_queue',
entityId: null,
details: {
count: insertedIds.length,
workflow_type: workflow_type,
finding_ids: findings.map((f) => f.finding_id.trim()),
},
ipAddress: req.ip,
});
return res.status(201).json({ items });
} catch (err) {
await client.query('ROLLBACK');
console.error('Batch insert error:', err);
return res.status(500).json({ error: 'Internal server error.' });
} finally {
client.release();
}
});
// POST /api/ivanti/todo-queue
router.post('/', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { finding_id, finding_title, cves, ip_address, hostname, vendor, workflow_type } = req.body;
if (!finding_id || typeof finding_id !== 'string' || finding_id.trim().length === 0) {
return res.status(400).json({ error: 'finding_id is required.' });
}
if (!VALID_WORKFLOW_TYPES.includes(workflow_type)) {
return res.status(400).json({ error: 'workflow_type must be FP, Archer, CARD, or GRANITE.' });
}
if (!['CARD', 'GRANITE'].includes(workflow_type) && !isValidVendor(vendor)) {
return res.status(400).json({ error: 'vendor is required for FP and Archer workflows.' });
}
if (vendor !== undefined && vendor !== '' && !isValidVendor(vendor)) {
return res.status(400).json({ error: 'vendor must be under 200 chars.' });
}
const vendorVal = ['CARD', 'GRANITE'].includes(workflow_type) ? '' : vendor.trim();
const cvesJson = Array.isArray(cves) ? JSON.stringify(cves) : null;
const ipVal = ip_address && typeof ip_address === 'string' ? ip_address.trim().slice(0, 64) : null;
const hostVal = hostname && typeof hostname === 'string' ? hostname.trim().slice(0, 255) : null;
const title = finding_title && typeof finding_title === 'string'
? finding_title.slice(0, 500) : null;
try {
const { rows } = await pool.query(
`INSERT INTO ivanti_todo_queue
(user_id, finding_id, finding_title, cves_json, ip_address, hostname, vendor, workflow_type)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING *`,
[req.user.id, finding_id.trim(), title, cvesJson, ipVal, hostVal, vendorVal, workflow_type]
);
const result = {
...rows[0],
cves: rows[0].cves_json ? JSON.parse(rows[0].cves_json) : [],
};
res.status(201).json(result);
} catch (err) {
console.error('Error adding to queue:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
// PUT /api/ivanti/todo-queue/:id
router.put('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params;
const { vendor, workflow_type, status } = req.body;
if (vendor !== undefined && !isValidVendor(vendor)) {
return res.status(400).json({ error: 'vendor must be a non-empty string (max 200 chars).' });
}
if (workflow_type !== undefined && !VALID_WORKFLOW_TYPES.includes(workflow_type)) {
return res.status(400).json({ error: 'workflow_type must be FP, Archer, CARD, or GRANITE.' });
}
if (status !== undefined && !VALID_STATUSES.includes(status)) {
return res.status(400).json({ error: 'status must be pending or complete.' });
}
try {
const { rows: existingRows } = await pool.query(
'SELECT * FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2',
[id, req.user.id]
);
if (!existingRows[0]) {
return res.status(404).json({ error: 'Queue item not found.' });
}
const updates = [];
const params = [];
let paramIndex = 1;
if (vendor !== undefined) {
updates.push(`vendor = $${paramIndex++}`);
params.push(vendor.trim());
}
if (workflow_type !== undefined) {
updates.push(`workflow_type = $${paramIndex++}`);
params.push(workflow_type);
}
if (status !== undefined) {
updates.push(`status = $${paramIndex++}`);
params.push(status);
}
if (updates.length === 0) {
return res.status(400).json({ error: 'No fields to update.' });
}
updates.push('updated_at = NOW()');
params.push(id, req.user.id);
await pool.query(
`UPDATE ivanti_todo_queue SET ${updates.join(', ')} WHERE id = $${paramIndex++} AND user_id = $${paramIndex}`,
params
);
const { rows } = await pool.query(
'SELECT * FROM ivanti_todo_queue WHERE id = $1', [id]
);
const result = {
...rows[0],
cves: rows[0].cves_json ? JSON.parse(rows[0].cves_json) : [],
};
res.json(result);
} catch (err) {
console.error(err);
res.status(500).json({ error: 'Internal server error.' });
}
});
// POST /api/ivanti/todo-queue/:id/redirect
router.post('/:id/redirect', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params;
const { workflow_type, vendor } = req.body;
if (!VALID_WORKFLOW_TYPES.includes(workflow_type)) {
return res.status(400).json({ error: 'workflow_type must be FP, Archer, CARD, or GRANITE.' });
}
if (!['CARD', 'GRANITE'].includes(workflow_type)) {
if (!isValidVendor(vendor)) {
return res.status(400).json({ error: 'vendor is required for FP and Archer workflows.' });
}
}
if (vendor !== undefined && vendor !== '' && typeof vendor === 'string' && vendor.trim().length > 200) {
return res.status(400).json({ error: 'vendor must be under 200 chars.' });
}
const vendorVal = ['CARD', 'GRANITE'].includes(workflow_type) ? '' : vendor.trim();
try {
const { rows: origRows } = await pool.query(
'SELECT * FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2',
[id, req.user.id]
);
const original = origRows[0];
if (!original) {
return res.status(404).json({ error: 'Queue item not found.' });
}
if (original.status !== 'complete') {
return res.status(400).json({ error: 'Only completed queue items can be redirected.' });
}
const { rows } = await pool.query(
`INSERT INTO ivanti_todo_queue
(user_id, finding_id, finding_title, cves_json, ip_address, hostname, vendor, workflow_type)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING *`,
[req.user.id, original.finding_id, original.finding_title, original.cves_json, original.ip_address, original.hostname, vendorVal, workflow_type]
);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'queue_item_redirected',
entityType: 'ivanti_todo_queue',
entityId: String(original.id),
details: {
original_workflow_type: original.workflow_type,
target_workflow_type: workflow_type,
new_item_id: rows[0].id,
vendor: vendorVal,
},
ipAddress: req.ip,
});
const result = {
...rows[0],
cves: rows[0].cves_json ? JSON.parse(rows[0].cves_json) : [],
};
return res.status(201).json(result);
} catch (err) {
console.error('Error redirecting queue item:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
// DELETE /api/ivanti/todo-queue/completed
router.delete('/completed', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
try {
const result = await pool.query(
"DELETE FROM ivanti_todo_queue WHERE user_id = $1 AND status = 'complete'",
[req.user.id]
);
res.json({ message: 'Completed items cleared.', deleted: result.rowCount });
} catch (err) {
console.error('Error clearing completed queue items:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
// DELETE /api/ivanti/todo-queue/:id
router.delete('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params;
try {
const { rows } = await pool.query(
'SELECT id FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2',
[id, req.user.id]
);
if (!rows[0]) {
return res.status(404).json({ error: 'Queue item not found.' });
}
await pool.query(
'DELETE FROM ivanti_todo_queue WHERE id = $1 AND user_id = $2',
[id, req.user.id]
);
res.json({ message: 'Queue item deleted.' });
} catch (err) {
console.error(err);
res.status(500).json({ error: 'Internal server error.' });
}
});
return router;
}
module.exports = createIvantiTodoQueueRouter;

View File

@@ -1,83 +1,17 @@
// Ivanti / RiskSense Workflow Routes // Ivanti / RiskSense Workflow Routes
// Data is cached in SQLite and refreshed on a daily schedule or on-demand. // Data is cached in PostgreSQL and refreshed on a daily schedule or on-demand.
// Auth: x-api-key header (confirmed via platform4.risksense.com/doc/swagger.json)
// Error codes: 401 bad key, 419 insufficient privileges, 429 rate limited
const express = require('express'); const express = require('express');
const https = require('https'); const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const { ivantiPost } = require('../helpers/ivantiApi');
const IVANTI_URL_BASE = 'https://platform4.risksense.com/api/v1';
const SYNC_INTERVAL_MS = 24 * 60 * 60 * 1000; // 24 hours const SYNC_INTERVAL_MS = 24 * 60 * 60 * 1000; // 24 hours
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// HTTP helper — uses Node's https module directly so we can toggle // Core sync — calls Ivanti API, stores result in PostgreSQL
// rejectUnauthorized for Charter's SSL inspection proxy (IVANTI_SKIP_TLS=true)
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
function ivantiPost(urlPath, body, apiKey, skipTls) { async function syncWorkflows() {
const bodyStr = JSON.stringify(body);
const fullUrl = new URL(IVANTI_URL_BASE + urlPath);
return new Promise((resolve, reject) => {
const options = {
hostname: fullUrl.hostname,
path: fullUrl.pathname + fullUrl.search,
method: 'POST',
headers: {
'accept': '*/*',
'content-type': 'application/json',
'x-api-key': apiKey,
'x-http-client-type': 'browser',
'content-length': Buffer.byteLength(bodyStr)
},
rejectUnauthorized: !skipTls,
timeout: 15000
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => resolve({ status: res.statusCode, body: data }));
});
req.on('timeout', () => req.destroy(new Error('Request timed out')));
req.on('error', reject);
req.write(bodyStr);
req.end();
});
}
// ---------------------------------------------------------------------------
// Ensure the sync state table exists (idempotent — safe to call on every start)
// ---------------------------------------------------------------------------
function initTable(db) {
return new Promise((resolve, reject) => {
db.serialize(() => {
db.run(`
CREATE TABLE IF NOT EXISTS ivanti_sync_state (
id INTEGER PRIMARY KEY CHECK (id = 1),
total INTEGER DEFAULT 0,
workflows_json TEXT DEFAULT '[]',
synced_at DATETIME,
sync_status TEXT DEFAULT 'never',
error_message TEXT
)
`, (err) => { if (err) return reject(err); });
db.run(`
INSERT OR IGNORE INTO ivanti_sync_state (id, total, workflows_json, sync_status)
VALUES (1, 0, '[]', 'never')
`, (err) => {
if (err) reject(err);
else resolve();
});
});
});
}
// ---------------------------------------------------------------------------
// Core sync — calls Ivanti API, stores result in SQLite
// ---------------------------------------------------------------------------
async function syncWorkflows(db) {
const apiKey = process.env.IVANTI_API_KEY; const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550'; const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const firstName = process.env.IVANTI_FIRST_NAME || ''; const firstName = process.env.IVANTI_FIRST_NAME || '';
@@ -87,12 +21,10 @@ async function syncWorkflows(db) {
if (!apiKey) { if (!apiKey) {
const errMsg = 'IVANTI_API_KEY not set in .env — skipping sync'; const errMsg = 'IVANTI_API_KEY not set in .env — skipping sync';
console.warn('[Ivanti]', errMsg); console.warn('[Ivanti]', errMsg);
await new Promise((resolve) => { await pool.query(
db.run( `UPDATE ivanti_sync_state SET sync_status='error', error_message=$1, synced_at=NOW() WHERE id=1`,
`UPDATE ivanti_sync_state SET sync_status='error', error_message=?, synced_at=datetime('now') WHERE id=1`, [errMsg]
[errMsg], resolve );
);
});
return; return;
} }
@@ -144,7 +76,6 @@ async function syncWorkflows(db) {
const data = JSON.parse(result.body); const data = JSON.parse(result.body);
// Spring Data REST format: { _embedded: { workflowBatches: [...] }, page: { totalElements, ... } }
let total = 0; let total = 0;
let workflows = []; let workflows = [];
@@ -164,105 +95,99 @@ async function syncWorkflows(db) {
total = data.length; total = data.length;
} }
await new Promise((resolve, reject) => { await pool.query(
db.run( `UPDATE ivanti_sync_state
`UPDATE ivanti_sync_state SET total=$1, workflows_json=$2, synced_at=NOW(), sync_status='success', error_message=NULL
SET total=?, workflows_json=?, synced_at=datetime('now'), sync_status='success', error_message=NULL WHERE id=1`,
WHERE id=1`, [total, JSON.stringify(workflows)]
[total, JSON.stringify(workflows)], );
(err) => { if (err) reject(err); else resolve(); }
);
});
console.log(`[Ivanti] Sync complete — ${total} workflows`); console.log(`[Ivanti] Sync complete — ${total} workflows`);
} catch (err) { } catch (err) {
const msg = err.message || 'Unknown error'; const msg = err.message || 'Unknown error';
console.error('[Ivanti] Sync failed:', msg); console.error('[Ivanti] Sync failed:', msg);
await new Promise((resolve) => { await pool.query(
db.run( `UPDATE ivanti_sync_state SET sync_status='error', error_message=$1, synced_at=NOW() WHERE id=1`,
`UPDATE ivanti_sync_state SET sync_status='error', error_message=?, synced_at=datetime('now') WHERE id=1`, [msg]
[msg], resolve );
);
});
} }
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Scheduler — runs sync immediately if >24h stale, then every 24h // Scheduler — runs sync immediately if >24h stale, then every 24h
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
function scheduleSync(db) { async function scheduleSync() {
db.get('SELECT synced_at FROM ivanti_sync_state WHERE id = 1', (err, row) => { try {
if (err || !row || !row.synced_at) { const { rows } = await pool.query('SELECT synced_at FROM ivanti_sync_state WHERE id = 1');
syncWorkflows(db); const row = rows[0];
if (!row || !row.synced_at) {
syncWorkflows();
} else { } else {
const lastSync = new Date(row.synced_at.replace(' ', 'T') + 'Z'); const lastSync = new Date(row.synced_at);
const hoursSince = (Date.now() - lastSync.getTime()) / (1000 * 60 * 60); const hoursSince = (Date.now() - lastSync.getTime()) / (1000 * 60 * 60);
if (hoursSince >= 24) { if (hoursSince >= 24) {
syncWorkflows(db); syncWorkflows();
} else { } else {
const hoursUntil = (24 - hoursSince).toFixed(1); const hoursUntil = (24 - hoursSince).toFixed(1);
console.log(`[Ivanti] Last sync ${hoursSince.toFixed(1)}h ago — next auto-sync in ${hoursUntil}h`); console.log(`[Ivanti] Last sync ${hoursSince.toFixed(1)}h ago — next auto-sync in ${hoursUntil}h`);
} }
} }
}); } catch (err) {
console.error('[Ivanti] Schedule check failed:', err);
syncWorkflows();
}
setInterval(() => syncWorkflows(db), SYNC_INTERVAL_MS); setInterval(() => syncWorkflows(), SYNC_INTERVAL_MS);
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Helper — read current state from DB and return as JSON-ready object // Helper — read current state from DB and return as JSON-ready object
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
function readState(db) { async function readState() {
return new Promise((resolve, reject) => { const { rows } = await pool.query(
db.get( 'SELECT total, workflows_json, synced_at, sync_status, error_message FROM ivanti_sync_state WHERE id = 1'
'SELECT total, workflows_json, synced_at, sync_status, error_message FROM ivanti_sync_state WHERE id = 1', );
(err, row) => { const row = rows[0];
if (err) return reject(err); if (!row) return { total: 0, workflows: [], synced_at: null, sync_status: 'never', error_message: null };
if (!row) return resolve({ total: 0, workflows: [], synced_at: null, sync_status: 'never', error_message: null });
let workflows = []; let workflows = [];
try { workflows = JSON.parse(row.workflows_json || '[]'); } catch (_) { /* leave empty */ } try { workflows = JSON.parse(row.workflows_json || '[]'); } catch (_) { /* leave empty */ }
resolve({ return {
total: row.total || 0, total: row.total || 0,
workflows, workflows,
synced_at: row.synced_at, synced_at: row.synced_at,
sync_status: row.sync_status, sync_status: row.sync_status,
error_message: row.error_message error_message: row.error_message
}); };
}
);
});
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Router // Router
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
function createIvantiWorkflowsRouter(db, requireAuth) { function createIvantiWorkflowsRouter() {
const router = express.Router(); const router = express.Router();
// Init table and kick off scheduler (fire-and-forget on startup) // Kick off scheduler (fire-and-forget on startup)
initTable(db) scheduleSync().catch((err) => console.error('[Ivanti] Init failed:', err));
.then(() => scheduleSync(db))
.catch((err) => console.error('[Ivanti] Init failed:', err));
// All routes require authentication // All routes require authentication
router.use(requireAuth(db)); router.use(requireAuth());
// GET / — return cached data (fast, no external call) // GET / — return cached data (fast, no external call)
router.get('/', async (req, res) => { router.get('/', async (req, res) => {
try { try {
res.json(await readState(db)); res.json(await readState());
} catch { } catch {
res.status(500).json({ error: 'Database error reading sync state' }); res.status(500).json({ error: 'Database error reading sync state' });
} }
}); });
// POST /sync — trigger an immediate sync, await completion, return fresh state // POST /sync — trigger an immediate sync, await completion, return fresh state
router.post('/sync', async (req, res) => { router.post('/sync', requireGroup('Admin', 'Standard_User'), async (req, res) => {
await syncWorkflows(db); await syncWorkflows();
try { try {
res.json(await readState(db)); res.json(await readState());
} catch { } catch {
res.status(500).json({ error: 'Sync ran but could not read updated state' }); res.status(500).json({ error: 'Sync ran but could not read updated state' });
} }

View File

@@ -0,0 +1,581 @@
// routes/jiraTickets.js
// Jira ticket CRUD + Jira REST API integration endpoints.
// Extracted from server.js inline endpoints and extended with live Jira
// operations (lookup, sync, create-in-jira, connection test).
//
// Charter Jira REST API compliance:
// - All GETs include explicit field lists (no /rest/api/2/field)
// - Sync uses bulk JQL search, not one-issue-at-a-time GETs
// - No /rest/api/2/issue/bulk — updates are one at a time
// - Inter-request delays enforced in jiraApi.js (1s GET, 2s write)
// - Rate limits enforced client-side (1440/day, 60/min burst)
const express = require('express');
const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog');
const jiraApi = require('../helpers/jiraApi');
// Validation helpers
const CVE_ID_PATTERN = /^CVE-\d{4}-\d{4,}$/;
const VALID_TICKET_STATUSES = ['Open', 'In Progress', 'Closed'];
function isValidCveId(cveId) {
return typeof cveId === 'string' && CVE_ID_PATTERN.test(cveId);
}
function isValidVendor(vendor) {
return typeof vendor === 'string' && vendor.trim().length > 0 && vendor.length <= 200;
}
function createJiraTicketsRouter() {
const router = express.Router();
// -----------------------------------------------------------------------
// Jira API integration endpoints
// -----------------------------------------------------------------------
router.get('/connection-test', requireAuth(), requireGroup('Admin'), async (req, res) => {
if (!jiraApi.isConfigured) {
return res.status(503).json({ error: 'Jira API is not configured. Set JIRA_BASE_URL and credentials in backend/.env.' });
}
try {
const result = await jiraApi.testConnection();
if (result.ok) {
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_connection_test',
entityType: 'jira_integration',
entityId: null,
details: { success: true, user: result.user.name },
ipAddress: req.ip
});
return res.json({ connected: true, user: result.user });
}
return res.status(502).json({ connected: false, status: result.status, error: result.body || result.error });
} catch (err) {
return res.status(502).json({ connected: false, error: err.message });
}
});
router.get('/rate-limit', requireAuth(), requireGroup('Admin'), (req, res) => {
res.json(jiraApi.getRateLimitStatus());
});
router.get('/lookup/:issueKey', requireAuth(), async (req, res) => {
if (!jiraApi.isConfigured) {
return res.status(503).json({ error: 'Jira API is not configured.' });
}
const { issueKey } = req.params;
if (!issueKey || !/^[A-Z][A-Z0-9_]+-\d+$/.test(issueKey)) {
return res.status(400).json({ error: 'Invalid Jira issue key format. Expected PROJECT-123.' });
}
try {
const result = await jiraApi.getIssue(issueKey);
if (result.ok) {
const issue = result.data;
return res.json({
key: issue.key,
summary: issue.fields.summary,
status: issue.fields.status ? issue.fields.status.name : null,
assignee: issue.fields.assignee ? issue.fields.assignee.displayName : null,
priority: issue.fields.priority ? issue.fields.priority.name : null,
issuetype: issue.fields.issuetype ? issue.fields.issuetype.name : null,
created: issue.fields.created,
updated: issue.fields.updated,
self: issue.self
});
}
if (result.rateLimited) {
return res.status(429).json({ error: 'Jira rate limit exceeded. Try again later.' });
}
return res.status(result.status === 404 ? 404 : 502).json({
error: result.status === 404 ? 'Issue not found in Jira.' : 'Jira API error.',
details: result.body
});
} catch (err) {
return res.status(502).json({ error: err.message });
}
});
router.post('/create-in-jira', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!jiraApi.isConfigured) {
return res.status(503).json({ error: 'Jira API is not configured.' });
}
const { cve_id, vendor, summary, description, project_key, issue_type } = req.body;
if (!cve_id || !isValidCveId(cve_id)) {
return res.status(400).json({ error: 'Valid CVE ID is required.' });
}
if (!vendor || !isValidVendor(vendor)) {
return res.status(400).json({ error: 'Valid vendor is required.' });
}
if (!summary || typeof summary !== 'string' || summary.trim().length === 0 || summary.length > 255) {
return res.status(400).json({ error: 'Summary is required (max 255 chars).' });
}
const projectKey = project_key || jiraApi.JIRA_PROJECT_KEY;
const issueType = issue_type || jiraApi.JIRA_ISSUE_TYPE;
if (!projectKey) {
return res.status(400).json({ error: 'Project key is required. Set JIRA_PROJECT_KEY in .env or provide project_key in request.' });
}
const fields = {
project: { key: projectKey },
summary: summary.trim(),
issuetype: { name: issueType }
};
if (description) {
fields.description = description;
}
try {
const result = await jiraApi.createIssue(fields);
if (!result.ok) {
if (result.rateLimited) {
return res.status(429).json({ error: 'Jira rate limit exceeded. Try again later.' });
}
return res.status(502).json({ error: 'Failed to create Jira issue.', details: result.body });
}
const jiraIssue = result.data;
const ticketKey = jiraIssue.key;
const jiraUrl = jiraIssue.self
? jiraIssue.self.replace(/\/rest\/api\/2\/issue\/.*/, `/browse/${ticketKey}`)
: null;
try {
const { rows } = await pool.query(
`INSERT INTO jira_tickets (cve_id, vendor, ticket_key, url, summary, status, jira_id, jira_status, last_synced_at, created_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, NOW(), $9)
RETURNING id`,
[cve_id, vendor, ticketKey, jiraUrl, summary.trim(), 'Open', jiraIssue.id, 'Open', req.user.id]
);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_ticket_create_via_api',
entityType: 'jira_ticket',
entityId: rows[0].id.toString(),
details: { cve_id, vendor, ticket_key: ticketKey, jira_id: jiraIssue.id, project_key: projectKey },
ipAddress: req.ip
});
res.status(201).json({
id: rows[0].id,
ticket_key: ticketKey,
jira_url: jiraUrl,
message: 'Jira issue created and linked successfully'
});
} catch (dbErr) {
console.error('Error saving local Jira ticket record:', dbErr);
return res.status(207).json({
warning: 'Issue created in Jira but local record failed to save.',
jira_key: ticketKey,
jira_url: jiraUrl,
error: dbErr.message
});
}
} catch (err) {
return res.status(502).json({ error: err.message });
}
});
router.post('/sync-all', requireAuth(), requireGroup('Admin'), async (req, res) => {
if (!jiraApi.isConfigured) {
return res.status(503).json({ error: 'Jira API is not configured.' });
}
try {
const { rows: tickets } = await pool.query(
"SELECT * FROM jira_tickets WHERE ticket_key IS NOT NULL AND ticket_key != ''"
);
if (tickets.length === 0) {
return res.json({ synced: 0, failed: 0, skipped: 0, unchanged: 0, errors: [] });
}
const results = { synced: 0, failed: 0, skipped: 0, unchanged: 0, errors: [] };
const BATCH_SIZE = 100;
const batches = [];
for (let i = 0; i < tickets.length; i += BATCH_SIZE) {
batches.push(tickets.slice(i, i + BATCH_SIZE));
}
for (const batch of batches) {
const rateStatus = jiraApi.getRateLimitStatus();
if (rateStatus.burst.remaining <= 5 || rateStatus.daily.remaining <= 10) {
const remaining = tickets.length - results.synced - results.failed - results.unchanged;
results.skipped += remaining;
results.errors.push('Rate limit approaching — stopped sync early to preserve budget.');
break;
}
const keys = batch.map(t => t.ticket_key);
try {
const result = await jiraApi.searchIssuesByKeys(keys);
if (!result.ok) {
if (result.rateLimited) {
results.skipped += batch.length;
results.errors.push('Jira rate limit hit during sync.');
break;
}
results.failed += batch.length;
results.errors.push(`Batch search failed: HTTP ${result.status}`);
continue;
}
const issueMap = {};
for (const issue of (result.data.issues || [])) {
issueMap[issue.key] = issue;
}
for (const ticket of batch) {
const issue = issueMap[ticket.ticket_key];
if (!issue) {
results.unchanged++;
continue;
}
const jiraStatus = issue.fields.status ? issue.fields.status.name : null;
const jiraSummary = issue.fields.summary || ticket.summary;
const localStatus = mapJiraStatusToLocal(jiraStatus);
try {
await pool.query(
`UPDATE jira_tickets SET summary = $1, status = $2, jira_status = $3, last_synced_at = NOW(), updated_at = NOW() WHERE id = $4`,
[jiraSummary, localStatus, jiraStatus, ticket.id]
);
results.synced++;
} catch (dbErr) {
results.failed++;
results.errors.push(`${ticket.ticket_key}: DB update failed — ${dbErr.message}`);
}
}
} catch (searchErr) {
results.failed += batch.length;
results.errors.push(`Batch search error: ${searchErr.message}`);
}
}
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_sync_all',
entityType: 'jira_integration',
entityId: null,
details: results,
ipAddress: req.ip
});
res.json(results);
} catch (err) {
console.error(err);
return res.status(500).json({ error: 'Internal server error.' });
}
});
router.post('/:id/sync', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
if (!jiraApi.isConfigured) {
return res.status(503).json({ error: 'Jira API is not configured.' });
}
const { id } = req.params;
try {
const { rows } = await pool.query('SELECT * FROM jira_tickets WHERE id = $1', [id]);
const ticket = rows[0];
if (!ticket) {
return res.status(404).json({ error: 'JIRA ticket not found.' });
}
if (!ticket.ticket_key) {
return res.status(400).json({ error: 'Ticket has no Jira key to sync.' });
}
const result = await jiraApi.getIssue(ticket.ticket_key);
if (!result.ok) {
if (result.rateLimited) {
return res.status(429).json({ error: 'Jira rate limit exceeded. Try again later.' });
}
return res.status(502).json({ error: 'Failed to fetch issue from Jira.', details: result.body });
}
const issue = result.data;
const jiraStatus = issue.fields.status ? issue.fields.status.name : null;
const jiraSummary = issue.fields.summary || ticket.summary;
const localStatus = mapJiraStatusToLocal(jiraStatus);
await pool.query(
`UPDATE jira_tickets SET summary = $1, status = $2, jira_status = $3, last_synced_at = NOW(), updated_at = NOW() WHERE id = $4`,
[jiraSummary, localStatus, jiraStatus, id]
);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_ticket_sync',
entityType: 'jira_ticket',
entityId: id,
details: { ticket_key: ticket.ticket_key, jira_status: jiraStatus, local_status: localStatus },
ipAddress: req.ip
});
res.json({
message: 'Ticket synced with Jira',
ticket_key: ticket.ticket_key,
jira_status: jiraStatus,
local_status: localStatus,
summary: jiraSummary
});
} catch (err) {
console.error(err);
return res.status(500).json({ error: 'Internal server error.' });
}
});
// -----------------------------------------------------------------------
// Local CRUD endpoints
// -----------------------------------------------------------------------
router.get('/', requireAuth(), async (req, res) => {
const { cve_id, vendor, status } = req.query;
let query = 'SELECT * FROM jira_tickets WHERE 1=1';
const params = [];
let paramIndex = 1;
if (cve_id) {
query += ` AND cve_id = $${paramIndex++}`;
params.push(cve_id);
}
if (vendor) {
query += ` AND vendor = $${paramIndex++}`;
params.push(vendor);
}
if (status) {
query += ` AND status = $${paramIndex++}`;
params.push(status);
}
query += ' ORDER BY created_at DESC';
try {
const { rows } = await pool.query(query, params);
res.json(rows);
} catch (err) {
console.error('Error fetching JIRA tickets:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
router.post('/', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { cve_id, vendor, ticket_key, url, summary, status } = req.body;
if (!cve_id || !isValidCveId(cve_id)) {
return res.status(400).json({ error: 'Valid CVE ID is required.' });
}
if (!vendor || !isValidVendor(vendor)) {
return res.status(400).json({ error: 'Valid vendor is required.' });
}
if (!ticket_key || typeof ticket_key !== 'string' || ticket_key.trim().length === 0 || ticket_key.length > 50) {
return res.status(400).json({ error: 'Ticket key is required (max 50 chars).' });
}
if (url && (typeof url !== 'string' || url.length > 500)) {
return res.status(400).json({ error: 'URL must be under 500 characters.' });
}
if (summary && (typeof summary !== 'string' || summary.length > 500)) {
return res.status(400).json({ error: 'Summary must be under 500 characters.' });
}
if (status && !VALID_TICKET_STATUSES.includes(status)) {
return res.status(400).json({ error: `Status must be one of: ${VALID_TICKET_STATUSES.join(', ')}` });
}
const ticketStatus = status || 'Open';
try {
const { rows } = await pool.query(
`INSERT INTO jira_tickets (cve_id, vendor, ticket_key, url, summary, status, created_by)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id`,
[cve_id, vendor, ticket_key.trim(), url || null, summary || null, ticketStatus, req.user.id]
);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_ticket_create',
entityType: 'jira_ticket',
entityId: rows[0].id.toString(),
details: { cve_id, vendor, ticket_key, status: ticketStatus },
ipAddress: req.ip
});
res.status(201).json({
id: rows[0].id,
message: 'JIRA ticket created successfully'
});
} catch (err) {
console.error('Error creating JIRA ticket:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
router.put('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params;
const { ticket_key, url, summary, status } = req.body;
if (ticket_key !== undefined && (typeof ticket_key !== 'string' || ticket_key.trim().length === 0 || ticket_key.length > 50)) {
return res.status(400).json({ error: 'Ticket key must be under 50 chars.' });
}
if (url !== undefined && url !== null && (typeof url !== 'string' || url.length > 500)) {
return res.status(400).json({ error: 'URL must be under 500 characters.' });
}
if (summary !== undefined && summary !== null && (typeof summary !== 'string' || summary.length > 500)) {
return res.status(400).json({ error: 'Summary must be under 500 characters.' });
}
if (status !== undefined && !VALID_TICKET_STATUSES.includes(status)) {
return res.status(400).json({ error: `Status must be one of: ${VALID_TICKET_STATUSES.join(', ')}` });
}
const fields = [];
const values = [];
let paramIndex = 1;
if (ticket_key !== undefined) { fields.push(`ticket_key = $${paramIndex++}`); values.push(ticket_key.trim()); }
if (url !== undefined) { fields.push(`url = $${paramIndex++}`); values.push(url); }
if (summary !== undefined) { fields.push(`summary = $${paramIndex++}`); values.push(summary); }
if (status !== undefined) { fields.push(`status = $${paramIndex++}`); values.push(status); }
if (fields.length === 0) {
return res.status(400).json({ error: 'No fields to update.' });
}
fields.push('updated_at = NOW()');
values.push(id);
try {
const { rows } = await pool.query('SELECT * FROM jira_tickets WHERE id = $1', [id]);
const existing = rows[0];
if (!existing) {
return res.status(404).json({ error: 'JIRA ticket not found.' });
}
const result = await pool.query(
`UPDATE jira_tickets SET ${fields.join(', ')} WHERE id = $${paramIndex}`,
values
);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_ticket_update',
entityType: 'jira_ticket',
entityId: id,
details: { before: existing, changes: req.body },
ipAddress: req.ip
});
res.json({ message: 'JIRA ticket updated successfully', changes: result.rowCount });
} catch (err) {
console.error('Error updating JIRA ticket:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
router.delete('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params;
try {
const { rows } = await pool.query('SELECT * FROM jira_tickets WHERE id = $1', [id]);
const ticket = rows[0];
if (!ticket) {
return res.status(404).json({ error: 'JIRA ticket not found.' });
}
// Admin bypasses all delete restrictions
if (req.user.group === 'Admin') {
return performJiraDelete();
}
// Standard_User: ownership check
if (ticket.created_by && ticket.created_by !== req.user.id) {
return res.status(403).json({ error: 'You can only delete resources you created' });
}
// Standard_User: compliance linkage check
const ticketKey = ticket.ticket_key;
try {
const { rows: compLinks } = await pool.query(
`SELECT ci.id, ci.extra_json
FROM compliance_items ci
JOIN compliance_uploads cu ON ci.upload_id = cu.id
WHERE ci.status = 'active' AND ci.extra_json ILIKE $1`,
[`%${ticketKey}%`]
);
const isLinked = (compLinks || []).some(cl => {
const json = cl.extra_json || '';
return json.includes(ticketKey);
});
if (isLinked) {
return res.status(403).json({ error: 'Cannot delete ticket linked to compliance report. Contact an admin.' });
}
} catch (compErr) {
if (!compErr.message.includes('does not exist')) throw compErr;
}
return performJiraDelete();
async function performJiraDelete() {
await pool.query('DELETE FROM jira_tickets WHERE id = $1', [id]);
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'jira_ticket_delete',
entityType: 'jira_ticket',
entityId: id,
details: { ticket_key: ticket.ticket_key, cve_id: ticket.cve_id, vendor: ticket.vendor },
ipAddress: req.ip
});
res.json({ message: 'JIRA ticket deleted successfully' });
}
} catch (err) {
console.error('Error deleting JIRA ticket:', err);
res.status(500).json({ error: 'Internal server error.' });
}
});
return router;
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function mapJiraStatusToLocal(jiraStatus) {
if (!jiraStatus) return 'Open';
const lower = jiraStatus.toLowerCase();
if (['closed', 'done', 'resolved', 'complete', 'completed', 'cancelled', 'canceled', "won't do", 'declined'].some(s => lower.includes(s))) {
return 'Closed';
}
if (['in progress', 'in review', 'in development', 'in testing', 'review', 'testing', 'dev', 'active', 'implementing'].some(s => lower.includes(s))) {
return 'In Progress';
}
return 'Open';
}
module.exports = createJiraTicketsRouter;

View File

@@ -1,10 +1,11 @@
const express = require('express'); const express = require('express');
const path = require('path'); const path = require('path');
const fs = require('fs'); const fs = require('fs');
const { requireAuth, requireRole } = require('../middleware/auth'); const pool = require('../db');
const { requireAuth, requireGroup } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog'); const logAudit = require('../helpers/auditLog');
function createKnowledgeBaseRouter(db, upload) { function createKnowledgeBaseRouter(upload) {
const router = express.Router(); const router = express.Router();
// Helper to sanitize filename // Helper to sanitize filename
@@ -39,8 +40,8 @@ function createKnowledgeBaseRouter(db, upload) {
return ALLOWED_EXTENSIONS.has(ext); return ALLOWED_EXTENSIONS.has(ext);
} }
// POST /api/knowledge-base/upload - Upload new document // POST /api/knowledge-base/upload
router.post('/upload', requireAuth(db), requireRole(db, 'editor', 'admin'), (req, res, next) => { router.post('/upload', requireAuth(), requireGroup('Admin', 'Standard_User'), (req, res, next) => {
upload.single('file')(req, res, (err) => { upload.single('file')(req, res, (err) => {
if (err) { if (err) {
console.error('[KB Upload] Multer error:', err); console.error('[KB Upload] Multer error:', err);
@@ -58,7 +59,6 @@ function createKnowledgeBaseRouter(db, upload) {
const uploadedFile = req.file; const uploadedFile = req.file;
const { title, description, category } = req.body; const { title, description, category } = req.body;
// Validate required fields
if (!title || !title.trim()) { if (!title || !title.trim()) {
console.error('[KB Upload] Error: Title is missing'); console.error('[KB Upload] Error: Title is missing');
if (uploadedFile) fs.unlinkSync(uploadedFile.path); if (uploadedFile) fs.unlinkSync(uploadedFile.path);
@@ -69,7 +69,6 @@ function createKnowledgeBaseRouter(db, upload) {
return res.status(400).json({ error: 'No file uploaded' }); return res.status(400).json({ error: 'No file uploaded' });
} }
// Validate file type
if (!isValidFileType(uploadedFile.originalname)) { if (!isValidFileType(uploadedFile.originalname)) {
fs.unlinkSync(uploadedFile.path); fs.unlinkSync(uploadedFile.path);
return res.status(400).json({ error: 'File type not allowed' }); return res.status(400).json({ error: 'File type not allowed' });
@@ -80,148 +79,125 @@ function createKnowledgeBaseRouter(db, upload) {
const slug = generateSlug(title); const slug = generateSlug(title);
const kbDir = path.join(__dirname, '..', 'uploads', 'knowledge_base'); const kbDir = path.join(__dirname, '..', 'uploads', 'knowledge_base');
// Create directory if it doesn't exist
if (!fs.existsSync(kbDir)) {
fs.mkdirSync(kbDir, { recursive: true });
}
const filename = `${timestamp}_${sanitizedName}`; const filename = `${timestamp}_${sanitizedName}`;
const filePath = path.join(kbDir, filename); const filePath = path.join(kbDir, filename);
try { try {
// Move uploaded file to permanent location
fs.renameSync(uploadedFile.path, filePath);
// Check if slug already exists // Check if slug already exists
db.get('SELECT id FROM knowledge_base WHERE slug = ?', [slug], (err, row) => { const { rows: existingRows } = await pool.query(
if (err) { 'SELECT id FROM knowledge_base WHERE slug = $1', [slug]
fs.unlinkSync(filePath); );
console.error('Error checking slug:', err);
return res.status(500).json({ error: 'Database error' }); const finalSlug = existingRows.length > 0 ? `${slug}-${timestamp}` : slug;
// Insert new knowledge base entry
const { rows } = await pool.query(
`INSERT INTO knowledge_base (
title, slug, description, category, file_path, file_name,
file_type, file_size, created_by
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
RETURNING id`,
[
title.trim(),
finalSlug,
description || null,
category || 'General',
filePath,
sanitizedName,
uploadedFile.mimetype,
uploadedFile.size,
req.user.id
]
);
// DB insert succeeded — now move file to permanent location
try {
if (!fs.existsSync(kbDir)) {
fs.mkdirSync(kbDir, { recursive: true });
} }
fs.renameSync(uploadedFile.path, filePath);
} catch (moveErr) {
console.error('Error moving file to permanent location:', moveErr);
}
// If slug exists, append timestamp to make it unique logAudit({
const finalSlug = row ? `${slug}-${timestamp}` : slug; userId: req.user.id,
username: req.user.username,
action: 'CREATE_KB_ARTICLE',
entityType: 'knowledge_base',
entityId: String(rows[0].id),
details: { title: title.trim(), filename: sanitizedName },
ipAddress: req.ip
});
// Insert new knowledge base entry res.json({
const insertSql = ` success: true,
INSERT INTO knowledge_base ( id: rows[0].id,
title, slug, description, category, file_path, file_name, title: title.trim(),
file_type, file_size, created_by slug: finalSlug,
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) category: category || 'General'
`;
db.run(
insertSql,
[
title.trim(),
finalSlug,
description || null,
category || 'General',
filePath,
sanitizedName,
uploadedFile.mimetype,
uploadedFile.size,
req.user.id
],
function (err) {
if (err) {
fs.unlinkSync(filePath);
console.error('Error inserting knowledge base entry:', err);
return res.status(500).json({ error: 'Failed to save document metadata' });
}
// Log audit entry
logAudit(
db,
req.user.id,
req.user.username,
'CREATE_KB_ARTICLE',
'knowledge_base',
this.lastID,
JSON.stringify({ title: title.trim(), filename: sanitizedName }),
req.ip
);
res.json({
success: true,
id: this.lastID,
title: title.trim(),
slug: finalSlug,
category: category || 'General'
});
}
);
}); });
} catch (error) { } catch (error) {
// Clean up file on error if (uploadedFile && fs.existsSync(uploadedFile.path)) fs.unlinkSync(uploadedFile.path);
if (fs.existsSync(filePath)) fs.unlinkSync(filePath);
console.error('Error uploading knowledge base document:', error); console.error('Error uploading knowledge base document:', error);
res.status(500).json({ error: error.message || 'Failed to upload document' }); res.status(500).json({ error: error.message || 'Failed to upload document' });
} }
}); });
// GET /api/knowledge-base - List all articles // GET /api/knowledge-base
router.get('/', requireAuth(db), (req, res) => { router.get('/', requireAuth(), async (req, res) => {
const sql = ` try {
SELECT const { rows } = await pool.query(`
kb.id, kb.title, kb.slug, kb.description, kb.category, SELECT
kb.file_name, kb.file_type, kb.file_size, kb.created_at, kb.updated_at, kb.id, kb.title, kb.slug, kb.description, kb.category,
u.username as created_by_username kb.file_name, kb.file_type, kb.file_size, kb.created_at, kb.updated_at,
FROM knowledge_base kb u.username as created_by_username
LEFT JOIN users u ON kb.created_by = u.id FROM knowledge_base kb
ORDER BY kb.created_at DESC LEFT JOIN users u ON kb.created_by = u.id
`; ORDER BY kb.created_at DESC
`);
db.all(sql, [], (err, rows) => {
if (err) {
console.error('Error fetching knowledge base articles:', err);
return res.status(500).json({ error: 'Failed to fetch articles' });
}
res.json(rows); res.json(rows);
}); } catch (err) {
console.error('Error fetching knowledge base articles:', err);
res.status(500).json({ error: 'Failed to fetch articles' });
}
}); });
// GET /api/knowledge-base/:id - Get single article details // GET /api/knowledge-base/:id
router.get('/:id', requireAuth(db), (req, res) => { router.get('/:id', requireAuth(), async (req, res) => {
const { id } = req.params; const { id } = req.params;
const sql = ` try {
SELECT const { rows } = await pool.query(`
kb.id, kb.title, kb.slug, kb.description, kb.category, SELECT
kb.file_name, kb.file_type, kb.file_size, kb.created_at, kb.updated_at, kb.id, kb.title, kb.slug, kb.description, kb.category,
u.username as created_by_username kb.file_name, kb.file_type, kb.file_size, kb.created_at, kb.updated_at,
FROM knowledge_base kb u.username as created_by_username
LEFT JOIN users u ON kb.created_by = u.id FROM knowledge_base kb
WHERE kb.id = ? LEFT JOIN users u ON kb.created_by = u.id
`; WHERE kb.id = $1
`, [id]);
db.get(sql, [id], (err, row) => { if (!rows[0]) {
if (err) {
console.error('Error fetching article:', err);
return res.status(500).json({ error: 'Failed to fetch article' });
}
if (!row) {
return res.status(404).json({ error: 'Article not found' }); return res.status(404).json({ error: 'Article not found' });
} }
res.json(row); res.json(rows[0]);
}); } catch (err) {
console.error('Error fetching article:', err);
res.status(500).json({ error: 'Failed to fetch article' });
}
}); });
// GET /api/knowledge-base/:id/content - Get document content for display // GET /api/knowledge-base/:id/content
router.get('/:id/content', requireAuth(db), (req, res) => { router.get('/:id/content', requireAuth(), async (req, res) => {
const { id } = req.params; const { id } = req.params;
const sql = 'SELECT file_path, file_name, file_type FROM knowledge_base WHERE id = ?'; try {
const { rows } = await pool.query(
db.get(sql, [id], (err, row) => { 'SELECT file_path, file_name, file_type FROM knowledge_base WHERE id = $1', [id]
if (err) { );
console.error('Error fetching document:', err); const row = rows[0];
return res.status(500).json({ error: 'Failed to fetch document' });
}
if (!row) { if (!row) {
return res.status(404).json({ error: 'Document not found' }); return res.status(404).json({ error: 'Document not found' });
@@ -231,49 +207,45 @@ function createKnowledgeBaseRouter(db, upload) {
return res.status(404).json({ error: 'File not found on disk' }); return res.status(404).json({ error: 'File not found on disk' });
} }
// Log audit entry logAudit({
logAudit( userId: req.user.id,
db, username: req.user.username,
req.user.id, action: 'VIEW_KB_ARTICLE',
req.user.username, entityType: 'knowledge_base',
'VIEW_KB_ARTICLE', entityId: String(id),
'knowledge_base', details: { filename: row.file_name },
id, ipAddress: req.ip
JSON.stringify({ filename: row.file_name }), });
req.ip
);
// Determine content type for inline display
let contentType = row.file_type || 'application/octet-stream'; let contentType = row.file_type || 'application/octet-stream';
// For markdown files, send as plain text so frontend can parse it
if (row.file_name.endsWith('.md')) { if (row.file_name.endsWith('.md')) {
contentType = 'text/plain; charset=utf-8'; contentType = 'text/plain; charset=utf-8';
} else if (row.file_name.endsWith('.txt')) { } else if (row.file_name.endsWith('.txt')) {
contentType = 'text/plain; charset=utf-8'; contentType = 'text/plain; charset=utf-8';
} }
const safeFileName = row.file_name.replace(/["\r\n\\]/g, '');
res.setHeader('Content-Type', contentType); res.setHeader('Content-Type', contentType);
// Use inline instead of attachment to allow browser to display res.setHeader('Content-Disposition', `inline; filename="${safeFileName}"`);
res.setHeader('Content-Disposition', `inline; filename="${row.file_name}"`);
// Allow iframe embedding from frontend origin
res.removeHeader('X-Frame-Options'); res.removeHeader('X-Frame-Options');
res.setHeader('Content-Security-Policy', "frame-ancestors 'self' http://71.85.90.9:3000 http://localhost:3000"); const corsOrigins = process.env.CORS_ORIGINS ? process.env.CORS_ORIGINS.split(',').join(' ') : 'http://localhost:3000';
res.setHeader('Content-Security-Policy', `frame-ancestors 'self' ${corsOrigins}`);
res.sendFile(row.file_path); res.sendFile(row.file_path);
}); } catch (err) {
console.error('Error fetching document:', err);
res.status(500).json({ error: 'Failed to fetch document' });
}
}); });
// GET /api/knowledge-base/:id/download - Download document // GET /api/knowledge-base/:id/download
router.get('/:id/download', requireAuth(db), (req, res) => { router.get('/:id/download', requireAuth(), async (req, res) => {
const { id } = req.params; const { id } = req.params;
const sql = 'SELECT file_path, file_name, file_type FROM knowledge_base WHERE id = ?'; try {
const { rows } = await pool.query(
db.get(sql, [id], (err, row) => { 'SELECT file_path, file_name, file_type FROM knowledge_base WHERE id = $1', [id]
if (err) { );
console.error('Error fetching document:', err); const row = rows[0];
return res.status(500).json({ error: 'Failed to fetch document' });
}
if (!row) { if (!row) {
return res.status(404).json({ error: 'Document not found' }); return res.status(404).json({ error: 'Document not found' });
@@ -283,67 +255,67 @@ function createKnowledgeBaseRouter(db, upload) {
return res.status(404).json({ error: 'File not found on disk' }); return res.status(404).json({ error: 'File not found on disk' });
} }
// Log audit entry logAudit({
logAudit( userId: req.user.id,
db, username: req.user.username,
req.user.id, action: 'DOWNLOAD_KB_ARTICLE',
req.user.username, entityType: 'knowledge_base',
'DOWNLOAD_KB_ARTICLE', entityId: String(id),
'knowledge_base', details: { filename: row.file_name },
id, ipAddress: req.ip
JSON.stringify({ filename: row.file_name }), });
req.ip
);
const safeDownloadName = row.file_name.replace(/["\r\n\\]/g, '');
res.setHeader('Content-Type', row.file_type || 'application/octet-stream'); res.setHeader('Content-Type', row.file_type || 'application/octet-stream');
res.setHeader('Content-Disposition', `attachment; filename="${row.file_name}"`); res.setHeader('Content-Disposition', `attachment; filename="${safeDownloadName}"`);
res.sendFile(row.file_path); res.sendFile(row.file_path);
}); } catch (err) {
console.error('Error fetching document:', err);
res.status(500).json({ error: 'Failed to fetch document' });
}
}); });
// DELETE /api/knowledge-base/:id - Delete article // DELETE /api/knowledge-base/:id
router.delete('/:id', requireAuth(db), requireRole(db, 'editor', 'admin'), (req, res) => { router.delete('/:id', requireAuth(), requireGroup('Admin', 'Standard_User'), async (req, res) => {
const { id } = req.params; const { id } = req.params;
const sql = 'SELECT file_path, title FROM knowledge_base WHERE id = ?'; try {
const { rows } = await pool.query(
db.get(sql, [id], (err, row) => { 'SELECT file_path, title, created_by FROM knowledge_base WHERE id = $1', [id]
if (err) { );
console.error('Error fetching article for deletion:', err); const row = rows[0];
return res.status(500).json({ error: 'Failed to fetch article' });
}
if (!row) { if (!row) {
return res.status(404).json({ error: 'Article not found' }); return res.status(404).json({ error: 'Article not found' });
} }
// Delete database record // Ownership check: Standard_User can only delete articles they created
db.run('DELETE FROM knowledge_base WHERE id = ?', [id], (err) => { if (req.user.group === 'Standard_User' && row.created_by !== req.user.id) {
if (err) { return res.status(403).json({ error: 'You can only delete resources you created' });
console.error('Error deleting article:', err); }
return res.status(500).json({ error: 'Failed to delete article' });
}
// Delete file await pool.query('DELETE FROM knowledge_base WHERE id = $1', [id]);
if (fs.existsSync(row.file_path)) {
fs.unlinkSync(row.file_path);
}
// Log audit entry // Delete file
logAudit( if (fs.existsSync(row.file_path)) {
db, fs.unlinkSync(row.file_path);
req.user.id, }
req.user.username,
'DELETE_KB_ARTICLE',
'knowledge_base',
id,
JSON.stringify({ title: row.title }),
req.ip
);
res.json({ success: true }); logAudit({
userId: req.user.id,
username: req.user.username,
action: 'DELETE_KB_ARTICLE',
entityType: 'knowledge_base',
entityId: String(id),
details: { title: row.title },
ipAddress: req.ip
}); });
});
res.json({ success: true });
} catch (err) {
console.error('Error deleting article:', err);
res.status(500).json({ error: 'Failed to delete article' });
}
}); });
return router; return router;

View File

@@ -1,13 +1,14 @@
// NVD CVE Lookup Routes // NVD CVE Lookup Routes
const express = require('express'); const express = require('express');
const { requireAuth } = require('../middleware/auth');
const CVE_ID_PATTERN = /^CVE-\d{4}-\d{4,}$/; const CVE_ID_PATTERN = /^CVE-\d{4}-\d{4,}$/;
function createNvdLookupRouter(db, requireAuth) { function createNvdLookupRouter() {
const router = express.Router(); const router = express.Router();
// All routes require authentication // All routes require authentication
router.use(requireAuth(db)); router.use(requireAuth());
// Lookup CVE details from NVD API 2.0 // Lookup CVE details from NVD API 2.0
router.get('/lookup/:cveId', async (req, res) => { router.get('/lookup/:cveId', async (req, res) => {

View File

@@ -1,27 +1,28 @@
// User Management Routes (Admin only) // User Management Routes (Admin only)
const express = require('express'); const express = require('express');
const bcrypt = require('bcryptjs'); const bcrypt = require('bcryptjs');
const pool = require('../db');
const { validateTeams } = require('../helpers/teams');
function createUsersRouter(db, requireAuth, requireRole, logAudit) { function createUsersRouter(requireAuth, requireGroup, logAudit) {
const router = express.Router(); const router = express.Router();
// All routes require admin role // All routes require Admin group
router.use(requireAuth(db), requireRole('admin')); router.use(requireAuth(), requireGroup('Admin'));
// Get all users // Get all users
router.get('/', async (req, res) => { router.get('/', async (req, res) => {
try { try {
const users = await new Promise((resolve, reject) => { const { rows: users } = await pool.query(
db.all( `SELECT id, username, email, user_group AS "group", bu_teams, is_active, created_at, last_login
`SELECT id, username, email, role, is_active, created_at, last_login FROM users ORDER BY created_at DESC`
FROM users ORDER BY created_at DESC`, );
(err, rows) => { // Parse bu_teams into teams array for each user
if (err) reject(err); const usersWithTeams = users.map(u => ({
else resolve(rows); ...u,
} teams: u.bu_teams ? u.bu_teams.split(',').filter(Boolean) : []
); }));
}); res.json(usersWithTeams);
res.json(users);
} catch (err) { } catch (err) {
console.error('Get users error:', err); console.error('Get users error:', err);
res.status(500).json({ error: 'Failed to fetch users' }); res.status(500).json({ error: 'Failed to fetch users' });
@@ -31,23 +32,22 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
// Get single user // Get single user
router.get('/:id', async (req, res) => { router.get('/:id', async (req, res) => {
try { try {
const user = await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.get( `SELECT id, username, email, user_group AS "group", bu_teams, is_active, created_at, last_login
`SELECT id, username, email, role, is_active, created_at, last_login FROM users WHERE id = $1`,
FROM users WHERE id = ?`, [req.params.id]
[req.params.id], );
(err, row) => {
if (err) reject(err); const user = rows[0];
else resolve(row);
}
);
});
if (!user) { if (!user) {
return res.status(404).json({ error: 'User not found' }); return res.status(404).json({ error: 'User not found' });
} }
res.json(user); res.json({
...user,
teams: user.bu_teams ? user.bu_teams.split(',').filter(Boolean) : []
});
} catch (err) { } catch (err) {
console.error('Get user error:', err); console.error('Get user error:', err);
res.status(500).json({ error: 'Failed to fetch user' }); res.status(500).json({ error: 'Failed to fetch user' });
@@ -56,38 +56,47 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
// Create new user // Create new user
router.post('/', async (req, res) => { router.post('/', async (req, res) => {
const { username, email, password, role } = req.body; const { username, email, password, group, bu_teams } = req.body;
const VALID_GROUPS = ['Admin', 'Standard_User', 'Leadership', 'Read_Only'];
if (!username || !email || !password) { if (!username || !email || !password) {
return res.status(400).json({ error: 'Username, email, and password are required' }); return res.status(400).json({ error: 'Username, email, and password are required' });
} }
if (role && !['admin', 'editor', 'viewer'].includes(role)) { const userGroup = group || 'Read_Only';
return res.status(400).json({ error: 'Invalid role. Must be admin, editor, or viewer' });
if (!VALID_GROUPS.includes(userGroup)) {
return res.status(400).json({ error: 'Invalid group. Must be one of: Admin, Standard_User, Leadership, Read_Only' });
}
// Validate bu_teams if provided
const teamsStr = bu_teams || '';
if (teamsStr) {
const teamsResult = validateTeams(teamsStr);
if (!teamsResult.valid) {
return res.status(400).json({ error: `Invalid team(s): ${teamsResult.invalid.join(', ')}. Must be one of: STEAM, ACCESS-ENG, ACCESS-OPS, INTELDEV` });
}
} }
try { try {
const passwordHash = await bcrypt.hash(password, 10); const passwordHash = await bcrypt.hash(password, 10);
const result = await new Promise((resolve, reject) => { const { rows } = await pool.query(
db.run( `INSERT INTO users (username, email, password_hash, user_group, bu_teams)
`INSERT INTO users (username, email, password_hash, role) VALUES ($1, $2, $3, $4, $5)
VALUES (?, ?, ?, ?)`, RETURNING id`,
[username, email, passwordHash, role || 'viewer'], [username, email, passwordHash, userGroup, teamsStr]
function(err) { );
if (err) reject(err);
else resolve({ id: this.lastID });
}
);
});
logAudit(db, { const result = rows[0];
logAudit({
userId: req.user.id, userId: req.user.id,
username: req.user.username, username: req.user.username,
action: 'user_create', action: 'user_create',
entityType: 'user', entityType: 'user',
entityId: String(result.id), entityId: String(result.id),
details: { created_username: username, role: role || 'viewer' }, details: { created_username: username, group: userGroup, bu_teams: teamsStr },
ipAddress: req.ip ipAddress: req.ip
}); });
@@ -97,12 +106,14 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
id: result.id, id: result.id,
username, username,
email, email,
role: role || 'viewer' group: userGroup,
bu_teams: teamsStr,
teams: teamsStr ? teamsStr.split(',').filter(Boolean) : []
} }
}); });
} catch (err) { } catch (err) {
console.error('Create user error:', err); console.error('Create user error:', err);
if (err.message.includes('UNIQUE constraint failed')) { if (err.code === '23505') { // Postgres unique violation
return res.status(409).json({ error: 'Username or email already exists' }); return res.status(409).json({ error: 'Username or email already exists' });
} }
res.status(500).json({ error: 'Failed to create user' }); res.status(500).json({ error: 'Failed to create user' });
@@ -111,46 +122,76 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
// Update user // Update user
router.patch('/:id', async (req, res) => { router.patch('/:id', async (req, res) => {
const { username, email, password, role, is_active } = req.body; const { username, email, password, group, is_active, bu_teams } = req.body;
const VALID_GROUPS = ['Admin', 'Standard_User', 'Leadership', 'Read_Only'];
const userId = req.params.id; const userId = req.params.id;
// Prevent self-demotion from admin // Validate group if provided
if (userId == req.user.id && role && role !== 'admin') { if (group && !VALID_GROUPS.includes(group)) {
return res.status(400).json({ error: 'Cannot remove your own admin role' }); return res.status(400).json({ error: 'Invalid group. Must be one of: Admin, Standard_User, Leadership, Read_Only' });
}
// Prevent admin self-demotion
if (String(userId) === String(req.user.id) && group && group !== 'Admin') {
return res.status(400).json({ error: 'Cannot remove your own admin group' });
} }
// Prevent self-deactivation // Prevent self-deactivation
if (userId == req.user.id && is_active === false) { if (String(userId) === String(req.user.id) && is_active === false) {
return res.status(400).json({ error: 'Cannot deactivate your own account' }); return res.status(400).json({ error: 'Cannot deactivate your own account' });
} }
// Validate bu_teams if provided
if (typeof bu_teams === 'string') {
if (bu_teams !== '') {
const teamsResult = validateTeams(bu_teams);
if (!teamsResult.valid) {
return res.status(400).json({ error: `Invalid team(s): ${teamsResult.invalid.join(', ')}. Must be one of: STEAM, ACCESS-ENG, ACCESS-OPS, INTELDEV` });
}
}
}
try { try {
// Fetch current user record before update (needed for group change audit)
const { rows: currentRows } = await pool.query(
'SELECT user_group, bu_teams FROM users WHERE id = $1',
[userId]
);
const currentUser = currentRows[0];
if (!currentUser) {
return res.status(404).json({ error: 'User not found' });
}
const updates = []; const updates = [];
const values = []; const values = [];
let paramIndex = 1;
if (username) { if (username) {
updates.push('username = ?'); updates.push(`username = $${paramIndex++}`);
values.push(username); values.push(username);
} }
if (email) { if (email) {
updates.push('email = ?'); updates.push(`email = $${paramIndex++}`);
values.push(email); values.push(email);
} }
if (password) { if (password) {
const passwordHash = await bcrypt.hash(password, 10); const passwordHash = await bcrypt.hash(password, 10);
updates.push('password_hash = ?'); updates.push(`password_hash = $${paramIndex++}`);
values.push(passwordHash); values.push(passwordHash);
} }
if (role) { if (group) {
if (!['admin', 'editor', 'viewer'].includes(role)) { updates.push(`user_group = $${paramIndex++}`);
return res.status(400).json({ error: 'Invalid role' }); values.push(group);
}
updates.push('role = ?');
values.push(role);
} }
if (typeof is_active === 'boolean') { if (typeof is_active === 'boolean') {
updates.push('is_active = ?'); updates.push(`is_active = $${paramIndex++}`);
values.push(is_active ? 1 : 0); values.push(is_active);
}
if (typeof bu_teams === 'string') {
updates.push(`bu_teams = $${paramIndex++}`);
values.push(bu_teams);
} }
if (updates.length === 0) { if (updates.length === 0) {
@@ -159,25 +200,20 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
values.push(userId); values.push(userId);
await new Promise((resolve, reject) => { await pool.query(
db.run( `UPDATE users SET ${updates.join(', ')} WHERE id = $${paramIndex}`,
`UPDATE users SET ${updates.join(', ')} WHERE id = ?`, values
values, );
function(err) {
if (err) reject(err);
else resolve({ changes: this.changes });
}
);
});
const updatedFields = {}; const updatedFields = {};
if (username) updatedFields.username = username; if (username) updatedFields.username = username;
if (email) updatedFields.email = email; if (email) updatedFields.email = email;
if (role) updatedFields.role = role; if (group) updatedFields.group = group;
if (typeof is_active === 'boolean') updatedFields.is_active = is_active; if (typeof is_active === 'boolean') updatedFields.is_active = is_active;
if (password) updatedFields.password_changed = true; if (password) updatedFields.password_changed = true;
if (typeof bu_teams === 'string') updatedFields.bu_teams = bu_teams;
logAudit(db, { logAudit({
userId: req.user.id, userId: req.user.id,
username: req.user.username, username: req.user.username,
action: 'user_update', action: 'user_update',
@@ -187,17 +223,47 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
ipAddress: req.ip ipAddress: req.ip
}); });
// Log specific audit entry for group changes
if (group && group !== currentUser.user_group) {
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'user_group_change',
entityType: 'user',
entityId: String(userId),
details: {
previous_group: currentUser.user_group,
new_group: group
},
ipAddress: req.ip
});
}
// Log specific audit entry for bu_teams changes
if (typeof bu_teams === 'string' && bu_teams !== (currentUser.bu_teams || '')) {
logAudit({
userId: req.user.id,
username: req.user.username,
action: 'user_teams_change',
entityType: 'user',
entityId: String(userId),
details: {
previous_teams: currentUser.bu_teams || '',
new_teams: bu_teams
},
ipAddress: req.ip
});
}
// If user was deactivated, delete their sessions // If user was deactivated, delete their sessions
if (is_active === false) { if (is_active === false) {
await new Promise((resolve) => { await pool.query('DELETE FROM sessions WHERE user_id = $1', [userId]);
db.run('DELETE FROM sessions WHERE user_id = ?', [userId], () => resolve());
});
} }
res.json({ message: 'User updated successfully' }); res.json({ message: 'User updated successfully' });
} catch (err) { } catch (err) {
console.error('Update user error:', err); console.error('Update user error:', err);
if (err.message.includes('UNIQUE constraint failed')) { if (err.code === '23505') { // Postgres unique violation
return res.status(409).json({ error: 'Username or email already exists' }); return res.status(409).json({ error: 'Username or email already exists' });
} }
res.status(500).json({ error: 'Failed to update user' }); res.status(500).json({ error: 'Failed to update user' });
@@ -209,37 +275,29 @@ function createUsersRouter(db, requireAuth, requireRole, logAudit) {
const userId = req.params.id; const userId = req.params.id;
// Prevent self-deletion // Prevent self-deletion
if (userId == req.user.id) { if (String(userId) === String(req.user.id)) {
return res.status(400).json({ error: 'Cannot delete your own account' }); return res.status(400).json({ error: 'Cannot delete your own account' });
} }
try { try {
// Look up the user before deleting // Look up the user before deleting
const targetUser = await new Promise((resolve, reject) => { const { rows: userRows } = await pool.query(
db.get('SELECT username FROM users WHERE id = ?', [userId], (err, row) => { 'SELECT username FROM users WHERE id = $1',
if (err) reject(err); [userId]
else resolve(row); );
}); const targetUser = userRows[0];
});
// Delete sessions first (foreign key) // Delete sessions first (foreign key)
await new Promise((resolve) => { await pool.query('DELETE FROM sessions WHERE user_id = $1', [userId]);
db.run('DELETE FROM sessions WHERE user_id = ?', [userId], () => resolve());
});
// Delete user // Delete user
const result = await new Promise((resolve, reject) => { const result = await pool.query('DELETE FROM users WHERE id = $1', [userId]);
db.run('DELETE FROM users WHERE id = ?', [userId], function(err) {
if (err) reject(err);
else resolve({ changes: this.changes });
});
});
if (result.changes === 0) { if (result.rowCount === 0) {
return res.status(404).json({ error: 'User not found' }); return res.status(404).json({ error: 'User not found' });
} }
logAudit(db, { logAudit({
userId: req.user.id, userId: req.user.id,
username: req.user.username, username: req.user.username,
action: 'user_delete', action: 'user_delete',

View File

@@ -1,261 +0,0 @@
const express = require('express');
const path = require('path');
const fs = require('fs');
const { requireAuth, requireRole } = require('../middleware/auth');
const logAudit = require('../helpers/auditLog');
const { processVulnerabilityReport } = require('../helpers/excelProcessor');
function createWeeklyReportsRouter(db, upload) {
const router = express.Router();
// Helper to sanitize filename
function sanitizePathSegment(segment) {
if (!segment || typeof segment !== 'string') return '';
return segment
.replace(/\0/g, '')
.replace(/\.\./g, '')
.replace(/[\/\\]/g, '')
.trim();
}
// Helper to generate week label
function getWeekLabel(date) {
const now = new Date();
const uploadDate = new Date(date);
const daysDiff = Math.floor((now - uploadDate) / (1000 * 60 * 60 * 24));
if (daysDiff < 7) {
return "This week's report";
} else if (daysDiff < 14) {
return "Last week's report";
} else {
const month = uploadDate.getMonth() + 1;
const day = uploadDate.getDate();
const year = uploadDate.getFullYear();
return `Week of ${month.toString().padStart(2, '0')}/${day.toString().padStart(2, '0')}/${year}`;
}
}
// POST /api/weekly-reports/upload - Upload and process vulnerability report
router.post('/upload', requireAuth(db), requireRole(db, 'editor', 'admin'), upload.single('file'), async (req, res) => {
const uploadedFile = req.file;
if (!uploadedFile) {
return res.status(400).json({ error: 'No file uploaded' });
}
// Validate file extension
const ext = path.extname(uploadedFile.originalname).toLowerCase();
if (ext !== '.xlsx') {
fs.unlinkSync(uploadedFile.path); // Clean up temp file
return res.status(400).json({ error: 'Only .xlsx files are allowed' });
}
const timestamp = Date.now();
const sanitizedName = sanitizePathSegment(uploadedFile.originalname);
const reportsDir = path.join(__dirname, '..', 'uploads', 'weekly_reports');
// Create directory if it doesn't exist
if (!fs.existsSync(reportsDir)) {
fs.mkdirSync(reportsDir, { recursive: true });
}
const originalFilename = `${timestamp}_original_${sanitizedName}`;
const processedFilename = `${timestamp}_processed_${sanitizedName}`;
const originalPath = path.join(reportsDir, originalFilename);
const processedPath = path.join(reportsDir, processedFilename);
try {
// Move uploaded file to permanent location
fs.renameSync(uploadedFile.path, originalPath);
// Process the file with Python script
const result = await processVulnerabilityReport(originalPath, processedPath);
const uploadDate = new Date().toISOString().split('T')[0];
// Update previous current reports to not current
db.run('UPDATE weekly_reports SET is_current = 0 WHERE is_current = 1', (err) => {
if (err) {
console.error('Error updating previous current reports:', err);
}
});
// Insert new report record
const insertSql = `
INSERT INTO weekly_reports (
upload_date, week_label, original_filename, processed_filename,
original_file_path, processed_file_path, row_count_original,
row_count_processed, uploaded_by, is_current
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1)
`;
const weekLabel = getWeekLabel(uploadDate);
db.run(
insertSql,
[
uploadDate,
weekLabel,
sanitizedName,
processedFilename,
originalPath,
processedPath,
result.original_rows,
result.processed_rows,
req.user.id
],
function (err) {
if (err) {
console.error('Error inserting weekly report:', err);
return res.status(500).json({ error: 'Failed to save report metadata' });
}
// Log audit entry
logAudit(
db,
req.user.id,
req.user.username,
'UPLOAD_WEEKLY_REPORT',
'weekly_reports',
this.lastID,
JSON.stringify({ filename: sanitizedName, rows: result.processed_rows }),
req.ip
);
res.json({
success: true,
id: this.lastID,
original_rows: result.original_rows,
processed_rows: result.processed_rows,
week_label: weekLabel
});
}
);
} catch (error) {
// Clean up files on error
if (fs.existsSync(originalPath)) fs.unlinkSync(originalPath);
if (fs.existsSync(processedPath)) fs.unlinkSync(processedPath);
console.error('Error processing vulnerability report:', error);
res.status(500).json({ error: error.message || 'Failed to process report' });
}
});
// GET /api/weekly-reports - List all reports
router.get('/', requireAuth(db), (req, res) => {
const sql = `
SELECT id, upload_date, week_label, original_filename, processed_filename,
row_count_original, row_count_processed, is_current, uploaded_at
FROM weekly_reports
ORDER BY upload_date DESC, uploaded_at DESC
`;
db.all(sql, [], (err, rows) => {
if (err) {
console.error('Error fetching weekly reports:', err);
return res.status(500).json({ error: 'Failed to fetch reports' });
}
res.json(rows);
});
});
// GET /api/weekly-reports/:id/download/:type - Download report file
router.get('/:id/download/:type', requireAuth(db), (req, res) => {
const { id, type } = req.params;
if (type !== 'original' && type !== 'processed') {
return res.status(400).json({ error: 'Invalid download type. Use "original" or "processed"' });
}
const sql = `SELECT original_file_path, processed_file_path, original_filename FROM weekly_reports WHERE id = ?`;
db.get(sql, [id], (err, row) => {
if (err) {
console.error('Error fetching report:', err);
return res.status(500).json({ error: 'Failed to fetch report' });
}
if (!row) {
return res.status(404).json({ error: 'Report not found' });
}
const filePath = type === 'original' ? row.original_file_path : row.processed_file_path;
if (!fs.existsSync(filePath)) {
return res.status(404).json({ error: 'File not found on disk' });
}
// Log audit entry
logAudit(
db,
req.user.id,
req.user.username,
'DOWNLOAD_WEEKLY_REPORT',
'weekly_reports',
id,
JSON.stringify({ type }),
req.ip
);
const downloadName = type === 'original' ? row.original_filename : row.original_filename.replace('.xlsx', '_processed.xlsx');
res.setHeader('Content-Type', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
res.setHeader('Content-Disposition', `attachment; filename="${downloadName}"`);
res.sendFile(filePath);
});
});
// DELETE /api/weekly-reports/:id - Delete report (admin only)
router.delete('/:id', requireAuth(db), requireRole(db, 'admin'), (req, res) => {
const { id } = req.params;
const sql = 'SELECT original_file_path, processed_file_path FROM weekly_reports WHERE id = ?';
db.get(sql, [id], (err, row) => {
if (err) {
console.error('Error fetching report for deletion:', err);
return res.status(500).json({ error: 'Failed to fetch report' });
}
if (!row) {
return res.status(404).json({ error: 'Report not found' });
}
// Delete database record
db.run('DELETE FROM weekly_reports WHERE id = ?', [id], (err) => {
if (err) {
console.error('Error deleting report:', err);
return res.status(500).json({ error: 'Failed to delete report' });
}
// Delete files
if (fs.existsSync(row.original_file_path)) {
fs.unlinkSync(row.original_file_path);
}
if (fs.existsSync(row.processed_file_path)) {
fs.unlinkSync(row.processed_file_path);
}
// Log audit entry
logAudit(
db,
req.user.id,
req.user.username,
'DELETE_WEEKLY_REPORT',
'weekly_reports',
id,
null,
req.ip
);
res.json({ success: true });
});
});
});
return router;
}
module.exports = createWeeklyReportsRouter;

View File

@@ -0,0 +1,388 @@
#!/usr/bin/env node
// ==========================================================================
// CARD → Granite Lookup Script (v2)
// ==========================================================================
// Queries CARD team assets endpoint (which returns full enriched records
// including ncim_discovery with EQUIP_INST_ID) for the 109 reassigned IPs
// from the findings-count investigation Appendix C.
//
// Generates:
// docs/card-lookup-results.csv — full CARD data for review
// docs/granite-reassignment-upload.csv — Team_Device Loader format
//
// Usage:
// cd backend
// node scripts/card-granite-lookup.js
// ==========================================================================
require('dotenv').config({ path: require('path').join(__dirname, '..', '.env') });
const cardApi = require('../helpers/cardApi');
const fs = require('fs');
const path = require('path');
// ---------------------------------------------------------------------------
// IP → hostname mapping from Appendix C
// ---------------------------------------------------------------------------
const REASSIGNED = {
// With approved FP workflows (58)
'98.120.0.78': 'syn-098-120-000-078', '98.120.32.185': 'syn-098-120-032-185',
'10.240.78.177': 'mon15-agg-sw', '10.240.78.176': 'mon16-agg-sw',
'10.240.78.133': 'mon15-sw14', '10.240.78.130': 'mon15-sw11',
'10.240.78.150': 'mon19-sw3', '10.240.78.107': 'mon16-sw2',
'10.240.78.110': 'mon16-sw5', '10.240.78.106': 'mon16-sw1',
'10.240.78.149': 'mon19-sw2', '10.240.78.154': 'mon19-sw7',
'10.240.78.111': 'mon16-sw6', '10.240.78.153': 'mon19-sw6',
'10.240.78.132': 'mon15-sw13', '10.240.78.115': 'mon16-sw10',
'10.240.78.109': 'mon16-sw4', '10.240.78.112': 'mon16-sw7',
'10.240.78.119': 'mon16-sw14', '10.240.78.114': 'mon16-sw9',
'10.240.78.118': 'mon16-sw13', '10.240.78.117': 'mon16-sw12',
'10.240.78.108': 'mon16-sw3', '10.240.78.155': 'mon19-sw8',
'10.240.78.157': 'mon19-sw10', '10.240.78.151': 'mon19-sw4',
'10.240.78.116': 'mon16-sw11', '10.240.78.152': 'mon19-sw5',
'10.240.78.161': 'mon19-sw14', '10.240.78.160': 'mon19-sw13',
'10.240.78.159': 'mon19-sw12', '10.240.78.158': 'mon19-sw11',
'10.240.78.123': 'mon15-sw4', '10.240.78.137': 'mon20-sw4',
'10.240.78.148': 'mon19-sw1', '10.240.78.125': 'mon15-sw6',
'10.240.78.156': 'mon19-sw9', '10.241.0.63': '',
'10.244.11.51': 'apc01se1shcc-n01-bmc', '172.27.72.1': '',
'96.37.185.145': '', '10.240.78.170': 'mon17-sw9',
'10.240.78.172': 'mon17-sw11', '10.240.78.169': 'mon17-sw8',
'10.240.78.166': 'mon17-sw5', '10.240.78.174': 'mon17-sw13',
'10.240.78.173': 'mon17-sw12', '10.240.78.167': 'mon17-sw6',
'10.240.78.175': 'mon17-sw14', '10.240.78.168': 'mon17-sw7',
'10.240.78.171': 'mon17-sw10', '66.61.128.10': 'syn-066-061-128-010',
'66.61.128.233': 'apa01se1shcc-bvi101-secondary',
'66.61.128.49': 'syn-066-061-128-049', '66.61.128.18': 'syn-066-061-128-018',
'10.244.4.26': '', '10.244.11.5': '', '10.244.11.6': '',
// With rejected FP workflows (8)
'10.244.4.55': 'apc15se1shcc-n03', '10.244.11.53': 'apc01se1shcc-n03-bmc',
'10.244.4.30': '', '10.244.11.63': 'apc04se1shcc-n01-cimc',
'24.28.208.125': '', '24.28.210.101': 'syn-024-028-210-101',
'10.244.11.27': '', '10.240.1.203': '',
// Without FP workflows (43)
'10.240.78.20': '', '172.16.1.229': '',
'10.244.11.96': '', '10.244.11.54': 'apc02se1shcc-n01-cimc',
'10.244.4.51': 'apc14se1shcc-n02', '10.244.11.86': '',
'10.244.11.55': 'apc02se1shcc-n02-cimc', '24.28.208.105': 'syn-024-028-208-105',
'10.244.4.50': 'apc14se1shcc-n01', '10.244.4.53': 'apc15se1shcc-n01',
'10.244.11.73': 'apc07se1shcc-n02-cimc', '10.244.11.64': 'apc04se1shcc-n02-cimc',
'10.244.4.54': 'apc15se1shcc-n02', '10.244.4.28': '',
'10.244.11.94': '', '10.241.0.43': 'c220-wzp27340ss5',
'10.244.11.56': 'apc02se1shcc-n03-cimc', '10.244.11.66': 'apc05se1shcc-n01-bmc',
'10.244.4.47': 'apc13se1shcc-n01', '10.244.4.49': 'apc13se1shcc-n03',
'10.244.4.52': 'apc14se1shcc-n03', '10.244.11.72': 'apc07se1shcc-n01-cimc',
'10.244.4.25': 'apc02ctsbcom7-n03-cimc', '10.244.4.29': '',
'10.244.11.74': 'apc07se1shcc-n03-cimc', '10.244.4.48': 'apc13se1shcc-n02',
'10.244.11.65': 'apc04se1shcc-n03-cimc', '10.244.4.24': 'apc02ctsbcom7-n02-cimc',
'10.244.11.87': '', '10.244.11.68': 'apc05se1shcc-n03-bmc',
'10.244.11.67': 'apc05se1shcc-n02-bmc', '10.244.4.23': 'apc02ctsbcom7-n01-cimc',
'10.244.11.57': '', '10.244.11.95': '',
'98.120.32.145': 'syn-098-120-032-145', '98.120.0.129': 'syn-098-120-000-129',
'68.114.184.84': 'rphy-runner-vecima',
};
const TARGET_IPS = new Set(Object.keys(REASSIGNED));
// ---------------------------------------------------------------------------
// Fetch all assets for both teams, then match against our IP list
// ---------------------------------------------------------------------------
async function fetchTeamAssets(teamName) {
const allAssets = [];
let page = 1;
const pageSize = 200;
while (true) {
// Fetch confirmed assets (these have the richest data)
const result = await cardApi.getTeamAssets(teamName, {
disposition: 'confirmed',
page,
pageSize,
});
if (!result.ok) {
console.error(` Failed to fetch ${teamName} page ${page}: HTTP ${result.status}`);
break;
}
let data;
try { data = JSON.parse(result.body); } catch (_) { break; }
const assets = Array.isArray(data) ? data : (data.assets || data.results || []);
allAssets.push(...assets);
const total = data.total || assets.length;
console.log(` ${teamName} page ${page}: ${assets.length} assets (total: ${total})`);
if (allAssets.length >= total || assets.length === 0) break;
page++;
}
return allAssets;
}
function extractIPFromAssetId(assetId) {
// Asset IDs are like "10.240.78.110-CTEC" — strip the suffix
if (!assetId) return null;
const parts = assetId.split('-');
// Rejoin all but the last part (the suffix like CTEC, NATL, etc.)
// But only if the last part looks like a suffix (not a number)
const last = parts[parts.length - 1];
if (/^\d+$/.test(last)) return assetId; // All numeric, probably just an IP
return parts.slice(0, -1).join('-');
}
function extractGraniteData(asset) {
const id = asset._id || '';
const ip = extractIPFromAssetId(id);
const flags = (asset.card_flags && asset.card_flags[0]) || {};
const ncim = asset.ncim_discovery || [];
const qualys = asset.qualys_hosts || [];
const ivanti = asset.ivanti_assets || [];
const granite = asset.netops_granite_allips || null;
const iseGranite = asset.ise_granite_equipment || null;
// Extract EQUIP_INST_ID from ncim_discovery (primary source)
let equipInstId = null;
let graniteTeam = null;
let entityId = null;
let sysLocation = null;
let ncimHostname = null;
if (ncim.length > 0) {
equipInstId = ncim[0].EQUIP_INST_ID || null;
graniteTeam = ncim[0].GRANITE_RESP_TEAM || ncim[0].RESPONSIBLE_TEAM || null;
entityId = ncim[0].ENTITYID || null;
sysLocation = ncim[0].SYSLOCATION || null;
ncimHostname = ncim[0].HOSTNAME || null;
}
// Fallback: check netops_granite_allips
if (!equipInstId && granite && Array.isArray(granite) && granite.length > 0) {
equipInstId = granite[0].EQUIP_INST_ID || null;
}
// Fallback: check ise_granite_equipment
if (!equipInstId && iseGranite && Array.isArray(iseGranite) && iseGranite.length > 0) {
equipInstId = iseGranite[0].EQUIP_INST_ID || null;
}
const hostname = ncimHostname
|| (flags.CARD_HOSTNAME && flags.CARD_HOSTNAME[0])
|| (qualys.length > 0 && qualys[0].HOSTNAME)
|| (ivanti.length > 0 && ivanti[0].hostName)
|| '';
const confirmedTeam = asset.owner && asset.owner.confirmed
? asset.owner.confirmed.name : null;
return {
ip,
assetId: id,
hostname,
equipInstId,
graniteTeam,
entityId,
sysLocation,
confirmedTeam,
deviceId: flags.CARD_DEVICE_ID || null,
asn: flags.CARD_ASN || null,
vendorModel: (flags.CARD_VENDOR_MODEL || []).map(v => v.vendor_model || v).join(', '),
status: flags.status || null,
};
}
// ---------------------------------------------------------------------------
// Main
// ---------------------------------------------------------------------------
async function main() {
console.log('=== CARD → Granite Lookup (v2 — team assets endpoint) ===');
console.log(`Target IPs: ${TARGET_IPS.size}`);
console.log(`CARD_API_URL: ${process.env.CARD_API_URL}`);
console.log('');
if (!cardApi.isConfigured) {
console.error('CARD API is not configured.');
process.exit(1);
}
// Fetch assets from both teams
const teams = ['NTS-AEO-STEAM', 'NTS-AEO-ACCESS-ENG'];
const allAssets = [];
for (const team of teams) {
console.log(`Fetching ${team}...`);
const assets = await fetchTeamAssets(team);
allAssets.push(...assets);
console.log(` Total: ${assets.length} assets\n`);
}
// Also fetch candidate/unconfirmed in case some were reassigned
for (const team of teams) {
for (const disp of ['candidate', 'unconfirmed']) {
console.log(`Fetching ${team} (${disp})...`);
try {
const result = await cardApi.getTeamAssets(team, { disposition: disp, pageSize: 200 });
if (result.ok) {
const data = JSON.parse(result.body);
const assets = Array.isArray(data) ? data : (data.assets || data.results || []);
allAssets.push(...assets);
console.log(` ${assets.length} assets`);
}
} catch (_) { /* skip */ }
}
}
console.log(`\nTotal assets fetched: ${allAssets.length}`);
// Build IP → asset map
const ipMap = new Map();
for (const asset of allAssets) {
const id = asset._id || '';
const ip = extractIPFromAssetId(id);
if (ip && !ipMap.has(ip)) {
ipMap.set(ip, asset);
}
}
console.log(`Unique IPs in CARD: ${ipMap.size}`);
// Match against our target IPs
const matched = [];
const notFound = [];
for (const ip of TARGET_IPS) {
const asset = ipMap.get(ip);
if (asset) {
matched.push(extractGraniteData(asset));
} else {
notFound.push(ip);
}
}
// For IPs not found in team assets, fall back to individual owner lookup
if (notFound.length > 0) {
console.log(`\n${notFound.length} IPs not in team assets — trying individual owner lookups...`);
const SUFFIXES = ['CTEC', 'NATL', 'TWC', 'BHN', 'CHTR'];
const stillNotFound = [];
for (const ip of notFound) {
let found = false;
for (const suffix of SUFFIXES) {
try {
const result = await cardApi.getOwner(`${ip}-${suffix}`);
if (result.ok) {
const data = JSON.parse(result.body);
// Owner endpoint is slim — extract what we can
const ncim = data.ncim_discovery || [];
matched.push({
ip,
assetId: data._id || `${ip}-${suffix}`,
hostname: REASSIGNED[ip] || '',
equipInstId: ncim.length > 0 ? (ncim[0].EQUIP_INST_ID || null) : null,
graniteTeam: ncim.length > 0 ? (ncim[0].GRANITE_RESP_TEAM || null) : null,
entityId: ncim.length > 0 ? (ncim[0].ENTITYID || null) : null,
sysLocation: ncim.length > 0 ? (ncim[0].SYSLOCATION || null) : null,
confirmedTeam: data.owner && data.owner.confirmed ? data.owner.confirmed.name : null,
deviceId: null,
asn: null,
vendorModel: '',
status: null,
});
found = true;
break;
}
} catch (_) { /* continue */ }
}
if (!found) stillNotFound.push(ip);
}
if (stillNotFound.length > 0) {
console.log(`\n${stillNotFound.length} IPs not found anywhere in CARD:`);
stillNotFound.forEach(ip => console.log(` ${ip} (${REASSIGNED[ip] || 'no hostname'})`));
}
}
// Sort by IP
matched.sort((a, b) => {
const aParts = a.ip.split('.').map(Number);
const bParts = b.ip.split('.').map(Number);
for (let i = 0; i < 4; i++) {
if (aParts[i] !== bParts[i]) return aParts[i] - bParts[i];
}
return 0;
});
// Summary
const withEquipId = matched.filter(r => r.equipInstId);
const withoutEquipId = matched.filter(r => !r.equipInstId);
console.log('\n=== Summary ===');
console.log(`Matched in CARD: ${matched.length}`);
console.log(`With EQUIP_INST_ID: ${withEquipId.length}`);
console.log(`Without EQUIP_INST_ID: ${withoutEquipId.length}`);
// Print results
console.log('\n=== Results with EQUIP_INST_ID ===');
console.log('IP Address | EQUIP_INST_ID | Hostname | Granite Team');
console.log('-'.repeat(100));
for (const r of withEquipId) {
console.log(`${r.ip.padEnd(20)} | ${String(r.equipInstId).padEnd(13)} | ${(r.hostname || '').padEnd(30)} | ${r.graniteTeam || '-'}`);
}
if (withoutEquipId.length > 0) {
console.log('\n=== Results WITHOUT EQUIP_INST_ID ===');
for (const r of withoutEquipId) {
console.log(` ${r.ip.padEnd(20)} ${(r.hostname || REASSIGNED[r.ip] || '').padEnd(30)} confirmed: ${r.confirmedTeam || '-'}`);
}
}
// Write full CSV
const csvPath = path.join(__dirname, '..', '..', 'docs', 'card-lookup-results.csv');
const csvHeader = 'IP Address,CARD Asset ID,Hostname,EQUIP_INST_ID,Granite Team,Entity ID,SysLocation,Confirmed Team,Device ID,ASN,Vendor Model,Status';
const csvRows = matched.map(r =>
[r.ip, r.assetId, r.hostname, r.equipInstId, r.graniteTeam, r.entityId, r.sysLocation, r.confirmedTeam, r.deviceId, r.asn, r.vendorModel, r.status]
.map(v => v === null || v === undefined ? '' : `"${String(v).replace(/"/g, '""')}"`)
.join(',')
);
fs.writeFileSync(csvPath, csvHeader + '\n' + csvRows.join('\n') + '\n', 'utf8');
console.log(`\nFull CSV: ${csvPath}`);
// Write Granite Team_Device Loader CSV
const graniteHeaders = [
'DELETE', 'SET_CONFIRMED', 'EQUIPMENT CLASS', 'EQUIP_INST_ID', 'SITE_NAME',
'EQUIP_NAME', 'EQUIP_TEMPLATE', 'EQUIP_STATUS',
'UDA#RESPONSIBLE ORGANIZATION#RESPONSIBLE TEAM',
'UDA#IP_ADDRESSING#IPV4_ADDRESS',
'UDA#IP_ADDRESSING#MAC ADDRESS', 'UDA#IP_ADDRESSING#MGMT_IP_ASN', 'SERIALNUMBER',
];
const graniteRows = withEquipId.map(r => [
'', // DELETE
'', // SET_CONFIRMED
'S', // EQUIPMENT CLASS (Shelf)
r.equipInstId, // EQUIP_INST_ID
'', // SITE_NAME
r.hostname || REASSIGNED[r.ip] || '', // EQUIP_NAME
'', // EQUIP_TEMPLATE
'', // EQUIP_STATUS
'NTS-AEO-STEAM', // RESPONSIBLE TEAM
r.ip, // IPV4_ADDRESS
'', // MAC ADDRESS
r.asn || '', // MGMT_IP_ASN
r.deviceId || '', // SERIALNUMBER
]);
const granitePath = path.join(__dirname, '..', '..', 'docs', 'granite-reassignment-upload.csv');
const graniteContent = [
graniteHeaders.join(','),
...graniteRows.map(r => r.map(v => `"${String(v).replace(/"/g, '""')}"`).join(','))
].join('\n');
fs.writeFileSync(granitePath, graniteContent + '\n', 'utf8');
console.log(`Granite upload CSV (${withEquipId.length} rows): ${granitePath}`);
}
main().catch(err => {
console.error('Unhandled error:', err);
process.exit(1);
});

View File

@@ -0,0 +1,44 @@
{
"metric_categories": {
"1.1.1": "Logging & Monitoring",
"1.1.3": "Logging & Monitoring",
"1.4.1": "Logging & Monitoring",
"2.3.4i": "Vulnerability Management",
"2.3.6i": "Vulnerability Management",
"2.3.8i": "Vulnerability Management",
"5.2.4": "Access & MFA",
"5.2.5": "Access & MFA",
"5.2.6": "Access & MFA",
"5.2.7": "Access & MFA",
"5.2.8": "Access & MFA",
"5.3.4": "Endpoint Protection",
"5.5.4i": "Vulnerability Management",
"5.5.5": "Decommissioned Assets",
"5.8.1": "Application Security",
"7.1.1": "Logging & Monitoring",
"7.1.4": "Logging & Monitoring",
"7.6.13": "Disaster Recovery",
"7.6.16": "Disaster Recovery",
"Missing_AppID": "Asset Data Quality",
"Missing_DF": "Asset Data Quality",
"Missing_OS": "Asset Data Quality",
"5.5.2": "Other"
},
"core_cols": [
"Preferred - Hostname",
"GRANITE - IPv4_Address",
"GRANITE - Type",
"Team",
"Compliant",
"Source_Network",
"Vertical",
"GRANITE - Equip_Inst_ID",
"GRANITE - RESPONSIBLE_TEAM"
],
"skip_sheets": [
"Summary",
"CMDB_9box",
"Vulns",
"Aging Dashboard"
]
}

View File

@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""
Dump the structural schema of a compliance xlsx file as JSON.
Usage: python3 dump_xlsx_schema.py <path_to_xlsx>
Output:
{
"sheets": [
{
"name": "SheetName",
"columns": ["Col A", "Col B", ...],
"row_count": 150,
"metric_values": ["2.3.4i", "5.2.4", ...] // only if a Metric column exists
},
...
]
}
Dependencies: openpyxl (already in requirements.txt)
"""
import sys
import json
from openpyxl import load_workbook
def main():
if len(sys.argv) < 2:
print(json.dumps({'error': 'No file path provided'}))
sys.exit(1)
filepath = sys.argv[1]
try:
wb = load_workbook(filepath, read_only=True, data_only=True)
except Exception as e:
print(json.dumps({'error': f'Cannot open file: {str(e)}'}))
sys.exit(1)
sheets = []
for sheet_name in wb.sheetnames:
ws = wb[sheet_name]
rows = list(ws.iter_rows(max_row=1, values_only=True))
columns = [str(c).strip() for c in rows[0] if c is not None] if rows else []
# Count data rows (excluding header)
row_count = 0
for _ in ws.iter_rows(min_row=2, values_only=True):
row_count += 1
# Extract metric values if a Metric column exists in the Summary sheet
metric_values = []
if sheet_name == 'Summary':
# Summary has header at row 4 (0-indexed row 3), read from row 5 onward
header_rows = list(ws.iter_rows(min_row=4, max_row=4, values_only=True))
if header_rows:
summary_cols = [str(c).strip() if c else '' for c in header_rows[0]]
metric_idx = None
for i, col in enumerate(summary_cols):
if col == 'Metric':
metric_idx = i
break
if metric_idx is not None:
for row in ws.iter_rows(min_row=5, values_only=True):
if row[metric_idx] is not None:
val = str(row[metric_idx]).strip()
if val and val != 'Metric':
metric_values.append(val)
entry = {
'name': sheet_name,
'columns': columns,
'row_count': row_count,
}
if metric_values:
entry['metric_values'] = sorted(set(metric_values))
sheets.append(entry)
wb.close()
print(json.dumps({'sheets': sheets}, indent=2))
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""
Extract the structural schema of a compliance xlsx file as JSON.
Usage: python3 extract_xlsx_schema.py <path_to_xlsx>
Output:
{
"sheets": [
{
"name": "Summary",
"columns": ["Metric", "Non-Compliant", "..."],
"metric_values": ["2.3.4i", "5.2.4", "..."]
},
{
"name": "2.3.4i",
"columns": ["Preferred - Hostname", "GRANITE - IPv4_Address", "..."]
}
]
}
- Uses openpyxl in read-only mode.
- Extracts sheet names, first-row column headers per sheet, and unique metric
values from the Summary sheet (header at row 4, data from row 5 onward).
- On error, returns { "error": "..." } on stdout and exits with non-zero code.
Dependencies: openpyxl (already in requirements.txt)
"""
import sys
import json
from openpyxl import load_workbook
def main():
if len(sys.argv) < 2:
print(json.dumps({"error": "No file path provided"}))
sys.exit(1)
filepath = sys.argv[1]
try:
wb = load_workbook(filepath, read_only=True, data_only=True)
except Exception as e:
print(json.dumps({"error": f"Cannot open file: {str(e)}"}))
sys.exit(1)
if not wb.sheetnames:
print(json.dumps({"error": "Workbook contains no sheets"}))
wb.close()
sys.exit(1)
sheets = []
for sheet_name in wb.sheetnames:
ws = wb[sheet_name]
# Extract first-row column headers
rows = list(ws.iter_rows(max_row=1, values_only=True))
columns = [str(c).strip() for c in rows[0] if c is not None] if rows else []
entry = {
"name": sheet_name,
"columns": columns,
}
# Extract metric values from the Summary sheet
# Summary has header at row 4, data from row 5 onward
if sheet_name == "Summary":
metric_values = []
header_rows = list(ws.iter_rows(min_row=4, max_row=4, values_only=True))
if header_rows:
summary_cols = [str(c).strip() if c else "" for c in header_rows[0]]
metric_idx = None
for i, col in enumerate(summary_cols):
if col == "Metric":
metric_idx = i
break
if metric_idx is not None:
for row in ws.iter_rows(min_row=5, values_only=True):
if row[metric_idx] is not None:
val = str(row[metric_idx]).strip()
if val and val != "Metric":
metric_values.append(val)
entry["metric_values"] = sorted(set(metric_values))
sheets.append(entry)
wb.close()
print(json.dumps({"sheets": sheets}))
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""
import_notes_from_csv.py
------------------------
Mass-import finding notes from a CSV file into the CVE dashboard database.
CSV format (header row required, column names are case-insensitive):
ID,NOTES
12345,EXC-5754
67890,EXC-6001 - pending review
Usage:
python3 import_notes_from_csv.py <csv_file> [--db <db_path>] [--dry-run]
Options:
--db <path> Path to cve_database.db (default: ../cve_database.db)
--dry-run Print what would change without touching the database
"""
import csv
import sqlite3
import sys
import os
import argparse
from datetime import datetime, timezone
NOTE_MAX_LEN = 255
DEFAULT_DB = os.path.join(os.path.dirname(__file__), '..', 'cve_database.db')
def parse_args():
p = argparse.ArgumentParser(description='Import finding notes from CSV into the dashboard DB.')
p.add_argument('csv_file', help='Path to the CSV file (must have ID and NOTES columns)')
p.add_argument('--db', default=DEFAULT_DB, help=f'Path to SQLite database (default: {DEFAULT_DB})')
p.add_argument('--dry-run', action='store_true', help='Preview changes without writing to DB')
return p.parse_args()
def load_csv(path):
"""Read CSV and return list of (finding_id, note) tuples."""
rows = []
with open(path, newline='', encoding='utf-8-sig') as f:
reader = csv.DictReader(f)
# Normalise header names to uppercase for case-insensitive matching
if reader.fieldnames is None:
print('ERROR: CSV file is empty or has no header row.')
sys.exit(1)
normalised = {k.strip().upper(): k for k in reader.fieldnames}
if 'ID' not in normalised or 'NOTES' not in normalised:
print(f'ERROR: CSV must have "ID" and "NOTES" columns.')
print(f' Found columns: {list(reader.fieldnames)}')
sys.exit(1)
id_col = normalised['ID']
notes_col = normalised['NOTES']
for i, row in enumerate(reader, start=2): # start=2 because row 1 is the header
finding_id = row[id_col].strip()
note = row[notes_col].strip()
if not finding_id:
print(f' WARNING row {i}: empty ID — skipping')
continue
if len(note) > NOTE_MAX_LEN:
print(f' WARNING row {i} ({finding_id}): note is {len(note)} chars, '
f'truncating to {NOTE_MAX_LEN}')
note = note[:NOTE_MAX_LEN]
rows.append((finding_id, note))
return rows
def run(args):
csv_path = os.path.abspath(args.csv_file)
db_path = os.path.abspath(args.db)
# ------------------------------------------------------------------ checks
if not os.path.exists(csv_path):
print(f'ERROR: CSV file not found: {csv_path}')
sys.exit(1)
if not os.path.exists(db_path):
print(f'ERROR: Database not found: {db_path}')
sys.exit(1)
print(f'CSV : {csv_path}')
print(f'DB : {db_path}')
if args.dry_run:
print('MODE: DRY RUN — no changes will be written\n')
else:
print()
# ----------------------------------------------------------------- load CSV
rows = load_csv(csv_path)
if not rows:
print('No valid rows found in CSV.')
sys.exit(0)
print(f'Loaded {len(rows)} row(s) from CSV.\n')
# ---------------------------------------------------------------- open DB
con = sqlite3.connect(db_path)
con.row_factory = sqlite3.Row
cur = con.cursor()
# Fetch all known finding IDs — only IDs present here will be processed
import json
cur.execute('SELECT findings_json FROM ivanti_findings_cache WHERE id = 1')
cache_row = cur.fetchone()
known_ids = set()
if cache_row and cache_row['findings_json']:
try:
known_ids = {str(f['id']) for f in json.loads(cache_row['findings_json'])}
except Exception:
pass
if not known_ids:
print('ERROR: No findings found in the database cache.')
print(' Run a Sync from the dashboard first, then re-run this script.')
con.close()
sys.exit(1)
print(f'{len(known_ids)} active finding(s) in cache.\n')
# ----------------------------------------------------------------- process
inserted = 0
updated = 0
skipped = 0
for finding_id, note in rows:
str_id = str(finding_id)
if str_id not in known_ids:
print(f' SKIP {str_id} — not in active findings (resolved or never synced)')
skipped += 1
continue
# Check if a note already exists
cur.execute('SELECT note FROM ivanti_finding_notes WHERE finding_id = ?', (str_id,))
existing = cur.fetchone()
if existing:
if existing['note'] == note:
print(f' SKIP {str_id} — note unchanged')
skipped += 1
continue
action = 'UPDATE'
updated += 1
else:
action = 'INSERT'
inserted += 1
print(f' {action:6s} {str_id}{note[:80]}{"" if len(note) > 80 else ""}')
if not args.dry_run:
cur.execute(
"""
INSERT INTO ivanti_finding_notes (finding_id, note, updated_at)
VALUES (?, ?, datetime('now'))
ON CONFLICT(finding_id) DO UPDATE
SET note = excluded.note, updated_at = datetime('now')
""",
(str_id, note)
)
# ----------------------------------------------------------------- summary
print()
if args.dry_run:
print(f'DRY RUN complete — would insert {inserted}, update {updated}, skip {skipped}.')
else:
con.commit()
print(f'Done — inserted {inserted}, updated {updated}, skipped {skipped} (unchanged).')
con.close()
if __name__ == '__main__':
run(parse_args())

View File

@@ -0,0 +1,928 @@
#!/usr/bin/env node
/**
* migrate-to-postgres.js — Data Migration Script
*
* Copies all data from the SQLite database (cve_database.db) to PostgreSQL.
* The SQLite file is opened READ-ONLY and is never modified.
*
* Special handling:
* - ivanti_findings_cache.findings_json → individual rows in ivanti_findings
* - ivanti_finding_notes → merged into ivanti_findings.note column
* - ivanti_finding_overrides → merged into ivanti_findings.override_host_name / override_dns
* - ivanti_sync_state and ivanti_counts_cache → populated from ivanti_findings_cache metadata
*
* Type conversions:
* - SQLite 0/1 integers → Postgres boolean
* - SQLite DATETIME strings → Postgres TIMESTAMPTZ (passed as-is)
* - SQLite NULL → Postgres NULL
*
* Uses ON CONFLICT DO NOTHING for idempotency (safe to re-run).
*
* Usage:
* node backend/scripts/migrate-to-postgres.js
*
* Requires:
* - DATABASE_URL env var (or .env file in backend/)
* - SQLite database at backend/cve_database.db
*/
const path = require('path');
require('dotenv').config({ path: path.join(__dirname, '..', '.env') });
const fs = require('fs');
const sqlite3 = require('sqlite3').verbose();
const { Pool } = require('pg');
// ---------------------------------------------------------------------------
// Configuration
// ---------------------------------------------------------------------------
const SQLITE_PATH = path.join(__dirname, '..', 'cve_database.db');
const SCHEMA_PATH = path.join(__dirname, '..', 'db-schema.sql');
const DATABASE_URL = process.env.DATABASE_URL;
if (!DATABASE_URL) {
console.error('ERROR: DATABASE_URL environment variable is not set.');
console.error('Expected format: postgresql://user:password@host:port/database');
process.exit(1);
}
if (!fs.existsSync(SQLITE_PATH)) {
console.error(`ERROR: SQLite database not found at ${SQLITE_PATH}`);
process.exit(1);
}
// ---------------------------------------------------------------------------
// SQLite helpers
// ---------------------------------------------------------------------------
function sqliteAll(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.all(sql, params, (err, rows) => {
if (err) reject(err);
else resolve(rows || []);
});
});
}
function sqliteGet(db, sql, params = []) {
return new Promise((resolve, reject) => {
db.get(sql, params, (err, row) => {
if (err) reject(err);
else resolve(row);
});
});
}
// ---------------------------------------------------------------------------
// Extract finding fields from raw JSON object (mirrors ivantiFindings.js)
// ---------------------------------------------------------------------------
function extractFinding(f) {
const rawDueDate = f.statusEmbedded?.dueDate || f.dueDate || '';
const dueDate = rawDueDate ? rawDueDate.split('T')[0] : null;
const buOwnership = f.assetCustomAttributes?.['1550_host_1']?.[0]
|| f.buOwnership || f.bu_ownership || '';
const cves = Array.isArray(f.cves)
? f.cves
: (f.vulnerabilities?.vulnInfoList || []).map(v => v.cve).filter(Boolean);
// Workflow extraction
let workflow = null;
if (f.workflow && typeof f.workflow === 'object') {
workflow = {
id: f.workflow.id || '',
state: f.workflow.state || '',
type: f.workflow.type || 'FP',
};
} else if (f.workflowDistribution) {
const wfDist = f.workflowDistribution || {};
const fpBuckets = [
...(wfDist.actionableWorkflows || []),
...(wfDist.requestedWorkflows || []),
...(wfDist.reworkedWorkflows || []),
...(wfDist.rejectedWorkflows || []),
...(wfDist.expiredWorkflows || []),
...(wfDist.approvedWorkflows || []),
].filter(w => (w.generatedId || '').startsWith('FP#'));
const fpEntry = fpBuckets[0] || null;
if (fpEntry) {
workflow = {
id: fpEntry.generatedId || '',
state: fpEntry.state || '',
type: 'FP',
};
}
}
return {
id: String(f.id),
hostId: f.hostId || f.host?.hostId || null,
title: f.title || '',
severity: typeof f.severity === 'number' ? f.severity : parseFloat(f.severity) || 0,
vrrGroup: f.vrrGroup || f.severityGroup || f.vrr_group || '',
hostName: f.hostName || f.host?.hostName || f.host_name || '',
ipAddress: f.ipAddress || f.host?.ipAddress || f.ip_address || '',
dns: f.dns || f.host?.fqdn || '',
status: f.status || '',
slaStatus: f.slaStatus || f.sla_status || '',
dueDate: dueDate,
lastFoundOn: f.lastFoundOn || f.last_found_on || null,
buOwnership,
cves,
workflow,
};
}
// ---------------------------------------------------------------------------
// Batch insert helper for Postgres
// ---------------------------------------------------------------------------
async function batchInsert(pool, tableName, columns, rows, conflictClause = 'DO NOTHING') {
if (rows.length === 0) return 0;
const BATCH_SIZE = 100;
let inserted = 0;
for (let i = 0; i < rows.length; i += BATCH_SIZE) {
const batch = rows.slice(i, i + BATCH_SIZE);
const values = [];
const placeholders = [];
batch.forEach((row, idx) => {
const offset = idx * columns.length;
const rowPlaceholders = columns.map((_, colIdx) => `$${offset + colIdx + 1}`);
placeholders.push(`(${rowPlaceholders.join(', ')})`);
values.push(...row);
});
const sql = `INSERT INTO ${tableName} (${columns.join(', ')})
VALUES ${placeholders.join(', ')}
ON CONFLICT ${conflictClause}`;
await pool.query(sql, values);
inserted += batch.length;
}
return inserted;
}
// ---------------------------------------------------------------------------
// Table migration definitions
// ---------------------------------------------------------------------------
/**
* Each entry defines how to copy a SQLite table to Postgres.
* - sqliteTable: source table name
* - pgTable: destination table name (defaults to sqliteTable)
* - columns: array of { src, dest, transform } objects
* - conflict: ON CONFLICT clause (default: DO NOTHING)
* - selectSql: optional custom SELECT (defaults to SELECT * FROM sqliteTable)
*/
function getTableMigrations() {
return [
{
sqliteTable: 'users',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'username', dest: 'username' },
{ src: 'email', dest: 'email' },
{ src: 'password_hash', dest: 'password_hash' },
{ src: 'role', dest: 'role' },
{ src: 'is_active', dest: 'is_active', transform: v => v === 1 || v === true },
{ src: 'created_at', dest: 'created_at' },
{ src: 'last_login', dest: 'last_login' },
{ src: 'user_group', dest: 'user_group' },
{ src: 'bu_teams', dest: 'bu_teams' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'sessions',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'session_id', dest: 'session_id' },
{ src: 'user_id', dest: 'user_id' },
{ src: 'expires_at', dest: 'expires_at' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'cves',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'cve_id', dest: 'cve_id' },
{ src: 'vendor', dest: 'vendor' },
{ src: 'severity', dest: 'severity' },
{ src: 'description', dest: 'description' },
{ src: 'published_date', dest: 'published_date' },
{ src: 'status', dest: 'status' },
{ src: 'created_at', dest: 'created_at' },
{ src: 'updated_at', dest: 'updated_at' },
{ src: 'created_by', dest: 'created_by' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'documents',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'cve_id', dest: 'cve_id' },
{ src: 'vendor', dest: 'vendor' },
{ src: 'name', dest: 'name' },
{ src: 'type', dest: 'type' },
{ src: 'file_path', dest: 'file_path' },
{ src: 'file_size', dest: 'file_size' },
{ src: 'mime_type', dest: 'mime_type' },
{ src: 'uploaded_at', dest: 'uploaded_at' },
{ src: 'notes', dest: 'notes' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'required_documents',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'vendor', dest: 'vendor' },
{ src: 'document_type', dest: 'document_type' },
{ src: 'is_mandatory', dest: 'is_mandatory', transform: v => v === 1 || v === true },
{ src: 'description', dest: 'description' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'jira_tickets',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'cve_id', dest: 'cve_id' },
{ src: 'vendor', dest: 'vendor' },
{ src: 'ticket_key', dest: 'ticket_key' },
{ src: 'url', dest: 'url' },
{ src: 'summary', dest: 'summary' },
{ src: 'status', dest: 'status' },
{ src: 'created_at', dest: 'created_at' },
{ src: 'updated_at', dest: 'updated_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'archer_tickets',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'exc_number', dest: 'exc_number' },
{ src: 'archer_url', dest: 'archer_url' },
{ src: 'status', dest: 'status' },
{ src: 'cve_id', dest: 'cve_id' },
{ src: 'vendor', dest: 'vendor' },
{ src: 'created_at', dest: 'created_at' },
{ src: 'updated_at', dest: 'updated_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'knowledge_base',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'title', dest: 'title' },
{ src: 'slug', dest: 'slug' },
{ src: 'description', dest: 'description' },
{ src: 'category', dest: 'category' },
{ src: 'file_path', dest: 'file_path' },
{ src: 'file_name', dest: 'file_name' },
{ src: 'file_type', dest: 'file_type' },
{ src: 'file_size', dest: 'file_size' },
{ src: 'created_at', dest: 'created_at' },
{ src: 'updated_at', dest: 'updated_at' },
{ src: 'created_by', dest: 'created_by' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'audit_logs',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'user_id', dest: 'user_id' },
{ src: 'username', dest: 'username' },
{ src: 'action', dest: 'action' },
{ src: 'entity_type', dest: 'entity_type' },
{ src: 'entity_id', dest: 'entity_id' },
{ src: 'details', dest: 'details' },
{ src: 'ip_address', dest: 'ip_address' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'compliance_uploads',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'filename', dest: 'filename' },
{ src: 'report_date', dest: 'report_date' },
{ src: 'uploaded_by', dest: 'uploaded_by' },
{ src: 'uploaded_at', dest: 'uploaded_at' },
{ src: 'new_count', dest: 'new_count' },
{ src: 'resolved_count', dest: 'resolved_count' },
{ src: 'recurring_count', dest: 'recurring_count' },
{ src: 'summary_json', dest: 'summary_json' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'compliance_items',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'upload_id', dest: 'upload_id' },
{ src: 'hostname', dest: 'hostname' },
{ src: 'ip_address', dest: 'ip_address' },
{ src: 'device_type', dest: 'device_type' },
{ src: 'team', dest: 'team' },
{ src: 'metric_id', dest: 'metric_id' },
{ src: 'metric_desc', dest: 'metric_desc' },
{ src: 'category', dest: 'category' },
{ src: 'extra_json', dest: 'extra_json' },
{ src: 'status', dest: 'status' },
{ src: 'first_seen_upload_id', dest: 'first_seen_upload_id' },
{ src: 'resolved_upload_id', dest: 'resolved_upload_id' },
{ src: 'seen_count', dest: 'seen_count' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'compliance_notes',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'hostname', dest: 'hostname' },
{ src: 'metric_id', dest: 'metric_id' },
{ src: 'note', dest: 'note' },
{ src: 'group_id', dest: 'group_id' },
{ src: 'created_by', dest: 'created_by' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_counts_history',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'open_count', dest: 'open_count' },
{ src: 'closed_count', dest: 'closed_count' },
{ src: 'recorded_at', dest: 'recorded_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_finding_archives',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'finding_id', dest: 'finding_id' },
{ src: 'finding_title', dest: 'finding_title' },
{ src: 'host_name', dest: 'host_name' },
{ src: 'ip_address', dest: 'ip_address' },
{ src: 'current_state', dest: 'current_state' },
{ src: 'last_severity', dest: 'last_severity' },
{ src: 'first_archived_at', dest: 'first_archived_at' },
{ src: 'last_transition_at', dest: 'last_transition_at' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_archive_transitions',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'archive_id', dest: 'archive_id' },
{ src: 'from_state', dest: 'from_state' },
{ src: 'to_state', dest: 'to_state' },
{ src: 'severity_at_transition', dest: 'severity_at_transition' },
{ src: 'reason', dest: 'reason' },
{ src: 'transitioned_at', dest: 'transitioned_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_sync_anomaly_log',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'sync_timestamp', dest: 'sync_timestamp' },
{ src: 'open_count_delta', dest: 'open_count_delta' },
{ src: 'closed_count_delta', dest: 'closed_count_delta' },
{ src: 'newly_archived_count', dest: 'newly_archived_count' },
{ src: 'returned_count', dest: 'returned_count' },
{ src: 'classification_json', dest: 'classification_json' },
{ src: 'return_classification_json', dest: 'return_classification_json' },
{ src: 'is_significant', dest: 'is_significant', transform: v => v === 1 || v === true },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_finding_bu_history',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'finding_id', dest: 'finding_id' },
{ src: 'finding_title', dest: 'finding_title' },
{ src: 'host_name', dest: 'host_name' },
{ src: 'previous_bu', dest: 'previous_bu' },
{ src: 'new_bu', dest: 'new_bu' },
{ src: 'detected_at', dest: 'detected_at' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'atlas_action_plans_cache',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'host_id', dest: 'host_id' },
{ src: 'has_action_plan', dest: 'has_action_plan', transform: v => v === 1 || v === true },
{ src: 'plan_count', dest: 'plan_count' },
{ src: 'plans_json', dest: 'plans_json' },
{ src: 'synced_at', dest: 'synced_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_fp_submissions',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'user_id', dest: 'user_id' },
{ src: 'username', dest: 'username' },
{ src: 'ivanti_workflow_batch_id', dest: 'ivanti_workflow_batch_id' },
{ src: 'ivanti_generated_id', dest: 'ivanti_generated_id' },
{ src: 'ivanti_workflow_batch_uuid', dest: 'ivanti_workflow_batch_uuid' },
{ src: 'workflow_name', dest: 'workflow_name' },
{ src: 'reason', dest: 'reason' },
{ src: 'description', dest: 'description' },
{ src: 'expiration_date', dest: 'expiration_date' },
{ src: 'scope_override', dest: 'scope_override' },
{ src: 'finding_ids_json', dest: 'finding_ids_json' },
{ src: 'queue_item_ids_json', dest: 'queue_item_ids_json' },
{ src: 'attachment_count', dest: 'attachment_count' },
{ src: 'attachment_results_json', dest: 'attachment_results_json' },
{ src: 'status', dest: 'status' },
{ src: 'lifecycle_status', dest: 'lifecycle_status' },
{ src: 'error_message', dest: 'error_message' },
{ src: 'created_at', dest: 'created_at' },
{ src: 'updated_at', dest: 'updated_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_fp_submission_history',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'submission_id', dest: 'submission_id' },
{ src: 'user_id', dest: 'user_id' },
{ src: 'username', dest: 'username' },
{ src: 'change_type', dest: 'change_type' },
{ src: 'change_details_json', dest: 'change_details_json' },
{ src: 'created_at', dest: 'created_at' },
],
conflict: '(id) DO NOTHING',
},
{
sqliteTable: 'ivanti_todo_queue',
columns: [
{ src: 'id', dest: 'id' },
{ src: 'user_id', dest: 'user_id' },
{ src: 'finding_id', dest: 'finding_id' },
{ src: 'finding_title', dest: 'finding_title' },
{ src: 'cves_json', dest: 'cves_json' },
{ src: 'ip_address', dest: 'ip_address' },
{ src: 'hostname', dest: 'hostname' },
{ src: 'vendor', dest: 'vendor' },
{ src: 'workflow_type', dest: 'workflow_type' },
{ src: 'status', dest: 'status' },
{ src: 'created_at', dest: 'created_at' },
{ src: 'updated_at', dest: 'updated_at' },
],
conflict: '(id) DO NOTHING',
},
];
}
// ---------------------------------------------------------------------------
// Main migration logic
// ---------------------------------------------------------------------------
async function migrate() {
console.log('╔════════════════════════════════════════════════════════╗');
console.log('║ CVE Dashboard — SQLite → PostgreSQL Migration ║');
console.log('╚════════════════════════════════════════════════════════╝\n');
// Open SQLite in READ-ONLY mode
const sqliteDb = new sqlite3.Database(SQLITE_PATH, sqlite3.OPEN_READONLY, (err) => {
if (err) {
console.error('ERROR: Failed to open SQLite database:', err.message);
process.exit(1);
}
});
console.log(`✓ Opened SQLite database (read-only): ${SQLITE_PATH}`);
// Connect to Postgres
const pool = new Pool({
connectionString: DATABASE_URL,
max: 5,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 10000,
});
try {
await pool.query('SELECT NOW()');
console.log('✓ Connected to PostgreSQL');
} catch (err) {
console.error('ERROR: Failed to connect to PostgreSQL:', err.message);
sqliteDb.close();
process.exit(1);
}
// Step 1: Run schema DDL
console.log('\n── Step 1: Creating schema (idempotent) ──');
try {
const schemaSql = fs.readFileSync(SCHEMA_PATH, 'utf8');
await pool.query(schemaSql);
console.log('✓ Schema created/verified');
} catch (err) {
console.error('ERROR: Schema creation failed:', err.message);
await cleanup(sqliteDb, pool);
process.exit(1);
}
// Step 2: Copy simple tables
console.log('\n── Step 2: Copying tables ──');
const migrations = getTableMigrations();
const migrationResults = {};
for (const migration of migrations) {
const tableName = migration.pgTable || migration.sqliteTable;
try {
// Check if table exists in SQLite
const tableCheck = await sqliteGet(
sqliteDb,
`SELECT name FROM sqlite_master WHERE type='table' AND name=?`,
[migration.sqliteTable]
);
if (!tableCheck) {
console.log(`${migration.sqliteTable} — table not found in SQLite, skipping`);
migrationResults[tableName] = { source: 0, dest: 0, skipped: true };
continue;
}
// Read all rows from SQLite
const selectSql = migration.selectSql || `SELECT * FROM ${migration.sqliteTable}`;
const sourceRows = await sqliteAll(sqliteDb, selectSql);
if (sourceRows.length === 0) {
console.log(`${tableName} — 0 rows (empty table)`);
migrationResults[tableName] = { source: 0, dest: 0 };
continue;
}
// Transform rows
const destColumns = migration.columns.map(c => c.dest);
const transformedRows = sourceRows.map(row => {
return migration.columns.map(col => {
let value = row[col.src];
if (value === undefined) value = null;
if (col.transform && value !== null) {
value = col.transform(value);
}
return value;
});
});
// Insert into Postgres
const inserted = await batchInsert(
pool,
tableName,
destColumns,
transformedRows,
migration.conflict
);
console.log(`${tableName}${inserted} rows copied`);
migrationResults[tableName] = { source: sourceRows.length, dest: inserted };
} catch (err) {
console.error(`${tableName} — ERROR: ${err.message}`);
migrationResults[tableName] = { source: 0, dest: 0, error: err.message };
}
}
// Reset sequences for SERIAL columns after bulk insert with explicit IDs
console.log('\n── Step 2b: Resetting sequences ──');
const serialTables = [
'users', 'sessions', 'cves', 'documents', 'required_documents',
'jira_tickets', 'archer_tickets', 'knowledge_base', 'audit_logs',
'compliance_uploads', 'compliance_items', 'compliance_notes',
'ivanti_counts_history', 'ivanti_finding_archives',
'ivanti_archive_transitions', 'ivanti_sync_anomaly_log',
'ivanti_finding_bu_history', 'atlas_action_plans_cache',
'ivanti_fp_submissions', 'ivanti_fp_submission_history',
'ivanti_todo_queue',
];
for (const table of serialTables) {
try {
await pool.query(`
SELECT setval(pg_get_serial_sequence('${table}', 'id'),
COALESCE((SELECT MAX(id) FROM ${table}), 0) + 1, false)
`);
} catch (err) {
// Non-fatal — sequence may not exist for some tables
console.log(` ⚠ Could not reset sequence for ${table}: ${err.message}`);
}
}
console.log('✓ Sequences reset');
// Step 3: Migrate findings from JSON blob
console.log('\n── Step 3: Migrating findings from JSON blob ──');
try {
const cacheRow = await sqliteGet(sqliteDb, 'SELECT * FROM ivanti_findings_cache WHERE id = 1');
if (!cacheRow || !cacheRow.findings_json) {
console.log(' ⚠ No findings_json data found in ivanti_findings_cache');
} else {
let findings;
try {
findings = JSON.parse(cacheRow.findings_json);
} catch (parseErr) {
console.error(' ✗ Failed to parse findings_json:', parseErr.message);
findings = [];
}
if (Array.isArray(findings) && findings.length > 0) {
console.log(` Parsing ${findings.length} findings from JSON blob...`);
// Extract and insert findings
const BATCH_SIZE = 100;
let insertedCount = 0;
for (let i = 0; i < findings.length; i += BATCH_SIZE) {
const batch = findings.slice(i, i + BATCH_SIZE);
const values = [];
const placeholders = [];
batch.forEach((rawFinding, idx) => {
const f = extractFinding(rawFinding);
const offset = idx * 18;
values.push(
f.id,
f.hostId,
f.title,
f.severity,
f.vrrGroup,
f.hostName,
f.ipAddress,
f.dns,
f.status,
f.slaStatus,
f.dueDate,
f.lastFoundOn,
f.buOwnership,
f.cves,
f.workflow ? f.workflow.id : null,
f.workflow ? f.workflow.state : null,
f.workflow ? f.workflow.type : null,
'open' // state = open for all findings from cache
);
placeholders.push(
`($${offset+1}, $${offset+2}, $${offset+3}, $${offset+4}, $${offset+5}, ` +
`$${offset+6}, $${offset+7}, $${offset+8}, $${offset+9}, $${offset+10}, ` +
`$${offset+11}, $${offset+12}, $${offset+13}, $${offset+14}, $${offset+15}, ` +
`$${offset+16}, $${offset+17}, $${offset+18})`
);
});
await pool.query(`
INSERT INTO ivanti_findings (
id, host_id, title, severity, vrr_group,
host_name, ip_address, dns, status, sla_status,
due_date, last_found_on, bu_ownership, cves,
workflow_id, workflow_state, workflow_type, state
)
VALUES ${placeholders.join(', ')}
ON CONFLICT (id) DO NOTHING
`, values);
insertedCount += batch.length;
}
console.log(` ✓ ivanti_findings — ${insertedCount} findings inserted (state='open')`);
migrationResults['ivanti_findings'] = { source: findings.length, dest: insertedCount };
} else {
console.log(' ○ findings_json is empty or not an array');
migrationResults['ivanti_findings'] = { source: 0, dest: 0 };
}
}
} catch (err) {
console.error(` ✗ Findings migration ERROR: ${err.message}`);
migrationResults['ivanti_findings'] = { source: 0, dest: 0, error: err.message };
}
// Step 4: Merge notes into ivanti_findings.note
console.log('\n── Step 4: Merging finding notes ──');
try {
const notes = await sqliteAll(sqliteDb, 'SELECT finding_id, note FROM ivanti_finding_notes');
if (notes.length === 0) {
console.log(' ○ No finding notes to merge');
} else {
let mergedCount = 0;
for (const { finding_id, note } of notes) {
if (!finding_id || !note) continue;
const result = await pool.query(
`UPDATE ivanti_findings SET note = $1 WHERE id = $2`,
[note, finding_id]
);
if (result.rowCount > 0) mergedCount++;
}
console.log(` ✓ Merged ${mergedCount}/${notes.length} notes into ivanti_findings.note`);
}
} catch (err) {
console.error(` ✗ Notes merge ERROR: ${err.message}`);
}
// Step 5: Merge overrides into ivanti_findings.override_host_name / override_dns
console.log('\n── Step 5: Merging finding overrides ──');
try {
const overrides = await sqliteAll(
sqliteDb,
'SELECT finding_id, field, value FROM ivanti_finding_overrides'
);
if (overrides.length === 0) {
console.log(' ○ No finding overrides to merge');
} else {
let mergedCount = 0;
for (const { finding_id, field, value } of overrides) {
if (!finding_id || !field) continue;
let pgColumn;
if (field === 'host_name' || field === 'hostName' || field === 'override_host_name') {
pgColumn = 'override_host_name';
} else if (field === 'dns' || field === 'override_dns') {
pgColumn = 'override_dns';
} else {
// Unknown field — skip
continue;
}
const result = await pool.query(
`UPDATE ivanti_findings SET ${pgColumn} = $1 WHERE id = $2`,
[value, finding_id]
);
if (result.rowCount > 0) mergedCount++;
}
console.log(` ✓ Merged ${mergedCount}/${overrides.length} overrides into ivanti_findings`);
}
} catch (err) {
console.error(` ✗ Overrides merge ERROR: ${err.message}`);
}
// Step 6: Populate ivanti_sync_state from ivanti_findings_cache metadata
console.log('\n── Step 6: Populating sync state and counts cache ──');
try {
const cacheRow = await sqliteGet(sqliteDb, 'SELECT * FROM ivanti_findings_cache WHERE id = 1');
if (cacheRow) {
await pool.query(`
UPDATE ivanti_sync_state SET
total = $1,
synced_at = $2,
sync_status = $3,
error_message = $4
WHERE id = 1
`, [
cacheRow.total || 0,
cacheRow.synced_at || null,
cacheRow.sync_status || 'never',
cacheRow.error_message || null,
]);
console.log(' ✓ ivanti_sync_state updated from ivanti_findings_cache metadata');
}
// Populate ivanti_counts_cache
const countsRow = await sqliteGet(sqliteDb, 'SELECT * FROM ivanti_counts_cache WHERE id = 1');
if (countsRow) {
await pool.query(`
UPDATE ivanti_counts_cache SET
open_count = $1,
closed_count = $2,
synced_at = $3,
fp_workflow_counts_json = $4,
fp_id_counts_json = $5
WHERE id = 1
`, [
countsRow.open_count || 0,
countsRow.closed_count || 0,
countsRow.synced_at || null,
countsRow.fp_workflow_counts_json || '{}',
countsRow.fp_id_counts_json || '{}',
]);
console.log(' ✓ ivanti_counts_cache updated');
}
} catch (err) {
console.error(` ✗ Sync state/counts migration ERROR: ${err.message}`);
}
// Step 7: Verification — compare row counts
console.log('\n── Step 7: Verification ──');
console.log('');
console.log('┌─────────────────────────────────┬──────────┬──────────┬────────┐');
console.log('│ Table │ SQLite │ Postgres │ Status │');
console.log('├─────────────────────────────────┼──────────┼──────────┼────────┤');
let hasDiscrepancy = false;
const verificationTables = [
...migrations.map(m => ({ sqlite: m.sqliteTable, pg: m.pgTable || m.sqliteTable })),
{ sqlite: null, pg: 'ivanti_findings', special: true },
];
for (const { sqlite: sqliteTable, pg: pgTable, special } of verificationTables) {
let sqliteCount = 0;
let pgCount = 0;
try {
if (special && pgTable === 'ivanti_findings') {
// For findings, source count is from the JSON blob
const cacheRow = await sqliteGet(sqliteDb, 'SELECT findings_json FROM ivanti_findings_cache WHERE id = 1');
if (cacheRow && cacheRow.findings_json) {
try {
const parsed = JSON.parse(cacheRow.findings_json);
sqliteCount = Array.isArray(parsed) ? parsed.length : 0;
} catch (e) {
sqliteCount = 0;
}
}
} else if (sqliteTable) {
const tableExists = await sqliteGet(
sqliteDb,
`SELECT name FROM sqlite_master WHERE type='table' AND name=?`,
[sqliteTable]
);
if (tableExists) {
const countRow = await sqliteGet(sqliteDb, `SELECT COUNT(*) as cnt FROM ${sqliteTable}`);
sqliteCount = countRow ? countRow.cnt : 0;
}
}
const pgCountRow = await pool.query(`SELECT COUNT(*) as cnt FROM ${pgTable}`);
pgCount = parseInt(pgCountRow.rows[0].cnt, 10);
} catch (err) {
// Table might not exist in one or both
}
const status = pgCount >= sqliteCount ? ' OK ' : ' WARN ';
if (pgCount < sqliteCount) hasDiscrepancy = true;
const tableDisplay = (pgTable || '').padEnd(31);
const srcDisplay = String(sqliteCount).padStart(6);
const destDisplay = String(pgCount).padStart(6);
console.log(`${tableDisplay}${srcDisplay}${destDisplay}${status}`);
}
console.log('└─────────────────────────────────┴──────────┴──────────┴────────┘');
if (hasDiscrepancy) {
console.log('\n⚠ WARNING: Some tables have fewer rows in Postgres than SQLite.');
console.log(' This may be due to ON CONFLICT DO NOTHING skipping existing rows,');
console.log(' or foreign key constraints preventing insertion.');
}
// Cleanup
await cleanup(sqliteDb, pool);
console.log('\n════════════════════════════════════════════════════════');
if (hasDiscrepancy) {
console.log('Migration completed with warnings. Review discrepancies above.');
} else {
console.log('✓ Migration completed successfully!');
}
console.log('════════════════════════════════════════════════════════\n');
process.exit(hasDiscrepancy ? 0 : 0); // Exit 0 even with warnings (data is safe)
}
// ---------------------------------------------------------------------------
// Cleanup helper
// ---------------------------------------------------------------------------
function cleanup(sqliteDb, pool) {
return new Promise((resolve) => {
sqliteDb.close((err) => {
if (err) console.error('Warning: Error closing SQLite:', err.message);
pool.end()
.then(() => resolve())
.catch(() => resolve());
});
});
}
// ---------------------------------------------------------------------------
// Entry point
// ---------------------------------------------------------------------------
migrate().catch((err) => {
console.error('\n✗ FATAL ERROR:', err.message);
console.error(err.stack);
process.exit(1);
});

View File

@@ -0,0 +1,208 @@
#!/usr/bin/env python3
"""
Parse NTS_AEO compliance xlsx file and write JSON to stdout.
Usage: python3 parse_compliance_xlsx.py <path_to_xlsx>
Output:
{
"items": [...], # non-compliant asset rows
"summary": { ... }, # metric health data from Summary sheet
"report_date": "YYYY-MM-DD" | null,
"total": int
}
"""
import sys
import os
import json
import re
import pandas as pd
from pathlib import Path
def load_config():
"""Load parser configuration from compliance_config.json."""
script_dir = os.path.dirname(os.path.abspath(__file__))
config_path = os.path.join(script_dir, 'compliance_config.json')
try:
with open(config_path, 'r') as f:
config = json.load(f)
except FileNotFoundError:
print(f"Error: Configuration file not found: {config_path}", file=sys.stderr)
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in configuration file {config_path}: {e}", file=sys.stderr)
sys.exit(1)
return config
_config = load_config()
METRIC_CATEGORIES = _config['metric_categories']
CORE_COLS = set(_config['core_cols'])
SKIP_SHEETS = set(_config['skip_sheets'])
def safe_str(val):
s = str(val).strip()
return '' if s == 'nan' else s
def parse_summary(xl):
"""Return { entries: [...], overall_scores: { customer_network, vertical } }"""
df_raw = pd.read_excel(xl, sheet_name='Summary', header=None)
overall_scores = {
'customer_network': float(df_raw.iloc[0, 4]) if pd.notna(df_raw.iloc[0, 4]) else None,
'vertical': float(df_raw.iloc[1, 4]) if pd.notna(df_raw.iloc[1, 4]) else None,
}
df = pd.read_excel(xl, sheet_name='Summary', header=3)
# Flatten any newlines in column names
df.columns = [str(c).replace('\n', ' ').strip() for c in df.columns]
# Locate the sub-vertical/team column robustly
team_col = next((c for c in df.columns if 'Sub-Vertical' in c or 'Purchase Group' in c), None)
entries = []
for _, row in df.iterrows():
metric_id = safe_str(row.get('Metric', ''))
if not metric_id or metric_id in ('Metric',):
continue
team = safe_str(row.get(team_col, '')) if team_col else ''
try:
non_compliant = int(row.get('Non-Compliant', 0) or 0)
compliant = int(row.get('Compliant', 0) or 0)
total = int(row.get('Total', 0) or 0)
compliance_pct = float(row.get('Current Compliance', 0) or 0)
target = float(row.get('Metric Target', 0) or 0)
except (ValueError, TypeError):
continue
entries.append({
'metric_id': metric_id,
'team': team,
'priority': safe_str(row.get('Priority / Non-Priority / IR', '')),
'non_compliant': non_compliant,
'compliant': compliant,
'total': total,
'compliance_pct': compliance_pct,
'target': target,
'status': safe_str(row.get('Status', '')),
'description': safe_str(row.get('Metric Description', '')),
'category': METRIC_CATEGORIES.get(metric_id, 'Other'),
})
return {'entries': entries, 'overall_scores': overall_scores}
def parse_sheet(xl, sheet_name, summary_entries):
"""Return list of non-compliant item dicts for a detail sheet."""
try:
df = pd.read_excel(xl, sheet_name=sheet_name, header=0)
except Exception:
return []
if df.empty:
return []
df.columns = [str(c).strip() for c in df.columns]
# Filter to non-compliant rows when the Compliant column exists
if 'Compliant' in df.columns:
df = df[df['Compliant'] == False]
if df.empty:
return []
# Look up description from summary
metric_desc = ''
for e in summary_entries:
if e['metric_id'] == sheet_name and e['description']:
metric_desc = e['description']
break
category = METRIC_CATEGORIES.get(sheet_name, 'Other')
items = []
for _, row in df.iterrows():
hostname = safe_str(row.get('Preferred - Hostname', ''))
if not hostname:
continue
ip = safe_str(row.get('GRANITE - IPv4_Address', ''))
device_type = safe_str(row.get('GRANITE - Type', ''))
team = safe_str(row.get('Team', ''))
# Everything non-core goes into extra_json
extra = {}
for col in df.columns:
if col in CORE_COLS:
continue
val = row.get(col)
if pd.isna(val) if not isinstance(val, str) else False:
continue
s = safe_str(val)
if s:
extra[col] = val.isoformat() if hasattr(val, 'isoformat') else s
items.append({
'hostname': hostname,
'ip_address': ip,
'device_type': device_type,
'team': team,
'metric_id': sheet_name,
'metric_desc': metric_desc,
'category': category,
'extra_json': extra,
})
return items
def extract_report_date(filepath):
"""Try to pull YYYY-MM-DD from the filename, e.g. NTS_AEO_2026_03_25.xlsx"""
stem = Path(filepath).stem
m = re.search(r'(\d{4})_(\d{2})_(\d{2})', stem)
if m:
return f"{m.group(1)}-{m.group(2)}-{m.group(3)}"
return None
def main():
if len(sys.argv) < 2:
print(json.dumps({'error': 'No file path provided'}))
sys.exit(1)
filepath = sys.argv[1]
try:
xl = pd.ExcelFile(filepath)
except Exception as e:
print(json.dumps({'error': f'Cannot open file: {str(e)}'}))
sys.exit(1)
try:
summary = parse_summary(xl)
except Exception as e:
summary = {'entries': [], 'overall_scores': {}, 'parse_error': str(e)}
all_items = []
for sheet_name in xl.sheet_names:
if sheet_name in SKIP_SHEETS:
continue
items = parse_sheet(xl, sheet_name, summary.get('entries', []))
all_items.extend(items)
print(json.dumps({
'items': all_items,
'summary': summary,
'report_date': extract_report_date(filepath),
'total': len(all_items),
}))
if __name__ == '__main__':
main()

View File

@@ -1,83 +0,0 @@
#!/usr/bin/env python3
"""
CVE Report Splitter
Splits multiple CVE IDs in a single row into separate rows for easier filtering and analysis.
"""
import pandas as pd
import sys
from pathlib import Path
def split_cve_report(input_file, output_file=None, sheet_name='Vulnerabilities', cve_column='CVE ID'):
"""
Split CVE IDs into separate rows.
Args:
input_file: Path to input Excel file
output_file: Path to output file (default: adds '_Split' to input filename)
sheet_name: Name of sheet with vulnerability data (default: 'Vulnerabilities')
cve_column: Name of column containing CVE IDs (default: 'CVE ID')
"""
input_path = Path(input_file)
if not input_path.exists():
print(f"Error: File not found: {input_file}")
sys.exit(1)
if output_file is None:
output_file = input_path.parent / f"{input_path.stem}_Split{input_path.suffix}"
print(f"Reading: {input_file}")
try:
df = pd.read_excel(input_file, sheet_name=sheet_name)
except ValueError as e:
print(f"Error: Sheet '{sheet_name}' not found in workbook")
print(f"Available sheets: {pd.ExcelFile(input_file).sheet_names}")
sys.exit(1)
if cve_column not in df.columns:
print(f"Error: Column '{cve_column}' not found")
print(f"Available columns: {list(df.columns)}")
sys.exit(1)
original_rows = len(df)
print(f"Original rows: {original_rows}")
# Split CVE IDs by comma
df[cve_column] = df[cve_column].astype(str).str.split(',')
# Explode to create separate rows
df_exploded = df.explode(cve_column)
# Clean up CVE IDs
df_exploded[cve_column] = df_exploded[cve_column].str.strip()
df_exploded = df_exploded[df_exploded[cve_column].notna()]
df_exploded = df_exploded[df_exploded[cve_column] != 'nan']
df_exploded = df_exploded[df_exploded[cve_column] != '']
# Reset index
df_exploded = df_exploded.reset_index(drop=True)
new_rows = len(df_exploded)
print(f"New rows: {new_rows}")
print(f"Added {new_rows - original_rows} rows from splitting CVEs")
# Save output
df_exploded.to_excel(output_file, index=False, sheet_name=sheet_name)
print(f"\n✓ Success! Saved to: {output_file}")
return output_file
if __name__ == "__main__":
if len(sys.argv) < 2:
print("Usage: python3 split_cve_report.py <input_file.xlsx> [output_file.xlsx]")
print("\nExample:")
print(" python3 split_cve_report.py 'Vulnerability Workbook.xlsx'")
print(" python3 split_cve_report.py 'input.xlsx' 'output.xlsx'")
sys.exit(1)
input_file = sys.argv[1]
output_file = sys.argv[2] if len(sys.argv) > 2 else None
split_cve_report(input_file, output_file)

File diff suppressed because it is too large Load Diff

49
backend/setup-postgres.js Normal file
View File

@@ -0,0 +1,49 @@
// Setup Script for CVE Dashboard — PostgreSQL
// Runs the db-schema.sql DDL against the Postgres instance configured in DATABASE_URL.
// Idempotent — safe to run multiple times.
//
// Usage: node backend/setup-postgres.js
//
// Requires DATABASE_URL in .env or environment.
require('dotenv').config({ path: require('path').join(__dirname, '.env') });
const fs = require('fs');
const path = require('path');
const pool = require('./db');
const SCHEMA_FILE = path.join(__dirname, 'db-schema.sql');
async function main() {
console.log('🚀 CVE Dashboard — PostgreSQL Schema Setup\n');
console.log('════════════════════════════════════════\n');
try {
// Verify connection
const { rows } = await pool.query('SELECT version()');
console.log(`✓ Connected to: ${rows[0].version.split(',')[0]}`);
console.log(` Database URL: ${process.env.DATABASE_URL.replace(/:[^:@]+@/, ':***@')}\n`);
// Read and execute schema
const schema = fs.readFileSync(SCHEMA_FILE, 'utf8');
await pool.query(schema);
console.log('✓ Schema created/verified (all tables and indexes)\n');
// Verify table count
const { rows: tables } = await pool.query(
"SELECT COUNT(*) as count FROM information_schema.tables WHERE table_schema = 'public'"
);
console.log(`${tables[0].count} tables in database\n`);
console.log('╔════════════════════════════════════════════════════════╗');
console.log('║ POSTGRESQL SCHEMA SETUP COMPLETE ║');
console.log('╚════════════════════════════════════════════════════════╝\n');
} catch (err) {
console.error('❌ Setup failed:', err.message);
process.exit(1);
} finally {
await pool.end();
}
}
main();

View File

@@ -1,333 +1,642 @@
// Setup Script for CVE Database // Setup Script for CVE Dashboard v1.0.0
// This creates a fresh database with multi-vendor support built-in // Creates a fresh database with the complete schema including all tables,
// indexes, triggers, and views needed for a new deployment.
//
// Usage: node backend/setup.js
//
// This consolidates the original schema plus all migration scripts into a
// single idempotent setup. Migration scripts in backend/migrations/ are
// retained for reference but are NOT needed on fresh deployments.
const sqlite3 = require('sqlite3').verbose(); const sqlite3 = require('sqlite3').verbose();
const bcrypt = require('bcryptjs'); const bcrypt = require('bcryptjs');
const crypto = require('crypto');
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const DB_FILE = './cve_database.db'; const DB_FILE = path.join(__dirname, 'cve_database.db');
const UPLOADS_DIR = './uploads'; const UPLOADS_DIR = path.join(__dirname, 'uploads');
// Initialize database with schema // ---------------------------------------------------------------------------
function initializeDatabase() { // Database helpers
// ---------------------------------------------------------------------------
function dbRun(db, sql, params = []) {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const db = new sqlite3.Database(DB_FILE, (err) => { db.run(sql, params, function (err) {
if (err) reject(err); if (err) reject(err);
}); else resolve(this);
const schema = `
CREATE TABLE IF NOT EXISTS cves (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
severity VARCHAR(20) NOT NULL,
description TEXT,
published_date DATE,
status VARCHAR(50) DEFAULT 'Open',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(cve_id, vendor)
);
CREATE TABLE IF NOT EXISTS documents (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
name VARCHAR(255) NOT NULL,
type VARCHAR(50) NOT NULL,
file_path VARCHAR(500) NOT NULL,
file_size VARCHAR(20),
mime_type VARCHAR(100),
uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
notes TEXT,
FOREIGN KEY (cve_id) REFERENCES cves(cve_id) ON DELETE CASCADE
);
CREATE TABLE IF NOT EXISTS required_documents (
id INTEGER PRIMARY KEY AUTOINCREMENT,
vendor VARCHAR(100) NOT NULL,
document_type VARCHAR(50) NOT NULL,
is_mandatory BOOLEAN DEFAULT 1,
description TEXT
);
CREATE INDEX IF NOT EXISTS idx_cve_id ON cves(cve_id);
CREATE INDEX IF NOT EXISTS idx_vendor ON cves(vendor);
CREATE INDEX IF NOT EXISTS idx_severity ON cves(severity);
CREATE INDEX IF NOT EXISTS idx_status ON cves(status);
CREATE INDEX IF NOT EXISTS idx_doc_cve_id ON documents(cve_id);
CREATE INDEX IF NOT EXISTS idx_doc_vendor ON documents(vendor);
CREATE INDEX IF NOT EXISTS idx_doc_type ON documents(type);
-- Users table for authentication
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username VARCHAR(50) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
role VARCHAR(20) NOT NULL DEFAULT 'viewer',
is_active BOOLEAN DEFAULT 1,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
last_login TIMESTAMP,
CHECK (role IN ('admin', 'editor', 'viewer'))
);
-- Sessions table for session management
CREATE TABLE IF NOT EXISTS sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id VARCHAR(255) UNIQUE NOT NULL,
user_id INTEGER NOT NULL,
expires_at TIMESTAMP NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_sessions_session_id ON sessions(session_id);
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions(user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_expires ON sessions(expires_at);
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
-- Audit log table for tracking user actions
CREATE TABLE IF NOT EXISTS audit_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER,
username VARCHAR(50) NOT NULL,
action VARCHAR(50) NOT NULL,
entity_type VARCHAR(50) NOT NULL,
entity_id VARCHAR(100),
details TEXT,
ip_address VARCHAR(45),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_id ON audit_logs(user_id);
CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_logs(action);
CREATE INDEX IF NOT EXISTS idx_audit_entity_type ON audit_logs(entity_type);
CREATE INDEX IF NOT EXISTS idx_audit_created_at ON audit_logs(created_at);
INSERT OR IGNORE INTO required_documents (vendor, document_type, is_mandatory, description) VALUES
('Microsoft', 'advisory', 1, 'Official Microsoft Security Advisory'),
('Microsoft', 'screenshot', 0, 'Proof of patch application'),
('Cisco', 'advisory', 1, 'Cisco Security Advisory'),
('Oracle', 'advisory', 1, 'Oracle Security Alert'),
('VMware', 'advisory', 1, 'VMware Security Advisory'),
('Adobe', 'advisory', 1, 'Adobe Security Bulletin');
CREATE VIEW IF NOT EXISTS cve_document_status AS
SELECT
c.id as record_id,
c.cve_id,
c.vendor,
c.severity,
c.status,
COUNT(DISTINCT d.id) as total_documents,
COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) as advisory_count,
COUNT(DISTINCT CASE WHEN d.type = 'email' THEN d.id END) as email_count,
COUNT(DISTINCT CASE WHEN d.type = 'screenshot' THEN d.id END) as screenshot_count,
CASE
WHEN COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) > 0
THEN 'Complete'
ELSE 'Missing Required Docs'
END as compliance_status
FROM cves c
LEFT JOIN documents d ON c.cve_id = d.cve_id AND c.vendor = d.vendor
GROUP BY c.id, c.cve_id, c.vendor, c.severity, c.status;
`;
db.exec(schema, (err) => {
if (err) {
reject(err);
} else {
console.log('✓ Database initialized successfully');
resolve(db);
}
}); });
}); });
} }
// Create uploads directory structure function dbGet(db, sql, params = []) {
function createUploadsDirectory() {
if (!fs.existsSync(UPLOADS_DIR)) {
fs.mkdirSync(UPLOADS_DIR, { recursive: true });
console.log('✓ Created uploads directory');
} else {
console.log('✓ Uploads directory already exists');
}
}
// Create default admin user
async function createDefaultAdmin(db) {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
// Check if admin already exists db.get(sql, params, (err, row) => {
db.get('SELECT id FROM users WHERE username = ?', ['admin'], async (err, row) => { if (err) reject(err);
if (err) { else resolve(row);
reject(err);
return;
}
if (row) {
console.log('✓ Default admin user already exists');
resolve();
return;
}
// Create admin user with password 'admin123'
const passwordHash = await bcrypt.hash('admin123', 10);
db.run(
`INSERT INTO users (username, email, password_hash, role, is_active)
VALUES (?, ?, ?, ?, ?)`,
['admin', 'admin@localhost', passwordHash, 'admin', 1],
(err) => {
if (err) {
reject(err);
} else {
console.log('✓ Created default admin user (admin/admin123)');
resolve();
}
}
);
}); });
}); });
} }
// Add sample CVE data (optional - for testing) function dbExec(db, sql) {
async function addSampleData(db) { return new Promise((resolve, reject) => {
console.log('\n📝 Adding sample CVE data for testing...'); db.exec(sql, (err) => {
if (err) reject(err);
const sampleCVEs = [ else resolve();
{ });
cve_id: 'CVE-2024-SAMPLE-1', });
vendor: 'Microsoft', }
severity: 'Critical',
description: 'Sample remote code execution vulnerability', // ---------------------------------------------------------------------------
published_date: '2024-01-15' // Schema — complete v1.0.0 database structure
}, // ---------------------------------------------------------------------------
{ async function initializeDatabase(db) {
cve_id: 'CVE-2024-SAMPLE-1', await dbExec(db, `
vendor: 'Cisco',
severity: 'High', -- =================================================================
description: 'Sample remote code execution vulnerability', -- Core CVE tracking tables
published_date: '2024-01-15' -- =================================================================
}
CREATE TABLE IF NOT EXISTS cves (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
severity VARCHAR(20) NOT NULL,
description TEXT,
published_date DATE,
status VARCHAR(50) DEFAULT 'Open',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(cve_id, vendor)
);
CREATE TABLE IF NOT EXISTS documents (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id VARCHAR(20) NOT NULL,
vendor VARCHAR(100) NOT NULL,
name VARCHAR(255) NOT NULL,
type VARCHAR(50) NOT NULL,
file_path VARCHAR(500) NOT NULL,
file_size VARCHAR(20),
mime_type VARCHAR(100),
uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
notes TEXT,
FOREIGN KEY (cve_id) REFERENCES cves(cve_id) ON DELETE CASCADE
);
CREATE TABLE IF NOT EXISTS required_documents (
id INTEGER PRIMARY KEY AUTOINCREMENT,
vendor VARCHAR(100) NOT NULL,
document_type VARCHAR(50) NOT NULL,
is_mandatory BOOLEAN DEFAULT 1,
description TEXT
);
CREATE INDEX IF NOT EXISTS idx_cve_id ON cves(cve_id);
CREATE INDEX IF NOT EXISTS idx_vendor ON cves(vendor);
CREATE INDEX IF NOT EXISTS idx_severity ON cves(severity);
CREATE INDEX IF NOT EXISTS idx_status ON cves(status);
CREATE INDEX IF NOT EXISTS idx_doc_cve_id ON documents(cve_id);
CREATE INDEX IF NOT EXISTS idx_doc_vendor ON documents(vendor);
CREATE INDEX IF NOT EXISTS idx_doc_type ON documents(type);
-- =================================================================
-- Authentication and session management
-- =================================================================
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username VARCHAR(50) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
role VARCHAR(20) NOT NULL DEFAULT 'viewer',
is_active BOOLEAN DEFAULT 1,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
last_login TIMESTAMP,
user_group VARCHAR(20) NOT NULL DEFAULT 'Read_Only',
bu_teams TEXT NOT NULL DEFAULT '',
CHECK (role IN ('admin', 'editor', 'viewer'))
);
CREATE TABLE IF NOT EXISTS sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id VARCHAR(255) UNIQUE NOT NULL,
user_id INTEGER NOT NULL,
expires_at TIMESTAMP NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_sessions_session_id ON sessions(session_id);
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions(user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_expires ON sessions(expires_at);
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
CREATE INDEX IF NOT EXISTS idx_users_user_group ON users(user_group);
-- =================================================================
-- Audit logging
-- =================================================================
CREATE TABLE IF NOT EXISTS audit_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER,
username VARCHAR(50) NOT NULL,
action VARCHAR(50) NOT NULL,
entity_type VARCHAR(50) NOT NULL,
entity_id VARCHAR(100),
details TEXT,
ip_address VARCHAR(45),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_id ON audit_logs(user_id);
CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_logs(action);
CREATE INDEX IF NOT EXISTS idx_audit_entity_type ON audit_logs(entity_type);
CREATE INDEX IF NOT EXISTS idx_audit_created_at ON audit_logs(created_at);
-- =================================================================
-- Jira integration
-- =================================================================
CREATE TABLE IF NOT EXISTS jira_tickets (
id INTEGER PRIMARY KEY AUTOINCREMENT,
cve_id TEXT NOT NULL,
vendor TEXT NOT NULL,
ticket_key TEXT NOT NULL,
url TEXT,
summary TEXT,
status TEXT DEFAULT 'Open' CHECK(status IN ('Open', 'In Progress', 'Closed')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (cve_id, vendor) REFERENCES cves(cve_id, vendor) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_jira_tickets_cve ON jira_tickets(cve_id, vendor);
CREATE INDEX IF NOT EXISTS idx_jira_tickets_status ON jira_tickets(status);
-- =================================================================
-- Archer integration
-- =================================================================
CREATE TABLE IF NOT EXISTS archer_tickets (
id INTEGER PRIMARY KEY AUTOINCREMENT,
exc_number TEXT NOT NULL UNIQUE,
archer_url TEXT,
status TEXT DEFAULT 'Draft' CHECK(status IN ('Draft', 'Open', 'Under Review', 'Accepted')),
cve_id TEXT NOT NULL,
vendor TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (cve_id, vendor) REFERENCES cves(cve_id, vendor) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_archer_tickets_cve ON archer_tickets(cve_id, vendor);
CREATE INDEX IF NOT EXISTS idx_archer_tickets_status ON archer_tickets(status);
CREATE INDEX IF NOT EXISTS idx_archer_tickets_exc ON archer_tickets(exc_number);
-- =================================================================
-- Knowledge base
-- =================================================================
CREATE TABLE IF NOT EXISTS knowledge_base (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title VARCHAR(255) NOT NULL,
slug VARCHAR(255) UNIQUE NOT NULL,
description TEXT,
category VARCHAR(100),
file_path VARCHAR(500),
file_name VARCHAR(255),
file_type VARCHAR(50),
file_size INTEGER,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
created_by INTEGER,
FOREIGN KEY (created_by) REFERENCES users(id)
);
CREATE INDEX IF NOT EXISTS idx_knowledge_base_slug ON knowledge_base(slug);
CREATE INDEX IF NOT EXISTS idx_knowledge_base_category ON knowledge_base(category);
CREATE INDEX IF NOT EXISTS idx_knowledge_base_created_at ON knowledge_base(created_at DESC);
-- =================================================================
-- Ivanti findings sync and cache
-- =================================================================
CREATE TABLE IF NOT EXISTS ivanti_sync_state (
id INTEGER PRIMARY KEY CHECK (id = 1),
total INTEGER DEFAULT 0,
workflows_json TEXT DEFAULT '[]',
synced_at DATETIME,
sync_status TEXT DEFAULT 'never',
error_message TEXT
);
CREATE TABLE IF NOT EXISTS ivanti_findings_cache (
id INTEGER PRIMARY KEY CHECK (id = 1),
total INTEGER DEFAULT 0,
findings_json TEXT DEFAULT '[]',
synced_at DATETIME,
sync_status TEXT DEFAULT 'never',
error_message TEXT
);
CREATE TABLE IF NOT EXISTS ivanti_finding_notes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL UNIQUE,
note TEXT NOT NULL DEFAULT '',
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_finding_notes_finding_id ON ivanti_finding_notes(finding_id);
CREATE TABLE IF NOT EXISTS ivanti_counts_cache (
id INTEGER PRIMARY KEY CHECK (id = 1),
open_count INTEGER DEFAULT 0,
closed_count INTEGER DEFAULT 0,
synced_at DATETIME,
fp_workflow_counts_json TEXT DEFAULT '{}',
fp_id_counts_json TEXT DEFAULT '{}'
);
CREATE TABLE IF NOT EXISTS ivanti_finding_overrides (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL,
field TEXT NOT NULL,
value TEXT NOT NULL DEFAULT '',
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
UNIQUE(finding_id, field)
);
CREATE INDEX IF NOT EXISTS idx_finding_overrides_finding_id ON ivanti_finding_overrides(finding_id);
CREATE TABLE IF NOT EXISTS ivanti_counts_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
open_count INTEGER NOT NULL,
closed_count INTEGER NOT NULL,
recorded_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
-- =================================================================
-- Ivanti FP (False Positive) submissions
-- =================================================================
CREATE TABLE IF NOT EXISTS ivanti_fp_submissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
username TEXT NOT NULL,
ivanti_workflow_batch_id INTEGER,
ivanti_generated_id TEXT,
ivanti_workflow_batch_uuid TEXT,
workflow_name TEXT NOT NULL,
reason TEXT NOT NULL,
description TEXT,
expiration_date TEXT NOT NULL,
scope_override TEXT NOT NULL DEFAULT 'Authorized',
finding_ids_json TEXT NOT NULL,
queue_item_ids_json TEXT NOT NULL,
attachment_count INTEGER DEFAULT 0,
attachment_results_json TEXT,
status TEXT NOT NULL DEFAULT 'success' CHECK(status IN ('success', 'partial', 'failed')),
lifecycle_status TEXT NOT NULL DEFAULT 'submitted' CHECK(lifecycle_status IN ('submitted', 'approved', 'rejected', 'rework', 'resubmitted')),
error_message TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT NULL
);
CREATE INDEX IF NOT EXISTS idx_fp_submissions_user ON ivanti_fp_submissions(user_id);
CREATE INDEX IF NOT EXISTS idx_fp_submissions_ivanti_id ON ivanti_fp_submissions(ivanti_generated_id);
CREATE TABLE IF NOT EXISTS ivanti_fp_submission_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
submission_id INTEGER NOT NULL,
user_id INTEGER NOT NULL,
username TEXT NOT NULL,
change_type TEXT NOT NULL CHECK(change_type IN (
'created', 'fields_updated', 'findings_added',
'attachments_added', 'status_changed'
)),
change_details_json TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (submission_id) REFERENCES ivanti_fp_submissions(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_fp_history_submission ON ivanti_fp_submission_history(submission_id);
-- =================================================================
-- Ivanti todo queue (FP, Archer, CARD, GRANITE workflows)
-- =================================================================
CREATE TABLE IF NOT EXISTS ivanti_todo_queue (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
finding_id TEXT NOT NULL,
finding_title TEXT,
cves_json TEXT,
ip_address TEXT,
hostname TEXT,
vendor TEXT NOT NULL,
workflow_type TEXT NOT NULL CHECK(workflow_type IN ('FP', 'Archer', 'CARD', 'GRANITE')),
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'complete')),
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_todo_queue_user ON ivanti_todo_queue(user_id, status);
-- =================================================================
-- Ivanti archive detection and anomaly tracking
-- =================================================================
CREATE TABLE IF NOT EXISTS ivanti_finding_archives (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL UNIQUE,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
ip_address TEXT NOT NULL DEFAULT '',
current_state TEXT NOT NULL CHECK(current_state IN ('ARCHIVED','RETURNED','CLOSED','CLOSED_GONE')),
last_severity REAL NOT NULL DEFAULT 0,
first_archived_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_transition_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_archive_finding_id ON ivanti_finding_archives(finding_id);
CREATE INDEX IF NOT EXISTS idx_archive_current_state ON ivanti_finding_archives(current_state);
CREATE TABLE IF NOT EXISTS ivanti_archive_transitions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
archive_id INTEGER NOT NULL,
from_state TEXT NOT NULL,
to_state TEXT NOT NULL,
severity_at_transition REAL NOT NULL DEFAULT 0,
reason TEXT NOT NULL DEFAULT '',
transitioned_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (archive_id) REFERENCES ivanti_finding_archives(id)
);
CREATE INDEX IF NOT EXISTS idx_transition_archive_id ON ivanti_archive_transitions(archive_id);
CREATE TABLE IF NOT EXISTS ivanti_sync_anomaly_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sync_timestamp DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
open_count_delta INTEGER NOT NULL DEFAULT 0,
closed_count_delta INTEGER NOT NULL DEFAULT 0,
newly_archived_count INTEGER NOT NULL DEFAULT 0,
returned_count INTEGER NOT NULL DEFAULT 0,
classification_json TEXT NOT NULL DEFAULT '{}',
return_classification_json TEXT NOT NULL DEFAULT '{}',
is_significant INTEGER NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_anomaly_sync_timestamp ON ivanti_sync_anomaly_log(sync_timestamp);
CREATE TABLE IF NOT EXISTS ivanti_finding_bu_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
finding_id TEXT NOT NULL,
finding_title TEXT NOT NULL DEFAULT '',
host_name TEXT NOT NULL DEFAULT '',
previous_bu TEXT NOT NULL,
new_bu TEXT NOT NULL,
detected_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_bu_history_finding_id ON ivanti_finding_bu_history(finding_id);
CREATE INDEX IF NOT EXISTS idx_bu_history_detected_at ON ivanti_finding_bu_history(detected_at);
-- =================================================================
-- Atlas action plans cache
-- =================================================================
CREATE TABLE IF NOT EXISTS atlas_action_plans_cache (
id INTEGER PRIMARY KEY AUTOINCREMENT,
host_id INTEGER NOT NULL UNIQUE,
has_action_plan INTEGER NOT NULL DEFAULT 0,
plan_count INTEGER NOT NULL DEFAULT 0,
plans_json TEXT NOT NULL DEFAULT '[]',
synced_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_atlas_cache_host_id ON atlas_action_plans_cache(host_id);
-- =================================================================
-- Compliance (NTS AEO) tracking
-- =================================================================
CREATE TABLE IF NOT EXISTS compliance_uploads (
id INTEGER PRIMARY KEY AUTOINCREMENT,
filename TEXT NOT NULL,
report_date TEXT,
uploaded_by INTEGER,
uploaded_at DATETIME DEFAULT CURRENT_TIMESTAMP,
new_count INTEGER DEFAULT 0,
resolved_count INTEGER DEFAULT 0,
recurring_count INTEGER DEFAULT 0,
summary_json TEXT,
FOREIGN KEY (uploaded_by) REFERENCES users(id) ON DELETE SET NULL
);
CREATE TABLE IF NOT EXISTS compliance_items (
id INTEGER PRIMARY KEY AUTOINCREMENT,
upload_id INTEGER NOT NULL,
hostname TEXT NOT NULL,
ip_address TEXT,
device_type TEXT,
team TEXT,
metric_id TEXT NOT NULL,
metric_desc TEXT,
category TEXT,
extra_json TEXT,
status TEXT NOT NULL DEFAULT 'active' CHECK(status IN ('active', 'resolved')),
first_seen_upload_id INTEGER,
resolved_upload_id INTEGER,
seen_count INTEGER DEFAULT 1,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (upload_id) REFERENCES compliance_uploads(id) ON DELETE CASCADE,
FOREIGN KEY (first_seen_upload_id) REFERENCES compliance_uploads(id) ON DELETE SET NULL,
FOREIGN KEY (resolved_upload_id) REFERENCES compliance_uploads(id) ON DELETE SET NULL
);
CREATE INDEX IF NOT EXISTS idx_compliance_items_upload ON compliance_items(upload_id);
CREATE INDEX IF NOT EXISTS idx_compliance_items_identity ON compliance_items(hostname, metric_id);
CREATE INDEX IF NOT EXISTS idx_compliance_items_team_status ON compliance_items(team, status);
CREATE TABLE IF NOT EXISTS compliance_notes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
hostname TEXT NOT NULL,
metric_id TEXT NOT NULL,
note TEXT NOT NULL,
group_id TEXT,
created_by INTEGER,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (created_by) REFERENCES users(id) ON DELETE SET NULL
);
CREATE INDEX IF NOT EXISTS idx_compliance_notes_identity ON compliance_notes(hostname, metric_id);
CREATE INDEX IF NOT EXISTS idx_compliance_notes_group ON compliance_notes(group_id);
-- =================================================================
-- Document compliance view
-- =================================================================
CREATE VIEW IF NOT EXISTS cve_document_status AS
SELECT
c.id as record_id,
c.cve_id,
c.vendor,
c.severity,
c.status,
COUNT(DISTINCT d.id) as total_documents,
COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) as advisory_count,
COUNT(DISTINCT CASE WHEN d.type = 'email' THEN d.id END) as email_count,
COUNT(DISTINCT CASE WHEN d.type = 'screenshot' THEN d.id END) as screenshot_count,
CASE
WHEN COUNT(DISTINCT CASE WHEN d.type = 'advisory' THEN d.id END) > 0
THEN 'Complete'
ELSE 'Missing Required Docs'
END as compliance_status
FROM cves c
LEFT JOIN documents d ON c.cve_id = d.cve_id AND c.vendor = d.vendor
GROUP BY c.id, c.cve_id, c.vendor, c.severity, c.status;
-- =================================================================
-- Seed data
-- =================================================================
INSERT OR IGNORE INTO required_documents (vendor, document_type, is_mandatory, description) VALUES
('Microsoft', 'advisory', 1, 'Official Microsoft Security Advisory'),
('Microsoft', 'screenshot', 0, 'Proof of patch application'),
('Cisco', 'advisory', 1, 'Cisco Security Advisory'),
('Oracle', 'advisory', 1, 'Oracle Security Alert'),
('VMware', 'advisory', 1, 'VMware Security Advisory'),
('Adobe', 'advisory', 1, 'Adobe Security Bulletin');
`);
console.log('✓ Database schema initialized');
// User group validation triggers (cannot be in db.exec multi-statement)
await dbRun(db, `
CREATE TRIGGER IF NOT EXISTS check_user_group_insert
BEFORE INSERT ON users
FOR EACH ROW
WHEN NEW.user_group NOT IN ('Admin', 'Standard_User', 'Leadership', 'Read_Only')
BEGIN
SELECT RAISE(ABORT, 'Invalid user_group value. Must be Admin, Standard_User, Leadership, or Read_Only');
END
`);
await dbRun(db, `
CREATE TRIGGER IF NOT EXISTS check_user_group_update
BEFORE UPDATE OF user_group ON users
FOR EACH ROW
WHEN NEW.user_group NOT IN ('Admin', 'Standard_User', 'Leadership', 'Read_Only')
BEGIN
SELECT RAISE(ABORT, 'Invalid user_group value. Must be Admin, Standard_User, Leadership, or Read_Only');
END
`);
console.log('✓ Triggers created');
}
// ---------------------------------------------------------------------------
// Directory setup
// ---------------------------------------------------------------------------
function createDirectories() {
const dirs = [
UPLOADS_DIR,
path.join(UPLOADS_DIR, 'temp'),
path.join(UPLOADS_DIR, 'knowledge_base'),
]; ];
for (const dir of dirs) {
for (const cve of sampleCVEs) { if (!fs.existsSync(dir)) {
await new Promise((resolve, reject) => { fs.mkdirSync(dir, { recursive: true });
db.run( console.log(`✓ Created directory: ${path.relative(__dirname, dir)}`);
`INSERT OR IGNORE INTO cves (cve_id, vendor, severity, description, published_date) }
VALUES (?, ?, ?, ?, ?)`,
[cve.cve_id, cve.vendor, cve.severity, cve.description, cve.published_date],
(err) => {
if (err) reject(err);
else {
console.log(` ✓ Added sample: ${cve.cve_id} / ${cve.vendor}`);
resolve();
}
}
);
});
} }
console.log(' Sample data added - demonstrates multi-vendor support');
} }
// Verify database structure // ---------------------------------------------------------------------------
async function verifySetup(db) { // Default admin user
return new Promise((resolve) => { // ---------------------------------------------------------------------------
db.get('SELECT sql FROM sqlite_master WHERE type="table" AND name="cves"', (err, row) => { async function createDefaultAdmin(db) {
if (err) { const existing = await dbGet(db, 'SELECT id FROM users WHERE username = ?', ['admin']);
console.error('Warning: Could not verify setup:', err); if (existing) {
} else { console.log('✓ Default admin user already exists');
console.log('\n📋 CVEs table structure:'); return;
console.log(row.sql); }
// Check if UNIQUE constraint is correct const generatedPassword = crypto.randomBytes(12).toString('base64url');
if (row.sql.includes('UNIQUE(cve_id, vendor)')) { const passwordHash = await bcrypt.hash(generatedPassword, 10);
console.log('\n✅ Multi-vendor support: ENABLED');
} else { await dbRun(db,
console.log('\n⚠ Warning: Multi-vendor constraint may not be set correctly'); `INSERT INTO users (username, email, password_hash, role, user_group, is_active)
} VALUES (?, ?, ?, ?, ?, ?)`,
} ['admin', 'admin@localhost', passwordHash, 'admin', 'Admin', 1]
resolve(); );
});
}); console.log('✓ Created default admin user');
console.log(`\n ╔══════════════════════════════════════════╗`);
console.log(` ║ Admin credentials (save these now!) ║`);
console.log(` ║ Username: admin ║`);
console.log(` ║ Password: ${generatedPassword.padEnd(29)}`);
console.log(` ╚══════════════════════════════════════════╝\n`);
} }
// Display setup summary // ---------------------------------------------------------------------------
// Setup summary
// ---------------------------------------------------------------------------
function displaySummary() { function displaySummary() {
console.log('\n╔════════════════════════════════════════════════════════╗'); console.log('\n╔════════════════════════════════════════════════════════╗');
console.log('║ CVE DATABASE SETUP COMPLETE! ║'); console.log('║ CVE DASHBOARD v1.0.0 — SETUP COMPLETE ║');
console.log('╚════════════════════════════════════════════════════════╝'); console.log('╚════════════════════════════════════════════════════════╝');
console.log('\n📊 What was created:'); console.log('\n📊 Tables created:');
console.log(' ✓ SQLite database (cve_database.db)'); console.log(' Core: cves, documents, required_documents');
console.log(' ✓ Tables: cves, documents, required_documents, users, sessions, audit_logs'); console.log(' Auth: users, sessions');
console.log(' ✓ Multi-vendor support with UNIQUE(cve_id, vendor)'); console.log(' Audit: audit_logs');
console.log(' ✓ Vendor column in documents table'); console.log(' Jira: jira_tickets');
console.log(' ✓ User authentication with session-based auth'); console.log(' Archer: archer_tickets');
console.log(' ✓ Indexes for fast queries'); console.log(' KB: knowledge_base');
console.log(' ✓ Document compliance view'); console.log(' Ivanti: ivanti_sync_state, ivanti_findings_cache,');
console.log(' ✓ Uploads directory for file storage'); console.log(' ivanti_finding_notes, ivanti_counts_cache,');
console.log(' ✓ Default admin user (admin/admin123)'); console.log(' ivanti_finding_overrides, ivanti_counts_history,');
console.log('\n📁 File structure will be:'); console.log(' ivanti_fp_submissions, ivanti_fp_submission_history,');
console.log(' uploads/'); console.log(' ivanti_todo_queue');
console.log(' └── CVE-XXXX-XXXX/'); console.log(' Archives: ivanti_finding_archives, ivanti_archive_transitions,');
console.log(' ├── Vendor1/'); console.log(' ivanti_sync_anomaly_log, ivanti_finding_bu_history');
console.log(' │ ├── advisory.pdf'); console.log(' Atlas: atlas_action_plans_cache');
console.log(' │ └── screenshot.png'); console.log(' Compliance: compliance_uploads, compliance_items, compliance_notes');
console.log(' └── Vendor2/');
console.log(' └── advisory.pdf');
console.log('\n🚀 Next steps:'); console.log('\n🚀 Next steps:');
console.log(' 1. Start the backend API:'); console.log(' 1. Copy .env.example to .env and configure API keys');
console.log(' → cd backend && node server.js'); console.log(' 2. Start the backend: node backend/server.js');
console.log(' 2. Start the frontend:'); console.log(' 3. Build the frontend: cd frontend && npm run build');
console.log(' → cd frontend && npm start'); console.log(' 4. Open the dashboard and log in with the admin credentials above\n');
console.log(' 3. Open http://localhost:3000');
console.log(' 4. Start adding CVEs with multiple vendors!');
console.log('\n💡 Key Features:');
console.log(' • Add same CVE-ID with different vendors');
console.log(' • Each vendor has separate document storage');
console.log(' • Quick Check shows all vendors for a CVE');
console.log(' • Document compliance tracking per vendor');
console.log(' • Required docs: Advisory (mandatory for most vendors)\n');
} }
// Main execution // ---------------------------------------------------------------------------
// Main
// ---------------------------------------------------------------------------
async function main() { async function main() {
console.log('🚀 CVE Database Setup (Multi-Vendor Support)\n'); console.log('🚀 CVE Dashboard v1.0.0 — Database Setup\n');
console.log('════════════════════════════════════════\n'); console.log('════════════════════════════════════════\n');
try {
// Create uploads directory
createUploadsDirectory();
// Initialize database
const db = await initializeDatabase();
// Create default admin user try {
createDirectories();
const db = new sqlite3.Database(DB_FILE);
await initializeDatabase(db);
await createDefaultAdmin(db); await createDefaultAdmin(db);
// Add sample data
await addSampleData(db);
// Verify setup
await verifySetup(db);
// Close database connection
db.close((err) => { db.close((err) => {
if (err) console.error('Error closing database:', err); if (err) console.error('Error closing database:', err);
else console.log('\n✓ Database connection closed'); else console.log('✓ Database connection closed');
// Display summary
displaySummary(); displaySummary();
}); });
} catch (error) { } catch (error) {
console.error('❌ Setup Error:', error); console.error('❌ Setup Error:', error);
process.exit(1); process.exit(1);
} }
} }
// Run the setup
main(); main();

BIN
cve_database.db Normal file

Binary file not shown.

0
cve_database.db.backup Normal file
View File

0
database.db Normal file
View File

26
docker-compose.yml Normal file
View File

@@ -0,0 +1,26 @@
# Docker Compose for CVE Dashboard PostgreSQL
# Run: docker compose up -d
# Stop: docker compose down
# View logs: docker compose logs -f postgres
services:
postgres:
image: postgres:16-alpine
container_name: steam-postgres
restart: unless-stopped
environment:
POSTGRES_DB: cve_dashboard
POSTGRES_USER: steam
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-sV4xmC9xAUCFop0ypxMVS056QgPqGrX}
ports:
- "5433:5432"
volumes:
- steam-pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U steam -d cve_dashboard"]
interval: 10s
timeout: 5s
retries: 5
volumes:
steam-pgdata:

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,194 @@
# Ivanti / RiskSense API Reference
Base URL: `https://platform4.risksense.com/api/v1`
Swagger: `https://platform4.risksense.com/doc/swagger.json`
Auth: `x-api-key` header. Error codes: 401 bad key, 419 insufficient privileges, 429 rate limited.
## Endpoints Used
### Search Workflow Batches
```
POST /client/{clientId}/workflowBatch/search
Content-Type: application/json
```
Standard JSON body with filters, projection, sort, page, size. Used by `ivantiWorkflows.js` for the daily sync.
### Create False Positive Workflow
```
POST /client/{clientId}/workflowBatch/falsePositive/request
Content-Type: multipart/form-data
```
This endpoint does NOT accept JSON. It requires `multipart/form-data` with the following fields:
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `name` | string | yes | Workflow batch name (max 255) |
| `reason` | string | yes | Reason for the FP determination |
| `description` | string | yes | Description (can be empty string but field must be present) |
| `expirationDate` | string | yes | ISO-8601 date, e.g. `2026-06-01` |
| `overrideControl` | string | yes | `AUTHORIZED`, `NONE`, or `AUTOMATED`. Use `AUTHORIZED` for standard FP workflows. `NONE` with `isEmptyWorkflow=true` is rejected (400). |
| `isEmptyWorkflow` | boolean | yes | `true` if no findings attached, `false` otherwise |
| `subjectFilterRequest` | string | yes | Stringified JSON (see format below) |
| `files` | file | no | Attachments sent inline in the same request |
#### subjectFilterRequest format
This is the critical field. It must be a stringified JSON object with this exact structure:
```json
{
"subject": "hostFinding",
"filterRequest": {
"filters": [
{
"field": "id",
"exclusive": false,
"operator": "IN",
"value": "2283734550,2283734551"
}
]
}
}
```
Key details:
- `subject` must be `"hostFinding"` — without this, the API returns 500
- `filters` is nested inside `filterRequest`, NOT at the top level — `{"filters":[]}` at the top level returns 500
- `value` for multiple IDs is comma-separated as a single string, not an array
- `operator` values: `EXACT`, `IN`, `LIKE`, `WILDCARD`, `RANGE`, `CIDR`
- For empty workflows, use `{"subject":"hostFinding","filterRequest":{"filters":[]}}` with `isEmptyWorkflow=true`
#### Response (200/202)
```json
{
"id": 33418832,
"created": "2026-04-08T18:16:08"
}
```
Returns HTTP 200 or 202 (Accepted — async job creation). Response contains a numeric `id` (the workflow batch job ID) and `created` timestamp. No `generatedId` or `uuid` in this response.
### Map Findings to Existing Workflow (tested 2026-04-13)
```
POST /client/{clientId}/workflowBatch/falsePositive/{workflowBatchUuid}/map
Content-Type: application/json
```
Maps additional host findings to an existing FP workflow batch. Used by the FP submission editing feature to add findings after initial creation.
**Critical: one finding per call.** The map endpoint only reliably maps one finding per request. Sending multiple finding IDs via the `IN` operator or comma-separated values results in only the first finding being mapped. The multipart/form-data format (used by the create endpoint) returns 500 on this endpoint.
#### Request body
```json
{
"subject": "hostFinding",
"filterRequest": {
"filters": [
{
"field": "id",
"exclusive": false,
"operator": "EXACT",
"value": "2283734550"
}
]
}
}
```
Key details:
- Must be `application/json` (NOT multipart/form-data — returns 500)
- Use `EXACT` operator with a single finding ID per call
- `IN` operator with comma-separated IDs only maps the first finding
- Loop through findings and make one API call per finding
- The `workflowBatchUuid` in the URL is the UUID from the search endpoint (not the numeric batch ID from create)
#### Response (200)
Returns the updated workflow batch object on success.
#### UUID resolution
The `workflowBatchUuid` required in the URL is NOT returned by the create endpoint. To obtain it:
1. Search via `POST /client/{clientId}/workflowBatch/search` with `{ field: 'name', operator: 'EXACT', value: '<workflow_name>' }`
2. Use `projection: 'internal'` to get full batch objects
3. The UUID is in the `uuid` field of the returned batch object
4. Cache the UUID locally after first resolution (stored in `ivanti_fp_submissions.ivanti_workflow_batch_uuid`)
#### Implementation in dashboard
The `resolveWorkflowBatchUuid()` helper in `backend/routes/ivantiFpWorkflow.js` handles UUID resolution:
- Returns cached UUID if available in the local submission record
- Otherwise searches Ivanti by workflow name, extracts `batch.uuid`, and caches it for future use
The findings map loop in the `POST /submissions/:id/findings` endpoint:
- Iterates through each finding ID individually
- Makes one JSON POST per finding with `EXACT` operator
- Tracks which findings succeeded vs failed
- Only marks queue items as complete for successfully mapped findings
- Returns both `addedFindings` and `failedFindings` arrays in the response
### Other Workflow Endpoints (from Swagger)
These are available but not all are currently used by the dashboard:
| Endpoint | Purpose | Status |
|----------|---------|--------|
| `/workflowBatch/acceptance/request` | Risk acceptance workflow | Not used |
| `/workflowBatch/remediation/request` | Remediation workflow | Not used |
| `/workflowBatch/severityChange/request` | Severity change workflow | Not used |
| `/workflowBatch/{workflowType}/approve` | Approve a workflow (needs `workflowBatchUuid`) | Not used |
| `/workflowBatch/{workflowType}/reject` | Reject a workflow | Not used |
| `/workflowBatch/{workflowType}/rework` | Send back for rework | Not used |
| `/workflowBatch/{workflowType}/update` | Update a workflow | Not used |
| `/workflowBatch/{workflowType}/{workflowBatchUuid}/map` | Map findings to workflow | Used (FP editing) |
| `/workflowBatch/{workflowType}/{workflowBatchUuid}/unmap` | Unmap findings | Not used |
| `/workflowBatch/{workflowType}/{workflowBatchUuid}/attach` | Attach file to existing workflow | **Broken — see note** |
| `/workflowBatch/{workflowType}/{workflowBatchUuid}/detach` | Detach file | Not used |
| `/workflowBatch/model` | Get model/schema | Not used |
| `/workflowBatch/filter` | Get available filter fields | Not used |
| `/workflowBatch/suggest` | Get suggested values for a filter field | Not used |
### Known Limitations
#### Attach endpoint does not work (tested 2026-04-13)
The `/workflowBatch/{workflowType}/{workflowBatchUuid}/attach` endpoint is listed in the Swagger spec but returns HTTP 400 (Bad Request) for all tested request formats:
- `multipart/form-data` with field name `file` (singular) — 400
- `multipart/form-data` with field name `files` (plural) — 400
- Tested with `Content-Type: application/octet-stream` and `image/png` — both 400
- Tested with both `ivantiMultipartPost` and `ivantiFormPost` helpers — both 400
The Ivanti response is a generic Spring Boot error with no detail message:
```json
{"timestamp":"...","status":400,"error":"Bad Request","path":"/api/v1/client/1550/workflowBatch/falsePositive/{uuid}/attach"}
```
**Workaround:** File attachments can only be uploaded during the initial workflow creation (sent inline with the `/workflowBatch/falsePositive/request` endpoint). To add attachments to an existing workflow, users must upload them directly in the Ivanti platform UI.
#### Search by numeric batch ID does not work
The `/workflowBatch/search` endpoint does not support filtering by the numeric `id` returned from the create endpoint. Searching with `{ field: 'id', operator: 'EXACT', value: '33432541' }` returns 0 results. Searching by `name` field works and returns the workflow batch object including the `uuid` field needed for map/attach operations.
#### UUID not returned by create endpoint
The `/workflowBatch/falsePositive/request` create endpoint returns only `{ id: <number>, created: <timestamp> }`. The `uuid` needed for map/attach/approve/reject operations must be obtained separately via the search endpoint.
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `IVANTI_API_KEY` | — | Required. API key for authentication |
| `IVANTI_CLIENT_ID` | `1550` | Client ID in the Ivanti platform |
| `IVANTI_SKIP_TLS` | `false` | Set `true` to skip TLS verification |
| `IVANTI_FIRST_NAME` | — | Used for workflow search filter (sync) |
| `IVANTI_LAST_NAME` | — | Used for workflow search filter (sync) |

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,171 @@
# Jira REST API Use Cases — STEAM Security Dashboard
## Overview
The STEAM Security Dashboard is a self-hosted vulnerability management tool used by the NTS-AEO-STEAM and NTS-AEO-ACCESS-ENG teams. It integrates with Jira Data Center to create, track, and sync vulnerability remediation tickets linked to CVE records.
All API calls are made from a single Node.js backend process. The integration uses Basic Auth with a service account and enforces Charter's posted rate limits client-side.
---
## Charter Compliance Summary
| Requirement | Implementation |
|---|---|
| Authentication | Basic Auth with service account (`JIRA_API_USER` + `JIRA_API_TOKEN`) |
| Rate limit — daily | Client-side enforced: 1 440 requests/day max |
| Rate limit — burst | Client-side enforced: 60 requests/minute max |
| Inter-request delay — GETs | 1 second minimum between GET requests |
| Inter-request delay — writes | 2 seconds minimum between PUT/POST/DELETE requests |
| Explicit field lists | Every GET includes `?fields=` parameter; `/rest/api/2/field` is blocked |
| No bulk updates | Issues are updated one at a time; `/rest/api/2/issue/bulk` is blocked |
| Bulk reads via JQL | Multi-ticket sync uses a single `GET /rest/api/2/search` with predefined key-based JQL query parameters, not per-issue GETs; no arbitrary JQL passthrough |
| Single-issue fetch via JQL | `GET /rest/api/2/search?jql=key="KEY" AND project=<KEY>&fields=...&maxResults=1` |
| JQL scoping | All recurring JQL queries use predefined scoped patterns with `updated >= -72h` clause and `project = <KEY>` scoping; no arbitrary JQL passthrough |
| `maxResults` cap | Search queries capped at 1 000 results per page |
---
## Use Cases
### 1. Connection Test
| | |
|---|---|
| **Endpoint** | `GET /rest/api/2/myself` |
| **Trigger** | Admin clicks "Test Connection" on the Jira settings panel |
| **Frequency** | Manual, infrequent (a few times per day at most) |
| **Purpose** | Verify service account credentials and connectivity |
| **Fields requested** | Default (myself endpoint returns user profile) |
### 2. Create Issue
| | |
|---|---|
| **Endpoint** | `POST /rest/api/2/issue` |
| **Trigger** | User clicks "Create in Jira" from a CVE detail panel |
| **Frequency** | Manual, estimated 520 per day |
| **Purpose** | Create a vulnerability remediation ticket linked to a CVE/vendor pair |
| **Fields sent** | `project.key`, `summary`, `issuetype.name`, `description` |
| **Notes** | A local record is also created in the dashboard database linking the Jira key to the CVE |
### 3. Get Single Issue
| | |
|---|---|
| **Endpoint** | `GET /rest/api/2/search?jql=key="ISSUE-KEY" AND project=<KEY>&fields=summary,status,assignee,created,updated,priority,issuetype,project,resolution&maxResults=1` |
| **Trigger** | User clicks "Sync" on a single Jira ticket row |
| **Frequency** | Manual, estimated 1030 per day |
| **Purpose** | Refresh a single ticket's status and summary from Jira via JQL search |
| **Notes** | Uses JQL-based lookup instead of single-issue GET per Charter compliance. Fields are always specified explicitly. |
### 4. Update Issue
| | |
|---|---|
| **Endpoint** | `PUT /rest/api/2/issue/{issueKey}` |
| **Trigger** | Future feature — local edits synced back to Jira |
| **Frequency** | Manual, estimated 510 per day when enabled |
| **Purpose** | Update issue summary or other fields from the dashboard |
| **Notes** | Issues are updated one at a time; bulk PUT is not used |
### 5. Add Comment
| | |
|---|---|
| **Endpoint** | `POST /rest/api/2/issue/{issueKey}/comment` |
| **Trigger** | Dashboard adds audit trail comments to linked tickets |
| **Frequency** | Automated on certain actions, estimated 515 per day |
| **Purpose** | Maintain an audit trail on the Jira ticket for compliance visibility |
### 6. Get Transitions
| | |
|---|---|
| **Endpoint** | `GET /rest/api/2/issue/{issueKey}/transitions` |
| **Trigger** | Dashboard checks available workflow transitions before moving a ticket |
| **Frequency** | Manual, paired with transition calls, estimated 510 per day |
| **Purpose** | Discover valid status transitions for the issue's current workflow state |
### 7. Transition Issue
| | |
|---|---|
| **Endpoint** | `POST /rest/api/2/issue/{issueKey}/transitions` |
| **Trigger** | User moves a ticket to a new status from the dashboard |
| **Frequency** | Manual, estimated 510 per day |
| **Purpose** | Move ticket through workflow states (e.g., Open to In Progress to Closed) |
### 8. Scoped Bulk Sync via JQL
| | |
|---|---|
| **Endpoint** | `GET /rest/api/2/search?jql=...&fields=...&maxResults=...&startAt=...` |
| **Trigger** | Admin clicks "Sync All" on the Jira tickets panel |
| **Frequency** | Manual, estimated 13 times per day |
| **Purpose** | Bulk-refresh all tracked tickets in a single request instead of per-issue GETs |
| **JQL pattern** | `key in ("KEY-1", "KEY-2", ...) AND updated >= -72h AND project = <KEY>` |
| **Fields requested** | `summary, status, assignee, created, updated, priority, issuetype, project, resolution` |
| **Batch size** | 100 keys per JQL query; multiple batches if needed |
| **Notes** | Uses GET with URL-encoded query parameters per Charter compliance. JQL is predefined and scoped — constructed from known tracked issue keys, a fixed 72-hour window, and the configured project key. No arbitrary JQL is accepted from the frontend. Stops early if rate limit budget is running low (burst remaining <= 5 or daily remaining <= 10) |
### 9. Issue Lookup
| | |
|---|---|
| **Endpoint** | `GET /rest/api/2/search?jql=key="ISSUE-KEY" AND project=<KEY>&fields=...&maxResults=1` |
| **Trigger** | User looks up a Jira issue by key from the dashboard search |
| **Frequency** | Manual, estimated 515 per day |
| **Purpose** | Quick lookup of any Jira issue to view its current state via JQL search |
---
## Estimated Daily API Usage
| Operation | Estimated calls/day | Method | Delay enforced |
|---|---|---|---|
| Connection test | 25 | GET | 1s |
| Create issue | 520 | POST | 2s |
| Get single issue | 1030 | GET | 1s |
| Update issue | 510 | PUT | 2s |
| Add comment | 515 | POST | 2s |
| Get transitions | 510 | GET | 1s |
| Transition issue | 510 | POST | 2s |
| Scoped bulk sync | 15 | GET | 1s |
| Issue lookup | 515 | GET | 1s |
| **Total estimated** | **43120** | | |
Well within the 1 440/day limit. Burst usage stays under 60/minute due to enforced inter-request delays.
---
## Blocked Endpoints
The integration explicitly blocks these endpoints to comply with Charter policy:
- `/rest/api/2/field` — field metadata is never queried; fields are specified in code
- `/rest/api/2/issue/bulk` — bulk updates are not used; issues are updated individually
- `POST /rest/api/2/search` — arbitrary JQL search via POST is not used; all searches use `GET /rest/api/2/search` with URL-encoded query parameters and predefined scoped JQL patterns
---
## Error Handling
- **429 responses**: Surfaced to the user as "Rate limit exceeded. Try again later." No automatic retry.
- **5xx responses**: Surfaced as "Jira API error" with the response body for debugging.
- **Network failures**: Caught and surfaced with the error message.
- **Timeout**: 15 second timeout per request; surfaced as a timeout error.
---
## UAT Test Evidence
The UAT test script (`backend/scripts/jira-uat-test.js`) exercises all use cases listed above and produces a log file at `backend/scripts/jira-uat-test.log`. This log can be attached to or referenced in the ATLSUP approval ticket.
To run:
```bash
cd backend
node scripts/jira-uat-test.js
```

Some files were not shown because too many files have changed in this diff Show More