15 Commits

Author SHA1 Message Date
Jordan Ramos
223b6f22b8 Fix BulkHideToolbar not sticking when scrolling on Reporting page
Add position: sticky, top: 0, zIndex: 20 to BulkHideToolbar so the
Atlas Action Plan button remains visible while scrolling through findings.
Matches the existing sticky behavior of SelectionToolbar.
2026-06-22 15:57:45 -06:00
Jordan Ramos
55795710d9 Add TLS/HTTPS support with auto-detection
- Server auto-detects cert/key in backend/certs/ and starts HTTPS
- Falls back to plain HTTP if no certs found or TLS_ENABLED=false
- Self-signed cert generated for dev (365-day, gitignored)
- Added TLS env vars to .env.example
- Frontend rebuilt with https:// API URLs for dev server
2026-06-19 14:44:04 -06:00
Jordan Ramos
e9d6038636 Add Granite Loader to AEO Compliance page with CARD enrichment and pagination
- Add checkbox selection + Granite Loader button to compliance device table
- Integrate LoaderModal for generating loader sheets from compliance devices
- Add direct IP resolve path (resolveAssetId + searchByAssetId) for CARD
  enrichment on compliance devices without Ivanti host IDs
- Add searchByAssetId helper for full enriched record via asset-search endpoint
- Include NTS-AEO-ACCESS-OPS in default enrich-batch team search
- Increase CARD quick-mode timeout from 15s to 30s
- Add timeout vs not-found distinction in enrichment error reporting
- Fix LoaderModal enriching state not resetting on modal reopen
- Add pagination to compliance device table (25/50/100/200 per page)
- Page resets on team, tab, filter, or search change
2026-06-19 13:49:26 -06:00
Jordan Ramos
c7274be66d Bump compliance upload limit to 100MB
NTS_AVVOC vertical xlsx is 72MB — 50MB was still too low.
2026-06-18 08:45:01 -06:00
Jordan Ramos
ba6e67c639 Increase compliance upload limit to 50MB
SDIT_CSD xlsx files exceed the 10MB general upload limit. Add a
separate multer instance (complianceUpload, 50MB) for the compliance
and VCL multi-vertical routes while keeping the 10MB cap for general
document/KB uploads.
2026-06-18 08:38:57 -06:00
Jordan Ramos
f257cfad88 Skip BU history entries when previous_bu is unknown
Only record BU reassignment in ivanti_finding_bu_history when the
previous_bu is a known managed BU (from EXPECTED_BUS). Findings that
were never in our sync cache show as UNKNOWN which provides no
actionable insight for asset movement tracking.

Closes #28
2026-06-17 14:58:01 -06:00
Jordan Ramos
a95fd03f5e Rebrand STEAM → AEGIS, fix BU drift checker previous_bu bug
- Replace all STEAM branding with AEGIS (Advanced Engineering Group
  Intelligence System) across login, header, nav drawer, manifest, and
  browser title
- Add shield logo to login page, main header, and nav drawer
- Fix BU drift checker recording incorrect previous_bu values by
  building a previousBuMap snapshot BEFORE the upsert/delete cycle
  instead of querying the DB after rows are already gone
- Clean 526 bogus BU history entries generated by the broken logic
- Add docs and scripts from prior session
2026-06-17 14:40:38 -06:00
Jordan Ramos
479c61b88f Restrict VCL/CCP Metrics page to Admin and Leadership groups
Add requireGroup('Admin', 'Leadership') as router-level middleware on all
VCL multi-vertical routes. Hide the CCP Metrics nav item from users not in
those groups and guard the page render in App.js with a redirect fallback.
2026-06-17 09:27:01 -06:00
Jordan Ramos
2fed9221f1 Fix test-backend — remove redundant CLI path arg conflicting with roots
The jest.roots config in package.json already restricts to backend/__tests__
and testPathIgnorePatterns excludes integration tests. The CLI path arg
was being interpreted as an additional ignore pattern, causing 0 matches.
2026-06-16 16:19:23 -06:00
Jordan Ramos
8b985a21f8 Restrict root Jest to backend/__tests__ only — stop scanning frontend
Jest 30 default test discovery was finding frontend/src/**/*.test.js
and __tests__/ files when running from the project root. These need
react-scripts (CRA's Babel config) to parse ESM imports and JSX.
Added jest.roots to confine root-level Jest to backend tests only.
Frontend tests run separately via react-scripts test in test-frontend.
2026-06-16 16:17:57 -06:00
Jordan Ramos
55a4d299ef Force npm ci in test-frontend to fix stale cache missing transforms
The cached node_modules was missing react-scripts babel config after
package-lock.json changed (remark-gfm addition). Tests failed with
'Cannot use import statement outside a module' and JSX parse errors.
Always run npm ci to ensure fresh dependencies match the lockfile.
2026-06-16 16:15:48 -06:00
Jordan Ramos
28714eed47 Cache plan IDs from Atlas create responses
Single-host PUT and bulk POST now extract and store the action_plan_id
from the Atlas API response in the local cache. Previously only a stub
with plan_type/commit_date was  now the actual plan ID iscached
included so it can be referenced for updates/display without re-fetching
from Atlas.
2026-06-16 16:10:54 -06:00
Jordan Ramos
c0e3139503 Fix atlas_known — parse response body to detect 'not found' hosts
Instead of blanket-marking managed BU hosts, now parses the Atlas API
response: if it returns a valid {active, inactive} structure, the host
is known. If it returns an error or 'not found' message (even with a
2xx status), the host is not known and won't show a badge.

This prevents the shield showing on hosts Atlas doesn't actually track,
while correctly showing it on hosts Atlas recognizes (with or without
plans).
2026-06-16 15:45:43 -06:00
Jordan Ramos
09db1c2ae9 Fix atlas_known — managed BU hosts always show badge regardless of plans
A STEAM/ACCESS-ENG host with zero Atlas plans but tracked in Atlas
(like olt01k7) wasn't showing the amber shield because atlas_known
was only true when plans existed. Now managed BU hosts always get
atlas_known=true so the '0 plans' warning badge renders. Non-managed
BU hosts only show badge if Atlas actually has plan data for them.
2026-06-16 15:40:51 -06:00
Jordan Ramos
c1a266f4f7 Skip integration tests in CI — no Postgres service available
The migrations-idempotency.integration.test.js requires a reachable
Postgres instance. The CI Docker container can't resolve the DATABASE_URL
hostname. Skip files matching 'integration' in the test-backend job.
2026-06-16 15:09:10 -06:00
25 changed files with 1429 additions and 74 deletions

View File

@@ -100,7 +100,7 @@ test-backend:
policy: pull
script:
- test -d node_modules || npm ci
- ./node_modules/.bin/jest --ci --forceExit backend/__tests__/
- ./node_modules/.bin/jest --ci --forceExit
timeout: 5 minutes
needs:
- install-backend
@@ -118,7 +118,7 @@ test-frontend:
- node_modules/
policy: pull
script:
- cd frontend && (test -d node_modules || npm ci) && cd .. && (test -d node_modules || npm ci) && cd frontend && CI=true npx react-scripts test --watchAll=false --ci
- cd frontend && npm ci && cd .. && (test -d node_modules || npm ci) && cd frontend && CI=true npx react-scripts test --watchAll=false --ci
timeout: 5 minutes
needs:
- install-frontend

View File

@@ -75,11 +75,36 @@ Python dependencies: `pandas>=2.0.0`, `openpyxl>=3.0.0` (install via apt or venv
## Environment Configuration
- `backend/.env` — PORT, CORS_ORIGINS, SESSION_SECRET, NVD_API_KEY, Ivanti API credentials
- `backend/.env` — PORT, CORS_ORIGINS, SESSION_SECRET, NVD_API_KEY, Ivanti API credentials, CARD API credentials
- `frontend/.env` — REACT_APP_API_BASE, REACT_APP_API_HOST
- Both `.env` files are gitignored; see `.env.example` files for templates.
- React env vars are baked in at **build time** — you must rebuild (`npm run build`) after changing them.
### Key Backend Env Vars
| Variable | Purpose |
|---|---|
| `IVANTI_API_KEY` | RiskSense platform API key |
| `IVANTI_CLIENT_ID` | RiskSense client ID (default: 1550) |
| `IVANTI_BU_FILTER` | Comma-separated BU teams to sync findings for (default: `NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM`) |
| `IVANTI_FIRST_NAME` / `IVANTI_LAST_NAME` | Fallback Ivanti identity for workflow sync (used only if no per-user identities configured) |
| `CARD_API_URL` | CARD API base URL (e.g., `https://card.charter.com`) |
| `CARD_API_USER` / `CARD_API_PASS` | CARD OAuth credentials for Bearer token acquisition |
| `CARD_SKIP_TLS` | Set to `true` to skip TLS verification (for SSL inspection proxies) |
| `DATABASE_URL` | PostgreSQL connection string |
### CARD API and Ivanti Integration Details
See `.kiro/steering/integrations.md` for full API contracts, response shapes, and quirks for CARD, Ivanti, Atlas, and Jira.
### Ivanti Findings IPv6 Handling
Some Ivanti findings have no IPv4 address. The sync captures fallback addresses:
- `qualys_ipv6` — from `hostAdditionalDetails[].["IPv6 Address"]` (resolves in CARD)
- `primary_ipv6` — from `assetCustomAttributes['1550_host_6'][0]` (may not resolve in CARD)
Display priority in the UI: IPv4 > Qualys IPv6 (amber "Q" badge) > Primary IPv6 (indigo "v6" badge)
## Code Style & Lint Rules
### Unused Variables

View File

@@ -80,3 +80,11 @@ GITLAB_PAT=
# Generate with: openssl rand -hex 20
GITLAB_WEBHOOK_SECRET=changeme_generate_a_random_secret
# TLS / HTTPS Configuration
# If cert and key files exist at the paths below, the server starts with HTTPS.
# Set TLS_ENABLED=false to force plain HTTP even when certs are present.
# Generate a self-signed cert: openssl req -x509 -newkey rsa:2048 -keyout certs/key.pem -out certs/cert.pem -days 365 -nodes -subj "/CN=cve-dashboard.local"
TLS_ENABLED=true
TLS_CERT=certs/cert.pem
TLS_KEY=certs/key.pem

3
backend/.gitignore vendored
View File

@@ -3,3 +3,6 @@
backend/fix_multivendor_constraint.js
backend/migrate_multivendor.js
backend/add_vendor_to_documents.js
# TLS certificates (self-signed or CA-issued)
certs/

View File

@@ -312,6 +312,23 @@ async function searchByIvantiHostId(ivantiHostId, options) {
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* GET /api/v2/asset-search/{assetId}?search_param=deep_search
* Search CARD by asset ID (e.g., "24.24.100.20-CTEC"). Returns the full
* enriched asset record including ncim_discovery, netops_granite_allips, etc.
*
* @param {string} assetId - CARD asset identifier (IP-SUFFIX format)
* @param {object} [options] - { timeout }
*/
async function searchByAssetId(assetId, options) {
const id = (assetId || '').trim();
if (!id) {
return { status: 400, body: '{"error":"Asset ID is required."}', ok: false };
}
const res = await cardGet(`/api/v2/asset-search/${encodeURIComponent(id)}?search_param=deep_search`, options);
return { status: res.status, body: res.body, ok: res.status >= 200 && res.status < 300 };
}
/**
* Resolve a bare IP address to a full CARD asset ID by trying known suffixes.
* Returns the first asset ID that returns a valid owner record, or null if none found.
@@ -322,7 +339,7 @@ async function searchByIvantiHostId(ivantiHostId, options) {
async function resolveAssetId(ip, options) {
const quick = options && options.quick;
const SUFFIXES = quick ? ['CTEC'] : ['CTEC', 'NATL', 'CHTR', 'COML', 'RESI', 'WIFI', 'VOIP'];
const timeout = quick ? 15000 : undefined; // 15s timeout for quick mode
const timeout = quick ? 30000 : undefined; // 30s timeout for quick mode
const trimmedIp = (ip || '').trim();
if (!trimmedIp) return null;
@@ -384,4 +401,5 @@ module.exports = {
invalidateToken,
resolveAssetId,
searchByIvantiHostId,
searchByAssetId,
};

View File

@@ -251,6 +251,19 @@ function createAtlasRouter() {
return res.json({ synced: 0, withPlans: 0, failed: 0 });
}
// Build a set of host IDs belonging to managed BUs — these always show the badge
const managedPatterns = managedBUs.map(b => `%${b}%`);
let managedHostIds = new Set();
try {
const { rows: managedRows } = await pool.query(
`SELECT DISTINCT host_id FROM ivanti_findings
WHERE host_id IS NOT NULL AND host_id > 0
AND bu_ownership ILIKE ANY($1::text[])`,
[managedPatterns]
);
managedHostIds = new Set(managedRows.map(r => r.host_id));
} catch (_) { /* non-fatal — fall back to plans-only logic */ }
let synced = 0;
let withPlans = 0;
let failed = 0;
@@ -273,30 +286,40 @@ function createAtlasRouter() {
}
const { hostId, result } = settled.value;
const isManagedHost = managedHostIds.has(hostId);
if (result.status >= 200 && result.status < 300) {
let allPlans = [];
let activePlans = [];
let atlasRecognizesHost = false;
try {
const parsed = JSON.parse(result.body);
if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {
activePlans = Array.isArray(parsed.active) ? parsed.active : [];
const inactive = Array.isArray(parsed.inactive) ? parsed.inactive : [];
allPlans = [...activePlans, ...inactive];
// Check for "not found" error responses that come back as 200
if (parsed.error || parsed.message?.includes('not found')) {
atlasRecognizesHost = false;
} else {
atlasRecognizesHost = true;
activePlans = Array.isArray(parsed.active) ? parsed.active : [];
const inactive = Array.isArray(parsed.inactive) ? parsed.inactive : [];
allPlans = [...activePlans, ...inactive];
}
} else if (Array.isArray(parsed)) {
atlasRecognizesHost = true;
allPlans = parsed;
activePlans = parsed;
}
} catch (e) {
allPlans = [];
activePlans = [];
atlasRecognizesHost = false;
}
const planCount = activePlans.length;
const hasActionPlan = planCount > 0;
// Atlas "knows" this host if it returned any plans (active or inactive).
// Hosts with completely empty responses are not tracked by Atlas.
const atlasKnown = allPlans.length > 0;
// Atlas knows this host if it returned a valid structured response
// (not "not found" or error). This determines whether the badge renders.
const atlasKnown = atlasRecognizesHost;
try {
if (!hasActionPlan) {
@@ -449,6 +472,45 @@ function createAtlasRouter() {
if (result.status >= 200 && result.status < 300) {
let body;
try { body = JSON.parse(result.body); } catch (e) { body = result.body; }
// Update local cache with the created plan
try {
const { rows: existingRows } = await pool.query(
`SELECT plan_count, plans_json FROM atlas_action_plans_cache WHERE host_id = $1`,
[hostId]
);
const existing = existingRows[0];
let existingPlans = [];
if (existing && existing.plans_json) {
try { existingPlans = JSON.parse(existing.plans_json); } catch (_) {}
}
// Include the plan ID from Atlas response if available
const newPlan = {
plan_type,
commit_date,
source: 'create',
created_at: new Date().toISOString(),
...(body?.action_plan_id ? { action_plan_id: body.action_plan_id } : {}),
...(body?.id ? { action_plan_id: body.id } : {}),
};
const updatedPlans = [...existingPlans, newPlan];
await pool.query(
`INSERT INTO atlas_action_plans_cache (host_id, has_action_plan, plan_count, plans_json, atlas_known, synced_at)
VALUES ($1, true, $2, $3, true, NOW())
ON CONFLICT(host_id) DO UPDATE SET
has_action_plan = true,
plan_count = EXCLUDED.plan_count,
plans_json = EXCLUDED.plans_json,
atlas_known = true,
synced_at = EXCLUDED.synced_at`,
[hostId, updatedPlans.length, JSON.stringify(updatedPlans)]
);
} catch (cacheErr) {
console.error('[Atlas] Cache update failed after plan create for host', hostId, ':', cacheErr.message);
}
res.status(result.status).json(body);
} else {
let errorBody;
@@ -580,7 +642,27 @@ function createAtlasRouter() {
try { existingPlans = JSON.parse(existing.plans_json); } catch (_) {}
}
const stubPlan = { plan_type, commit_date, source: 'bulk-create', created_at: new Date().toISOString() };
// Extract plan ID from bulk response if available (keyed by host_id)
let planId = null;
if (body && typeof body === 'object') {
// Response may be { results: [{host_id, action_plan_id}] } or { [hostId]: {id} }
if (Array.isArray(body.results)) {
const match = body.results.find(r => r.host_id === hid || r.host_id === String(hid));
if (match) planId = match.action_plan_id || match.id;
} else if (body[hid]) {
planId = body[hid].action_plan_id || body[hid].id;
} else if (body[String(hid)]) {
planId = body[String(hid)].action_plan_id || body[String(hid)].id;
}
}
const stubPlan = {
plan_type,
commit_date,
source: 'bulk-create',
created_at: new Date().toISOString(),
...(planId ? { action_plan_id: planId } : {}),
};
const updatedPlans = [...existingPlans, stubPlan];
const newCount = updatedPlans.length;

View File

@@ -17,6 +17,7 @@ const {
redirectAsset,
resolveAssetId,
searchByIvantiHostId,
searchByAssetId,
} = require('../helpers/cardApi');
// ---------------------------------------------------------------------------
@@ -1025,16 +1026,66 @@ function createCardApiRouter() {
}
}
let foundCount = Object.keys(resultMap).length;
// Direct resolve path: for IPs not found via ivanti_findings, resolve
// the asset ID via suffix guessing (CTEC first) and fetch the full asset
// record via asset-search. This returns ncim_discovery, netops_granite, etc.
const unresolvedIps = ipsArray.filter(ip => !resultMap[ip]);
if (unresolvedIps.length > 0) {
const CONCURRENCY = 5;
for (let i = 0; i < unresolvedIps.length; i += CONCURRENCY) {
const batch = unresolvedIps.slice(i, i + CONCURRENCY);
await Promise.all(batch.map(async (ip) => {
if (resultMap[ip]) return;
try {
const assetId = await resolveAssetId(ip, { quick: true });
if (assetId) {
// Use asset-search to get the full enriched record (30s timeout)
const searchResult = await searchByAssetId(assetId, { timeout: 30000 });
if (searchResult.ok) {
const searchData = JSON.parse(searchResult.body);
const assets = searchData.assets || [];
if (assets.length > 0) {
resultMap[ip] = extractGraniteFields(assets[0], ip);
} else {
// Fallback: asset-search returned empty, try owner record
const ownerResult = await getOwner(assetId);
if (ownerResult.ok) {
const ownerData = JSON.parse(ownerResult.body);
resultMap[ip] = extractGraniteFields(ownerData, ip);
}
}
} else {
// asset-search failed, fall back to owner endpoint
const ownerResult = await getOwner(assetId);
if (ownerResult.ok) {
const ownerData = JSON.parse(ownerResult.body);
resultMap[ip] = extractGraniteFields(ownerData, ip);
}
}
}
} catch (err) {
const isTimeout = err.message && (err.message.includes('CARD_TIMEOUT') || err.message.includes('timed out'));
if (isTimeout) {
console.warn(`[card-api] enrich-batch: Direct resolve for ${ip} timed out`);
resultMap[ip] = { _timeout: true };
} else {
console.warn(`[card-api] enrich-batch: Direct resolve for ${ip} failed:`, err.message);
}
}
}));
}
}
let foundCount = Object.keys(resultMap).filter(k => !resultMap[k]._timeout).length;
// Fallback: paginated team-assets loop for any IPs not resolved by fast path
// The team assets endpoint returns the full enriched record with ncim_discovery,
// card_flags, netops_granite_allips, etc.
const teams = team ? [team] : ['NTS-AEO-STEAM', 'NTS-AEO-ACCESS-ENG'];
// Skip if all unresolved IPs already timed out (heavier calls will also timeout)
const teams = team ? [team] : ['NTS-AEO-STEAM', 'NTS-AEO-ACCESS-ENG', 'NTS-AEO-ACCESS-OPS'];
const dispositions = ['confirmed', 'unconfirmed', 'candidate'];
const stillUnresolved = [...targetIps].filter(ip => !resultMap[ip]);
for (const teamName of teams) {
if (foundCount >= targetIps.size) break;
if (stillUnresolved.length === 0 || foundCount >= targetIps.size) break;
for (const disposition of dispositions) {
if (foundCount >= targetIps.size) break;
@@ -1097,8 +1148,13 @@ function createCardApiRouter() {
}
if (resultMap[trimmedIp]) {
results.push({ ip: trimmedIp, found: true, ...resultMap[trimmedIp] });
enrichedCount++;
if (resultMap[trimmedIp]._timeout) {
results.push({ ip: trimmedIp, found: false, equip_inst_id: null, hostname: null, error: 'CARD lookup timed out — try again' });
notFoundCount++;
} else {
results.push({ ip: trimmedIp, found: true, ...resultMap[trimmedIp] });
enrichedCount++;
}
} else {
results.push({ ip: trimmedIp, found: false, equip_inst_id: null, hostname: null, error: 'IP not found in CARD' });
notFoundCount++;

View File

@@ -681,7 +681,7 @@ async function syncFPWorkflowCounts(openFindings, apiKey, clientId, skipTls) {
const MANAGED_BUS_VALUE = process.env.IVANTI_MANAGED_BUS || 'NTS-AEO-ACCESS-ENG,NTS-AEO-STEAM';
const EXPECTED_BUS = new Set(MANAGED_BUS_VALUE.split(',').map(b => b.trim()).filter(Boolean));
async function runBUDriftChecker(newlyArchivedIds, apiKey, clientId, skipTls) {
async function runBUDriftChecker(newlyArchivedIds, apiKey, clientId, skipTls, previousBuMap) {
const summary = { bu_reassignment: 0, severity_drift: 0, closed_on_platform: 0, decommissioned: 0 };
if (!newlyArchivedIds || newlyArchivedIds.length === 0) return summary;
@@ -797,17 +797,18 @@ async function runBUDriftChecker(newlyArchivedIds, apiKey, clientId, skipTls) {
// Record BU reassignment in ivanti_finding_bu_history for detail view
if (classification === 'bu_reassignment' && found) {
try {
// Determine previous BU — look up from the cached finding record
const { rows: prevRows } = await pool.query(
`SELECT bu_ownership FROM ivanti_findings WHERE id = $1`,
[id]
);
const previousBu = prevRows[0]?.bu_ownership || 'UNKNOWN';
await pool.query(
`INSERT INTO ivanti_finding_bu_history (finding_id, finding_title, host_name, previous_bu, new_bu, detected_at)
VALUES ($1, $2, $3, $4, $5, NOW())`,
[id, found.title || '', found.hostName || '', previousBu, found.bu]
);
// Determine previous BU from the pre-sync snapshot (passed in from syncFindings)
const previousBu = (previousBuMap && previousBuMap.get(id)) || '';
// Only record if we have a known previous BU — "UNKNOWN → X" entries
// provide no actionable insight for asset movement tracking.
if (previousBu && EXPECTED_BUS.has(previousBu)) {
await pool.query(
`INSERT INTO ivanti_finding_bu_history (finding_id, finding_title, host_name, previous_bu, new_bu, detected_at)
VALUES ($1, $2, $3, $4, $5, NOW())`,
[id, found.title || '', found.hostName || '', previousBu, found.bu]
);
}
} catch (err) {
console.error(`[BU Drift Checker] Error recording BU change for finding ${id}:`, err.message);
}
@@ -897,12 +898,14 @@ async function syncFindings() {
// Read previous open findings from DB for archive detection
let previousFindings = [];
let previousBuMap = new Map(); // id → bu_ownership snapshot BEFORE upsert
try {
const { rows } = await pool.query(
`SELECT id, title, host_name AS "hostName", ip_address AS "ipAddress", severity, bu_ownership AS "buOwnership"
FROM ivanti_findings WHERE state = 'open'`
);
previousFindings = rows;
previousBuMap = new Map(rows.map(f => [String(f.id), f.buOwnership || '']));
} catch (err) {
console.error('[Ivanti Findings] Failed to read previous findings for archive detection:', err.message);
}
@@ -1004,7 +1007,7 @@ async function syncFindings() {
console.log(`[BU Drift Checker] ${idsToCheck.length} disappeared total, ${newlyArchivedOnly.length} genuinely new (${alreadyArchivedSet.size} already archived, skipped)`);
idsToCheck = newlyArchivedOnly;
}
classificationBreakdown = await runBUDriftChecker(idsToCheck, apiKey, clientId, skipTls);
classificationBreakdown = await runBUDriftChecker(idsToCheck, apiKey, clientId, skipTls, previousBuMap);
} catch (err) {
console.error('[Ivanti Findings] BU drift checker failed (non-fatal):', err.message);
}

View File

@@ -186,8 +186,9 @@ function isSafeTempPath(filePath) {
function createVCLMultiVerticalRouter(upload) {
const router = express.Router();
// All routes require authentication
// All routes require authentication + Leadership or Admin group
router.use(requireAuth());
router.use(requireGroup('Admin', 'Leadership'));
/**
* POST /preview

View File

@@ -0,0 +1,34 @@
#!/usr/bin/env node
// Temporary diagnostic script — fetch a specific finding and dump host fields
require('dotenv').config();
const { ivantiPost } = require('../helpers/ivantiApi');
const apiKey = process.env.IVANTI_API_KEY;
const clientId = process.env.IVANTI_CLIENT_ID || '1550';
const skipTls = process.env.IVANTI_SKIP_TLS === 'true';
const findingId = process.argv[2] || '2814870699';
const urlPath = `/client/${encodeURIComponent(clientId)}/hostFinding/search`;
const body = {
filters: [
{ field: 'id', exclusive: false, operator: 'EXACT', orWithPrevious: false, implicitFilters: [], value: findingId, caseSensitive: false }
],
projection: 'internal',
sort: [{ field: 'severity', direction: 'ASC' }],
page: 0,
size: 1
};
ivantiPost(urlPath, body, apiKey, skipTls).then(r => {
const data = JSON.parse(r.body);
const finding = (data._embedded && data._embedded.hostFindings || [])[0];
if (!finding) { console.log('Finding not found'); process.exit(0); }
console.log('=== host object ===');
console.log(JSON.stringify(finding.host, null, 2));
console.log('');
console.log('=== hostAdditionalDetails ===');
console.log(JSON.stringify(finding.hostAdditionalDetails, null, 2));
process.exit(0);
}).catch(e => { console.error(e.message); process.exit(1); });

View File

@@ -196,6 +196,13 @@ const upload = multer({
limits: { fileSize: 10 * 1024 * 1024 } // 10MB limit
});
// Separate multer instance for compliance xlsx uploads — these can be 75MB+ for large verticals
const complianceUpload = multer({
storage: storage,
fileFilter: fileFilter,
limits: { fileSize: 100 * 1024 * 1024 } // 100MB limit for compliance spreadsheets
});
// Knowledge base routes (editor/admin for upload/delete, all authenticated for view)
app.use('/api/knowledge-base', createKnowledgeBaseRouter(upload));
@@ -223,10 +230,10 @@ app.use('/api/ivanti/fp-workflow', createIvantiFpWorkflowRouter());
// VCL multi-vertical routes — cross-organizational compliance reporting
// Must be mounted BEFORE the general compliance router since both share the /api/compliance prefix
app.use('/api/compliance/vcl-multi', createVCLMultiVerticalRouter(upload));
app.use('/api/compliance/vcl-multi', createVCLMultiVerticalRouter(complianceUpload));
// AEO compliance routes — xlsx upload, non-compliant item tracking, notes
app.use('/api/compliance', createComplianceRouter(upload));
app.use('/api/compliance', createComplianceRouter(complianceUpload));
// Atlas InfoSec action plan routes — proxy CRUD to Atlas API, local cache for badges
app.use('/api/atlas', createAtlasRouter());
@@ -1196,8 +1203,30 @@ if (fs.existsSync(frontendBuild)) {
});
}
// Start server
app.listen(PORT, () => {
console.log(`CVE API server running on http://${API_HOST}:${PORT}`);
console.log(`CORS origins: ${CORS_ORIGINS.join(', ')}`);
});
// Start server — use HTTPS if TLS cert/key are available, otherwise plain HTTP
const TLS_CERT = process.env.TLS_CERT || path.join(__dirname, 'certs', 'cert.pem');
const TLS_KEY = process.env.TLS_KEY || path.join(__dirname, 'certs', 'key.pem');
const TLS_ENABLED = process.env.TLS_ENABLED !== 'false' && fs.existsSync(TLS_CERT) && fs.existsSync(TLS_KEY);
if (TLS_ENABLED) {
const https = require('https');
const httpsOptions = {
cert: fs.readFileSync(TLS_CERT),
key: fs.readFileSync(TLS_KEY),
};
https.createServer(httpsOptions, app).listen(PORT, () => {
console.log(`CVE API server running on https://${API_HOST}:${PORT}`);
console.log(`TLS: enabled (cert: ${TLS_CERT})`);
console.log(`CORS origins: ${CORS_ORIGINS.join(', ')}`);
});
} else {
app.listen(PORT, () => {
console.log(`CVE API server running on http://${API_HOST}:${PORT}`);
if (!fs.existsSync(TLS_CERT) || !fs.existsSync(TLS_KEY)) {
console.log('TLS: disabled (no certs found in backend/certs/)');
} else {
console.log('TLS: disabled (TLS_ENABLED=false)');
}
console.log(`CORS origins: ${CORS_ORIGINS.join(', ')}`);
});
}

View File

@@ -0,0 +1,354 @@
# AD/SAML Integration Architecture
## Overview
This document describes the architecture for integrating Active Directory (AD) authentication via SAML 2.0 into the STEAM Security Dashboard. The integration adds Single Sign-On (SSO) as the primary authentication method while retaining local password login as a break-glass fallback for administrators. AD group memberships drive automatic permission assignment and BU team scoping through a configurable mapping layer.
---
## Authentication Model
The dashboard supports two authentication paths simultaneously:
| Path | Users | Mechanism | Session |
|---|---|---|---|
| Local | Break-glass admins, service accounts | Username + bcrypt password | Cookie-based, PostgreSQL sessions table |
| SAML SSO | All AD users | SP-initiated SAML 2.0 via AD FS | Same cookie-based session (identical to local) |
Both paths produce the same session artifact — an httpOnly cookie containing a `session_id` that maps to a row in the `sessions` table. Downstream middleware (`requireAuth`, `requireGroup`) is unaware of how the session was created.
---
## SAML 2.0 Authentication Flow
### SP-Initiated Login (Success Path)
```mermaid
sequenceDiagram
participant B as Browser
participant SP as Dashboard (SP)
participant IdP as AD FS (IdP)
participant DB as PostgreSQL
B->>SP: GET /api/auth/saml/login
SP->>SP: Generate AuthnRequest XML
SP->>B: HTTP 302 Redirect to IdP SSO URL (with AuthnRequest)
B->>IdP: Follow redirect (user sees AD FS login page)
IdP->>IdP: Authenticate user against AD
IdP->>IdP: Build assertion (NameID, email, groups)
IdP->>IdP: Sign assertion with IdP private key
IdP->>B: HTTP 200 with auto-submit form (POST to SP callback)
B->>SP: POST /api/auth/saml/callback (SAMLResponse in body)
SP->>SP: Base64-decode SAMLResponse
SP->>SP: Validate XML signature against IdP certificate
SP->>SP: Check NotBefore/NotOnOrAfter (120s clock skew tolerance)
SP->>SP: Extract NameID, email, displayName, group claims
SP->>DB: Look up user by external_id (NameID)
alt New user (no matching external_id)
SP->>DB: INSERT into users (JIT provisioning)
SP->>DB: INSERT audit_log (saml_user_provisioned)
else Existing user
SP->>DB: UPDATE user group, teams, email
SP->>DB: INSERT audit_log (saml_user_updated) if changed
end
SP->>DB: INSERT into sessions (session_id, user_id, expires_at)
SP->>DB: INSERT audit_log (saml_login)
SP->>B: Set-Cookie: session_id=xxx; HttpOnly; SameSite=Lax
SP->>B: HTTP 302 Redirect to /?saml_success=true
B->>SP: GET /api/auth/me (with cookie)
SP->>B: 200 { user: { id, username, group, teams, authSource } }
```
### Assertion Rejection Path
```mermaid
sequenceDiagram
participant B as Browser
participant SP as Dashboard (SP)
participant IdP as AD FS (IdP)
B->>SP: GET /api/auth/saml/login
SP->>B: HTTP 302 Redirect to IdP
B->>IdP: Authenticate
IdP->>B: POST assertion to SP callback
B->>SP: POST /api/auth/saml/callback
SP->>SP: Validate assertion
alt Invalid signature
SP->>SP: Log audit (saml_auth_failed, reason: invalid_signature)
SP->>B: Redirect /?saml_error=Invalid+assertion+signature
else Expired assertion
SP->>SP: Log audit (saml_auth_failed, reason: assertion_expired)
SP->>B: Redirect /?saml_error=Assertion+expired
else Account disabled
SP->>SP: Log audit (saml_auth_failed, reason: account_disabled)
SP->>B: Redirect /?saml_error=Account+is+disabled
end
```
---
## Component Architecture
```
┌───────────────────────────────────────────────────────────────────┐
│ Express Backend (port 3001) │
│ │
│ ┌────────────────┐ ┌─────────────────┐ ┌──────────────────┐ │
│ │ routes/saml.js │ │ routes/auth.js │ │ middleware/auth.js│ │
│ │ │ │ │ │ │ │
│ │ GET /status │ │ POST /login │ │ requireAuth() │ │
│ │ GET /login │ │ POST /logout │ │ requireGroup() │ │
│ │ POST /callback │ │ GET /me │ │ │ │
│ │ GET /metadata │ │ POST /change-pw │ │ (unchanged — │ │
│ └───────┬────────┘ └────────┬────────┘ │ reads session │ │
│ │ │ │ cookie only) │ │
│ │ │ └──────────────────┘ │
│ ▼ ▼ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ helpers/samlProvisioning.js │ │
│ │ │ │
│ │ resolveGroup(adGroups, config) → dashboardGroup │ │
│ │ resolveTeams(adGroups, config) → "STEAM,..." │ │
│ │ deriveUsername(nameId) → username │ │
│ │ provisionOrUpdateUser(assertion, config, ip) │ │
│ └───────────────────────┬───────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ helpers/samlConfig.js │ │
│ │ │ │
│ │ loadGroupMappingConfig() → validated config obj │ │
│ │ (reads config/adGroupMapping.json or env var) │ │
│ └───────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────┘
```
### Key Design Decisions
1. **Session reuse**: SAML login creates the exact same session format as local login. No changes to `requireAuth()` middleware.
2. **Feature flag isolation**: When `SAML_ENABLED=false`, SAML routes return 404 and no SAML library is loaded. Zero runtime cost when disabled.
3. **Config-driven mapping**: AD group names are externalized in `config/adGroupMapping.json`. Changing the mapping requires only a file edit and backend restart — no code changes.
4. **JIT provisioning**: Users are created on first login, updated on each subsequent login. AD is the source of truth for SSO users.
5. **Separation of concerns**: The provisioning logic (`samlProvisioning.js`) is a pure module with no HTTP dependencies — fully unit-testable without a web server.
---
## AD Group-to-Permission Mapping
### Configuration Structure
```json
{
"groups": {
"<AD-GROUP-CN>": "<Dashboard-Group>",
"CVE-Dashboard-Admins": "Admin",
"CVE-Dashboard-Users": "Standard_User",
"CVE-Dashboard-Leadership": "Leadership",
"CVE-Dashboard-ReadOnly": "Read_Only"
},
"teams": {
"<AD-GROUP-CN>": "<BU-Team-ID>",
"NTS-AEO-STEAM": "STEAM",
"NTS-AEO-ACCESS-ENG": "ACCESS-ENG",
"NTS-AEO-ACCESS-OPS": "ACCESS-OPS",
"NTS-AEO-INTELDEV": "INTELDEV"
},
"groupPriority": ["Admin", "Standard_User", "Leadership", "Read_Only"],
"defaultGroup": "Read_Only",
"attributes": {
"email": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
"displayName": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name",
"groups": "http://schemas.xmlsoap.org/claims/Group"
}
}
```
### Privilege Hierarchy
When a user belongs to multiple AD groups that map to different dashboard groups, the highest-privilege group wins:
```
Admin > Standard_User > Leadership > Read_Only
```
### Multi-Team Assignment
When a user belongs to multiple AD groups that map to BU teams, all matching teams are assigned:
```
AD Groups: ["NTS-AEO-STEAM", "NTS-AEO-ACCESS-ENG", "CVE-Dashboard-Users"]
→ user_group: "Standard_User"
→ bu_teams: "ACCESS-ENG,STEAM" (sorted alphabetically, deduplicated)
```
### Placeholder Group Names
The AD group names in the `groups` and `teams` sections (e.g., "CVE-Dashboard-Admins") are placeholders. When real group CNs are obtained from the AD administrators, update only this configuration file. No code changes required.
---
## Database Schema Changes
### Migration: `add_saml_auth_columns.js`
| Column | Type | Nullable | Default | Purpose |
|---|---|---|---|---|
| `auth_source` | VARCHAR(10) | NOT NULL | `'local'` | Discriminates local vs SSO users |
| `external_id` | VARCHAR(256) | NULL | NULL | SAML NameID for IdP correlation |
Additional changes:
- `password_hash` becomes nullable (SAML users have no local password)
- Partial unique index on `external_id WHERE external_id IS NOT NULL`
### Impact on Existing Data
- All existing users receive `auth_source = 'local'` and `external_id = NULL`
- No existing functionality is affected
- Migration is idempotent (safe to re-run)
---
## Environment Variables
| Variable | Required | Description | Example |
|---|---|---|---|
| `SAML_ENABLED` | Always | Master feature flag for SAML authentication | `false` |
| `SAML_IDP_METADATA_URL` | When SAML_ENABLED=true | AD FS federation metadata endpoint | `https://adfs.corp.local/FederationMetadata/2007-06/FederationMetadata.xml` |
| `SAML_SP_ENTITY_ID` | When SAML_ENABLED=true | Unique identifier for this Service Provider | `http://71.85.90.6:3001` |
| `SAML_SP_CALLBACK_URL` | When SAML_ENABLED=true | Assertion consumer service URL | `http://71.85.90.6:3001/api/auth/saml/callback` |
| `SAML_IDP_CERT_PATH` | When SAML_ENABLED=true | File path to IdP signing certificate (PEM format) | `/etc/cve-dashboard/idp-cert.pem` |
| `SESSION_LIFETIME_HOURS` | Optional | Session duration (1-720 hours, default: 24) | `8` |
| `AD_GROUP_MAPPING_JSON` | Optional | JSON string override for adGroupMapping.json | `{"groups":{...},"teams":{...}}` |
### Startup Validation
When `SAML_ENABLED=true`, the server validates at startup:
1. All required SAML env vars are set (fails with descriptive error if missing)
2. Certificate file exists and is readable (fails if not)
3. Group mapping config parses as valid JSON (fails if not)
4. All mapped team names exist in KNOWN_TEAMS (fails if not)
5. All mapped dashboard groups are valid (fails if not)
This fail-fast approach prevents silent misconfiguration in production.
---
## Build vs Wait: Phase Breakdown
### Phase 1 — Build Now (No AD Access Required)
| Component | File | Test Strategy |
|---|---|---|
| Database migration | `backend/migrations/add_saml_auth_columns.js` | Run against test DB, verify columns |
| Group mapping config | `backend/config/adGroupMapping.json` | Startup validation tests |
| Config loader | `backend/helpers/samlConfig.js` | Unit test with mock JSON files |
| JIT provisioner | `backend/helpers/samlProvisioning.js` | Unit test all paths with mock pool |
| SAML routes (skeleton) | `backend/routes/saml.js` | Integration test: feature flag, status, metadata |
| Session lifetime | `server.js` (startup block) | Unit test env var parsing |
| Auth route changes | `backend/routes/auth.js` | Integration test: SAML user login rejection |
| User route changes | `backend/routes/users.js` | Integration test: auth_source in responses, password block |
| Frontend SSO button | `frontend/src/components/LoginForm.js` | Render test: button hidden when flag=false |
| Admin auth_source badges | `frontend/src/components/UserManagement.js` | Render test: badge displays |
| Architecture doc | `docs/architecture/ad-saml-integration.md` | N/A (documentation) |
### Phase 2 — Requires Live AD FS Connection
| Component | Dependency | Who Provides It |
|---|---|---|
| SAML library installation | Package selection (`@node-saml/passport-saml` or `saml2-js`) | Development team |
| IdP metadata URL | AD FS federation metadata endpoint | AD administrators |
| IdP signing certificate | Token-signing cert exported from AD FS | AD administrators |
| SP registration | Relying party trust created in AD FS console | AD administrators |
| Real AD group names | Actual CNs of permission/team groups | AD administrators |
| Assertion parsing implementation | Fill in `routes/saml.js` callback | Development team |
| End-to-end flow testing | Working AD user accounts | AD administrators |
| Session lifetime tuning | AD FS token lifetime policy value | AD administrators |
---
## Security Considerations
### Certificate Management
- The IdP signing certificate is stored on disk at `SAML_IDP_CERT_PATH`
- When the IdP rotates its certificate, replace the file and restart the backend
- No database migration required for certificate rotation
- Consider monitoring certificate expiry dates (AD FS certs typically rotate annually)
### Assertion Replay Prevention
- Each SAML assertion is consumed exactly once by the callback handler
- The JIT provisioner's idempotent update pattern means replayed assertions would simply re-update the same user record (no escalation possible)
- For additional protection in Phase 2, implement InResponseTo validation and a short-lived assertion ID cache
### Trust Boundary
```
┌─────────────────────────────────┐ ┌─────────────────────────────────┐
│ Trust Zone A │ │ Trust Zone B │
│ │ │ │
│ Dashboard (SP) │ │ AD FS (IdP) │
│ - Validates assertions │ │ - Authenticates users │
│ - Trusts ONLY signed assertions │ │ - Signs assertions with │
│ - Creates local sessions │ │ private key │
│ - Enforces local authorization │ │ - Asserts group memberships │
│ │ │ │
└────────────────┬─────────────────┘ └────────────────┬────────────────┘
│ │
└─── SAML 2.0 over HTTPS (HTTP-POST) ────┘
```
- The SP trusts assertions only when cryptographically signed by the IdP
- Group memberships in the assertion drive permission assignment — the SP does not query AD directly
- If the IdP is compromised, an attacker could forge assertions. Mitigate with certificate pinning and monitoring assertion patterns in audit logs.
- The SP never sends credentials to the IdP — authentication happens entirely on the IdP side
### Break-Glass Protection
- The last local Admin account cannot be deleted or deactivated
- If the IdP is unavailable, local Admin users can still log in with username/password
- SAML users cannot authenticate via password (and vice versa) — the two paths are isolated per user record
### Transport Security
- Production deployments should serve the SP callback over HTTPS
- The SAMLResponse is transmitted via HTTP-POST binding (browser-mediated, not direct server-to-server)
- The assertion is signed — even if transmitted over HTTP, it cannot be tampered with without detection
- For defense in depth, HTTPS prevents assertion interception by network observers
---
## Hybrid Login User Experience
```
┌─────────────────────────────────────────────────────────────┐
│ Login Page │
│ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ ┌─────────────────────────────┐ │ │
│ │ │ Sign in with SSO │ │ │
│ │ │ (redirects to AD FS) │ │ │
│ │ └─────────────────────────────┘ │ │
│ │ │ │
│ │ ─── or sign in with local account ─── │ │
│ │ │ │
│ │ Username: [________________] │ │
│ │ Password: [________________] │ │
│ │ │ │
│ │ [Sign In] │ │
│ └────────────────────────────────────────────────────────┘ │
│ │
│ Note: SSO button only visible when SAML_ENABLED=true │
│ Local login always available (break-glass for admins) │
└─────────────────────────────────────────────────────────────┘
```
---
## Related Specifications
- `.kiro/specs/ad-saml-integration/requirements.md` — detailed acceptance criteria
- `.kiro/specs/ad-saml-integration/design.md` — implementation design with code examples
- `.kiro/specs/group-based-access-control/requirements.md` — existing RBAC system
- `.kiro/specs/multi-bu-tenancy/design.md` — BU team scoping (leveraged by AD integration)

View File

@@ -0,0 +1,547 @@
# Split Architecture Proposal: Collector + Indexer
**Author:** Infrastructure Team
**Date:** 2026-06-08
**Status:** Draft — Pending Review
**Scope:** Scale CVE Dashboard from 2 teams / ~15 users to company-wide deployment (100+ users, 15+ teams)
---
## Executive Summary
The STEAM Security Dashboard currently runs as a monolithic single-process Express application on CT107 (dashboard-dev, 71.85.90.9). This single process simultaneously serves the frontend, handles all API requests, and performs background data collection from Ivanti, Jira, CARD, Atlas, and NVD APIs.
At current scale (2 teams, <15 users, daily sync), this architecture works. At company-wide scale (15+ teams, hundreds of users, sub-hourly sync), it will not. This document proposes a phased transition to a **Collector + API Server** architecture that separates data ingestion from request serving.
**Critical constraint:** CT107 (71.85.90.9) has the firewall rules granting access to the production Ivanti, Jira, and CARD APIs. The collector component must remain on this machine or firewall rules must be extended.
---
## Table of Contents
- [Current Architecture](#current-architecture)
- [Problem Statement](#problem-statement)
- [Proposed Architecture](#proposed-architecture)
- [Phase Plan](#phase-plan)
- [Infrastructure Requirements](#infrastructure-requirements)
- [Risk Assessment](#risk-assessment)
- [Decision Points](#decision-points)
- [Appendix: Current Data Flow Analysis](#appendix-current-data-flow-analysis)
---
## Current Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ CT107 (dashboard-dev) │
│ 71.85.90.9 — 48 GB RAM, 250 GB Disk │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ Express Process (port 3001/3100) │ │
│ │ │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │ │
│ │ │ React SPA │ │ API Routes │ │ Sync Workers │ │ │
│ │ │ (static) │ │ (50+ endpts)│ │ (setInterval) │ │ │
│ │ └─────────────┘ └──────────────┘ └────────────────┘ │ │
│ │ │ │ │ │
│ │ │ Shared PG Pool (10 conn) │ │
│ │ │ │ │ │
│ └──────────────────────────┼──────────┼─────────────────────┘ │
│ │ │ │
│ ┌──────────────────────────▼──────────▼─────────────────────┐ │
│ │ PostgreSQL 16 (Docker, port 5433) │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ Firewall Access: Ivanti API, Jira DC, CARD API, Atlas API │
└─────────────────────────────────────────────────────────────────┘
```
### Key Metrics (Current)
| Metric | Current Value | Company-Wide Projection |
|--------|--------------|------------------------|
| Concurrent users | 515 | 100300 |
| Teams tracked | 2 | 15+ |
| Ivanti findings (open) | ~200500 | 2,00010,000+ |
| Ivanti sync frequency | 24h | 14h desired |
| PG connection pool | 10 | Insufficient |
| Jira API rate limit | 1,440/day | Shared across all users |
| Data sources | 5 (Ivanti, NVD, Jira, Atlas, CARD) | 8+ (add CrowdStrike, Qualys, Tanium) |
---
## Problem Statement
### 1. Sync Blocks the API Server
`syncFindings()` runs sequentially through:
1. Fetch all open findings pages (100/page)
2. Upsert findings batch into PostgreSQL
3. Detect archive changes (compare all previous vs current)
4. Fetch all closed findings pages
5. Upsert closed findings
6. Run BU drift checker (makes additional API calls per disappeared finding)
7. Sync FP workflow counts (sweeps all closed pages again)
8. Compute and store anomaly summary
9. Record counts history
At 500 findings, this takes 25 minutes. At 10,000 findings across 15 teams, this could take 1530 minutes. During sync, the Express process is saturated — API responses slow, the connection pool contends.
### 2. Single Point of Failure
One process handles everything. A memory leak during sync, an unhandled promise rejection in the BU drift checker, or a runaway loop in archive detection crashes the entire dashboard for all users.
### 3. Connection Pool Exhaustion
10 connections shared between:
- User-facing read queries (findings list, compliance items, charts)
- Sync bulk upserts (batches of 100 rows × 18 columns)
- User writes (notes, overrides, queue operations)
The pool already logs warnings at 8/10 active. At 100+ concurrent users issuing reads while a sync writes thousands of rows, this will deadlock or time out.
### 4. Rate Limits Shared Across Functions
Jira's 1,440/day limit is consumed by both background sync and user-initiated operations (lookups, ticket creation). A bulk sync could exhaust the daily budget, blocking users from creating tickets the rest of the day.
### 5. No Horizontal Scaling Path
Cannot add a second API server without also duplicating the sync scheduler, which would cause duplicate syncs, double-writes, and race conditions.
### 6. Firewall Constraint
CT107 has the only firewall access to production Ivanti, Jira, and CARD APIs. The collector (data fetcher) must run on this machine. The API server could potentially move elsewhere, but the collector cannot without firewall changes.
---
## Proposed Architecture
### Target State
```
┌─────────────────────────────────────────────────────────────────┐
│ CT107 (dashboard-dev) │
│ 71.85.90.9 — 48 GB RAM, 250 GB Disk │
│ ★ Firewall access to prod APIs ★ │
│ │
│ ┌───────────────────────────────────┐ ┌─────────────────────┐│
│ │ API Server (Express, port 3001) │ │ Collector Service ││
│ │ │ │ (Node.js worker) ││
│ │ • React SPA serving │ │ ││
│ │ • All /api/* read endpoints │ │ • Ivanti sync ││
│ │ • User writes (notes, queue) │ │ • Jira bulk sync ││
│ │ • On-demand lookups (proxied) │ │ • CARD cache sync ││
│ │ • Triggers collector via │ │ • Atlas cache sync ││
│ │ pg NOTIFY │ │ • NVD bulk sync ││
│ │ │ │ • Archive detect ││
│ │ Pool: 15 conn (reads + writes) │ │ • BU drift checker ││
│ │ │ │ • Anomaly compute ││
│ └───────────────┬───────────────────┘ │ • Compliance parse ││
│ │ │ ││
│ │ │ Pool: 10 conn ││
│ │ │ (bulk upserts) ││
│ │ │ ││
│ │ │ Listens: ││
│ │ │ pg LISTEN ││
│ │ │ 'sync_trigger' ││
│ │ └──────────┬──────────┘│
│ │ │ │
│ ┌───────────────▼──────────────────────────────────▼─────────┐│
│ │ PostgreSQL 16 (Docker, port 5433) ││
│ │ Pool: 25 total connections allocated ││
│ └────────────────────────────────────────────────────────────┘│
│ │
└─────────────────────────────────────────────────────────────────┘
```
### Component Responsibilities
#### API Server (`cve-api.service`)
| Responsibility | Details |
|---|---|
| Frontend serving | Static React build via `express.static` |
| Read endpoints | All GET routes — findings, compliance, charts, exports |
| User writes | Notes, overrides, queue items, ticket CRUD, KB uploads |
| On-demand lookups | Single NVD lookup, single Jira issue lookup, CARD real-time queries |
| Sync trigger | `SELECT pg_notify('sync_trigger', '{"type":"findings","user":"admin"}')` |
| Health/status | Expose collector status via sync_state table reads |
#### Collector (`cve-collector.service`)
| Responsibility | Details |
|---|---|
| Scheduled syncs | Ivanti findings (configurable interval), workflows (24h) |
| Bulk API operations | Jira JQL sync-all, Atlas cache refresh, NVD bulk sync |
| Post-sync processing | Archive detection, BU drift classification, closed-gone detection |
| Anomaly computation | Open/closed deltas, classification breakdown, significance flagging |
| Compliance parsing | Spawns Python subprocess for xlsx parsing on upload commit |
| Event-driven triggers | Listens on `pg LISTEN sync_trigger` for on-demand requests |
| Rate budget management | Owns the Jira daily/burst counters; API server gets a reserved allocation |
### Communication Pattern
```
User clicks "Sync" in UI
API Server receives POST /api/ivanti/findings/sync
API Server: SELECT pg_notify('sync_trigger', '{"type":"findings"}')
API Server responds: { status: 'sync_started', message: 'Check /sync-status' }
Collector receives NOTIFY, starts syncFindings()
Collector updates ivanti_sync_state (status='syncing')
Collector completes, updates ivanti_sync_state (status='success')
Frontend polls GET /api/ivanti/findings/sync-status → sees 'success' → refreshes
```
No Redis. No message broker. Just PostgreSQL LISTEN/NOTIFY — zero new infrastructure.
---
## Phase Plan
### Phase 0: Immediate Improvements (Week 12)
**Goal:** Reduce risk within the current monolith. No architectural changes.
| Task | Effort | Impact |
|------|--------|--------|
| Make `POST /sync` non-blocking — return immediately, let sync run in background | 2h | Unblocks users during sync |
| Add `GET /api/ivanti/findings/sync-status` endpoint | 1h | Frontend can poll for completion |
| Increase PG pool from 10 → 20 connections | 10min | Headroom for concurrent operations |
| Add `pg_stat_activity` monitoring query to health endpoint | 30min | Visibility into pool pressure |
| Update frontend to poll sync-status instead of waiting | 2h | UX improvement |
**Deliverables:**
- Updated `ivantiFindings.js` with async sync dispatch
- New sync-status polling endpoint
- Frontend ReportingPage sync UX updated
- Pool configuration change in `db.js`
---
### Phase 1: Extract Collector (Weeks 34)
**Goal:** Separate data collection into its own process on CT107.
| Task | Effort | Impact |
|------|--------|--------|
| Create `backend/collector.js` — standalone Node process | 4h | Fault isolation |
| Move sync functions from route files into shared `lib/sync/` modules | 3h | Code reuse between collector and API |
| Implement pg LISTEN/NOTIFY trigger mechanism | 2h | API → Collector communication |
| Create `cve-collector.service` systemd unit | 30min | Process management |
| Add collector health check and status reporting | 1h | Observability |
| Update `POST /sync` routes to use pg_notify instead of inline sync | 1h | Complete decoupling |
| Add `sync_jobs` table for job tracking (queued, running, complete, failed) | 1h | Multi-user sync coordination |
| Update CI/CD pipeline to deploy collector service | 2h | Automated deployment |
**Deliverables:**
- `backend/collector.js` — entry point for collector process
- `backend/lib/sync/` — shared sync logic (extracted from routes)
- `systemd/cve-collector.service` — systemd unit
- Updated `.gitlab-ci.yml` with collector deploy stage
- `sync_jobs` table for job state tracking
**File structure after Phase 1:**
```
backend/
├── server.js # API server (unchanged entry point)
├── collector.js # NEW — collector entry point
├── db.js # Shared pool config
├── lib/
│ └── sync/
│ ├── ivantiFindings.js # Extracted from routes/ivantiFindings.js
│ ├── ivantiWorkflows.js # Extracted from routes/ivantiWorkflows.js
│ ├── jiraBulkSync.js # Extracted from routes/jiraTickets.js
│ ├── atlasCache.js # Extracted from routes/atlas.js
│ ├── nvdBulkSync.js # New — bulk NVD operations
│ ├── archiveDetection.js # Extracted from routes/ivantiFindings.js
│ └── anomalyCompute.js # Extracted from routes/ivantiFindings.js
├── routes/ # API routes — now thin, read-heavy
└── helpers/ # Shared API client helpers (unchanged)
```
---
### Phase 2: Multi-Tenancy & Scale Hardening (Weeks 58)
**Goal:** Prepare for 15 teams and hundreds of users.
| Task | Effort | Impact |
|------|--------|--------|
| Per-team sync scheduling — stagger syncs to avoid API burst | 3h | Spreads load |
| Jira rate budget partitioning (collector gets 80%, API gets 20%) | 2h | Prevents sync from starving users |
| Per-BU finding isolation — team users only see their findings | 4h | Data scoping |
| Add connection pooling metrics endpoint (`/api/admin/pool-stats`) | 1h | Operational visibility |
| Implement sync queue with priority (user-triggered > scheduled) | 3h | Better UX |
| Add retry logic with exponential backoff to collector | 2h | Resilience |
| Partial-progress persistence — don't lose work on mid-sync failure | 4h | Data integrity |
| PG connection pool separation — API pool (15) + Collector pool (10) | 1h | Isolation |
| Add `pg_bouncer` or similar for connection multiplexing (optional) | 4h | Scale past 50 concurrent |
**Deliverables:**
- Team-scoped sync scheduler in collector
- Rate budget allocation system
- Retry/backoff logic
- Partial progress tracking
- Pool separation
---
### Phase 3: Additional Data Sources (Weeks 912)
**Goal:** Integrate CrowdStrike, Qualys, and Tanium feeds.
| Task | Effort | Impact |
|------|--------|--------|
| CrowdStrike Falcon API integration in collector | 8h | New vulnerability source |
| Qualys VMDR API integration in collector | 8h | New vulnerability source |
| Tanium asset inventory sync | 6h | Asset correlation |
| Cross-source finding deduplication logic | 6h | Data quality |
| Unified findings view (merged from all sources) | 4h | Single pane of glass |
| Source-specific sync schedules (configurable per source) | 2h | Flexibility |
**Note:** All new API integrations go into the collector. The API server never makes outbound calls to external vulnerability platforms except for single-item on-demand lookups.
**Firewall implications:** CrowdStrike, Qualys, and Tanium API access will need firewall rules added to CT107 (71.85.90.9). Submit firewall requests in advance.
---
### Phase 4: Horizontal Scaling (Weeks 13+)
**Goal:** Support 300+ concurrent users if company-wide adoption materializes.
| Task | Effort | Impact |
|------|--------|--------|
| Move API server to a separate LXC container (with more resources) | 4h | Dedicated API resources |
| Run multiple API server instances behind a load balancer | 8h | Horizontal scale |
| Keep collector on CT107 (firewall access) | 0h | No change needed |
| Add Redis for session store (replace PG sessions) | 4h | Multi-instance sessions |
| Add read replicas if PG becomes the bottleneck | 8h | Read scale |
| Evaluate moving PG to CT109 (zbl-indexer, 32GB/500GB) | 2h | Larger DB host |
**Architecture at Phase 4:**
```
┌─────────────────┐
│ Load Balancer │
│ (nginx/HAProxy)│
└────┬───────┬────┘
│ │
┌─────────────▼─┐ ┌─▼─────────────┐
│ API Server 1 │ │ API Server 2 │ (New LXC or CT103)
│ (Express) │ │ (Express) │
└───────┬───────┘ └───────┬───────┘
│ │
└─────────┬─────────┘
┌────────────────────────────▼──────────────────────────────────────┐
│ CT107 (71.85.90.9) │
│ │
│ ┌─────────────────────────┐ ┌──────────────────────────────┐ │
│ │ Collector Service │ │ PostgreSQL 16 │ │
│ │ (sole process with │ │ (or moved to CT109) │ │
│ │ firewall API access) │ │ │ │
│ └─────────────────────────┘ └──────────────────────────────┘ │
│ │
│ ★ Firewall: Ivanti, Jira, CARD, Atlas, CrowdStrike, Qualys ★ │
└───────────────────────────────────────────────────────────────────┘
```
---
## Infrastructure Requirements
### CT107 Resource Allocation (Current → Phase 2)
| Resource | Current | Phase 2 Target | Notes |
|----------|---------|---------------|-------|
| RAM | 48 GB | 48 GB (sufficient) | Node processes use <2GB each |
| CPU | Shared | May need 4+ dedicated cores | Sync is CPU-intensive during transform |
| Disk | 250 GB | 250 GB (sufficient) | PG data + uploads + logs |
| PG connections | 10 | 25 (15 API + 10 collector) | Configure in `postgresql.conf` |
| Systemd services | 2 (backend + frontend) | 3 (api + collector + postgres) | Frontend served by API |
### PostgreSQL Tuning (for 15 teams / hundreds of users)
```
# postgresql.conf changes
max_connections = 50 # Up from default 100 is fine, need headroom
shared_buffers = 4GB # 25% of available RAM for PG
effective_cache_size = 12GB # 75% of RAM PG can expect from OS
work_mem = 64MB # Per-sort/hash operation
maintenance_work_mem = 512MB # For VACUUM, CREATE INDEX
wal_level = replica # If read replicas needed later
```
### Firewall Dependencies
| Service | Endpoint | Required By | Current Access |
|---------|----------|-------------|----------------|
| Ivanti/RiskSense | platform4.risksense.com:443 | Collector | ✅ CT107 only |
| Jira Data Center | jira.charter.com:443 | Collector + API (lookups) | ✅ CT107 only |
| CARD API | card.charter.com:443 | API (real-time) | ✅ CT107 only |
| Atlas InfoSec | (internal) | Collector | ✅ CT107 only |
| NVD API | services.nvd.nist.gov:443 | Collector + API | ✅ Public |
| CrowdStrike | api.crowdstrike.com:443 | Collector | ❌ Firewall request needed |
| Qualys | qualysapi.qualys.com:443 | Collector | ❌ Firewall request needed |
| Tanium | (internal) | Collector | ❌ Firewall request needed |
**Key constraint:** If the API server moves off CT107 in Phase 4, you'll need firewall rules for the new host to reach Jira (for user lookups) and CARD (for real-time queries). Alternatively, the collector could proxy those on-demand requests — adds latency but avoids firewall changes.
---
## Risk Assessment
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Collector crash doesn't affect API users | — | — | This is the primary benefit of splitting |
| Collector and API race on DB writes | Medium | Low | Collector does bulk upserts; API does single-row writes. Different tables mostly. Use advisory locks for sync_state. |
| Sync trigger lost (pg NOTIFY missed) | Low | Medium | Collector also runs on a schedule. Missed trigger just delays to next interval. |
| Phase 1 introduces bugs in extraction | Medium | Medium | Comprehensive test suite exists. Run parallel (old monolith + new split) in staging for 1 week. |
| Firewall change delays block Phase 4 | High | Medium | Start firewall requests early. Phase 4 is optional — single-machine split (Phases 13) works fine at 15 teams. |
| PG becomes bottleneck at 300+ users | Low | High | Phase 4 addresses with read replicas. CT109 (500GB, 32GB) available as larger DB host. |
---
## Decision Points
These require team/leadership input before proceeding:
1. **Sync frequency target:** Is 1-hour sync acceptable, or do teams need near-real-time (15 min)? This affects collector design complexity and API rate budget math.
2. **API server location:** Keep everything on CT107, or move the API server to a separate container? Keeping it on CT107 is simpler (no firewall changes for CARD/Jira lookups) but limits scaling options.
3. **Database location:** Keep PG on CT107, or move to CT109 (zbl-indexer, 500GB disk, 32GB RAM)? Moving adds network latency but gives more room for growth.
4. **CrowdStrike/Qualys/Tanium priority:** Which new data sources are most urgent? This affects Phase 3 ordering and firewall request timing.
5. **Session management:** At 300+ users, PG-backed sessions will be high-churn. Acceptable, or invest in Redis? Redis adds infrastructure but is the industry standard for session stores at scale.
6. **Multi-instance API:** Is the goal to survive a single API server restart without downtime? If yes, Phase 4 (load balancer + multiple instances) is needed. If brief restarts during deploys are acceptable, single-instance on CT107 works through Phase 3.
---
## Appendix: Current Data Flow Analysis
### Data Collection Patterns
| Source | Trigger | Frequency | Data Volume | Processing |
|--------|---------|-----------|-------------|------------|
| Ivanti Findings | Schedule + manual | 24h | 100500 findings (all pages) | Extract, upsert, archive detect, BU drift, anomaly |
| Ivanti Workflows | Schedule + manual | 24h | 50 workflow batches | Store as JSON blob |
| Ivanti Closed Findings | During findings sync | 24h | All closed pages | Upsert + closed archive detection |
| Jira Bulk Sync | Manual (admin) | On-demand | All tracked tickets via JQL | Status/summary update per ticket |
| Jira Single Lookup | User action | Real-time | 1 issue | Proxy + display |
| NVD Lookup | User action | Real-time | 1 CVE | Proxy + optional save |
| NVD Bulk Sync | Manual | On-demand | All CVEs in DB | Batch update metadata |
| Atlas Action Plans | Cache refresh | Background | Per-host plan data | Cache in `atlas_action_plans_cache` |
| CARD Operations | User action | Real-time | 1 asset at a time | Proxy (confirm/decline/redirect) |
| Compliance xlsx | Manual upload | Weekly | 1 file → hundreds of rows | Python parse → PG upsert (transactional) |
### What Moves to Collector vs Stays in API
| Operation | Collector | API Server | Rationale |
|-----------|-----------|------------|-----------|
| Ivanti findings sync (all pages) | ✅ | | Heavy, multi-page, post-processing |
| Ivanti workflows sync | ✅ | | Scheduled background task |
| Ivanti closed sweep | ✅ | | Part of findings sync pipeline |
| Archive detection | ✅ | | CPU-intensive comparison |
| BU drift checker | ✅ | | Makes additional API calls |
| Anomaly computation | ✅ | | Depends on sync completion |
| Jira bulk sync-all | ✅ | | Consumes rate budget, multi-issue |
| NVD bulk sync | ✅ | | Multi-CVE, rate-limited |
| Atlas cache refresh | ✅ | | Background, per-host API calls |
| Compliance xlsx parse | ✅ | | Spawns Python, heavy DB writes |
| Single Jira lookup | | ✅ | User-initiated, real-time, 1 call |
| Single NVD lookup | | ✅ | User-initiated, real-time, 1 call |
| CARD operations | | ✅ | User-initiated, real-time |
| All GET /api/* reads | | ✅ | Pure DB queries, user-facing |
| Notes/overrides/queue | | ✅ | Small writes, user-facing |
| File uploads | | ✅ | User-initiated, disk I/O |
### Sync Pipeline Detail (becomes collector's core loop)
```
┌──────────────────────────────────────────────────────────────────┐
│ Collector Sync Pipeline │
│ │
│ ┌────────────────┐ │
│ │ 1. Fetch Open │ ← Ivanti API (paginated, 100/page) │
│ │ Findings │ │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 2. Extract & │ ← Transform raw API → normalized rows │
│ │ Transform │ │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 3. Upsert to │ ← Batch INSERT ON CONFLICT (100/batch) │
│ │ PG │ Preserves notes + overrides │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 4. Archive │ ← Compare previous IDs vs current IDs │
│ │ Detection │ Detect disappeared + returned findings │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 5. Fetch Closed│ ← Ivanti API (all closed pages) │
│ │ Findings │ Upsert as state='closed' │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 6. BU Drift │ ← Re-query Ivanti for disappeared IDs │
│ │ Checker │ Classify: BU reassign / severity / decom │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 7. FP Workflow │ ← Sweep closed findings for FP# tickets │
│ │ Counts │ Aggregate by state │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 8. Anomaly │ ← Compute deltas, write to anomaly_log │
│ │ Summary │ │
│ └───────┬────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ 9. Update │ ← sync_state status='success' │
│ │ Sync State │ Notify API server: pg_notify('sync_done') │
│ └────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘
```
---
## Timeline Summary
| Phase | Timeframe | Key Outcome | Required For |
|-------|-----------|-------------|--------------|
| **0** | Weeks 12 | Non-blocking sync, pool increase | Immediate UX fix |
| **1** | Weeks 34 | Collector extracted, fault isolation | Multi-team onboarding |
| **2** | Weeks 58 | Multi-tenancy, rate budgeting, retries | 15 teams / 100+ users |
| **3** | Weeks 912 | New data sources (CS/Qualys/Tanium) | Full vuln coverage |
| **4** | Weeks 13+ | Horizontal scaling, load balancing | 300+ users (if needed) |
Phases 02 are recommended regardless of company-wide rollout. Phase 3 depends on data source priority decisions. Phase 4 is contingent on actual adoption numbers.
---
## Next Steps
1. Review this document and provide input on [Decision Points](#decision-points)
2. Approve Phase 0 for immediate implementation
3. Schedule Phase 1 kickoff once Phase 0 is validated in staging
4. Submit firewall requests for CrowdStrike/Qualys/Tanium access to CT107 (long lead time)

View File

@@ -7,7 +7,7 @@
<meta name="theme-color" content="#000000" />
<meta
name="description"
content="Steam CVE Dashboard - Vulnerability tracking and documentation"
content="AEGIS — Advanced Engineering Group Intelligence System"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
<!--
@@ -24,7 +24,7 @@
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>CVE Dashboard</title>
<title>AEGIS</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500;600;700&family=Outfit:wght@300;400;500;600;700&display=swap" rel="stylesheet">

View File

@@ -1,6 +1,6 @@
{
"short_name": "SCD",
"name": "Steam CVE Dashboard",
"short_name": "AEGIS",
"name": "AEGIS — Advanced Engineering Group Intelligence System",
"icons": [
{
"src": "favicon.ico",

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

View File

@@ -186,7 +186,7 @@ const getSeverityDotColor = (severity) => {
const severityLevels = ['All Severities', 'Critical', 'High', 'Medium', 'Low'];
export default function App() {
const { isAuthenticated, loading: authLoading, canWrite, canDelete, canExport, isAdmin, getActiveTeamsParam, adminScope } = useAuth();
const { isAuthenticated, loading: authLoading, canWrite, canDelete, canExport, isAdmin, isInGroup, getActiveTeamsParam, adminScope } = useAuth();
const [searchQuery, setSearchQuery] = useState('');
const [selectedVendor, setSelectedVendor] = useState('All Vendors');
const [selectedSeverity, setSelectedSeverity] = useState('All Severities');
@@ -1022,10 +1022,15 @@ export default function App() {
<Menu className="w-5 h-5" />
</button>
<div>
<h1 className="text-4xl font-bold text-intel-accent mb-1 font-mono tracking-tight">
STEAM Security Dashboard
</h1>
<p className="text-gray-400 text-sm font-sans">NTS Threat Intelligence and Metric Aggregation</p>
<div className="flex items-center gap-3">
<img src="/shieldlogo.jpeg" alt="AEGIS" style={{ width: '44px', height: '44px', borderRadius: '6px' }} />
<div>
<h1 className="text-4xl font-bold text-intel-accent mb-1 font-mono tracking-tight">
AEGIS
</h1>
<p className="text-gray-400 text-sm font-sans">Advanced Engineering Group Intelligence System</p>
</div>
</div>
</div>
</div>
<div className="flex items-center gap-3">
@@ -1102,7 +1107,8 @@ export default function App() {
{/* Page content */}
{currentPage === 'triage' && <VulnerabilityTriagePage filterDate={calendarFilter} filterEXC={reportingExcFilter} />}
{currentPage === 'compliance' && <CompliancePage onNavigate={setCurrentPage} />}
{currentPage === 'ccp-metrics' && <CCPMetricsPage />}
{currentPage === 'ccp-metrics' && isInGroup('Admin', 'Leadership') && <CCPMetricsPage />}
{currentPage === 'ccp-metrics' && !isInGroup('Admin', 'Leadership') && (() => { setCurrentPage('home'); return null; })()}
{currentPage === 'knowledge-base' && <KnowledgeBasePage />}
{currentPage === 'exports' && <ExportsPage />}
{currentPage === 'jira' && <JiraPage />}

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

View File

@@ -95,6 +95,7 @@ export default function LoaderModal({ isOpen, onClose, initialDevices }) {
setBulkDefaults({});
setEnrichErrors([]);
setValidationWarnings([]);
setEnriching(false);
}, [isOpen, initialDevices]);
// Auto-select required columns + useful defaults when operation type changes
@@ -417,11 +418,27 @@ export default function LoaderModal({ isOpen, onClose, initialDevices }) {
{/* Enrich errors */}
{enrichErrors.length > 0 && (
<div style={{ marginBottom: '0.75rem', padding: '0.5rem 0.75rem', background: 'rgba(239, 68, 68, 0.1)', border: '1px solid rgba(239, 68, 68, 0.3)', borderRadius: '0.375rem' }}>
<div style={{ fontSize: '0.7rem', color: '#EF4444', display: 'flex', alignItems: 'center', gap: '0.3rem' }}>
<AlertCircle style={{ width: '12px', height: '12px' }} />
{enrichErrors.length === 1 && enrichErrors[0].ip === 'all'
? enrichErrors[0].error
: `${enrichErrors.length} device(s) not found in CARD`}
<div style={{ fontSize: '0.7rem', color: '#EF4444', display: 'flex', alignItems: 'center', gap: '0.3rem', justifyContent: 'space-between' }}>
<span style={{ display: 'flex', alignItems: 'center', gap: '0.3rem' }}>
<AlertCircle style={{ width: '12px', height: '12px' }} />
{enrichErrors.length === 1 && enrichErrors[0].ip === 'all'
? enrichErrors[0].error
: enrichErrors.some(e => e.error && e.error.includes('timed out'))
? `${enrichErrors.length} device(s) timed out — CARD may be slow`
: `${enrichErrors.length} device(s) not found in CARD`}
</span>
<button
onClick={enrichFromCard}
disabled={enriching}
style={{
background: 'rgba(239, 68, 68, 0.15)', border: '1px solid rgba(239, 68, 68, 0.4)',
borderRadius: '0.25rem', padding: '0.2rem 0.5rem',
color: '#F87171', cursor: 'pointer',
fontSize: '0.65rem', fontFamily: 'monospace', fontWeight: '600',
}}
>
Retry
</button>
</div>
</div>
)}

View File

@@ -30,11 +30,10 @@ export default function LoginForm() {
<div className="intel-card rounded-lg shadow-2xl max-w-md w-full p-8 border-intel-accent relative z-10">
<div className="text-center mb-8">
<div className="w-16 h-16 bg-gradient-to-br from-intel-accent to-intel-accent-dim rounded-full flex items-center justify-center mx-auto mb-4 shadow-lg" style={{boxShadow: '0 0 30px rgba(0, 217, 255, 0.4)'}}>
<Lock className="w-8 h-8 text-intel-darkest" />
</div>
<h1 className="text-3xl font-bold text-intel-accent font-mono tracking-tight">CVE INTEL</h1>
<p className="text-gray-400 mt-2 font-sans text-sm">Threat Intelligence Access Portal</p>
{/* ⚠️ CONVENTION: Use lucide-react icons instead of <img> tags for iconography */}
<img src="/shieldlogo.jpeg" alt="AEGIS" style={{ width: '64px', height: '64px', borderRadius: '50%', margin: '0 auto 1rem', display: 'block', boxShadow: '0 0 30px rgba(0, 217, 255, 0.4)' }} />
<h1 className="text-3xl font-bold text-intel-accent font-mono tracking-tight">AEGIS</h1>
<p className="text-gray-400 mt-2 font-sans text-sm">Advanced Engineering Group Intelligence System</p>
</div>
{error && (

View File

@@ -6,7 +6,7 @@ const NAV_ITEMS = [
{ id: 'home', label: 'Home', icon: Home, color: '#0EA5E9', description: 'Main dashboard' },
{ id: 'triage', label: 'Vuln Triage', icon: BarChart2, color: '#F59E0B', description: 'Active findings & CVE triage' },
{ id: 'compliance', label: 'Compliance', icon: ShieldCheck, color: '#14B8A6', description: 'AEO posture & metrics' },
{ id: 'ccp-metrics', label: 'CCP Metrics', icon: Building2, color: '#A78BFA', description: 'Cross-vertical VCL reporting' },
{ id: 'ccp-metrics', label: 'CCP Metrics', icon: Building2, color: '#A78BFA', description: 'Cross-vertical VCL reporting', requiredGroups: ['Admin', 'Leadership'] },
{ id: 'knowledge-base', label: 'Knowledge Base', icon: BookOpen, color: '#10B981', description: 'Articles & documentation' },
{ id: 'exports', label: 'Exports', icon: Download, color: '#8B5CF6', description: 'Export data & reports' },
{ id: 'jira', label: 'Jira Tickets', icon: Ticket, color: '#6366F1', description: 'Jira issue tracking & sync' },
@@ -16,7 +16,7 @@ const NAV_ITEMS = [
const ADMIN_ITEM = { id: 'admin', label: 'Admin Panel', icon: Settings, color: '#EF4444', description: 'User management & audit' };
export default function NavDrawer({ isOpen, onClose, currentPage, onNavigate }) {
const { isAdmin } = useAuth();
const { isAdmin, isInGroup } = useAuth();
if (!isOpen) return null;
@@ -46,11 +46,16 @@ export default function NavDrawer({ isOpen, onClose, currentPage, onNavigate })
{/* Drawer header */}
<div style={{ display: 'flex', justifyContent: 'space-between', alignItems: 'flex-start', marginBottom: '2rem' }}>
<div>
<div style={{ fontFamily: 'monospace', fontSize: '1rem', fontWeight: '700', color: '#0EA5E9', textTransform: 'uppercase', letterSpacing: '0.15em', textShadow: '0 0 12px rgba(14, 165, 233, 0.5)' }}>
STEAM
</div>
<div style={{ fontFamily: 'monospace', fontSize: '0.65rem', color: '#475569', textTransform: 'uppercase', letterSpacing: '0.1em', marginTop: '2px' }}>
Security Dashboard
<div style={{ display: 'flex', alignItems: 'center', gap: '0.5rem' }}>
<img src="/shieldlogo.jpeg" alt="AEGIS" style={{ width: '28px', height: '28px', borderRadius: '4px' }} />
<div>
<div style={{ fontFamily: 'monospace', fontSize: '1rem', fontWeight: '700', color: '#0EA5E9', textTransform: 'uppercase', letterSpacing: '0.15em', textShadow: '0 0 12px rgba(14, 165, 233, 0.5)' }}>
AEGIS
</div>
<div style={{ fontFamily: 'monospace', fontSize: '0.65rem', color: '#475569', textTransform: 'uppercase', letterSpacing: '0.1em', marginTop: '2px' }}>
Security Dashboard
</div>
</div>
</div>
</div>
<button
@@ -65,7 +70,7 @@ export default function NavDrawer({ isOpen, onClose, currentPage, onNavigate })
{/* Nav items */}
<nav style={{ display: 'flex', flexDirection: 'column', gap: '0.375rem' }}>
{NAV_ITEMS.map(({ id, label, icon: Icon, color, description }) => {
{NAV_ITEMS.filter(({ requiredGroups }) => !requiredGroups || isInGroup(...requiredGroups)).map(({ id, label, icon: Icon, color, description }) => {
const active = currentPage === id;
return (
<button

View File

@@ -1,11 +1,12 @@
import React, { useState, useEffect, useCallback, useRef } from 'react';
import { Upload, MessageSquare, RefreshCw, AlertCircle, Loader, RotateCcw, Info } from 'lucide-react';
import { Upload, MessageSquare, RefreshCw, AlertCircle, Loader, RotateCcw, Info, FileSpreadsheet } from 'lucide-react';
import { useAuth } from '../../contexts/AuthContext';
import ComplianceUploadModal from './ComplianceUploadModal';
import ComplianceDetailPanel from './ComplianceDetailPanel';
import ComplianceChartsPanel from './ComplianceChartsPanel';
import MetricInfoPanel from './MetricInfoPanel';
import VCLReportPage from './VCLReportPage';
import LoaderModal from '../LoaderModal';
import metricDefinitionsRaw from '../../data/metricDefinitions.json';
import metricCategoriesConfig from '../../data/complianceCategories.json';
@@ -361,6 +362,10 @@ export default function CompliancePage({ onNavigate }) {
const [rollbackResult, setRollbackResult] = useState(null);
const [infoMetric, setInfoMetric] = useState(null);
const [hoveredMetric, setHoveredMetric] = useState(null);
const [selectedDevices, setSelectedDevices] = useState(new Set());
const [showLoaderModal, setShowLoaderModal] = useState(false);
const [currentPage, setCurrentPage] = useState(1);
const [pageSize, setPageSize] = useState(50);
const hoverTimeoutRef = useRef(null);
const hoveredCardRef = useRef(null);
@@ -392,6 +397,8 @@ export default function CompliancePage({ onNavigate }) {
setFilterState(null);
setHostSearch('');
setSelectedHost(null);
setSelectedDevices(new Set());
setCurrentPage(1);
fetchSummary(activeTeam);
fetchDevices(activeTeam, activeTab);
}, [activeTeam]); // eslint-disable-line react-hooks/exhaustive-deps
@@ -406,14 +413,50 @@ export default function CompliancePage({ onNavigate }) {
useEffect(() => {
setFilterState(null);
setSelectedDevices(new Set());
setCurrentPage(1);
fetchDevices(activeTeam, activeTab);
}, [activeTab]); // eslint-disable-line react-hooks/exhaustive-deps
// Reset page when filter changes
useEffect(() => { setCurrentPage(1); }, [filterState]);
const refresh = () => {
fetchSummary(activeTeam);
fetchDevices(activeTeam, activeTab);
};
const toggleDeviceSelection = (hostname) => {
setSelectedDevices(prev => {
const next = new Set(prev);
if (next.has(hostname)) next.delete(hostname);
else next.add(hostname);
return next;
});
};
const toggleSelectAll = () => {
if (selectedDevices.size === paginatedDevices.length && paginatedDevices.every(d => selectedDevices.has(d.hostname))) {
setSelectedDevices(new Set());
} else {
setSelectedDevices(new Set(paginatedDevices.map(d => d.hostname)));
}
};
const openGraniteLoader = () => {
setShowLoaderModal(true);
};
const getLoaderDevices = () => {
return filteredDevices
.filter(d => selectedDevices.has(d.hostname))
.map(d => ({
ip_address: d.ip_address || '',
hostname: d.hostname || '',
host_id: null,
}));
};
const handleRollback = async () => {
if (!lastUpload) return;
setRollbackLoading(true);
@@ -446,6 +489,11 @@ export default function CompliancePage({ onNavigate }) {
})
.filter(d => !hostSearch || d.hostname.toLowerCase().includes(hostSearch.toLowerCase()));
// Pagination
const totalPages = Math.max(1, Math.ceil(filteredDevices.length / pageSize));
const safePage = Math.min(currentPage, totalPages);
const paginatedDevices = filteredDevices.slice((safePage - 1) * pageSize, safePage * pageSize);
const families = groupByMetricFamily(summary.entries, activeTeam);
const lastUpload = summary.upload;
@@ -724,7 +772,7 @@ export default function CompliancePage({ onNavigate }) {
padding: '0.875rem 1rem', borderBottom: '1px solid rgba(255,255,255,0.05)',
}}>
{/* Active / Resolved tabs */}
<div style={{ display: 'flex', gap: '0.25rem' }}>
<div style={{ display: 'flex', gap: '0.25rem', alignItems: 'center' }}>
{['active', 'resolved'].map(tab => {
const isActive = activeTab === tab;
return (
@@ -747,12 +795,36 @@ export default function CompliancePage({ onNavigate }) {
</button>
);
})}
{selectedDevices.size > 0 && canWrite() && (
<button
onClick={openGraniteLoader}
title="Generate Granite Loader Sheet from selected devices"
style={{
marginLeft: '0.75rem',
padding: '0.35rem 0.75rem',
display: 'flex', alignItems: 'center', gap: '0.35rem',
background: 'rgba(124, 58, 237, 0.12)',
border: '1px solid rgba(124, 58, 237, 0.5)',
borderRadius: '0.25rem',
color: '#A78BFA',
cursor: 'pointer',
fontFamily: 'monospace', fontSize: '0.72rem', fontWeight: '600',
textTransform: 'uppercase', letterSpacing: '0.04em',
transition: 'all 0.15s',
}}
onMouseEnter={e => { e.currentTarget.style.background = 'rgba(124, 58, 237, 0.2)'; e.currentTarget.style.borderColor = 'rgba(124, 58, 237, 0.8)'; }}
onMouseLeave={e => { e.currentTarget.style.background = 'rgba(124, 58, 237, 0.12)'; e.currentTarget.style.borderColor = 'rgba(124, 58, 237, 0.5)'; }}
>
<FileSpreadsheet style={{ width: '13px', height: '13px' }} />
Granite ({selectedDevices.size})
</button>
)}
</div>
{/* Hostname search */}
<input
value={hostSearch}
onChange={e => setHostSearch(e.target.value)}
onChange={e => { setHostSearch(e.target.value); setCurrentPage(1); }}
placeholder="Search hostname…"
style={{
background: 'rgba(15,23,42,0.8)', border: '1px solid rgba(20,184,166,0.2)',
@@ -768,12 +840,22 @@ export default function CompliancePage({ onNavigate }) {
{/* Column headers */}
<div style={{
display: 'grid',
gridTemplateColumns: '2fr 1fr 0.8fr 1.8fr 1fr 1.2fr 0.5fr 0.4fr',
gridTemplateColumns: '0.3fr 2fr 1fr 0.8fr 1.8fr 1fr 1.2fr 0.5fr 0.4fr',
padding: '0.5rem 1rem',
borderBottom: '1px solid rgba(255,255,255,0.05)',
fontSize: '0.62rem', color: '#334155',
fontFamily: 'monospace', textTransform: 'uppercase', letterSpacing: '0.08em',
alignItems: 'center',
}}>
<span style={{ display: 'flex', justifyContent: 'center' }}>
<input
type="checkbox"
checked={paginatedDevices.length > 0 && paginatedDevices.every(d => selectedDevices.has(d.hostname))}
onChange={toggleSelectAll}
style={{ cursor: 'pointer', accentColor: TEAL }}
title="Select all on this page"
/>
</span>
<span>Hostname</span>
<span>IP Address</span>
<span>Type</span>
@@ -798,15 +880,75 @@ export default function CompliancePage({ onNavigate }) {
{lastUpload === null ? 'No reports uploaded yet' : filterState ? 'No devices match the selected filter' : activeTab === 'active' ? 'No non-compliant devices' : 'No resolved items'}
</div>
) : (
filteredDevices.map(device => (
paginatedDevices.map(device => (
<DeviceRow
key={device.hostname}
device={device}
selected={selectedHost === device.hostname}
checked={selectedDevices.has(device.hostname)}
onCheck={() => toggleDeviceSelection(device.hostname)}
onClick={() => setSelectedHost(selectedHost === device.hostname ? null : device.hostname)}
/>
))
)}
{/* Pagination controls */}
{filteredDevices.length > 0 && (
<div style={{
display: 'flex', justifyContent: 'space-between', alignItems: 'center',
padding: '0.75rem 1rem', borderTop: '1px solid rgba(255,255,255,0.05)',
}}>
<div style={{ display: 'flex', alignItems: 'center', gap: '0.5rem' }}>
<span style={{ fontSize: '0.68rem', color: '#475569', fontFamily: 'monospace' }}>
Showing {((safePage - 1) * pageSize) + 1}{Math.min(safePage * pageSize, filteredDevices.length)} of {filteredDevices.length}
</span>
<select
value={pageSize}
onChange={e => { setPageSize(Number(e.target.value)); setCurrentPage(1); }}
style={{
background: 'rgba(15,23,42,0.8)', border: '1px solid rgba(20,184,166,0.2)',
borderRadius: '0.25rem', color: '#94A3B8', fontSize: '0.68rem',
fontFamily: 'monospace', padding: '0.2rem 0.4rem', cursor: 'pointer',
}}
>
<option value={25}>25</option>
<option value={50}>50</option>
<option value={100}>100</option>
<option value={200}>200</option>
</select>
<span style={{ fontSize: '0.68rem', color: '#334155', fontFamily: 'monospace' }}>per page</span>
</div>
<div style={{ display: 'flex', alignItems: 'center', gap: '0.25rem' }}>
<button
onClick={() => setCurrentPage(p => Math.max(1, p - 1))}
disabled={safePage <= 1}
style={{
padding: '0.3rem 0.6rem', borderRadius: '0.25rem',
border: '1px solid rgba(20,184,166,0.2)', background: 'transparent',
color: safePage <= 1 ? '#1E293B' : '#94A3B8', cursor: safePage <= 1 ? 'default' : 'pointer',
fontFamily: 'monospace', fontSize: '0.72rem',
}}
>
</button>
<span style={{ fontSize: '0.68rem', color: '#64748B', fontFamily: 'monospace', padding: '0 0.5rem' }}>
{safePage} / {totalPages}
</span>
<button
onClick={() => setCurrentPage(p => Math.min(totalPages, p + 1))}
disabled={safePage >= totalPages}
style={{
padding: '0.3rem 0.6rem', borderRadius: '0.25rem',
border: '1px solid rgba(20,184,166,0.2)', background: 'transparent',
color: safePage >= totalPages ? '#1E293B' : '#94A3B8', cursor: safePage >= totalPages ? 'default' : 'pointer',
fontFamily: 'monospace', fontSize: '0.72rem',
}}
>
</button>
</div>
</div>
)}
</div>}
{/* ── Detail panel ─────────────────────────────────────────── */}
@@ -949,11 +1091,18 @@ export default function CompliancePage({ onNavigate }) {
)}
</div>
)}
{/* ── Granite Loader Modal ─────────────────────────────────── */}
<LoaderModal
isOpen={showLoaderModal}
onClose={() => setShowLoaderModal(false)}
initialDevices={showLoaderModal ? getLoaderDevices() : null}
/>
</div>
);
}
function DeviceRow({ device, selected, onClick }) {
function DeviceRow({ device, selected, checked, onCheck, onClick }) {
const truncateText = (text, maxLen = 80) => {
if (!text) return '—';
return text.length > maxLen ? text.slice(0, maxLen) + '…' : text;
@@ -964,7 +1113,7 @@ function DeviceRow({ device, selected, onClick }) {
onClick={onClick}
style={{
display: 'grid',
gridTemplateColumns: '2fr 1fr 0.8fr 1.8fr 1fr 1.2fr 0.5fr 0.4fr',
gridTemplateColumns: '0.3fr 2fr 1fr 0.8fr 1.8fr 1fr 1.2fr 0.5fr 0.4fr',
padding: '0.625rem 1rem',
borderBottom: '1px solid rgba(255,255,255,0.04)',
cursor: 'pointer',
@@ -976,6 +1125,15 @@ function DeviceRow({ device, selected, onClick }) {
onMouseEnter={e => { if (!selected) e.currentTarget.style.background = 'rgba(255,255,255,0.025)'; }}
onMouseLeave={e => { if (!selected) e.currentTarget.style.background = 'transparent'; }}
>
{/* Checkbox */}
<div style={{ display: 'flex', justifyContent: 'center' }} onClick={e => e.stopPropagation()}>
<input
type="checkbox"
checked={checked}
onChange={onCheck}
style={{ cursor: 'pointer', accentColor: TEAL }}
/>
</div>
{/* Hostname */}
<div style={{ fontFamily: 'monospace', fontSize: '0.78rem', color: selected ? TEAL : '#E2E8F0', overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }}>
{device.hostname}

View File

@@ -5378,6 +5378,7 @@ function RowVisibilityManager({ hiddenRowIds, findings, onRestore, onRestoreAll
function BulkHideToolbar({ count, onHide, onClear, onAtlasBulk, canWrite }) {
return (
<div style={{
position: 'sticky', top: 0, zIndex: 20,
display: 'flex', alignItems: 'center', gap: '0.75rem',
padding: '0.5rem 1rem',
background: 'linear-gradient(135deg, rgba(15,23,42,0.95), rgba(30,41,59,0.95))',

View File

@@ -25,5 +25,14 @@
"devDependencies": {
"fast-check": "^4.8.0",
"jest": "^30.3.0"
},
"jest": {
"roots": [
"<rootDir>/backend/__tests__"
],
"testPathIgnorePatterns": [
"/node_modules/",
"integration"
]
}
}

BIN
shieldlogo.jpeg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB