Add NVD sync implementation plan

Detailed plan for the NVD lookup + retroactive sync feature covering stash resolution, backend endpoints, frontend NvdSyncModal component, and App.js integration. Note: claude_status.md is gitignored but has been updated locally with full session context including stash state, conflict resolution steps, and task list. Copy it manually to the offsite machine if needed. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 15:11:30 -07:00
parent 41c8a1ef27
commit fbdf05392a
1 changed files with 297 additions and 0 deletions
--- a/plan.md
+++ b/plan.md
@@ -0,0 +1,297 @@
+# NVD Lookup + Retroactive Sync — Implementation Plan
+
+## Overview
+Two capabilities on `feature/nvd-lookup` branch:
+1. **Auto-fill on Add CVE** (DONE, stashed) — onBlur NVD lookup fills description/severity/date in the Add CVE modal
+2. **Sync with NVD** (TO DO) — bulk tool for editors/admins to retroactively update existing CVE entries from NVD, with per-CVE choice to keep or replace description
+
+## Current State
+
+### Git State
+- **Branch:** `feature/nvd-lookup` (branched from master post-audit-merge)
+- **Stash:** `stash@{0}` contains the auto-fill implementation (4 files)
+- **Master** now has audit logging (merged from feature/audit on 2026-01-30)
+- Offsite repo is up to date through the feature/audit merge to master
+
+### What's in the Stash
+The stash contains working NVD auto-fill code that needs to be popped and conflict-resolved before continuing:
+
+**`backend/routes/nvdLookup.js` (NEW file)**
+- Factory function: `createNvdLookupRouter(db, requireAuth)`
+- `GET /lookup/:cveId` endpoint
+- Validates CVE ID format (regex: `CVE-YYYY-NNNNN`)
+- Calls `https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=...`
+- 10-second timeout via `AbortSignal.timeout(10000)`
+- Optional `apiKey` header from `NVD_API_KEY` env var
+- CVSS severity cascade: v3.1 → v3.0 → v2.0
+- Maps NVD uppercase severity to app format (CRITICAL→Critical, etc.)
+- Returns: `{ description, severity, published_date }`
+
+**`backend/server.js` (MODIFIED)**
+- Adds `const createNvdLookupRouter = require('./routes/nvdLookup');`
+- Adds `app.use('/api/nvd', createNvdLookupRouter(db, requireAuth));`
+
+**`frontend/src/App.js` (MODIFIED)**
+- New state: `nvdLoading`, `nvdError`, `nvdAutoFilled`
+- New function: `lookupNVD(cveId)` — calls backend, auto-fills form fields
+- CVE ID input: `onBlur` triggers lookup, `onChange` resets NVD feedback
+- Spinner (Loader icon) in CVE ID field while loading
+- Green "Auto-filled from NVD" with CheckCircle on success
+- Amber warning with AlertCircle on errors (non-blocking)
+- Description only fills if currently empty; severity + published_date always update
+- NVD state resets on modal close (X, Cancel) and form submit
+
+**`backend/.env.example` (MODIFIED)**
+- Adds `NVD_API_KEY=` with comment about rate limits
+
+### Stash Conflict Resolution
+Popping the stash will conflict in `server.js` because master now has audit imports that didn't exist when the stash was created. Resolution:
+
+The conflict is in the imports section. Keep ALL existing audit lines from master:
+```js
+const createAuditLogRouter = require('./routes/auditLog');
+const logAudit = require('./helpers/auditLog');
+```
+AND add the NVD line:
+```js
+const createNvdLookupRouter = require('./routes/nvdLookup');
+```
+
+Similarly, keep the audit route mount and add the NVD mount after it:
+```js
+app.use('/api/audit-logs', createAuditLogRouter(db, requireAuth, requireRole));
+app.use('/api/nvd', createNvdLookupRouter(db, requireAuth));
+```
+
+Then `git add backend/server.js` to mark resolved and `git stash drop`.
+
+---
+
+## Step 1: Resolve Stash + Rebase onto Master
+
+```bash
+git checkout feature/nvd-lookup
+git rebase master                    # Get audit changes into the branch
+git stash pop                        # Apply NVD changes (will conflict in server.js)
+# Resolve conflict in server.js as described above
+git add backend/server.js
+git stash drop
+```
+
+Verify: `backend/routes/nvdLookup.js` exists, `server.js` has both audit AND NVD imports/mounts.
+
+---
+
+## Step 2: Backend — New Endpoints in `server.js`
+
+### 2A: `GET /api/cves/distinct-ids`
+Place BEFORE `GET /api/cves/check/:cveId` (to avoid route param conflict):
+```js
+app.get('/api/cves/distinct-ids', requireAuth(db), (req, res) => {
+    db.all('SELECT DISTINCT cve_id FROM cves ORDER BY cve_id', [], (err, rows) => {
+        if (err) return res.status(500).json({ error: err.message });
+        res.json(rows.map(r => r.cve_id));
+    });
+});
+```
+
+### 2B: `POST /api/cves/nvd-sync`
+Place after the existing `PATCH /api/cves/:cveId/status`:
+```js
+app.post('/api/cves/nvd-sync', requireAuth(db), requireRole('editor', 'admin'), (req, res) => {
+    const { updates } = req.body;
+    if (!Array.isArray(updates) || updates.length === 0) {
+        return res.status(400).json({ error: 'No updates provided' });
+    }
+
+    let updated = 0;
+    const errors = [];
+    let completed = 0;
+
+    db.serialize(() => {
+        updates.forEach((entry) => {
+            const fields = [];
+            const values = [];
+            if (entry.description !== null && entry.description !== undefined) {
+                fields.push('description = ?');
+                values.push(entry.description);
+            }
+            if (entry.severity !== null && entry.severity !== undefined) {
+                fields.push('severity = ?');
+                values.push(entry.severity);
+            }
+            if (entry.published_date !== null && entry.published_date !== undefined) {
+                fields.push('published_date = ?');
+                values.push(entry.published_date);
+            }
+            if (fields.length === 0) {
+                completed++;
+                if (completed === updates.length) sendResponse();
+                return;
+            }
+            fields.push('updated_at = CURRENT_TIMESTAMP');
+            values.push(entry.cve_id);
+
+            db.run(
+                `UPDATE cves SET ${fields.join(', ')} WHERE cve_id = ?`,
+                values,
+                function(err) {
+                    if (err) {
+                        errors.push({ cve_id: entry.cve_id, error: err.message });
+                    } else {
+                        updated += this.changes;
+                    }
+                    completed++;
+                    if (completed === updates.length) sendResponse();
+                }
+            );
+        });
+    });
+
+    function sendResponse() {
+        logAudit(db, {
+            userId: req.user.id,
+            username: req.user.username,
+            action: 'cve_nvd_sync',
+            entityType: 'cve',
+            entityId: null,
+            details: { count: updated, cve_ids: updates.map(u => u.cve_id) },
+            ipAddress: req.ip
+        });
+        const result = { message: 'NVD sync completed', updated };
+        if (errors.length > 0) result.errors = errors;
+        res.json(result);
+    }
+});
+```
+
+**How "keep existing description" works:** If the user chooses to keep the existing description, the frontend sends `description: null` for that CVE. The backend skips null fields, so the description is not overwritten. Severity and published_date are always sent (auto-update).
+
+---
+
+## Step 3: Frontend — New `NvdSyncModal.js` Component
+
+**File:** `frontend/src/components/NvdSyncModal.js`
+
+### Props
+```jsx
+<NvdSyncModal onClose={fn} onSyncComplete={fn} />
+```
+
+### Phase Machine
+| Phase | What's shown |
+|-------|-------------|
+| `idle` | CVE count + "Fetch NVD Data" button |
+| `fetching` | Progress bar, current CVE being fetched, cancel button |
+| `review` | Comparison table with per-CVE description choice |
+| `applying` | Spinner |
+| `done` | Summary (X updated, Y errors) + Close button |
+
+### Fetching Logic
+- Iterate CVE IDs sequentially
+- Call `GET /api/nvd/lookup/:cveId` for each
+- 7-second delay between requests (safe for 5 req/30s without API key)
+- On 429: wait 35 seconds, retry once
+- On 404: mark as "Not found in NVD" (gray, skipped)
+- On timeout/error: mark with warning (skipped)
+- Support cancellation via AbortController
+
+### Comparison Table Columns
+| Column | Content |
+|--------|---------|
+| CVE ID | The identifier |
+| Status | Icon: check=found, warning=error, dash=no changes |
+| Severity | `[Current] → [NVD]` with color badges, or "No change" |
+| Published Date | `Current → NVD` or "No change" |
+| Description | Truncated preview with expand toggle. Current (red bg) vs NVD (green bg) when different |
+| Choice | Radio: "Keep existing" (default) / "Use NVD" — only shown when descriptions differ |
+
+### Bulk Controls
+Above the table:
+- Summary: `Found: N | Up to date: N | Changes: N | Not in NVD: N | Errors: N`
+- Bulk toggle: "Keep All Existing" / "Use All NVD Descriptions"
+
+Below the table:
+- "Apply N Changes" button (count updates dynamically)
+- "Cancel" button
+
+### Apply Logic
+Build updates array:
+- For each CVE with NVD data (no error):
+  - Always include `severity` and `published_date` if different from current
+  - Include `description` only if user chose "Use NVD" — otherwise send `null`
+  - Skip CVEs where nothing changed
+- POST to `/api/cves/nvd-sync`
+- On success: call `onSyncComplete()` to refresh CVE list, then show done phase
+
+---
+
+## Step 4: Frontend — App.js Integration
+
+Minimal changes following `AuditLog`/`UserManagement` pattern:
+
+1. **Import:** Add `NvdSyncModal` and `RefreshCw` icon
+2. **State:** Add `const [showNvdSync, setShowNvdSync] = useState(false);`
+3. **Header button** (next to "Add CVE/Vendor", visible to editors/admins):
+```jsx
+{canWrite() && (
+  <button onClick={() => setShowNvdSync(true)}
+    className="px-4 py-2 bg-green-600 text-white rounded-lg hover:bg-green-700 transition-colors flex items-center gap-2 shadow-md">
+    <RefreshCw className="w-5 h-5" />
+    Sync with NVD
+  </button>
+)}
+```
+4. **Modal render** (alongside other modals):
+```jsx
+{showNvdSync && (
+  <NvdSyncModal onClose={() => setShowNvdSync(false)} onSyncComplete={() => fetchCVEs()} />
+)}
+```
+
+---
+
+## Step 5: AuditLog Badge
+
+**File:** `frontend/src/components/AuditLog.js`
+
+Add to the `ACTION_BADGES` object:
+```js
+cve_nvd_sync: { bg: 'bg-green-100', text: 'text-green-800' },
+```
+
+---
+
+## Step 6: .env.example (already in stash)
+```
+# NVD API Key (optional - increases rate limit from 5 to 50 requests per 30s)
+# Request one at https://nvd.nist.gov/developers/request-an-api-key
+NVD_API_KEY=
+```
+
+---
+
+## File Summary
+
+| File | Action | Lines Changed (est.) |
+|------|--------|---------------------|
+| `backend/server.js` | Modify | +40 (NVD mount + 2 new endpoints) |
+| `backend/routes/nvdLookup.js` | From stash | 0 (already complete) |
+| `backend/.env.example` | From stash | +3 |
+| `frontend/src/components/NvdSyncModal.js` | New | ~350-400 |
+| `frontend/src/App.js` | Modify | +10 (import, state, button, modal) |
+| `frontend/src/components/AuditLog.js` | Modify | +1 (badge entry) |
+
+---
+
+## Verification Checklist
+1. Pop stash, resolve conflict, verify `nvdLookup.js` and server.js are correct
+2. Test NVD lookup via curl: `curl -b cookie.txt http://localhost:3001/api/nvd/lookup/CVE-2024-3094`
+3. Test distinct-ids: `curl -b cookie.txt http://localhost:3001/api/cves/distinct-ids`
+4. Open Add CVE modal, type CVE ID, tab out → verify auto-fill works
+5. Click "Sync with NVD" button → modal opens with CVE count
+6. Click "Fetch NVD Data" → progress bar, rate-limited fetching
+7. Review comparison table → verify diffs shown correctly
+8. Toggle description choices, click "Apply" → verify database updated
+9. Confirm main CVE list refreshes with new data
+10. Check audit log for `cve_nvd_sync` entry