Compare commits
1 Commits
feature/iv
...
feature/we
| Author | SHA1 | Date | |
|---|---|---|---|
| 0d67a99c7e |
211
WEEKLY_REPORT_FEATURE.md
Normal file
211
WEEKLY_REPORT_FEATURE.md
Normal file
@@ -0,0 +1,211 @@
|
||||
# Weekly Vulnerability Report Upload Feature
|
||||
|
||||
## Overview
|
||||
|
||||
A new feature has been added to the CVE Dashboard that allows users to upload their weekly vulnerability reports in Excel format (.xlsx) and automatically process them to split multiple CVE IDs into separate rows for easier filtering and analysis.
|
||||
|
||||
## What Was Implemented
|
||||
|
||||
### Backend Changes
|
||||
|
||||
1. **Database Migration** (`backend/migrations/add_weekly_reports_table.js`)
|
||||
- Created `weekly_reports` table to store report metadata
|
||||
- Tracks upload date, file paths, row counts, and which report is current
|
||||
- Indexed for fast queries
|
||||
|
||||
2. **Excel Processor** (`backend/helpers/excelProcessor.js`)
|
||||
- Executes Python script via Node.js child_process
|
||||
- Parses row counts from Python output
|
||||
- Handles errors, timeouts (30 seconds), and validation
|
||||
|
||||
3. **API Routes** (`backend/routes/weeklyReports.js`)
|
||||
- `POST /api/weekly-reports/upload` - Upload and process Excel file
|
||||
- `GET /api/weekly-reports` - List all reports
|
||||
- `GET /api/weekly-reports/:id/download/:type` - Download original or processed file
|
||||
- `DELETE /api/weekly-reports/:id` - Delete report (admin only)
|
||||
|
||||
4. **Python Script** (`backend/scripts/split_cve_report.py`)
|
||||
- Moved from ~/Documents to backend/scripts
|
||||
- Splits comma-separated CVE IDs into separate rows
|
||||
- Duplicates device/IP data for each CVE
|
||||
|
||||
### Frontend Changes
|
||||
|
||||
1. **Weekly Report Modal** (`frontend/src/components/WeeklyReportModal.js`)
|
||||
- Phase-based UI: idle → uploading → processing → success
|
||||
- File upload with .xlsx validation
|
||||
- Display existing reports with current report indicator (★)
|
||||
- Download buttons for both original and processed files
|
||||
|
||||
2. **App.js Integration**
|
||||
- Added "Weekly Report" button next to NVD Sync button
|
||||
- State management for modal visibility
|
||||
- Modal rendering
|
||||
|
||||
## How to Use
|
||||
|
||||
### Starting the Application
|
||||
|
||||
1. **Backend:**
|
||||
```bash
|
||||
cd /home/admin/cve-dashboard/backend
|
||||
node server.js
|
||||
```
|
||||
|
||||
2. **Frontend:**
|
||||
```bash
|
||||
cd /home/admin/cve-dashboard/frontend
|
||||
npm start
|
||||
```
|
||||
|
||||
### Using the Feature
|
||||
|
||||
1. **Access the Feature**
|
||||
- Login as an editor or admin user
|
||||
- Look for the "Weekly Report" button in the top header (next to "NVD Sync")
|
||||
|
||||
2. **Upload a Report**
|
||||
- Click the "Weekly Report" button
|
||||
- Click "Choose File" and select your .xlsx file
|
||||
- Click "Upload & Process"
|
||||
- Wait for processing to complete (usually 5-10 seconds)
|
||||
|
||||
3. **Download Processed Report**
|
||||
- After upload succeeds, you'll see row counts (e.g., "45 → 67 rows")
|
||||
- Click "Download Processed" to get the split version
|
||||
- The current week's report is marked with a ★ star icon
|
||||
|
||||
4. **Access Previous Reports**
|
||||
- All previous reports are listed below the upload section
|
||||
- Click the download icons to get original or processed versions
|
||||
- Reports are labeled as "This week's report", "Last week's report", or by date
|
||||
|
||||
### What the Processing Does
|
||||
|
||||
**Before Processing:**
|
||||
| HOSTNAME | IP | CVE ID |
|
||||
|----------|------------|---------------------------|
|
||||
| server01 | 10.0.0.1 | CVE-2024-1234, CVE-2024-5678 |
|
||||
|
||||
**After Processing:**
|
||||
| HOSTNAME | IP | CVE ID |
|
||||
|----------|------------|---------------------------|
|
||||
| server01 | 10.0.0.1 | CVE-2024-1234 |
|
||||
| server01 | 10.0.0.1 | CVE-2024-5678 |
|
||||
|
||||
Each CVE now has its own row, making it easy to:
|
||||
- Sort by CVE ID
|
||||
- Filter for specific CVEs
|
||||
- Research CVEs one by one per device
|
||||
|
||||
## File Locations
|
||||
|
||||
### New Files Created
|
||||
|
||||
```
|
||||
backend/
|
||||
scripts/
|
||||
split_cve_report.py # Python script for CVE splitting
|
||||
requirements.txt # Python dependencies
|
||||
routes/
|
||||
weeklyReports.js # API endpoints
|
||||
helpers/
|
||||
excelProcessor.js # Python integration
|
||||
migrations/
|
||||
add_weekly_reports_table.js # Database migration
|
||||
uploads/
|
||||
weekly_reports/ # Uploaded and processed files
|
||||
|
||||
frontend/
|
||||
src/
|
||||
components/
|
||||
WeeklyReportModal.js # Upload modal UI
|
||||
```
|
||||
|
||||
### Modified Files
|
||||
|
||||
```
|
||||
backend/
|
||||
server.js # Added route mounting
|
||||
|
||||
frontend/
|
||||
src/
|
||||
App.js # Added button and modal
|
||||
```
|
||||
|
||||
## Security & Permissions
|
||||
|
||||
- **Upload**: Requires editor or admin role
|
||||
- **Download**: Any authenticated user
|
||||
- **Delete**: Admin only
|
||||
- **File Validation**: Only .xlsx files accepted, 10MB limit
|
||||
- **Audit Logging**: All uploads, downloads, and deletions are logged
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Backend Issues
|
||||
|
||||
**Python not found:**
|
||||
```bash
|
||||
# Install Python 3
|
||||
sudo apt-get install python3
|
||||
```
|
||||
|
||||
**Missing dependencies:**
|
||||
```bash
|
||||
# Install pandas and openpyxl
|
||||
pip3 install pandas openpyxl
|
||||
```
|
||||
|
||||
**Port already in use:**
|
||||
```bash
|
||||
# Find and kill process using port 3001
|
||||
lsof -i :3001
|
||||
kill -9 <PID>
|
||||
```
|
||||
|
||||
### Frontend Issues
|
||||
|
||||
**Button not visible:**
|
||||
- Make sure you're logged in as editor or admin
|
||||
- Viewer role cannot upload reports
|
||||
|
||||
**Upload fails:**
|
||||
- Check file is .xlsx format (not .xls or .csv)
|
||||
- Ensure file has "Vulnerabilities" sheet with "CVE ID" column
|
||||
- Check file size is under 10MB
|
||||
|
||||
**Processing timeout:**
|
||||
- Large files (10,000+ rows) may timeout
|
||||
- Try reducing file size or increase timeout in `excelProcessor.js`
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [x] Backend starts without errors
|
||||
- [x] Frontend compiles successfully
|
||||
- [x] Database migration completed
|
||||
- [x] Python dependencies installed
|
||||
- [ ] Upload .xlsx file (manual test in browser)
|
||||
- [ ] Verify processed file has split CVEs (manual test)
|
||||
- [ ] Download original and processed files (manual test)
|
||||
- [ ] Verify current report marked with star (manual test)
|
||||
- [ ] Test as viewer - button should be hidden (manual test)
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
Possible improvements:
|
||||
- Progress bar during Python processing
|
||||
- Email notifications when processing completes
|
||||
- Scheduled automatic uploads
|
||||
- Report comparison (diff between weeks)
|
||||
- Export to other formats (CSV, JSON)
|
||||
- Bulk delete old reports
|
||||
- Report validation before upload
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Check the troubleshooting section above
|
||||
2. Review audit logs for error details
|
||||
3. Check browser console for frontend errors
|
||||
4. Review backend server logs for API errors
|
||||
93
backend/helpers/excelProcessor.js
Normal file
93
backend/helpers/excelProcessor.js
Normal file
@@ -0,0 +1,93 @@
|
||||
const { spawn } = require('child_process');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
|
||||
/**
|
||||
* Process vulnerability report Excel file by splitting CVE IDs into separate rows
|
||||
* @param {string} inputPath - Path to original Excel file
|
||||
* @param {string} outputPath - Path for processed Excel file
|
||||
* @returns {Promise<{original_rows: number, processed_rows: number, output_path: string}>}
|
||||
*/
|
||||
function processVulnerabilityReport(inputPath, outputPath) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const scriptPath = path.join(__dirname, '..', 'scripts', 'split_cve_report.py');
|
||||
|
||||
// Verify script exists
|
||||
if (!fs.existsSync(scriptPath)) {
|
||||
return reject(new Error(`Python script not found: ${scriptPath}`));
|
||||
}
|
||||
|
||||
// Verify input file exists
|
||||
if (!fs.existsSync(inputPath)) {
|
||||
return reject(new Error(`Input file not found: ${inputPath}`));
|
||||
}
|
||||
|
||||
const python = spawn('python3', [scriptPath, inputPath, outputPath]);
|
||||
|
||||
let stdout = '';
|
||||
let stderr = '';
|
||||
let timedOut = false;
|
||||
|
||||
// 30 second timeout
|
||||
const timeout = setTimeout(() => {
|
||||
timedOut = true;
|
||||
python.kill();
|
||||
reject(new Error('Processing timed out. File may be too large or corrupted.'));
|
||||
}, 30000);
|
||||
|
||||
python.stdout.on('data', (data) => {
|
||||
stdout += data.toString();
|
||||
});
|
||||
|
||||
python.stderr.on('data', (data) => {
|
||||
stderr += data.toString();
|
||||
});
|
||||
|
||||
python.on('close', (code) => {
|
||||
clearTimeout(timeout);
|
||||
|
||||
if (timedOut) return;
|
||||
|
||||
if (code !== 0) {
|
||||
// Parse Python error messages
|
||||
if (stderr.includes('Sheet') && stderr.includes('not found')) {
|
||||
return reject(new Error('Invalid Excel file. Expected "Vulnerabilities" sheet with "CVE ID" column.'));
|
||||
}
|
||||
if (stderr.includes('pandas') || stderr.includes('openpyxl')) {
|
||||
return reject(new Error('Python dependencies missing. Run: pip3 install pandas openpyxl'));
|
||||
}
|
||||
return reject(new Error(`Python script failed: ${stderr || 'Unknown error'}`));
|
||||
}
|
||||
|
||||
// Parse output for row counts
|
||||
const originalMatch = stdout.match(/Original rows:\s*(\d+)/);
|
||||
const newMatch = stdout.match(/New rows:\s*(\d+)/);
|
||||
|
||||
if (!originalMatch || !newMatch) {
|
||||
return reject(new Error('Failed to parse row counts from Python output'));
|
||||
}
|
||||
|
||||
// Verify output file was created
|
||||
if (!fs.existsSync(outputPath)) {
|
||||
return reject(new Error('Processed file was not created'));
|
||||
}
|
||||
|
||||
resolve({
|
||||
original_rows: parseInt(originalMatch[1]),
|
||||
processed_rows: parseInt(newMatch[1]),
|
||||
output_path: outputPath
|
||||
});
|
||||
});
|
||||
|
||||
python.on('error', (err) => {
|
||||
clearTimeout(timeout);
|
||||
if (err.code === 'ENOENT') {
|
||||
reject(new Error('Python 3 is required but not found. Please install Python.'));
|
||||
} else {
|
||||
reject(err);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { processVulnerabilityReport };
|
||||
59
backend/migrations/add_weekly_reports_table.js
Normal file
59
backend/migrations/add_weekly_reports_table.js
Normal file
@@ -0,0 +1,59 @@
|
||||
// Migration: Add weekly_reports table for vulnerability report uploads
|
||||
|
||||
const sqlite3 = require('sqlite3').verbose();
|
||||
const path = require('path');
|
||||
|
||||
const dbPath = path.join(__dirname, '..', 'cve_database.db');
|
||||
const db = new sqlite3.Database(dbPath);
|
||||
|
||||
console.log('Running migration: add_weekly_reports_table');
|
||||
|
||||
db.serialize(() => {
|
||||
db.run(`
|
||||
CREATE TABLE IF NOT EXISTS weekly_reports (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
upload_date DATE NOT NULL,
|
||||
week_label VARCHAR(50),
|
||||
original_filename VARCHAR(255),
|
||||
processed_filename VARCHAR(255),
|
||||
original_file_path VARCHAR(500),
|
||||
processed_file_path VARCHAR(500),
|
||||
row_count_original INTEGER,
|
||||
row_count_processed INTEGER,
|
||||
uploaded_by INTEGER,
|
||||
uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
is_current BOOLEAN DEFAULT 0,
|
||||
FOREIGN KEY (uploaded_by) REFERENCES users(id)
|
||||
)
|
||||
`, (err) => {
|
||||
if (err) {
|
||||
console.error('Error creating weekly_reports table:', err);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('✓ Created weekly_reports table');
|
||||
});
|
||||
|
||||
db.run(`
|
||||
CREATE INDEX IF NOT EXISTS idx_weekly_reports_date
|
||||
ON weekly_reports(upload_date DESC)
|
||||
`, (err) => {
|
||||
if (err) {
|
||||
console.error('Error creating date index:', err);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('✓ Created index on upload_date');
|
||||
});
|
||||
|
||||
db.run(`
|
||||
CREATE INDEX IF NOT EXISTS idx_weekly_reports_current
|
||||
ON weekly_reports(is_current)
|
||||
`, (err) => {
|
||||
if (err) {
|
||||
console.error('Error creating current index:', err);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('✓ Created index on is_current');
|
||||
console.log('\nMigration completed successfully!');
|
||||
db.close();
|
||||
});
|
||||
});
|
||||
261
backend/routes/weeklyReports.js
Normal file
261
backend/routes/weeklyReports.js
Normal file
@@ -0,0 +1,261 @@
|
||||
const express = require('express');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const { requireAuth, requireRole } = require('../middleware/auth');
|
||||
const logAudit = require('../helpers/auditLog');
|
||||
const { processVulnerabilityReport } = require('../helpers/excelProcessor');
|
||||
|
||||
function createWeeklyReportsRouter(db, upload) {
|
||||
const router = express.Router();
|
||||
|
||||
// Helper to sanitize filename
|
||||
function sanitizePathSegment(segment) {
|
||||
if (!segment || typeof segment !== 'string') return '';
|
||||
return segment
|
||||
.replace(/\0/g, '')
|
||||
.replace(/\.\./g, '')
|
||||
.replace(/[\/\\]/g, '')
|
||||
.trim();
|
||||
}
|
||||
|
||||
// Helper to generate week label
|
||||
function getWeekLabel(date) {
|
||||
const now = new Date();
|
||||
const uploadDate = new Date(date);
|
||||
const daysDiff = Math.floor((now - uploadDate) / (1000 * 60 * 60 * 24));
|
||||
|
||||
if (daysDiff < 7) {
|
||||
return "This week's report";
|
||||
} else if (daysDiff < 14) {
|
||||
return "Last week's report";
|
||||
} else {
|
||||
const month = uploadDate.getMonth() + 1;
|
||||
const day = uploadDate.getDate();
|
||||
const year = uploadDate.getFullYear();
|
||||
return `Week of ${month.toString().padStart(2, '0')}/${day.toString().padStart(2, '0')}/${year}`;
|
||||
}
|
||||
}
|
||||
|
||||
// POST /api/weekly-reports/upload - Upload and process vulnerability report
|
||||
router.post('/upload', requireAuth(db), requireRole(db, 'editor', 'admin'), upload.single('file'), async (req, res) => {
|
||||
const uploadedFile = req.file;
|
||||
|
||||
if (!uploadedFile) {
|
||||
return res.status(400).json({ error: 'No file uploaded' });
|
||||
}
|
||||
|
||||
// Validate file extension
|
||||
const ext = path.extname(uploadedFile.originalname).toLowerCase();
|
||||
if (ext !== '.xlsx') {
|
||||
fs.unlinkSync(uploadedFile.path); // Clean up temp file
|
||||
return res.status(400).json({ error: 'Only .xlsx files are allowed' });
|
||||
}
|
||||
|
||||
const timestamp = Date.now();
|
||||
const sanitizedName = sanitizePathSegment(uploadedFile.originalname);
|
||||
const reportsDir = path.join(__dirname, '..', 'uploads', 'weekly_reports');
|
||||
|
||||
// Create directory if it doesn't exist
|
||||
if (!fs.existsSync(reportsDir)) {
|
||||
fs.mkdirSync(reportsDir, { recursive: true });
|
||||
}
|
||||
|
||||
const originalFilename = `${timestamp}_original_${sanitizedName}`;
|
||||
const processedFilename = `${timestamp}_processed_${sanitizedName}`;
|
||||
const originalPath = path.join(reportsDir, originalFilename);
|
||||
const processedPath = path.join(reportsDir, processedFilename);
|
||||
|
||||
try {
|
||||
// Move uploaded file to permanent location
|
||||
fs.renameSync(uploadedFile.path, originalPath);
|
||||
|
||||
// Process the file with Python script
|
||||
const result = await processVulnerabilityReport(originalPath, processedPath);
|
||||
|
||||
const uploadDate = new Date().toISOString().split('T')[0];
|
||||
|
||||
// Update previous current reports to not current
|
||||
db.run('UPDATE weekly_reports SET is_current = 0 WHERE is_current = 1', (err) => {
|
||||
if (err) {
|
||||
console.error('Error updating previous current reports:', err);
|
||||
}
|
||||
});
|
||||
|
||||
// Insert new report record
|
||||
const insertSql = `
|
||||
INSERT INTO weekly_reports (
|
||||
upload_date, week_label, original_filename, processed_filename,
|
||||
original_file_path, processed_file_path, row_count_original,
|
||||
row_count_processed, uploaded_by, is_current
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1)
|
||||
`;
|
||||
|
||||
const weekLabel = getWeekLabel(uploadDate);
|
||||
|
||||
db.run(
|
||||
insertSql,
|
||||
[
|
||||
uploadDate,
|
||||
weekLabel,
|
||||
sanitizedName,
|
||||
processedFilename,
|
||||
originalPath,
|
||||
processedPath,
|
||||
result.original_rows,
|
||||
result.processed_rows,
|
||||
req.user.id
|
||||
],
|
||||
function (err) {
|
||||
if (err) {
|
||||
console.error('Error inserting weekly report:', err);
|
||||
return res.status(500).json({ error: 'Failed to save report metadata' });
|
||||
}
|
||||
|
||||
// Log audit entry
|
||||
logAudit(
|
||||
db,
|
||||
req.user.id,
|
||||
req.user.username,
|
||||
'UPLOAD_WEEKLY_REPORT',
|
||||
'weekly_reports',
|
||||
this.lastID,
|
||||
JSON.stringify({ filename: sanitizedName, rows: result.processed_rows }),
|
||||
req.ip
|
||||
);
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
id: this.lastID,
|
||||
original_rows: result.original_rows,
|
||||
processed_rows: result.processed_rows,
|
||||
week_label: weekLabel
|
||||
});
|
||||
}
|
||||
);
|
||||
} catch (error) {
|
||||
// Clean up files on error
|
||||
if (fs.existsSync(originalPath)) fs.unlinkSync(originalPath);
|
||||
if (fs.existsSync(processedPath)) fs.unlinkSync(processedPath);
|
||||
|
||||
console.error('Error processing vulnerability report:', error);
|
||||
res.status(500).json({ error: error.message || 'Failed to process report' });
|
||||
}
|
||||
});
|
||||
|
||||
// GET /api/weekly-reports - List all reports
|
||||
router.get('/', requireAuth(db), (req, res) => {
|
||||
const sql = `
|
||||
SELECT id, upload_date, week_label, original_filename, processed_filename,
|
||||
row_count_original, row_count_processed, is_current, uploaded_at
|
||||
FROM weekly_reports
|
||||
ORDER BY upload_date DESC, uploaded_at DESC
|
||||
`;
|
||||
|
||||
db.all(sql, [], (err, rows) => {
|
||||
if (err) {
|
||||
console.error('Error fetching weekly reports:', err);
|
||||
return res.status(500).json({ error: 'Failed to fetch reports' });
|
||||
}
|
||||
|
||||
res.json(rows);
|
||||
});
|
||||
});
|
||||
|
||||
// GET /api/weekly-reports/:id/download/:type - Download report file
|
||||
router.get('/:id/download/:type', requireAuth(db), (req, res) => {
|
||||
const { id, type } = req.params;
|
||||
|
||||
if (type !== 'original' && type !== 'processed') {
|
||||
return res.status(400).json({ error: 'Invalid download type. Use "original" or "processed"' });
|
||||
}
|
||||
|
||||
const sql = `SELECT original_file_path, processed_file_path, original_filename FROM weekly_reports WHERE id = ?`;
|
||||
|
||||
db.get(sql, [id], (err, row) => {
|
||||
if (err) {
|
||||
console.error('Error fetching report:', err);
|
||||
return res.status(500).json({ error: 'Failed to fetch report' });
|
||||
}
|
||||
|
||||
if (!row) {
|
||||
return res.status(404).json({ error: 'Report not found' });
|
||||
}
|
||||
|
||||
const filePath = type === 'original' ? row.original_file_path : row.processed_file_path;
|
||||
|
||||
if (!fs.existsSync(filePath)) {
|
||||
return res.status(404).json({ error: 'File not found on disk' });
|
||||
}
|
||||
|
||||
// Log audit entry
|
||||
logAudit(
|
||||
db,
|
||||
req.user.id,
|
||||
req.user.username,
|
||||
'DOWNLOAD_WEEKLY_REPORT',
|
||||
'weekly_reports',
|
||||
id,
|
||||
JSON.stringify({ type }),
|
||||
req.ip
|
||||
);
|
||||
|
||||
const downloadName = type === 'original' ? row.original_filename : row.original_filename.replace('.xlsx', '_processed.xlsx');
|
||||
|
||||
res.setHeader('Content-Type', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${downloadName}"`);
|
||||
res.sendFile(filePath);
|
||||
});
|
||||
});
|
||||
|
||||
// DELETE /api/weekly-reports/:id - Delete report (admin only)
|
||||
router.delete('/:id', requireAuth(db), requireRole(db, 'admin'), (req, res) => {
|
||||
const { id } = req.params;
|
||||
|
||||
const sql = 'SELECT original_file_path, processed_file_path FROM weekly_reports WHERE id = ?';
|
||||
|
||||
db.get(sql, [id], (err, row) => {
|
||||
if (err) {
|
||||
console.error('Error fetching report for deletion:', err);
|
||||
return res.status(500).json({ error: 'Failed to fetch report' });
|
||||
}
|
||||
|
||||
if (!row) {
|
||||
return res.status(404).json({ error: 'Report not found' });
|
||||
}
|
||||
|
||||
// Delete database record
|
||||
db.run('DELETE FROM weekly_reports WHERE id = ?', [id], (err) => {
|
||||
if (err) {
|
||||
console.error('Error deleting report:', err);
|
||||
return res.status(500).json({ error: 'Failed to delete report' });
|
||||
}
|
||||
|
||||
// Delete files
|
||||
if (fs.existsSync(row.original_file_path)) {
|
||||
fs.unlinkSync(row.original_file_path);
|
||||
}
|
||||
if (fs.existsSync(row.processed_file_path)) {
|
||||
fs.unlinkSync(row.processed_file_path);
|
||||
}
|
||||
|
||||
// Log audit entry
|
||||
logAudit(
|
||||
db,
|
||||
req.user.id,
|
||||
req.user.username,
|
||||
'DELETE_WEEKLY_REPORT',
|
||||
'weekly_reports',
|
||||
id,
|
||||
null,
|
||||
req.ip
|
||||
);
|
||||
|
||||
res.json({ success: true });
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
return router;
|
||||
}
|
||||
|
||||
module.exports = createWeeklyReportsRouter;
|
||||
2
backend/scripts/requirements.txt
Normal file
2
backend/scripts/requirements.txt
Normal file
@@ -0,0 +1,2 @@
|
||||
pandas>=2.0.0
|
||||
openpyxl>=3.0.0
|
||||
83
backend/scripts/split_cve_report.py
Executable file
83
backend/scripts/split_cve_report.py
Executable file
@@ -0,0 +1,83 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
CVE Report Splitter
|
||||
Splits multiple CVE IDs in a single row into separate rows for easier filtering and analysis.
|
||||
"""
|
||||
|
||||
import pandas as pd
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
def split_cve_report(input_file, output_file=None, sheet_name='Vulnerabilities', cve_column='CVE ID'):
|
||||
"""
|
||||
Split CVE IDs into separate rows.
|
||||
|
||||
Args:
|
||||
input_file: Path to input Excel file
|
||||
output_file: Path to output file (default: adds '_Split' to input filename)
|
||||
sheet_name: Name of sheet with vulnerability data (default: 'Vulnerabilities')
|
||||
cve_column: Name of column containing CVE IDs (default: 'CVE ID')
|
||||
"""
|
||||
input_path = Path(input_file)
|
||||
|
||||
if not input_path.exists():
|
||||
print(f"Error: File not found: {input_file}")
|
||||
sys.exit(1)
|
||||
|
||||
if output_file is None:
|
||||
output_file = input_path.parent / f"{input_path.stem}_Split{input_path.suffix}"
|
||||
|
||||
print(f"Reading: {input_file}")
|
||||
|
||||
try:
|
||||
df = pd.read_excel(input_file, sheet_name=sheet_name)
|
||||
except ValueError as e:
|
||||
print(f"Error: Sheet '{sheet_name}' not found in workbook")
|
||||
print(f"Available sheets: {pd.ExcelFile(input_file).sheet_names}")
|
||||
sys.exit(1)
|
||||
|
||||
if cve_column not in df.columns:
|
||||
print(f"Error: Column '{cve_column}' not found")
|
||||
print(f"Available columns: {list(df.columns)}")
|
||||
sys.exit(1)
|
||||
|
||||
original_rows = len(df)
|
||||
print(f"Original rows: {original_rows}")
|
||||
|
||||
# Split CVE IDs by comma
|
||||
df[cve_column] = df[cve_column].astype(str).str.split(',')
|
||||
|
||||
# Explode to create separate rows
|
||||
df_exploded = df.explode(cve_column)
|
||||
|
||||
# Clean up CVE IDs
|
||||
df_exploded[cve_column] = df_exploded[cve_column].str.strip()
|
||||
df_exploded = df_exploded[df_exploded[cve_column].notna()]
|
||||
df_exploded = df_exploded[df_exploded[cve_column] != 'nan']
|
||||
df_exploded = df_exploded[df_exploded[cve_column] != '']
|
||||
|
||||
# Reset index
|
||||
df_exploded = df_exploded.reset_index(drop=True)
|
||||
|
||||
new_rows = len(df_exploded)
|
||||
print(f"New rows: {new_rows}")
|
||||
print(f"Added {new_rows - original_rows} rows from splitting CVEs")
|
||||
|
||||
# Save output
|
||||
df_exploded.to_excel(output_file, index=False, sheet_name=sheet_name)
|
||||
print(f"\n✓ Success! Saved to: {output_file}")
|
||||
|
||||
return output_file
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python3 split_cve_report.py <input_file.xlsx> [output_file.xlsx]")
|
||||
print("\nExample:")
|
||||
print(" python3 split_cve_report.py 'Vulnerability Workbook.xlsx'")
|
||||
print(" python3 split_cve_report.py 'input.xlsx' 'output.xlsx'")
|
||||
sys.exit(1)
|
||||
|
||||
input_file = sys.argv[1]
|
||||
output_file = sys.argv[2] if len(sys.argv) > 2 else None
|
||||
|
||||
split_cve_report(input_file, output_file)
|
||||
@@ -18,6 +18,7 @@ const createUsersRouter = require('./routes/users');
|
||||
const createAuditLogRouter = require('./routes/auditLog');
|
||||
const logAudit = require('./helpers/auditLog');
|
||||
const createNvdLookupRouter = require('./routes/nvdLookup');
|
||||
const createWeeklyReportsRouter = require('./routes/weeklyReports');
|
||||
|
||||
const app = express();
|
||||
const PORT = process.env.PORT || 3001;
|
||||
@@ -167,6 +168,9 @@ const upload = multer({
|
||||
limits: { fileSize: 10 * 1024 * 1024 } // 10MB limit
|
||||
});
|
||||
|
||||
// Weekly reports routes (editor/admin for upload, all authenticated for download)
|
||||
app.use('/api/weekly-reports', createWeeklyReportsRouter(db, upload));
|
||||
|
||||
// ========== CVE ENDPOINTS ==========
|
||||
|
||||
// Get all CVEs with optional filters (authenticated users)
|
||||
|
||||
@@ -6,6 +6,7 @@ import UserMenu from './components/UserMenu';
|
||||
import UserManagement from './components/UserManagement';
|
||||
import AuditLog from './components/AuditLog';
|
||||
import NvdSyncModal from './components/NvdSyncModal';
|
||||
import WeeklyReportModal from './components/WeeklyReportModal';
|
||||
import './App.css';
|
||||
|
||||
const API_BASE = process.env.REACT_APP_API_BASE || 'http://localhost:3001/api';
|
||||
@@ -173,6 +174,7 @@ export default function App() {
|
||||
const [showUserManagement, setShowUserManagement] = useState(false);
|
||||
const [showAuditLog, setShowAuditLog] = useState(false);
|
||||
const [showNvdSync, setShowNvdSync] = useState(false);
|
||||
const [showWeeklyReport, setShowWeeklyReport] = useState(false);
|
||||
const [newCVE, setNewCVE] = useState({
|
||||
cve_id: '',
|
||||
vendor: '',
|
||||
@@ -757,6 +759,15 @@ export default function App() {
|
||||
NVD Sync
|
||||
</button>
|
||||
)}
|
||||
{canWrite() && (
|
||||
<button
|
||||
onClick={() => setShowWeeklyReport(true)}
|
||||
className="intel-button intel-button-success relative z-10 flex items-center gap-2"
|
||||
>
|
||||
<Upload className="w-4 h-4" />
|
||||
Weekly Report
|
||||
</button>
|
||||
)}
|
||||
{canWrite() && (
|
||||
<button
|
||||
onClick={() => setShowAddCVE(true)}
|
||||
@@ -810,6 +821,11 @@ export default function App() {
|
||||
<NvdSyncModal onClose={() => setShowNvdSync(false)} onSyncComplete={() => fetchCVEs()} />
|
||||
)}
|
||||
|
||||
{/* Weekly Report Modal */}
|
||||
{showWeeklyReport && (
|
||||
<WeeklyReportModal onClose={() => setShowWeeklyReport(false)} />
|
||||
)}
|
||||
|
||||
{/* Add CVE Modal */}
|
||||
{showAddCVE && (
|
||||
<div className="fixed inset-0 modal-overlay flex items-center justify-center z-50 p-4">
|
||||
|
||||
291
frontend/src/components/WeeklyReportModal.js
Normal file
291
frontend/src/components/WeeklyReportModal.js
Normal file
@@ -0,0 +1,291 @@
|
||||
import React, { useState, useEffect } from 'react';
|
||||
import { X, Loader, AlertCircle, CheckCircle, Upload as UploadIcon, Download, Star } from 'lucide-react';
|
||||
|
||||
const API_BASE = process.env.REACT_APP_API_BASE || 'http://localhost:3001/api';
|
||||
|
||||
export default function WeeklyReportModal({ onClose }) {
|
||||
const [phase, setPhase] = useState('idle'); // idle, uploading, processing, success, error
|
||||
const [selectedFile, setSelectedFile] = useState(null);
|
||||
const [uploadProgress, setUploadProgress] = useState(0);
|
||||
const [result, setResult] = useState(null);
|
||||
const [existingReports, setExistingReports] = useState([]);
|
||||
const [error, setError] = useState('');
|
||||
|
||||
// Fetch existing reports on mount
|
||||
useEffect(() => {
|
||||
fetchExistingReports();
|
||||
}, []);
|
||||
|
||||
const fetchExistingReports = async () => {
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/weekly-reports`, { credentials: 'include' });
|
||||
if (!response.ok) throw new Error('Failed to fetch reports');
|
||||
const data = await response.json();
|
||||
setExistingReports(data);
|
||||
} catch (err) {
|
||||
console.error('Error fetching reports:', err);
|
||||
}
|
||||
};
|
||||
|
||||
const handleFileSelect = (e) => {
|
||||
const file = e.target.files[0];
|
||||
if (file) {
|
||||
if (!file.name.endsWith('.xlsx')) {
|
||||
setError('Please select an Excel file (.xlsx)');
|
||||
return;
|
||||
}
|
||||
setSelectedFile(file);
|
||||
setError('');
|
||||
}
|
||||
};
|
||||
|
||||
const handleUpload = async () => {
|
||||
if (!selectedFile) return;
|
||||
|
||||
setPhase('uploading');
|
||||
setUploadProgress(0);
|
||||
|
||||
const formData = new FormData();
|
||||
formData.append('file', selectedFile);
|
||||
|
||||
try {
|
||||
setUploadProgress(50); // Simulated progress
|
||||
setPhase('processing');
|
||||
|
||||
const response = await fetch(`${API_BASE}/weekly-reports/upload`, {
|
||||
method: 'POST',
|
||||
body: formData,
|
||||
credentials: 'include'
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json();
|
||||
throw new Error(errorData.error || 'Upload failed');
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
setResult(data);
|
||||
setPhase('success');
|
||||
|
||||
// Refresh the list of existing reports
|
||||
await fetchExistingReports();
|
||||
} catch (err) {
|
||||
setError(err.message);
|
||||
setPhase('error');
|
||||
}
|
||||
};
|
||||
|
||||
const handleDownload = async (id, type) => {
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/weekly-reports/${id}/download/${type}`, {
|
||||
credentials: 'include'
|
||||
});
|
||||
|
||||
if (!response.ok) throw new Error('Download failed');
|
||||
|
||||
const blob = await response.blob();
|
||||
const url = window.URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `vulnerability_report_${type}.xlsx`;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
window.URL.revokeObjectURL(url);
|
||||
document.body.removeChild(a);
|
||||
} catch (err) {
|
||||
console.error('Error downloading file:', err);
|
||||
setError(`Failed to download ${type} file`);
|
||||
}
|
||||
};
|
||||
|
||||
const resetForm = () => {
|
||||
setPhase('idle');
|
||||
setSelectedFile(null);
|
||||
setUploadProgress(0);
|
||||
setResult(null);
|
||||
setError('');
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="modal-overlay" onClick={onClose}>
|
||||
<div className="modal-content" onClick={(e) => e.stopPropagation()}>
|
||||
{/* Header */}
|
||||
<div className="modal-header">
|
||||
<h2 className="modal-title">Weekly Vulnerability Report</h2>
|
||||
<button onClick={onClose} className="modal-close">
|
||||
<X className="w-5 h-5" />
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Body */}
|
||||
<div className="modal-body">
|
||||
{/* Idle Phase - File Selection */}
|
||||
{phase === 'idle' && (
|
||||
<div className="space-y-6">
|
||||
<div>
|
||||
<label className="block text-sm font-medium mb-2" style={{ color: '#94A3B8' }}>
|
||||
Upload Excel File (.xlsx)
|
||||
</label>
|
||||
<input
|
||||
type="file"
|
||||
accept=".xlsx"
|
||||
onChange={handleFileSelect}
|
||||
className="intel-input w-full"
|
||||
/>
|
||||
{selectedFile && (
|
||||
<p className="mt-2 text-sm" style={{ color: '#10B981' }}>
|
||||
Selected: {selectedFile.name}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<button
|
||||
onClick={handleUpload}
|
||||
disabled={!selectedFile}
|
||||
className={`intel-button w-full ${selectedFile ? 'intel-button-success' : 'opacity-50 cursor-not-allowed'}`}
|
||||
>
|
||||
<UploadIcon className="w-4 h-4 mr-2" />
|
||||
Upload & Process
|
||||
</button>
|
||||
|
||||
{error && (
|
||||
<div className="flex items-start gap-2 p-3 rounded" style={{ background: 'rgba(239, 68, 68, 0.1)', border: '1px solid #EF4444' }}>
|
||||
<AlertCircle className="w-5 h-5 flex-shrink-0" style={{ color: '#EF4444' }} />
|
||||
<p style={{ color: '#FCA5A5' }}>{error}</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Uploading Phase */}
|
||||
{phase === 'uploading' && (
|
||||
<div className="text-center py-8">
|
||||
<Loader className="w-12 h-12 animate-spin mx-auto mb-4" style={{ color: '#0EA5E9' }} />
|
||||
<p style={{ color: '#94A3B8' }}>Uploading file...</p>
|
||||
<div className="w-full bg-gray-700 rounded-full h-2 mt-4">
|
||||
<div
|
||||
className="h-2 rounded-full transition-all"
|
||||
style={{ width: `${uploadProgress}%`, background: '#0EA5E9' }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Processing Phase */}
|
||||
{phase === 'processing' && (
|
||||
<div className="text-center py-8">
|
||||
<Loader className="w-12 h-12 animate-spin mx-auto mb-4" style={{ color: '#0EA5E9' }} />
|
||||
<p style={{ color: '#94A3B8' }}>Processing vulnerability report...</p>
|
||||
<p className="text-sm mt-2" style={{ color: '#64748B' }}>Splitting CVE IDs into separate rows</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Success Phase */}
|
||||
{phase === 'success' && result && (
|
||||
<div className="space-y-4">
|
||||
<div className="flex items-center gap-2 p-4 rounded" style={{ background: 'rgba(16, 185, 129, 0.1)', border: '1px solid #10B981' }}>
|
||||
<CheckCircle className="w-6 h-6" style={{ color: '#10B981' }} />
|
||||
<div>
|
||||
<p className="font-medium" style={{ color: '#34D399' }}>Upload Successful!</p>
|
||||
<p className="text-sm mt-1" style={{ color: '#94A3B8' }}>
|
||||
Original: {result.original_rows} rows → Processed: {result.processed_rows} rows
|
||||
<span className="ml-2" style={{ color: '#10B981' }}>
|
||||
(+{result.processed_rows - result.original_rows} rows from splitting CVEs)
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex gap-3">
|
||||
<button
|
||||
onClick={() => handleDownload(result.id, 'original')}
|
||||
className="intel-button flex-1"
|
||||
>
|
||||
<Download className="w-4 h-4 mr-2" />
|
||||
Download Original
|
||||
</button>
|
||||
<button
|
||||
onClick={() => handleDownload(result.id, 'processed')}
|
||||
className="intel-button intel-button-success flex-1"
|
||||
>
|
||||
<Download className="w-4 h-4 mr-2" />
|
||||
Download Processed
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<button onClick={resetForm} className="intel-button w-full">
|
||||
Upload Another Report
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Error Phase */}
|
||||
{phase === 'error' && (
|
||||
<div className="space-y-4">
|
||||
<div className="flex items-start gap-2 p-4 rounded" style={{ background: 'rgba(239, 68, 68, 0.1)', border: '1px solid #EF4444' }}>
|
||||
<AlertCircle className="w-6 h-6 flex-shrink-0" style={{ color: '#EF4444' }} />
|
||||
<div>
|
||||
<p className="font-medium" style={{ color: '#FCA5A5' }}>Upload Failed</p>
|
||||
<p className="text-sm mt-1" style={{ color: '#94A3B8' }}>{error}</p>
|
||||
</div>
|
||||
</div>
|
||||
<button onClick={resetForm} className="intel-button w-full">
|
||||
Try Again
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Existing Reports Section */}
|
||||
{(phase === 'idle' || phase === 'success') && existingReports.length > 0 && (
|
||||
<div className="mt-8">
|
||||
<h3 className="text-lg font-medium mb-4" style={{ color: '#94A3B8' }}>
|
||||
Previous Reports
|
||||
</h3>
|
||||
<div className="space-y-3">
|
||||
{existingReports.map((report) => (
|
||||
<div
|
||||
key={report.id}
|
||||
className="intel-card p-4"
|
||||
>
|
||||
<div className="flex items-start justify-between">
|
||||
<div className="flex-1">
|
||||
<div className="flex items-center gap-2 mb-2">
|
||||
{report.is_current && (
|
||||
<Star className="w-4 h-4 fill-current" style={{ color: '#F59E0B' }} />
|
||||
)}
|
||||
<p className="font-medium" style={{ color: report.is_current ? '#F59E0B' : '#94A3B8' }}>
|
||||
{report.week_label}
|
||||
</p>
|
||||
</div>
|
||||
<p className="text-sm" style={{ color: '#64748B' }}>
|
||||
{new Date(report.upload_date).toLocaleDateString()} •
|
||||
{report.row_count_original} → {report.row_count_processed} rows
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
onClick={() => handleDownload(report.id, 'original')}
|
||||
className="intel-button intel-button-small"
|
||||
title="Download Original"
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
</button>
|
||||
<button
|
||||
onClick={() => handleDownload(report.id, 'processed')}
|
||||
className="intel-button intel-button-success intel-button-small"
|
||||
title="Download Processed"
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user