Sync .kiro/ from master — v2.2.0 release batch
New specs: archer-template-library, ccp-metrics-view-restructure, compliance-list-stale-after-sidebar-edit, compliance-metric-estimated-resolution-date, compliance-remediation-display-fix, flexible-jira-ticket-creation, forecast-burndown-chart, granite-loader-export, ivanti-queue-clear-completed-fix, multi-item-jira-ticket, queue-collapsible-sections, vendor-issue-type-dropdown New steering: archer-template-gen.md Updated: migration-registration-check hook, remediation-plan-history spec, gitlab-workflow, tech, versioning steering files
This commit is contained in:
Jordan Ramos
319
.kiro/specs/granite-loader-export/design.md
Normal file
319
.kiro/specs/granite-loader-export/design.md
Normal file
@@ -0,0 +1,319 @@
|
||||
# Design Document: Granite Loader Sheet Export
|
||||
|
||||
## Overview
|
||||
|
||||
This feature adds a Granite Team_Device Loader xlsx generator to the STEAM Security Dashboard. It integrates with the existing Ivanti Queue (for CARD/GRANITE items) and provides a standalone mode for ad-hoc device lists. The system enriches device data from the CARD API, allows bulk defaults with per-row overrides in an editable preview table, and generates a properly formatted xlsx for upload to SNIP XperLoad.
|
||||
|
||||
Key design decisions:
|
||||
- **Frontend-driven xlsx generation**: The xlsx is generated client-side using the `xlsx` library (already a project dependency) to avoid backend file I/O and temp file cleanup.
|
||||
- **Backend CARD enrichment endpoint**: A single batch endpoint accepts an array of IPs and returns enriched Granite fields from CARD. This keeps the frontend simple and centralizes CARD API token management.
|
||||
- **Reuse queue data model**: Queue items already have `ip_address`, `hostname`, and `workflow_type` — no schema changes needed.
|
||||
- **Modal-based UI**: The loader configuration lives in a modal (like the Consolidation Modal pattern) to avoid adding a new page while keeping the queue page clean.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph Frontend
|
||||
QP[IvantiTodoQueuePage] -->|selected CARD/GRANITE items| LM[LoaderModal]
|
||||
NAV[Nav Drawer / Standalone] -->|manual IP list| LM
|
||||
LM -->|enrich request| API[POST /api/card/enrich-batch]
|
||||
LM -->|generate xlsx| XLSX[xlsx library - client-side]
|
||||
XLSX -->|download| FILE[Loader_Change_TEAM_DATE.xlsx]
|
||||
end
|
||||
|
||||
subgraph Backend
|
||||
API --> CARD[CARD API Helper]
|
||||
CARD -->|per-IP lookup| EXT[card.charter.com]
|
||||
end
|
||||
```
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
### Backend: Batch Enrichment Endpoint
|
||||
|
||||
**File:** `backend/routes/cardApi.js` (added to existing router)
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/card/enrich-batch` | Admin, Standard_User | Batch lookup IPs in CARD, return Granite-relevant fields |
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"ips": ["10.240.78.110", "10.240.78.111", "172.16.5.20"]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"ip": "10.240.78.110",
|
||||
"found": true,
|
||||
"equip_inst_id": "1931008",
|
||||
"hostname": "NDCW-SW-CORE-01",
|
||||
"site_name": "ENWDCOCD-PEAKVIEW-SRDC",
|
||||
"mgmt_ip_asn": "11427",
|
||||
"responsible_team": "NTS-AEO-STEAM",
|
||||
"equipment_class": "S",
|
||||
"equip_template": "DISC-CISCO NEXUS 9300",
|
||||
"equip_status": "Active"
|
||||
},
|
||||
{
|
||||
"ip": "172.16.5.20",
|
||||
"found": false,
|
||||
"equip_inst_id": null,
|
||||
"hostname": null,
|
||||
"error": "IP not found in CARD"
|
||||
}
|
||||
],
|
||||
"enriched_count": 1,
|
||||
"not_found_count": 1,
|
||||
"total": 2
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation notes:**
|
||||
- Accepts up to 200 IPs per request (matches Requirement 1.4 limit).
|
||||
- For each IP, constructs asset ID candidates with known suffixes (CTEC, NATL, CHTR, etc.) and queries `GET /api/v1/owner/{assetId}`.
|
||||
- Falls back to team asset search if direct owner lookup fails.
|
||||
- Extracts fields from `ncim_discovery`, `netops_granite_allips`, `card_flags`, and `owner` on the asset record.
|
||||
- Returns partial results on CARD API errors (best-effort enrichment).
|
||||
|
||||
### Frontend: LoaderModal Component
|
||||
|
||||
**File:** `frontend/src/components/LoaderModal.js`
|
||||
|
||||
A modal component that handles the full loader sheet workflow:
|
||||
|
||||
**Props:**
|
||||
```javascript
|
||||
{
|
||||
isOpen: boolean,
|
||||
onClose: () => void,
|
||||
// Pre-populated from queue selection (null in standalone mode)
|
||||
initialDevices: Array<{ ip_address: string, hostname: string }> | null,
|
||||
}
|
||||
```
|
||||
|
||||
**Internal State:**
|
||||
```javascript
|
||||
{
|
||||
operationType: 'Change' | 'Add' | 'Delete' | 'Move',
|
||||
selectedColumns: Set<string>, // checked column IDs
|
||||
devices: Array<DeviceRow>, // the editable row data
|
||||
bulkDefaults: Record<string, string>, // column → default value
|
||||
overrides: Record<string, Record<string, string>>, // rowIndex → column → value
|
||||
enriching: boolean,
|
||||
enrichErrors: Array<{ ip: string, error: string }>,
|
||||
}
|
||||
```
|
||||
|
||||
**DeviceRow shape:**
|
||||
```javascript
|
||||
{
|
||||
ip_address: string,
|
||||
hostname: string,
|
||||
// CARD-enriched fields (populated after enrichment)
|
||||
equip_inst_id: string | null,
|
||||
site_name: string | null,
|
||||
mgmt_ip_asn: string | null,
|
||||
responsible_team: string | null,
|
||||
equipment_class: string | null,
|
||||
equip_template: string | null,
|
||||
equip_status: string | null,
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend: Column Configuration
|
||||
|
||||
**File:** `frontend/src/utils/graniteLoaderConfig.js`
|
||||
|
||||
Pure data module defining the 41 columns, their groupings, and operation-type requirements:
|
||||
|
||||
```javascript
|
||||
export const LOADER_COLUMNS = [
|
||||
{ id: 'DELETE', label: 'DELETE', group: 'Identification', requiredFor: ['Delete'] },
|
||||
{ id: 'SET_CONFIRMED', label: 'SET_CONFIRMED', group: 'Identification', requiredFor: [] },
|
||||
{ id: 'EQUIPMENT_CLASS', label: 'EQUIPMENT CLASS', group: 'Identification', requiredFor: ['Add'] },
|
||||
{ id: 'EQUIP_INST_ID', label: 'EQUIP_INST_ID', group: 'Identification', requiredFor: ['Change', 'Move', 'Delete'] },
|
||||
{ id: 'SITE_NAME', label: 'SITE_NAME', group: 'Identification', requiredFor: ['Add', 'Move'] },
|
||||
{ id: 'EQUIP_NAME', label: 'EQUIP_NAME', group: 'Identification', requiredFor: ['Add'] },
|
||||
{ id: 'EQUIP_TEMPLATE', label: 'EQUIP_TEMPLATE', group: 'Identification', requiredFor: ['Add'] },
|
||||
{ id: 'EQUIP_STATUS', label: 'EQUIP_STATUS', group: 'Identification', requiredFor: ['Add'] },
|
||||
{ id: 'RESPONSIBLE_TEAM', label: 'UDA#RESPONSIBLE ORGANIZATION#RESPONSIBLE TEAM', group: 'Responsible Org', requiredFor: ['Add'] },
|
||||
{ id: 'IPV4_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV4_ADDRESS', group: 'IP Addressing', requiredFor: ['Add'] },
|
||||
{ id: 'MAC_ADDRESS', label: 'UDA#IP_ADDRESSING#MAC ADDRESS', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'MGMT_IP_ASN', label: 'UDA#IP_ADDRESSING#MGMT_IP_ASN', group: 'IP Addressing', requiredFor: ['Add'] },
|
||||
{ id: 'SERIALNUMBER', label: 'SERIALNUMBER', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'EXCLUDED_DISCOVERY', label: 'UDA#AUTO DISCOVERY#EXCLUDED FROM DISCOVERY', group: 'Discovery', requiredFor: [] },
|
||||
{ id: 'EXCLUDED_REASON', label: 'UDA#AUTO DISCOVERY#EXCLUDED FROM DISCOVERY REASON', group: 'Discovery', requiredFor: [] },
|
||||
{ id: 'IPV6_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV6_ADDRESS', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'ILOM_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV4_ILOM_ADDRESS', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'APP_ID_ASSET_TAG', label: 'UDA#CHERWELL_CMDB#APP ID ASSET TAG', group: 'Cyber Metrics', requiredFor: [] },
|
||||
{ id: 'DEVICE_FUNCTION', label: 'UDA#CHERWELL_CMDB#DEVICE_FUNCTION', group: 'Cyber Metrics', requiredFor: [] },
|
||||
{ id: 'ENVIRONMENT', label: 'UDA#CHERWELL_CMDB#ENVIRONMENT', group: 'Cyber Metrics', requiredFor: [] },
|
||||
{ id: 'SECONDARY_MGMT_IP', label: 'UDA#IP_ADDRESSING#SECONDARY_MGMT_IP_ADDRESS', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'VIP', label: 'UDA#IP_ADDRESSING#VIP', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'FLOATING_IP', label: 'UDA#IP_ADDRESSING#FLOATING IP ADDRESS', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'SCAN_IP_1', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 1', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'SCAN_IP_2', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 2', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'SCAN_IP_3', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 3', group: 'IP Addressing', requiredFor: [] },
|
||||
{ id: 'EQUIP_MODEL', label: 'EQUIP_MODEL', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'EQUIP_COMMENTS', label: 'EQUIP_COMMENTS', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'EQUIP_PARTNUMBER', label: 'EQUIP_PARTNUMBER', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'OS', label: 'UDA#CONTROLLER CONFIG#OS', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'OS_VERSION', label: 'UDA#CONTROLLER CONFIG#OS VERSION', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'CPU_CORES', label: 'UDA#CONTROLLER CONFIG#TOTAL CPU CORES', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'RAM_GB', label: 'UDA#CONTROLLER CONFIG#RAM IN GB', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'STORAGE_GB', label: 'UDA#CONTROLLER CONFIG#STORAGE IN GB', group: 'Equipment Info', requiredFor: [] },
|
||||
{ id: 'ARCHER_ID', label: 'UDA#WIFI EQUIP INFO#ARCHER ID', group: 'Other', requiredFor: [] },
|
||||
{ id: 'INSTALL_LOCATION', label: 'UDA#WIFI EQUIP INFO#INSTALL LOCATION', group: 'Other', requiredFor: [] },
|
||||
{ id: 'SYSNAME', label: 'UDA#EQUIPMENT_INFO#SYSNAME', group: 'Other', requiredFor: [] },
|
||||
{ id: 'LATITUDE', label: 'UDA#WIFI EQUIP INFO#LATITUDE', group: 'Other', requiredFor: [] },
|
||||
{ id: 'LONGITUDE', label: 'UDA#WIFI EQUIP INFO#LONGITUDE', group: 'Other', requiredFor: [] },
|
||||
{ id: 'OSTYPE', label: 'UDA#EQUIP MIGRATION#OSTYPE', group: 'Other', requiredFor: [] },
|
||||
{ id: 'OSVERSION', label: 'UDA#EQUIP MIGRATION#OSVERSION', group: 'Other', requiredFor: [] },
|
||||
];
|
||||
|
||||
export const COLUMN_GROUPS = [
|
||||
'Identification',
|
||||
'IP Addressing',
|
||||
'Responsible Org',
|
||||
'Discovery',
|
||||
'Cyber Metrics',
|
||||
'Equipment Info',
|
||||
'Other',
|
||||
];
|
||||
|
||||
export const OPERATION_TYPES = ['Change', 'Add', 'Delete', 'Move'];
|
||||
```
|
||||
|
||||
### Frontend: XLSX Generation
|
||||
|
||||
**File:** `frontend/src/utils/graniteLoaderExport.js`
|
||||
|
||||
Pure function that takes the configured state and produces an xlsx workbook:
|
||||
|
||||
```javascript
|
||||
/**
|
||||
* @param {Object} config
|
||||
* @param {string} config.operationType - 'Change' | 'Add' | 'Delete' | 'Move'
|
||||
* @param {Array<string>} config.columnIds - ordered list of selected column IDs
|
||||
* @param {Array<Object>} config.rows - device rows with resolved values (bulk + overrides merged)
|
||||
* @returns {Blob} xlsx file as a Blob for download
|
||||
*/
|
||||
export function generateLoaderXlsx(config) { ... }
|
||||
```
|
||||
|
||||
Uses the `xlsx` library (already in `frontend/package.json`) to create a workbook with a single "Load_Sheet" worksheet.
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Queue-Initiated Flow
|
||||
|
||||
1. User selects CARD/GRANITE items on IvantiTodoQueuePage
|
||||
2. User clicks "Generate Loader Sheet" in floating action bar
|
||||
3. LoaderModal opens with `initialDevices` populated from selected items
|
||||
4. User selects Operation Type (defaults to "Change")
|
||||
5. User checks desired columns (required columns pre-checked)
|
||||
6. User optionally clicks "Enrich from CARD" → `POST /api/card/enrich-batch`
|
||||
7. EQUIP_INST_ID and other fields populate in the preview table
|
||||
8. User sets bulk defaults and per-row overrides as needed
|
||||
9. User clicks "Download" → client-side xlsx generation → browser download
|
||||
|
||||
### Standalone Flow
|
||||
|
||||
1. User navigates to standalone access point (nav drawer link or CARD page section)
|
||||
2. LoaderModal opens with empty device list
|
||||
3. User pastes IPs (textarea, one per line or comma-separated) → rows populate
|
||||
4. Steps 4–9 same as above
|
||||
|
||||
### CARD Enrichment Flow (Backend)
|
||||
|
||||
1. Frontend sends `POST /api/card/enrich-batch` with array of IPs
|
||||
2. Backend iterates IPs, for each:
|
||||
a. Try `GET /api/v1/owner/{ip}-CTEC`, then `-NATL`, then `-CHTR` (known suffixes)
|
||||
b. If found, extract fields from the asset record
|
||||
c. If not found via owner lookup, search team assets for the IP
|
||||
3. Return results array with found/not-found status per IP
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Scenario | Behavior |
|
||||
|----------|----------|
|
||||
| CARD API not configured | "Enrich from CARD" button hidden; tooltip explains why |
|
||||
| CARD API timeout on individual IP | Mark that IP as not-found, continue with others |
|
||||
| CARD API auth failure | Show error toast, abort enrichment, preserve any already-enriched data |
|
||||
| All IPs not found in CARD | Show warning banner "No devices found in CARD — enter EQUIP_INST_ID manually" |
|
||||
| Required field missing on download | Highlight cells, show warning count, allow download with acknowledgment |
|
||||
| xlsx generation failure | Show error toast with message |
|
||||
| More than 200 IPs submitted | Frontend truncates to 200 with warning message |
|
||||
|
||||
## UI Layout
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Generate Granite Loader Sheet [X] │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Operation: [Change ▾] Devices: 24 items │
|
||||
│ │
|
||||
│ ┌─ Columns ──────────────────────────────────────────────────┐ │
|
||||
│ │ ▸ Identification (4 selected) │ │
|
||||
│ │ ▸ IP Addressing (2 selected) │ │
|
||||
│ │ ▸ Responsible Org (1 selected) │ │
|
||||
│ │ ▸ Cyber Metrics (0 selected) │ │
|
||||
│ │ ▸ Equipment Info (0 selected) │ │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Enrich from CARD] │
|
||||
│ │
|
||||
│ ┌─ Bulk Defaults ────────────────────────────────────────────┐ │
|
||||
│ │ RESPONSIBLE TEAM: [NTS-AEO-STEAM ] │ │
|
||||
│ │ EQUIP_STATUS: [Active ] │ │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─ Preview (24 rows) ────────────────────────────────────────┐ │
|
||||
│ │ IP Address │ EQUIP_INST_ID │ RESP TEAM │ STATUS │ │
|
||||
│ │───────────────┼───────────────┼────────────────┼───────────│ │
|
||||
│ │ 10.240.78.110 │ 1931008 │ NTS-AEO-STEAM │ Active │ │
|
||||
│ │ 10.240.78.111 │ 1931009 │ NTS-AEO-STEAM │ Active │ │
|
||||
│ │ 172.16.5.20 │ ⚠ (not found) │ ACCESS-ENG ● │ Active │ │
|
||||
│ │ 172.16.5.21 │ 2045112 │ NTS-AEO-STEAM │ Active │ │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ⚠ 1 row missing EQUIP_INST_ID │
|
||||
│ │
|
||||
│ [Cancel] [Download Loader Sheet] │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- ● = per-row override indicator (amber dot)
|
||||
- ⚠ = missing required field or CARD lookup failure
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Property-Based Tests
|
||||
|
||||
**File:** `backend/__tests__/granite-loader-enrichment.property.test.js`
|
||||
|
||||
- **Property 1: Enrichment result count** — For any array of N IPs (1 ≤ N ≤ 200), the response contains exactly N result objects.
|
||||
- **Property 2: Found results have required fields** — For any result where `found === true`, `equip_inst_id` is a non-empty string.
|
||||
- **Property 3: Not-found results have null fields** — For any result where `found === false`, `equip_inst_id` is null.
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- Column configuration: required columns for each operation type
|
||||
- XLSX generation: correct headers, correct row data, empty cells handled
|
||||
- Bulk default + override merge logic
|
||||
- IP validation
|
||||
- EQUIP_INST_ID numeric validation
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- `POST /api/card/enrich-batch` with mocked CARD API responses
|
||||
- Auth requirement enforcement
|
||||
- 200-IP limit enforcement
|
||||
Reference in New Issue
Block a user