cve-dashboard/.kiro/specs/granite-loader-export/design.md

# Design Document: Granite Loader Sheet Export

## Overview

This feature adds a Granite Team_Device Loader xlsx generator to the STEAM Security Dashboard. It integrates with the existing Ivanti Queue (for CARD/GRANITE items) and provides a standalone mode for ad-hoc device lists. The system enriches device data from the CARD API, allows bulk defaults with per-row overrides in an editable preview table, and generates a properly formatted xlsx for upload to SNIP XperLoad.

Key design decisions:
- **Frontend-driven xlsx generation**: The xlsx is generated client-side using the `xlsx` library (already a project dependency) to avoid backend file I/O and temp file cleanup.
- **Backend CARD enrichment endpoint**: A single batch endpoint accepts an array of IPs and returns enriched Granite fields from CARD. This keeps the frontend simple and centralizes CARD API token management.
- **Reuse queue data model**: Queue items already have `ip_address`, `hostname`, and `workflow_type` — no schema changes needed.
- **Modal-based UI**: The loader configuration lives in a modal (like the Consolidation Modal pattern) to avoid adding a new page while keeping the queue page clean.

## Architecture

```mermaid
flowchart TD
    subgraph Frontend
        QP[IvantiTodoQueuePage] -->|selected CARD/GRANITE items| LM[LoaderModal]
        NAV[Nav Drawer / Standalone] -->|manual IP list| LM
        LM -->|enrich request| API[POST /api/card/enrich-batch]
        LM -->|generate xlsx| XLSX[xlsx library - client-side]
        XLSX -->|download| FILE[Loader_Change_TEAM_DATE.xlsx]
    end

    subgraph Backend
        API --> CARD[CARD API Helper]
        CARD -->|per-IP lookup| EXT[card.charter.com]
    end
```

## Components and Interfaces

### Backend: Batch Enrichment Endpoint

**File:** `backend/routes/cardApi.js` (added to existing router)

| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/api/card/enrich-batch` | Admin, Standard_User | Batch lookup IPs in CARD, return Granite-relevant fields |

**Request:**
```json
{
  "ips": ["10.240.78.110", "10.240.78.111", "172.16.5.20"]
}
```

**Response:**
```json
{
  "results": [
    {
      "ip": "10.240.78.110",
      "found": true,
      "equip_inst_id": "1931008",
      "hostname": "NDCW-SW-CORE-01",
      "site_name": "ENWDCOCD-PEAKVIEW-SRDC",
      "mgmt_ip_asn": "11427",
      "responsible_team": "NTS-AEO-STEAM",
      "equipment_class": "S",
      "equip_template": "DISC-CISCO NEXUS 9300",
      "equip_status": "Active"
    },
    {
      "ip": "172.16.5.20",
      "found": false,
      "equip_inst_id": null,
      "hostname": null,
      "error": "IP not found in CARD"
    }
  ],
  "enriched_count": 1,
  "not_found_count": 1,
  "total": 2
}
```

**Implementation notes:**
- Accepts up to 200 IPs per request (matches Requirement 1.4 limit).
- For each IP, constructs asset ID candidates with known suffixes (CTEC, NATL, CHTR, etc.) and queries `GET /api/v1/owner/{assetId}`.
- Falls back to team asset search if direct owner lookup fails.
- Extracts fields from `ncim_discovery`, `netops_granite_allips`, `card_flags`, and `owner` on the asset record.
- Returns partial results on CARD API errors (best-effort enrichment).

### Frontend: LoaderModal Component

**File:** `frontend/src/components/LoaderModal.js`

A modal component that handles the full loader sheet workflow:

**Props:**
```javascript
{
  isOpen: boolean,
  onClose: () => void,
  // Pre-populated from queue selection (null in standalone mode)
  initialDevices: Array<{ ip_address: string, hostname: string }> | null,
}
```

**Internal State:**
```javascript
{
  operationType: 'Change' | 'Add' | 'Delete' | 'Move',
  selectedColumns: Set<string>,        // checked column IDs
  devices: Array<DeviceRow>,           // the editable row data
  bulkDefaults: Record<string, string>, // column → default value
  overrides: Record<string, Record<string, string>>, // rowIndex → column → value
  enriching: boolean,
  enrichErrors: Array<{ ip: string, error: string }>,
}
```

**DeviceRow shape:**
```javascript
{
  ip_address: string,
  hostname: string,
  // CARD-enriched fields (populated after enrichment)
  equip_inst_id: string | null,
  site_name: string | null,
  mgmt_ip_asn: string | null,
  responsible_team: string | null,
  equipment_class: string | null,
  equip_template: string | null,
  equip_status: string | null,
}
```

### Frontend: Column Configuration

**File:** `frontend/src/utils/graniteLoaderConfig.js`

Pure data module defining the 41 columns, their groupings, and operation-type requirements:

```javascript
export const LOADER_COLUMNS = [
  { id: 'DELETE', label: 'DELETE', group: 'Identification', requiredFor: ['Delete'] },
  { id: 'SET_CONFIRMED', label: 'SET_CONFIRMED', group: 'Identification', requiredFor: [] },
  { id: 'EQUIPMENT_CLASS', label: 'EQUIPMENT CLASS', group: 'Identification', requiredFor: ['Add'] },
  { id: 'EQUIP_INST_ID', label: 'EQUIP_INST_ID', group: 'Identification', requiredFor: ['Change', 'Move', 'Delete'] },
  { id: 'SITE_NAME', label: 'SITE_NAME', group: 'Identification', requiredFor: ['Add', 'Move'] },
  { id: 'EQUIP_NAME', label: 'EQUIP_NAME', group: 'Identification', requiredFor: ['Add'] },
  { id: 'EQUIP_TEMPLATE', label: 'EQUIP_TEMPLATE', group: 'Identification', requiredFor: ['Add'] },
  { id: 'EQUIP_STATUS', label: 'EQUIP_STATUS', group: 'Identification', requiredFor: ['Add'] },
  { id: 'RESPONSIBLE_TEAM', label: 'UDA#RESPONSIBLE ORGANIZATION#RESPONSIBLE TEAM', group: 'Responsible Org', requiredFor: ['Add'] },
  { id: 'IPV4_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV4_ADDRESS', group: 'IP Addressing', requiredFor: ['Add'] },
  { id: 'MAC_ADDRESS', label: 'UDA#IP_ADDRESSING#MAC ADDRESS', group: 'IP Addressing', requiredFor: [] },
  { id: 'MGMT_IP_ASN', label: 'UDA#IP_ADDRESSING#MGMT_IP_ASN', group: 'IP Addressing', requiredFor: ['Add'] },
  { id: 'SERIALNUMBER', label: 'SERIALNUMBER', group: 'Equipment Info', requiredFor: [] },
  { id: 'EXCLUDED_DISCOVERY', label: 'UDA#AUTO DISCOVERY#EXCLUDED FROM DISCOVERY', group: 'Discovery', requiredFor: [] },
  { id: 'EXCLUDED_REASON', label: 'UDA#AUTO DISCOVERY#EXCLUDED FROM DISCOVERY REASON', group: 'Discovery', requiredFor: [] },
  { id: 'IPV6_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV6_ADDRESS', group: 'IP Addressing', requiredFor: [] },
  { id: 'ILOM_ADDRESS', label: 'UDA#IP_ADDRESSING#IPV4_ILOM_ADDRESS', group: 'IP Addressing', requiredFor: [] },
  { id: 'APP_ID_ASSET_TAG', label: 'UDA#CHERWELL_CMDB#APP ID ASSET TAG', group: 'Cyber Metrics', requiredFor: [] },
  { id: 'DEVICE_FUNCTION', label: 'UDA#CHERWELL_CMDB#DEVICE_FUNCTION', group: 'Cyber Metrics', requiredFor: [] },
  { id: 'ENVIRONMENT', label: 'UDA#CHERWELL_CMDB#ENVIRONMENT', group: 'Cyber Metrics', requiredFor: [] },
  { id: 'SECONDARY_MGMT_IP', label: 'UDA#IP_ADDRESSING#SECONDARY_MGMT_IP_ADDRESS', group: 'IP Addressing', requiredFor: [] },
  { id: 'VIP', label: 'UDA#IP_ADDRESSING#VIP', group: 'IP Addressing', requiredFor: [] },
  { id: 'FLOATING_IP', label: 'UDA#IP_ADDRESSING#FLOATING IP ADDRESS', group: 'IP Addressing', requiredFor: [] },
  { id: 'SCAN_IP_1', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 1', group: 'IP Addressing', requiredFor: [] },
  { id: 'SCAN_IP_2', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 2', group: 'IP Addressing', requiredFor: [] },
  { id: 'SCAN_IP_3', label: 'UDA#IP_ADDRESSING#SCAN IP ADDRESS 3', group: 'IP Addressing', requiredFor: [] },
  { id: 'EQUIP_MODEL', label: 'EQUIP_MODEL', group: 'Equipment Info', requiredFor: [] },
  { id: 'EQUIP_COMMENTS', label: 'EQUIP_COMMENTS', group: 'Equipment Info', requiredFor: [] },
  { id: 'EQUIP_PARTNUMBER', label: 'EQUIP_PARTNUMBER', group: 'Equipment Info', requiredFor: [] },
  { id: 'OS', label: 'UDA#CONTROLLER CONFIG#OS', group: 'Equipment Info', requiredFor: [] },
  { id: 'OS_VERSION', label: 'UDA#CONTROLLER CONFIG#OS VERSION', group: 'Equipment Info', requiredFor: [] },
  { id: 'CPU_CORES', label: 'UDA#CONTROLLER CONFIG#TOTAL CPU CORES', group: 'Equipment Info', requiredFor: [] },
  { id: 'RAM_GB', label: 'UDA#CONTROLLER CONFIG#RAM IN GB', group: 'Equipment Info', requiredFor: [] },
  { id: 'STORAGE_GB', label: 'UDA#CONTROLLER CONFIG#STORAGE IN GB', group: 'Equipment Info', requiredFor: [] },
  { id: 'ARCHER_ID', label: 'UDA#WIFI EQUIP INFO#ARCHER ID', group: 'Other', requiredFor: [] },
  { id: 'INSTALL_LOCATION', label: 'UDA#WIFI EQUIP INFO#INSTALL LOCATION', group: 'Other', requiredFor: [] },
  { id: 'SYSNAME', label: 'UDA#EQUIPMENT_INFO#SYSNAME', group: 'Other', requiredFor: [] },
  { id: 'LATITUDE', label: 'UDA#WIFI EQUIP INFO#LATITUDE', group: 'Other', requiredFor: [] },
  { id: 'LONGITUDE', label: 'UDA#WIFI EQUIP INFO#LONGITUDE', group: 'Other', requiredFor: [] },
  { id: 'OSTYPE', label: 'UDA#EQUIP MIGRATION#OSTYPE', group: 'Other', requiredFor: [] },
  { id: 'OSVERSION', label: 'UDA#EQUIP MIGRATION#OSVERSION', group: 'Other', requiredFor: [] },
];

export const COLUMN_GROUPS = [
  'Identification',
  'IP Addressing',
  'Responsible Org',
  'Discovery',
  'Cyber Metrics',
  'Equipment Info',
  'Other',
];

export const OPERATION_TYPES = ['Change', 'Add', 'Delete', 'Move'];
```

### Frontend: XLSX Generation

**File:** `frontend/src/utils/graniteLoaderExport.js`

Pure function that takes the configured state and produces an xlsx workbook:

```javascript
/**
 * @param {Object} config
 * @param {string} config.operationType - 'Change' | 'Add' | 'Delete' | 'Move'
 * @param {Array<string>} config.columnIds - ordered list of selected column IDs
 * @param {Array<Object>} config.rows - device rows with resolved values (bulk + overrides merged)
 * @returns {Blob} xlsx file as a Blob for download
 */
export function generateLoaderXlsx(config) { ... }
```

Uses the `xlsx` library (already in `frontend/package.json`) to create a workbook with a single "Load_Sheet" worksheet.

## Data Flow

### Queue-Initiated Flow

1. User selects CARD/GRANITE items on IvantiTodoQueuePage
2. User clicks "Generate Loader Sheet" in floating action bar
3. LoaderModal opens with `initialDevices` populated from selected items
4. User selects Operation Type (defaults to "Change")
5. User checks desired columns (required columns pre-checked)
6. User optionally clicks "Enrich from CARD" → `POST /api/card/enrich-batch`
7. EQUIP_INST_ID and other fields populate in the preview table
8. User sets bulk defaults and per-row overrides as needed
9. User clicks "Download" → client-side xlsx generation → browser download

### Standalone Flow

1. User navigates to standalone access point (nav drawer link or CARD page section)
2. LoaderModal opens with empty device list
3. User pastes IPs (textarea, one per line or comma-separated) → rows populate
4. Steps 4–9 same as above

### CARD Enrichment Flow (Backend)

1. Frontend sends `POST /api/card/enrich-batch` with array of IPs
2. Backend iterates IPs, for each:
   a. Try `GET /api/v1/owner/{ip}-CTEC`, then `-NATL`, then `-CHTR` (known suffixes)
   b. If found, extract fields from the asset record
   c. If not found via owner lookup, search team assets for the IP
3. Return results array with found/not-found status per IP

## Error Handling

| Scenario | Behavior |
|----------|----------|
| CARD API not configured | "Enrich from CARD" button hidden; tooltip explains why |
| CARD API timeout on individual IP | Mark that IP as not-found, continue with others |
| CARD API auth failure | Show error toast, abort enrichment, preserve any already-enriched data |
| All IPs not found in CARD | Show warning banner "No devices found in CARD — enter EQUIP_INST_ID manually" |
| Required field missing on download | Highlight cells, show warning count, allow download with acknowledgment |
| xlsx generation failure | Show error toast with message |
| More than 200 IPs submitted | Frontend truncates to 200 with warning message |

## UI Layout

```
┌─────────────────────────────────────────────────────────────────┐
│ Generate Granite Loader Sheet                              [X]  │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│ Operation: [Change ▾]     Devices: 24 items                     │
│                                                                 │
│ ┌─ Columns ──────────────────────────────────────────────────┐  │
│ │ ▸ Identification (4 selected)                              │  │
│ │ ▸ IP Addressing (2 selected)                               │  │
│ │ ▸ Responsible Org (1 selected)                             │  │
│ │ ▸ Cyber Metrics (0 selected)                               │  │
│ │ ▸ Equipment Info (0 selected)                              │  │
│ └────────────────────────────────────────────────────────────┘  │
│                                                                 │
│ [Enrich from CARD]                                              │
│                                                                 │
│ ┌─ Bulk Defaults ────────────────────────────────────────────┐  │
│ │ RESPONSIBLE TEAM: [NTS-AEO-STEAM    ]                      │  │
│ │ EQUIP_STATUS:     [Active           ]                      │  │
│ └────────────────────────────────────────────────────────────┘  │
│                                                                 │
│ ┌─ Preview (24 rows) ────────────────────────────────────────┐  │
│ │ IP Address    │ EQUIP_INST_ID │ RESP TEAM      │ STATUS    │  │
│ │───────────────┼───────────────┼────────────────┼───────────│  │
│ │ 10.240.78.110 │ 1931008       │ NTS-AEO-STEAM  │ Active    │  │
│ │ 10.240.78.111 │ 1931009       │ NTS-AEO-STEAM  │ Active    │  │
│ │ 172.16.5.20   │ ⚠ (not found) │ ACCESS-ENG ●   │ Active    │  │
│ │ 172.16.5.21   │ 2045112       │ NTS-AEO-STEAM  │ Active    │  │
│ └────────────────────────────────────────────────────────────┘  │
│                                                                 │
│ ⚠ 1 row missing EQUIP_INST_ID                                  │
│                                                                 │
│                              [Cancel]  [Download Loader Sheet]  │
└─────────────────────────────────────────────────────────────────┘
```

- ● = per-row override indicator (amber dot)
- ⚠ = missing required field or CARD lookup failure

## Testing Strategy

### Property-Based Tests

**File:** `backend/__tests__/granite-loader-enrichment.property.test.js`

- **Property 1: Enrichment result count** — For any array of N IPs (1 ≤ N ≤ 200), the response contains exactly N result objects.
- **Property 2: Found results have required fields** — For any result where `found === true`, `equip_inst_id` is a non-empty string.
- **Property 3: Not-found results have null fields** — For any result where `found === false`, `equip_inst_id` is null.

### Unit Tests

- Column configuration: required columns for each operation type
- XLSX generation: correct headers, correct row data, empty cells handled
- Bulk default + override merge logic
- IP validation
- EQUIP_INST_ID numeric validation

### Integration Tests

- `POST /api/card/enrich-batch` with mocked CARD API responses
- Auth requirement enforcement
- 200-IP limit enforcement