# Design Document: Finding Archive Tracking

## Overview

The Finding Archive Tracking system adds a detection layer to the existing Ivanti sync pipeline that identifies findings which disappear from sync results due to severity score drift. It tracks these findings through a four-state lifecycle (ACTIVE → ARCHIVED → RETURNED → CLOSED) with full transition history stored in two new SQLite tables. Three new API endpoints expose archive data, and an Archive Summary Bar UI component provides at-a-glance state counts on the Ivanti dashboard.

The system integrates directly into the existing `syncFindings()` function in `ivantiFindings.js`, comparing current sync results against the previous set to detect disappearances and reappearances. This approach requires no additional API calls to Ivanti and leverages the already-cached findings data.

## Architecture

```mermaid
flowchart TD
    subgraph Ivanti Sync Pipeline
        A[syncFindings] --> B[Fetch all pages from Ivanti API]
        B --> C[Store findings in ivanti_findings_cache]
        C --> D[Archive Detection]
    end

    subgraph Archive Detection
        D --> E{Compare previous vs current finding IDs}
        E -->|Missing from current| F[Create/Update Archive Record → ARCHIVED]
        E -->|Returned in current| G[Update Archive Record → RETURNED]
        E -->|Closed in Ivanti| H[Update Archive Record → CLOSED]
        F --> I[Insert Transition History]
        G --> I
        H --> I
    end

    subgraph Archive API
        J[GET /api/ivanti/archive] --> K[(ivanti_finding_archives)]
        L[GET /api/ivanti/archive/:findingId/history] --> M[(ivanti_archive_transitions)]
        N[GET /api/ivanti/archive/stats] --> K
    end

    subgraph Frontend
        O[Archive Summary Bar] -->|fetch stats| N
        O -->|click state| J
        P[Transition History Panel] -->|fetch history| L
    end
```

### Integration Points

1. **Sync Pipeline Hook**: Archive detection runs after `syncFindings()` successfully stores new findings in the cache. It reads the previous findings from the cache before the update, then compares against the new set.
2. **Route Registration**: The archive router is mounted at `/api/ivanti/archive` in `server.js`, following the same factory pattern as existing Ivanti routes.
3. **Frontend Integration**: The Archive Summary Bar is rendered on the existing Ivanti findings page, above the findings table.

## Components and Interfaces

### 1. Archive Detection Module (`detectArchiveChanges`)

Located within `backend/routes/ivantiFindings.js`, this async function runs after a successful sync.

```javascript
/**
 * Compare previous and current finding sets to detect archive state changes.
 * @param {sqlite3.Database} db - SQLite database instance
 * @param {Array} previousFindings - Findings from before the sync update
 * @param {Array} currentFindings - Findings from the latest sync
 */
async function detectArchiveChanges(db, previousFindings, currentFindings) {
    // 1. Build ID sets from previous and current
    // 2. Disappeared = in previous but not in current → ARCHIVED
    // 3. Returned = in current AND has existing ARCHIVED record → RETURNED
    // 4. For each state change, upsert archive record + insert transition
}
```

### 2. Closed Finding Detection (`detectClosedFindings`)

Runs during the closed count sync to detect findings that transitioned to CLOSED in Ivanti.

```javascript
/**
 * Check archived findings against Ivanti closed findings to detect remediation.
 * @param {sqlite3.Database} db - SQLite database instance
 * @param {Array} closedFindingIds - IDs of findings confirmed closed in Ivanti
 */
async function detectClosedFindings(db, closedFindingIds) {
    // For each archived/returned finding, if it appears in closed set → CLOSED
}
```

### 3. Archive API Router (`createIvantiArchiveRouter`)

Located at `backend/routes/ivantiArchive.js`, follows the existing factory pattern.

```javascript
/**
 * @param {sqlite3.Database} db - SQLite database instance
 * @param {Function} requireAuth - Auth middleware factory
 * @returns {express.Router}
 */
function createIvantiArchiveRouter(db, requireAuth) {
    const router = express.Router();
    router.use(requireAuth(db));

    // GET / - List archive records, optional ?state= filter
    // GET /stats - Summary counts by state
    // GET /:findingId/history - Transition history for a finding

    return router;
}
```

### 4. Archive Summary Bar Component (`ArchiveSummaryBar`)

Located at `frontend/src/components/pages/ArchiveSummaryBar.js`.

```javascript
/**
 * Displays four stat cards for ACTIVE, ARCHIVED, RETURNED, CLOSED counts.
 * @param {Object} props
 * @param {Function} props.onStateClick - Callback when a state card is clicked
 * @param {string|null} props.activeFilter - Currently selected state filter
 */
function ArchiveSummaryBar({ onStateClick, activeFilter }) { ... }
```

### API Endpoint Specifications

| Endpoint | Method | Auth | Query Params | Response |
|----------|--------|------|-------------|----------|
| `/api/ivanti/archive` | GET | Required | `state` (optional: ACTIVE, ARCHIVED, RETURNED, CLOSED) | `{ archives: [...], total: N }` |
| `/api/ivanti/archive/stats` | GET | Required | None | `{ ARCHIVED: N, RETURNED: N, CLOSED: N, total: N }` |
| `/api/ivanti/archive/:findingId/history` | GET | Required | None | `{ finding_id: "...", transitions: [...] }` |

## Data Models

### `ivanti_finding_archives` Table

| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Row ID |
| `finding_id` | TEXT | NOT NULL UNIQUE | Ivanti finding identifier |
| `finding_title` | TEXT | NOT NULL DEFAULT '' | Finding title at time of archival |
| `host_name` | TEXT | NOT NULL DEFAULT '' | Host name at time of archival |
| `ip_address` | TEXT | NOT NULL DEFAULT '' | IP address at time of archival |
| `current_state` | TEXT | NOT NULL CHECK(IN ('ARCHIVED','RETURNED','CLOSED')) | Current lifecycle state |
| `last_severity` | REAL | NOT NULL DEFAULT 0 | Last known severity score |
| `first_archived_at` | DATETIME | NOT NULL DEFAULT CURRENT_TIMESTAMP | When first archived |
| `last_transition_at` | DATETIME | NOT NULL DEFAULT CURRENT_TIMESTAMP | When last state change occurred |
| `created_at` | DATETIME | DEFAULT CURRENT_TIMESTAMP | Row creation time |

**Indexes:**
- `idx_archive_finding_id` on `finding_id`
- `idx_archive_current_state` on `current_state`

### `ivanti_archive_transitions` Table

| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Row ID |
| `archive_id` | INTEGER | NOT NULL, FK → ivanti_finding_archives(id) | Parent archive record |
| `from_state` | TEXT | NOT NULL | Previous state (or 'NONE' for initial) |
| `to_state` | TEXT | NOT NULL | New state |
| `severity_at_transition` | REAL | NOT NULL DEFAULT 0 | Severity score at time of transition |
| `reason` | TEXT | NOT NULL DEFAULT '' | Human-readable reason |
| `transitioned_at` | DATETIME | NOT NULL DEFAULT CURRENT_TIMESTAMP | When transition occurred |

**Indexes:**
- `idx_transition_archive_id` on `archive_id`

### State Transition Diagram

Archive records are only created when a finding first disappears from sync results. Findings that remain present in sync results do not get archive records — they are simply "active" in the findings cache. The three database states are ARCHIVED, RETURNED, and CLOSED.

```mermaid
stateDiagram-v2
    [*] --> ARCHIVED : Finding disappears from sync (score drift)
    ARCHIVED --> RETURNED : Reappeared in sync
    ARCHIVED --> CLOSED : Confirmed remediated in Ivanti
    RETURNED --> ARCHIVED : Disappeared again
    RETURNED --> CLOSED : Confirmed remediated in Ivanti
```

### Valid State Transitions

| From State | To State | Reason |
|-----------|----------|--------|
| NONE → | ARCHIVED | `severity_score_drift` (first disappearance) |
| ARCHIVED → | RETURNED | `reappeared_in_sync` |
| ARCHIVED → | CLOSED | `remediated_in_ivanti` |
| RETURNED → | ARCHIVED | `severity_score_drift` |
| RETURNED → | CLOSED | `remediated_in_ivanti` |


## Correctness Properties

*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*

### Property 1: Disappeared findings are archived with complete metadata

*For any* set of previous findings and current findings, every finding present in the previous set but absent from the current set should have an Archive_Record with state ARCHIVED, and that record should contain the correct finding_id, finding_title, host_name, ip_address, and last_severity matching the original finding's data.

**Validates: Requirements 1.1, 1.2, 2.2**

### Property 2: Returned findings transition from ARCHIVED to RETURNED

*For any* finding that has an Archive_Record with state ARCHIVED, if that finding reappears in the current sync results, the Archive_Record state should be updated to RETURNED and the last_severity should reflect the finding's current severity score.

**Validates: Requirements 1.3**

### Property 3: Re-disappeared findings transition from RETURNED to ARCHIVED

*For any* finding that has an Archive_Record with state RETURNED, if that finding disappears from the current sync results, the Archive_Record state should be updated back to ARCHIVED.

**Validates: Requirements 1.4**

### Property 4: Every state transition produces a history record with all required fields

*For any* state transition on an Archive_Record, a Transition_History row should be inserted containing a valid archive_id, the correct from_state and to_state, a severity_at_transition value, a non-empty reason string, and a transitioned_at timestamp.

**Validates: Requirements 2.1**

### Property 5: Closed findings transition to CLOSED state

*For any* finding that has an Archive_Record with state ARCHIVED or RETURNED, if that finding appears in the Ivanti closed findings set, the Archive_Record state should be updated to CLOSED and the transition reason should be "remediated_in_ivanti".

**Validates: Requirements 2.3**

### Property 6: State filter returns only matching records

*For any* set of Archive_Records with various states, querying the archive list endpoint with a state filter should return only records whose current_state matches the filter, and the count should equal the number of records in that state.

**Validates: Requirements 4.1**

### Property 7: Transition history is ordered by timestamp descending

*For any* finding with multiple Transition_History entries, the history endpoint should return entries ordered by transitioned_at descending, such that each entry's timestamp is greater than or equal to the next entry's timestamp.

**Validates: Requirements 4.2**

### Property 8: Stats counts match actual record distribution

*For any* set of Archive_Records, the stats endpoint should return counts where the sum of all state counts equals the total number of Archive_Records, and each individual state count matches the actual number of records in that state.

**Validates: Requirements 4.3**

### Property 9: Migration idempotency

*For any* number of consecutive executions of the migration script, the resulting database schema should be identical and no errors should occur on subsequent runs.

**Validates: Requirements 6.2**

## Error Handling

| Scenario | Handling |
|----------|----------|
| Sync fails (API error, timeout) | Archive detection is skipped entirely for that cycle. No archive records are created or modified. The sync error is logged as usual. |
| Database error during archive upsert | Log the error, continue processing remaining findings. Do not abort the entire archive detection pass. |
| Database error during transition insert | Log the error. The archive record state may have been updated but the transition history may be incomplete. This is acceptable as the current state is the source of truth. |
| Invalid state transition attempted | The detection logic only performs valid transitions per the state diagram. Invalid transitions (e.g., CLOSED → ARCHIVED) are not possible by design since closed findings are excluded from the sync pipeline. |
| Missing finding metadata | Use empty string defaults for finding_title, host_name, ip_address if the finding object lacks these fields. Severity defaults to 0. |
| Archive API query with invalid state parameter | Return a 400 status code with message "Invalid state parameter. Valid values: ACTIVE, ARCHIVED, RETURNED, CLOSED". Explicit errors surface frontend bugs faster than silent fallbacks. |
| History query for non-existent finding | Return 200 with empty transitions array (not 404), per requirement 4.5. |

## Testing Strategy

### Unit Tests

Unit tests cover specific examples and edge cases:

- Migration script creates both tables and all indexes (example, Req 3.1–3.4)
- Archive detection skips when sync errors occur (example, Req 1.5)
- Unauthenticated requests return 401 (example, Req 4.4)
- History endpoint returns empty array for unknown finding (edge case, Req 4.5)
- Archive Summary Bar renders four stat cards (example, Req 5.1)
- Archive Summary Bar fetches stats on mount (example, Req 5.2)
- Clicking a state card triggers filter callback (example, Req 5.3)

### Property-Based Tests

Property-based tests use a PBT library (e.g., `fast-check`) to verify universal properties across generated inputs. Each test runs a minimum of 100 iterations.

| Property | Test Description | Tag |
|----------|-----------------|-----|
| Property 1 | Generate random previous/current finding sets, run detection, verify all disappeared findings have correct ARCHIVED records | **Feature: finding-archive-tracking, Property 1: Disappeared findings are archived with complete metadata** |
| Property 2 | Generate archived findings, add some back to current set, verify RETURNED state | **Feature: finding-archive-tracking, Property 2: Returned findings transition from ARCHIVED to RETURNED** |
| Property 3 | Generate returned findings, remove some from current set, verify ARCHIVED state | **Feature: finding-archive-tracking, Property 3: Re-disappeared findings transition from RETURNED to ARCHIVED** |
| Property 4 | Generate random state transitions, verify each produces a complete history row | **Feature: finding-archive-tracking, Property 4: Every state transition produces a history record** |
| Property 5 | Generate archived/returned findings, mark some as closed, verify CLOSED state and reason | **Feature: finding-archive-tracking, Property 5: Closed findings transition to CLOSED state** |
| Property 6 | Generate archive records with random states, query with filter, verify only matching records returned | **Feature: finding-archive-tracking, Property 6: State filter returns only matching records** |
| Property 7 | Generate multiple transitions for a finding, query history, verify descending order | **Feature: finding-archive-tracking, Property 7: Transition history is ordered by timestamp descending** |
| Property 8 | Generate archive records with random states, query stats, verify counts match | **Feature: finding-archive-tracking, Property 8: Stats counts match actual record distribution** |
| Property 9 | Run migration N times, verify no errors and schema is consistent | **Feature: finding-archive-tracking, Property 9: Migration idempotency** |

### Testing Tools

- **Test runner**: Jest (via react-scripts for frontend, direct for backend)
- **Property-based testing**: `fast-check` library
- **Database**: In-memory SQLite (`:memory:`) for isolated test runs
- **HTTP testing**: `supertest` for API endpoint tests