`docs/02-architecture.md`

61 lines
# Architecture

Polyglot by design — public search, transactional claims, immutable source files, and
analytics have different consistency, latency, and durability needs; forcing them into
one engine is the wrong call.

## Recommended stack

| Role | Technology | Why |
|---|---|---|
| System of record | PostgreSQL (managed) | ACID, constraints, joins, JSON, trigram fuzzy for review tools |
| National masked search | OpenSearch | distributed fuzzy search, faceting, near-real-time; **eventually consistent, not a claim store** |
| Raw / replay / history | Object storage + open table format | cheap, immutable, versioned; mandatory forensic layer |
| Analytics / model training | Warehouse (Snowflake/BigQuery/lakehouse) | large scans, workload isolation |
| Cache / rate limits / sessions | Redis | low-latency atomic counters; not a store of record |
| Fraud / entity investigations | Graph DB (later) | owners↔addresses↔holders↔claims |

**Prototype substitution:** this repo runs the PostgreSQL + object-store + search roles
against **SQLite + the local filesystem + an in-DB search fallback** so the whole thing
runs at $0 with zero infrastructure. The service interfaces are unchanged; only the
concrete adapters differ.

## Service boundaries (anonymous search is isolated from claims)

| API | Responsibility |
|---|---|
| Search API | query, filters, **masked** result cards, pagination tokens, rate limits |
| Property-selection API | short-lived signed reference for one selected result |
| Claim API | create case, eligibility, evidence metadata, status |
| State-integration API | submit case, receive status, apply corrections & suppressions |
| Administration API | per-jurisdiction field/policy config, support queues, reports |
| Privacy API | access records, deletion, correction, restrictions |
| Audit API | restricted evidence export for state & compliance reviewers |

The authoritative property table is **never** directly queryable from the internet.

## Two matching systems, deliberately separate

| System | Objective | Error preference |
|---|---|---|
| Public search ranking | retrieve plausible records a user may recognize | favor **recall** |
| Internal entity resolution | decide whether records refer to the same real owner | favor **precision**; keep a manual-review band |
| Claimant↔owner comparison | support evidence review for one selected property | **never** auto-decide entitlement from a score |

Fuzzy search must help a person *find* a record; it must never be the basis for
automatically *approving* a claim.

## Refresh cadence (per-contract, not one-size-fits-all)

| Feed type | Cadence | Rule |
|---|---|---|
| Full snapshot | monthly/quarterly | reconcile totals; don't delete absent records until snapshot semantics confirmed |
| Incremental adds | daily–weekly | idempotent upsert keyed by (jurisdiction, state record id) |
| Corrections | daily | preserve prior version + effective timestamp |
| Suppressions | near-real-time/daily | remove from public search promptly; retain restricted audit history |
| Claim dispositions | daily–weekly | mark claimed/paid/denied/inactive per state definitions |

Every batch carries: checksum, source URI, jurisdiction, parser version, record/error/
accepted counts, effective date, transformation version. Reprocessing the same batch is
idempotent unless the transformation version changes.