[object Object]

← back to Unclaimed Property Platform

Cycle 3: NAUPA II adapter + candidate-blocking + guardrail tripwire + doc reconcile

9fbefae0afc0921a4cec1155516a1c077bc199e1 · 2026-08-01 11:14:35 -0700 · Steve Abrams

Product:
- NAUPA II fixed-width parser via lazy adapter registry (_get_parser); same CanonicalProperty
  as CSV so dedup/masking/search are format-agnostic. build_naupa2_line keeps fixtures aligned.
- tests/test_cycle3_naupa_and_blocking.py: NAUPA parity + idempotency + blocking-key bucketing.
Matching:
- blocking_keys fairness fix: empty-norm (CJK/Arabic) names now fall through to geo keys so
  they're never un-indexable; tests/test_cycle4_fairness.py asserts the coverage floor.
Claims:
- FIX regression from M1 refactor: complete_state_submission namespaced its transition key
  (:transition) so it no longer collides with queue's SUBMITTING event (C3-variant, caught
  by the Cycle-2 regression test).
Guardrails:
- scripts/pre-commit-tripwire.sh: blocks SSN-shaped strings + real-feed formats at commit.
Docs:
- docs/jurisdictions.json (machine-readable, 50+DC); docs/03 roadmap/budget.
- RECONCILED docs/03 revenue model from consumer finder-fees -> B2G government-paid, matching
  the thesis + brief (finder-fee model was explicitly excluded at launch). Fixed data/ -> docs/
  jurisdictions.json path (data/* is gitignored).
Tests: 4/4 suites green. All local/$0.

TK-10097

Files touched

Diff

commit 9fbefae0afc0921a4cec1155516a1c077bc199e1
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Sat Aug 1 11:14:35 2026 -0700

    Cycle 3: NAUPA II adapter + candidate-blocking + guardrail tripwire + doc reconcile
    
    Product:
    - NAUPA II fixed-width parser via lazy adapter registry (_get_parser); same CanonicalProperty
      as CSV so dedup/masking/search are format-agnostic. build_naupa2_line keeps fixtures aligned.
    - tests/test_cycle3_naupa_and_blocking.py: NAUPA parity + idempotency + blocking-key bucketing.
    Matching:
    - blocking_keys fairness fix: empty-norm (CJK/Arabic) names now fall through to geo keys so
      they're never un-indexable; tests/test_cycle4_fairness.py asserts the coverage floor.
    Claims:
    - FIX regression from M1 refactor: complete_state_submission namespaced its transition key
      (:transition) so it no longer collides with queue's SUBMITTING event (C3-variant, caught
      by the Cycle-2 regression test).
    Guardrails:
    - scripts/pre-commit-tripwire.sh: blocks SSN-shaped strings + real-feed formats at commit.
    Docs:
    - docs/jurisdictions.json (machine-readable, 50+DC); docs/03 roadmap/budget.
    - RECONCILED docs/03 revenue model from consumer finder-fees -> B2G government-paid, matching
      the thesis + brief (finder-fee model was explicitly excluded at launch). Fixed data/ -> docs/
      jurisdictions.json path (data/* is gitignored).
    Tests: 4/4 suites green. All local/$0.
    
    TK-10097
---
 docs/03-roadmap-and-budget.md           | 45 ++++++++++++++++--------
 docs/jurisdictions.json                 | 61 +++++++++++++++++++++++++++++++++
 scripts/pre-commit-tripwire.sh          | 49 ++++++++++++++++++++++++++
 services/claims/claim_workflow.py       |  7 +++-
 services/matching/entity_match.py       | 23 +++++++------
 tests/test_cycle3_naupa_and_blocking.py |  9 +++--
 tests/test_cycle4_fairness.py           |  8 ++---
 7 files changed, 169 insertions(+), 33 deletions(-)

diff --git a/docs/03-roadmap-and-budget.md b/docs/03-roadmap-and-budget.md
index 06493aa..69efb01 100644
--- a/docs/03-roadmap-and-budget.md
+++ b/docs/03-roadmap-and-budget.md
@@ -21,7 +21,7 @@ Key gates:
 
 ### Phase 1 — Pilot (Months 3–9)
 **Goal:** live with ≤3 states, SAMPLE→real data, first 100 rightful-owner matches confirmed
-by state adjudication, finder fee revenue positive.
+by state adjudication, first paid government participation contract signed.
 
 Milestone checklist:
 - [ ] Replace synthetic SAMPLE jurisdiction with ≥1 real state feed (NAUPA II)
@@ -71,17 +71,29 @@ Key work:
 | Contingency (15%) | $600k | $1.0M |
 | **Total** | **$4.8M** | **$7.6M** |
 
-### Revenue Model (Year 3 Steady State)
-
-| Revenue stream | Unit | Volume | Rate | ARR |
-|---|---|---|---|---|
-| Finder fees (consumer) | per successful claim | 40,000 / yr | avg $350 recovered × 15% avg fee cap | $2.1M |
-| B2B API (law firms / CPAs) | per query | 2M / yr | $0.05 / query | $100k |
-| SaaS seat (estate attorneys) | per seat / yr | 500 seats | $2,400 / yr | $1.2M |
-| State partner revenue share | per state referral | 20 states × 2k claims | $20 referral fee | $800k |
-| **Total ARR** | | | | **$4.2M** |
-
-Break-even: ~Month 28 (conservative capital) / Month 32 (aggressive).
+### Revenue Model (Year 3 Steady State) — GOVERNMENT-FIRST (B2G)
+
+> **Reconciled 2026-08 to match `docs/00-thesis-and-guardrails.md` and the source brief.**
+> An earlier draft of this table modeled **consumer finder fees** as the primary revenue
+> stream. That is explicitly EXCLUDED at launch: taking a % of a claimant's recovery makes
+> the platform a regulated *finder* (finder-law exposure in every state, FTC scam-adjacency,
+> and direct conflict with the state agencies that are the real customer). Consumer search
+> and claim initiation stay **free**. Revenue comes from **government**. If a finder/estate
+> line is ever added, it is a separate, later, compliance-gated decision — never the base model.
+
+| Revenue stream | Buyer | Basis | ARR (illustrative) |
+|---|---|---|---:|
+| Annual platform participation | State / NAUPA | population/volume tier | 35 jur × ~$250k = $8.75M |
+| Feed onboarding & migration | State | one-time implementation | (amortized separately) |
+| White-label state search portal | State | annual software + support | included above / upsell |
+| Claim-workflow module | State | annual or per-claim | upsell on participation |
+| Identity & document verification | State | pass-through + margin | upsell |
+| Authorized partner API (gov/regulated institution) | Gov / regulated FI | contracted usage tiers | modest, contract-gated |
+| **Total recurring (base)** | | | **~$8.75M+ ARR** |
+
+At 35 participating jurisdictions the government-subscription base alone clears the earlier
+$4.2M target ~2×, without any contingent consumer fee. Break-even is driven by how fast
+jurisdictions sign, not by claimant volume.
 
 ---
 
@@ -100,6 +112,9 @@ state-partner termination:
    a Computer Fraud and Abuse Act exposure.
 4. **Masked results only on anonymous search.** Rightful owners see their own data after
    identity verification; unauthenticated search shows C••• O••• (constant-width mask).
-5. **Finder-fee cap compliance.** Per `data/jurisdictions.json`, caps range from 8% (CA)
-   to 25% (DC/FL/KY). The billing engine must enforce the lowest applicable cap for each
-   claim — not the highest.
+5. **No contingent finder fees at launch.** The base model is government-paid (B2G); the
+   platform does NOT take a percentage of a claimant's recovery. Statutory finder-fee caps
+   are catalogued in `docs/jurisdictions.json` (e.g. CA 10%, IL 10%, TX 10%, FL ≤30%) ONLY
+   so that IF a compliance-gated finder/estate line is ever added later, the billing engine
+   can enforce the lowest applicable cap per claim. Until then, this limit means: don't
+   build a finder product by default.
diff --git a/docs/jurisdictions.json b/docs/jurisdictions.json
new file mode 100644
index 0000000..bcb2f35
--- /dev/null
+++ b/docs/jurisdictions.json
@@ -0,0 +1,61 @@
+{
+  "_meta": {
+    "note": "Machine-readable companion to docs/01-regulatory-state-acquisition-matrix.md.",
+    "recommended_method_default": ["direct_feed", "records_request", "public_handoff"],
+    "public_bulk_api": "none established by NAUPA directory for any jurisdiction",
+    "finder_fee_cap_pct": "known statutory cap on third-party finder fees; null = not encoded here (verify per statute)"
+  },
+  "jurisdictions": [
+    {"code": "AL", "name": "Alabama", "portal": "AL Treasury Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "AK", "name": "Alaska", "portal": "AK Treasury", "finder_fee_cap_pct": null},
+    {"code": "AZ", "name": "Arizona", "portal": "AZ Dept of Revenue", "finder_fee_cap_pct": null},
+    {"code": "AR", "name": "Arkansas", "portal": "ClaimItAR (Auditor of State)", "finder_fee_cap_pct": null},
+    {"code": "CA", "name": "California", "portal": "CA State Controller", "finder_fee_cap_pct": 10, "note": "field-level legal review before any records request"},
+    {"code": "CO", "name": "Colorado", "portal": "Great Colorado Payback", "finder_fee_cap_pct": null},
+    {"code": "CT", "name": "Connecticut", "portal": "CT Big List", "finder_fee_cap_pct": null},
+    {"code": "DE", "name": "Delaware", "portal": "DE Office of Unclaimed Property", "finder_fee_cap_pct": null, "note": "aggressive holder-audit state"},
+    {"code": "DC", "name": "District of Columbia", "portal": "DC Office of Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "FL", "name": "Florida", "portal": "FL Treasure Hunt", "finder_fee_cap_pct": 30, "note": "professional requirements on recovery agreements"},
+    {"code": "GA", "name": "Georgia", "portal": "GA Dept of Revenue", "finder_fee_cap_pct": null},
+    {"code": "HI", "name": "Hawaii", "portal": "HI Budget & Finance", "finder_fee_cap_pct": null},
+    {"code": "ID", "name": "Idaho", "portal": "Your Money Idaho", "finder_fee_cap_pct": null},
+    {"code": "IL", "name": "Illinois", "portal": "IL Treasurer I-Cash", "finder_fee_cap_pct": 10, "note": "finders regulated"},
+    {"code": "IN", "name": "Indiana", "portal": "IN Unclaimed Property (AG)", "finder_fee_cap_pct": null},
+    {"code": "IA", "name": "Iowa", "portal": "Great Iowa Treasure Hunt", "finder_fee_cap_pct": null},
+    {"code": "KS", "name": "Kansas", "portal": "KS Treasurer", "finder_fee_cap_pct": null},
+    {"code": "KY", "name": "Kentucky", "portal": "KY Treasury", "finder_fee_cap_pct": null},
+    {"code": "LA", "name": "Louisiana", "portal": "LA Treasury / LaCashClaim", "finder_fee_cap_pct": null},
+    {"code": "ME", "name": "Maine", "portal": "ME Treasurer", "finder_fee_cap_pct": null},
+    {"code": "MD", "name": "Maryland", "portal": "MD Comptroller", "finder_fee_cap_pct": null},
+    {"code": "MA", "name": "Massachusetts", "portal": "Find Mass Money", "finder_fee_cap_pct": null},
+    {"code": "MI", "name": "Michigan", "portal": "MI Treasury", "finder_fee_cap_pct": null},
+    {"code": "MN", "name": "Minnesota", "portal": "MN Commerce", "finder_fee_cap_pct": null},
+    {"code": "MS", "name": "Mississippi", "portal": "MS Treasury", "finder_fee_cap_pct": null},
+    {"code": "MO", "name": "Missouri", "portal": "MO Treasurer", "finder_fee_cap_pct": null},
+    {"code": "MT", "name": "Montana", "portal": "MT Dept of Revenue", "finder_fee_cap_pct": null},
+    {"code": "NE", "name": "Nebraska", "portal": "Nebraska Lost Cash", "finder_fee_cap_pct": null},
+    {"code": "NV", "name": "Nevada", "portal": "NV Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "NH", "name": "New Hampshire", "portal": "NH Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "NJ", "name": "New Jersey", "portal": "NJ Treasury", "finder_fee_cap_pct": null},
+    {"code": "NM", "name": "New Mexico", "portal": "NM Taxation & Revenue", "finder_fee_cap_pct": null},
+    {"code": "NY", "name": "New York", "portal": "NY Office of the State Comptroller", "finder_fee_cap_pct": null, "note": "records request subject to NY disclosure rules; paid location arrangements restricted"},
+    {"code": "NC", "name": "North Carolina", "portal": "NCCash", "finder_fee_cap_pct": null},
+    {"code": "ND", "name": "North Dakota", "portal": "ND Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "OH", "name": "Ohio", "portal": "OH Dept of Commerce", "finder_fee_cap_pct": null},
+    {"code": "OK", "name": "Oklahoma", "portal": "OK Treasurer", "finder_fee_cap_pct": null},
+    {"code": "OR", "name": "Oregon", "portal": "OR Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "PA", "name": "Pennsylvania", "portal": "PA Treasury", "finder_fee_cap_pct": null},
+    {"code": "RI", "name": "Rhode Island", "portal": "Find RI Money", "finder_fee_cap_pct": null},
+    {"code": "SC", "name": "South Carolina", "portal": "SC Treasurer", "finder_fee_cap_pct": null},
+    {"code": "SD", "name": "South Dakota", "portal": "SD Unclaimed Property", "finder_fee_cap_pct": null},
+    {"code": "TN", "name": "Tennessee", "portal": "ClaimItTN", "finder_fee_cap_pct": null},
+    {"code": "TX", "name": "Texas", "portal": "ClaimItTexas / TX Comptroller", "finder_fee_cap_pct": 10},
+    {"code": "UT", "name": "Utah", "portal": "MyCash Utah", "finder_fee_cap_pct": null},
+    {"code": "VT", "name": "Vermont", "portal": "VT Treasurer", "finder_fee_cap_pct": null},
+    {"code": "VA", "name": "Virginia", "portal": "Virginia Money Search", "finder_fee_cap_pct": null},
+    {"code": "WA", "name": "Washington", "portal": "WA Dept of Revenue", "finder_fee_cap_pct": null},
+    {"code": "WV", "name": "West Virginia", "portal": "WV Treasurer", "finder_fee_cap_pct": null},
+    {"code": "WI", "name": "Wisconsin", "portal": "WI Dept of Revenue", "finder_fee_cap_pct": null},
+    {"code": "WY", "name": "Wyoming", "portal": "WY Treasurer", "finder_fee_cap_pct": null}
+  ]
+}
diff --git a/scripts/pre-commit-tripwire.sh b/scripts/pre-commit-tripwire.sh
new file mode 100644
index 0000000..33fa7ce
--- /dev/null
+++ b/scripts/pre-commit-tripwire.sh
@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Pre-commit tripwire — the ACTIVE half of the synthetic-only guardrail.
+#
+# .gitignore is passive (it only ignores untracked paths). This hook BLOCKS a commit that
+# would introduce (a) an SSN-shaped string, or (b) a real-feed file format anywhere outside
+# data/sample/. Born from Security audit finding 2.4 + Steve's standing "one accidental
+# commit is how a key/feed leaks" lesson.
+#
+# Install (from repo root):
+#   ln -sf ../../scripts/pre-commit-tripwire.sh .git/hooks/pre-commit
+#   chmod +x scripts/pre-commit-tripwire.sh
+# Bypass (only with a human decision): git commit --no-verify
+set -euo pipefail
+
+fail=0
+staged=$(git diff --cached --name-only --diff-filter=ACM)
+
+# (a) SSN-shaped content in any staged text file
+ssn_re='[0-9]{3}-[0-9]{2}-[0-9]{4}'
+for f in $staged; do
+  [ -f "$f" ] || continue
+  # skip obvious binaries
+  if file "$f" | grep -qi 'text\|json\|csv\|ascii'; then
+    if grep -nEq "$ssn_re" "$f"; then
+      echo "TRIPWIRE: SSN-shaped string found in staged file: $f"
+      grep -nE "$ssn_re" "$f" | head -3 | sed 's/^/    /'
+      fail=1
+    fi
+  fi
+done
+
+# (b) real-feed file formats must never be committed outside data/sample/
+while IFS= read -r f; do
+  [ -n "$f" ] || continue
+  case "$f" in
+    data/sample/*) : ;;                       # the ONE allowed synthetic location
+    *.naupa|*.dat|*.tab) echo "TRIPWIRE: real-feed format staged: $f"; fail=1 ;;
+    data/*.txt|data/*.xml|data/*.zip|data/**/*.txt|data/**/*.xml|data/**/*.zip)
+      echo "TRIPWIRE: data-feed file staged outside data/sample/: $f"; fail=1 ;;
+  esac
+done <<< "$staged"
+
+if [ "$fail" -ne 0 ]; then
+  echo ""
+  echo "Commit BLOCKED by pre-commit-tripwire. This repo is synthetic-only."
+  echo "If this is a false positive, a HUMAN may bypass with: git commit --no-verify"
+  exit 1
+fi
+exit 0
diff --git a/services/claims/claim_workflow.py b/services/claims/claim_workflow.py
index 3beb729..e1d4138 100644
--- a/services/claims/claim_workflow.py
+++ b/services/claims/claim_workflow.py
@@ -130,9 +130,14 @@ def complete_state_submission(repository: ClaimRepository, adapter: StateAdapter
     # Route through transition_claim — picks up ALLOWED_TRANSITIONS guard + STATE_ONLY
     # guard + the standard 'claim_status_changed' event. transition_claim does its own
     # get_for_update so optimistic locking remains in play.
+    # Namespace THIS transition's idempotency key. queue_state_submission already wrote a
+    # 'claim_status_changed' event under the bare `idempotency_key` for its SUBMITTING
+    # transition; reusing it here collides on UNIQUE(claim_id, idempotency_key). The base
+    # key is one business token spanning queue+complete, so each internal transition must
+    # namespace its own event (regression caught by test_cycle2_hardening).
     claim = transition_claim(
         repository, claim_id, ClaimStatus.SUBMITTED_TO_STATE,
-        actor_id="system:state-adapter", idempotency_key=idempotency_key,
+        actor_id="system:state-adapter", idempotency_key=f"{idempotency_key}:transition",
     )
     # Append the submission-specific event (namespaced key avoids collision with the
     # 'claim_status_changed' event written by transition_claim — C3 fix preserved).
diff --git a/services/matching/entity_match.py b/services/matching/entity_match.py
index 9a262a2..d75acba 100644
--- a/services/matching/entity_match.py
+++ b/services/matching/entity_match.py
@@ -77,16 +77,19 @@ def blocking_keys(inp: MatchInput) -> list[str]:
     """
     norm = normalize_business(inp.name) if inp.is_business else normalize_text(inp.name)
     keys: list[str] = []
-    if not norm:
-        return keys
-    pk = phonetic_key(norm)
-    if pk:
-        keys.append(f"ph:{pk}")
-    toks = norm.split()
-    if toks:
-        keys.append(f"tok0:{toks[0]}")
-        if len(toks) > 1:
-            keys.append(f"tok01:{toks[0]}_{toks[1]}")
+    # Name-derived keys only when the name yields latin-script tokens. A name that
+    # normalizes to empty (pure CJK/Arabic script under the Soundex-era normalizer) must
+    # STILL fall through to the geo keys below — otherwise those owners are un-indexable
+    # even with a valid address, a coverage/fairness gap (see tests/test_cycle4_fairness.py).
+    if norm:
+        pk = phonetic_key(norm)
+        if pk:
+            keys.append(f"ph:{pk}")
+        toks = norm.split()
+        if toks:
+            keys.append(f"tok0:{toks[0]}")
+            if len(toks) > 1:
+                keys.append(f"tok01:{toks[0]}_{toks[1]}")
     if inp.postal_code:
         z5 = (normalize_postal(inp.postal_code) or "")[:5]
         if z5:
diff --git a/tests/test_cycle3_naupa_and_blocking.py b/tests/test_cycle3_naupa_and_blocking.py
index ced8e31..97df0e0 100644
--- a/tests/test_cycle3_naupa_and_blocking.py
+++ b/tests/test_cycle3_naupa_and_blocking.py
@@ -82,9 +82,12 @@ def main() -> int:
     ok(f"re-ingest -> duplicate; count stable at {n_before}")
 
     print("4) blocking keys put same-owner variants in a shared bucket")
-    a = MatchInput("Catherine O'Neil", postal_code="00001", region="SM")
-    b = MatchInput("Kathryn ONeill", postal_code="00001", region="SM")
-    c = MatchInput("Jonathan Doe", postal_code="99999", region="XX")
+    a = MatchInput("Catherine O'Neil", address="100 Test St", city="Springfield",
+                   postal_code="00001", region="SM")
+    b = MatchInput("Kathryn ONeill", address="100 Test Street", city="Springfield",
+                   postal_code="00001", region="SM")
+    c = MatchInput("Jonathan Doe", address="9 Elsewhere Rd", city="Rivertown",
+                   postal_code="99999", region="XX")
     ka, kb, kc = blocking_keys(a), blocking_keys(b), blocking_keys(c)
     phon_a = {k for k in ka if k.startswith("ph:")}
     phon_b = {k for k in kb if k.startswith("ph:")}
diff --git a/tests/test_cycle4_fairness.py b/tests/test_cycle4_fairness.py
index ca2a87f..59fb7f7 100644
--- a/tests/test_cycle4_fairness.py
+++ b/tests/test_cycle4_fairness.py
@@ -33,16 +33,16 @@ def ok(msg: str) -> None:
 
 
 def main() -> int:
-    print("1) Coverage floor: every non-empty name yields >=1 blocking key")
+    print("1) Coverage floor: every record with an address is indexable (name-key OR geo-key)")
     unindexable = []
     for name in DIVERSE_NAMES:
         keys = blocking_keys(MatchInput(name, postal_code="12345", region="SM"))
         # A record with an address is ALWAYS indexable via zip/region even when the name
-        # script produces no phonetic/token key — that's the coverage guarantee.
+        # script produces no phonetic/token key (pure CJK/Arabic) — the coverage guarantee.
         if not keys:
             unindexable.append(name)
-    assert not unindexable, f"these names produced NO blocking key (un-findable): {unindexable}"
-    ok(f"all {len(DIVERSE_NAMES)} names indexable (name-key or geo-key)")
+    assert not unindexable, f"these records produced NO blocking key (un-findable): {unindexable}"
+    ok(f"all {len(DIVERSE_NAMES)} records indexable (incl. non-latin scripts via geo)")
 
     print("2) A name with NO geo still needs a name-derived key when it has letters")
     # A latin-script name with no address must still be indexable by a name key.

← c0f1c68 auto-save: 2026-08-01T11:06:15 (4 files) — services/ingestio  ·  back to Unclaimed Property Platform  ·  chore: mark pre-commit-tripwire.sh executable; TK-10097 (hoo 509f657 →