`BROKER-COVERAGE-WAVE2.md`

82 lines
# Broker registry — wave-2 state coverage (M-B2)

Probed 2026-07-21, curl-only, same OPEN-FILES-ONLY rule as wave 1 (no portal
scraping fights, no paid data, no browser sessions). Statuses match the wave-1
matrix vocabulary. This file is wave-2 only — merge into BROKER-COVERAGE.md later.

## Newly ingested (OPEN-INGESTED)

| State | Source | URL | Rows ingested | Notes |
|---|---|---|---|---|
| IL | IDFPR "Professional Licensing" Socrata `pzzh-kp68` on illinois-edp.data.socrata.com | https://illinois-edp.data.socrata.com/resource/pzzh-kp68.csv?$where=license_type='REAL ESTATE' | **549,380 brokers + 30,295 firms** | 4.26M-row all-professions dataset; RE slice = 663,185 rows / 593,494 distinct license numbers (renewal-era duplicate rows — adapter orders by license_number,lastmodifieddate so engine last-write-wins keeps newest). `description` = specific credential (SALESPERSON 362k / BROKER 184k / MANAGING BROKER 26k / ASSOCIATE BROKER 5k / LEASING AGENT 40k individuals; BROKER CORPORATION 15.5k / LLF 5.2k / BRANCH OFFICE 9.7k / PARTNERSHIP / ASSOCIATION / VIRTUAL OFFICE firms). Education credentials (CE/pre-license courses, schools, instructors, sponsors) skipped. All statuses kept (ACTIVE 77k, NOT RENEWED 329k, EXPIRED 136k…), license_status preserved. No employer linkage columns → individuals carry no firm (CO-style). SODA CSV paginated $limit=300000, ≤2 req/s. Adapter `il_idfpr.ts`. |
| DE | Division of Professional Regulation "Professional and Occupational Licensing" Socrata `pjnv-eaih` on data.delaware.gov | https://data.delaware.gov/resource/pjnv-eaih.csv?$where=profession_id='Real Estate' | **29,895 brokers + 997 firms** | 351k-row all-professions dataset; RE slice = 30,895 rows. Individual types: (Resident/Nonresident) Salesperson 24.9k, (Resident/Nonresident/Associate) Broker 5.9k. Main Office Permit (638) + Branch Office Permit (361) = brokerage offices → firm rows (company name in last_name/combined_name). All statuses kept. No employer linkage. Adapter `de_dpr.ts`. |

Registry after wave 2: **1,506,248 brokers / 172,883 firms** across 7 states
(TX, FL, NY, CO, CT, IL, DE). Both ingest_runs green; `all` regression re-run
green across all 7 adapters. Cost $0 (public records).

## Probed and blocked this pass

| State | Source | Status | Detail |
|---|---|---|---|
| AZ | ADRE PdbWeb (services.azre.gov/PdbWeb) | **GATED-confirmed** | Wave-1's "underlying JSON API" candidate does not exist: PdbWeb is classic ASP.NET MVC + Kendo, not a JSON SPA. Search pages (SearchIndividualLicenses / SearchEntityLicenses) are form-POSTs with `__RequestVerificationToken` anti-forgery + session cookie, REQUIRE ≥1 populated text field (prefix-match), and the "export" is built client-side with jszip from the returned search-result table. Bulk access would mean enumerating name/license-number prefixes through the search form = portal-scraping fight. |
| PA | data.pa.gov `fwj2-whnj` "Pennsylvania Professional Licensee Data County State" | **GATED-portal** (counts only) | Looked licensee-level from the title, is actually per-county × license-type ACTIVE COUNTS (6,052 rows). PALS lookup remains the only per-licensee surface. |
| IA | data.iowa.gov | **GATED (platform migration)** | The Socrata portal is GONE — data.iowa.gov is now a custom Next.js data hub (catalog renders client-side; no discoverable JSON API from the RSC payload or JS chunks; /resource/ endpoints 404; Socrata discovery resultSetSize=0 for the domain). "Iowa Professional Licenses" dataset unreachable without a browser. Recheck when the new portal ships a public API (CSP hints data files live on storage.googleapis.com). |
| OH | eLicense (Salesforce) + com.ohio.gov DRE | **NOT-FOUND (recheck confirmed dead)** | eLicense is a Salesforce community (Aura endpoints = portal fight); the old com.ohio.gov DRE roster pages 404. No RE dataset on any Socrata/CKAN surface. |
| WA | data.wa.gov | **GATED-portal (recheck confirmed)** | Searched `q=licenses` across the domain: only COUNTS (`a2n7-rij5`), transactions, and — bizarrely — individual-level professional licenses for ASOTIN COUNTY ONLY (`c738-vqng`). No statewide licensee-level RE roster. |
| NJ | data.nj.gov + mylicense portal | **GATED-portal (recheck confirmed)** | Socrata domain exists but has no professional-licensee datasets (child care, vehicle inspection…). |
| IN | PLA bulk download (secure.in.gov/apps/pla/license/bulk/pla_bulk_ia) | **GATED-paid** | Beautiful trap: free-looking public bulk form (agree → pick "Real Estate *" types → search) returns "33,606 records for a total of **$500.01**" + credit-card form. IN.gov charges per record. |
| NV | red.nv.gov | **GATED (bot-wall)** | Entire site serves a JS challenge shell (base64 placeholder images, no real HTML) to curl. |
| AK | commerce.alaska.gov CBPL | **GATED (WAF)** | The known free professional-license DB download page 403s curl (even with browser UA). Candidate for a later real-browser pass — the underlying CSV is genuinely free. |
| SC | LLR (llr.sc.gov / verify.llronline.com) | **GATED (WAF)** | 403 on verify; datasets page redirects home. |
| MA | mass.gov DOL licensee lists | **GATED (WAF)** | Akamai 403s curl even with full browser headers. MA does publish licensee lists — real-browser candidate for wave 3. |
| NH | oplc.nh.gov license-data | **GATED (WAF)** | 403. |
| MO | pr.mo.gov | **GATED-portal** | Division of Professional Registration migrated to Salesforce Lightning ("/license/s/sfsites/") — old listings.asp gone. |
| MI | LARA / data.michigan.gov | **GATED** | Socrata domain has only boilerplate report PDFs; LARA lookup pages moved/404; BPL custom lists are a paid service. |
| OR | Oregon REA (oregon.gov/rea) | **GATED-portal** | eLicense = Micropact portal; no data/roster links on the licensing pages; data.oregon.gov has no RE datasets. |
| MD | opendata.maryland.gov + onestop.md.gov | **GATED** | Socrata has property assessments, not licensees. OneStop is a "Formability" JS platform (JSON endpoints exist but registries are behind hashed IDs + search UI). |
| VT | data.vermont.gov + sos.vermont.gov OPR | **NOT-FOUND** | "DFS Licensing MasterList" is Division of FIRE SAFETY trades (gas installers…), not RE. OPR publishes only monthly discipline reports. |
| MT | data.mt.gov / ebiz.mt.gov | **GATED-portal** | No Socrata licensee data; ebiz redirects to Accela portal. |
| AR | arec.arkansas.gov / labor.arkansas.gov roster search | **GATED (WAF)** | Roster-search page 403s curl. |
| KY | krec.ky.gov (data.ky.gov doesn't exist) | **NOT-FOUND** | No roster/data downloads on the site. |
| LA | lrec.gov | **NOT-FOUND** | No data/roster page in nav; licensee search only. |
| KS | krec.ks.gov | **GATED (WAF)** | 403. |
| NE | nrec.nebraska.gov | **GATED-portal** | igovsolution verification portal only. |
| WY | realestate.wyo.gov | **NOT-FOUND** | No roster downloads. |
| SD | dlr.sd.gov/realestate | **NOT-FOUND** | Application forms only; sdrec.sd.gov is a login portal. |
| TN | tn.gov/commerce/regboards/trec | **UNREACHABLE** | Connection failed this pass (000); recheck next pass. |
| HI | data.hawaii.gov | **NOT-FOUND** | Socrata domain returns 0 datasets (also migrated/dead). |
| MN / WI / MS / AL / SC / DC | various | **NOT-FOUND** | No state Socrata domains (`data.mn.gov`, `data.wi.gov`, `data.ms.gov`, `data.alabama.gov`, `data.sc.gov`, `opendata.dc.gov` all absent from Socrata discovery). WI DSPS + MN Commerce are known paid/portal. |

## Framework changes

None to the shared engine/harness beyond registry appends (`ilIdfpr`, `deDpr`
in engine.ts ADAPTERS). Both new adapters are plain StateAdapter
implementations; IL introduced the SODA-paginated-CSV fetch pattern
($limit/$offset pages, header-only page = stop, 500ms inter-page delay)
reusable for any future big Socrata slice.

## Wave-3 done

- **CT upgrade (2026-07-26)** — `ct_dcp` re-pointed from the affiliation-only feed
  `6tja-6vdt` (18,367 active salespersons, no status/city) to the FULL DCP
  salesperson credential roster `eqtn-rppv` (99,396 rows). CT salespersons
  **18,367 → 96,989** (5.3×), now carrying `license_status` (96,981) and `city`
  (96,956) on every row + `supervising_broker` firm linkage (20,460). Same
  `source`/source_id (`fullcredentialcode` = "RES.#") → in-place upsert, no
  double-count. Salespersons only; the broker-side `REB` credential roster
  remains a follow-up. Cost $0 (SODA CSV, `$limit=200000`).

## Wave-3 candidates

1. **AK CBPL CSV** — genuinely free full professional-license DB; only a WAF
   between us and it. One real-browser (openclaw/Browserbase) fetch would do it.
2. **MA DOL licensee lists** — published Excel lists behind an Akamai wall;
   same real-browser treatment.
3. **CT upgrade** — `eqtn-rppv` (status w/ broker) + DCP full credential
   extract to add CT firm HQ cities + broker-side roster.
4. **IA new portal API** — watch for the Next.js data hub exposing its catalog
   API; files live on GCS per CSP.
5. **TN TREC** — retry (network blip this pass); TN C&I historically posts
   roster files.