specs COMPLETE: contractors (CSLB direct CSV + AZ ROC commercial 47k verified) + lenders (DFPI CFL 2565, NMLS=ToS DROP); source matrix all-verified

eaa8236378d7fb97c4c38c279e3abf9ed4faf4ec · 2026-07-30 21:24:11 -0700 · Steve
Files touched

M README.md
A specs/contractors.md
Diff

commit eaa8236378d7fb97c4c38c279e3abf9ed4faf4ec
Author: Steve <steve@designerwallcoverings.com>
Date:   Thu Jul 30 21:24:11 2026 -0700

    specs COMPLETE: contractors (CSLB direct CSV + AZ ROC commercial 47k verified) + lenders (DFPI CFL 2565, NMLS=ToS DROP); source matrix all-verified
---
 README.md            | 23 ++++++++++++-----------
 specs/contractors.md | 15 +++++++++++++++
 2 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/README.md b/README.md
index 35029ad..49a4957 100644
--- a/README.md
+++ b/README.md
@@ -10,17 +10,18 @@ way the DRE registry backs Brokers/Owners.
 - **Geography:** near-term markets — **SoCal (LA/OC/IE/SD/Ventura) + Arizona (Scottsdale/Maricopa)**, within ~300 mi of the LA core + Scottsdale.
 - **Filter:** **commercial-relevant only** (e.g. CSLB class B general-building + relevant C-classes; CRE lenders; commercial title/escrow).
 
-## Sources (per-role authoritative public registries) — status: DISCOVERY IN FLIGHT
-| Role | Source | Status |
-|------|--------|--------|
-| Contractors | CA **CSLB** + **Arizona ROC** | discovery agent running |
-| Title | CA **DOI** (title insurers / UTCs) + AZ DIFI | discovery agent running |
-| Escrow | CA **DFPI** (Escrow Law licensees) + AZ DIFI | discovery agent running |
-| Lenders | CA **DFPI** (CFL/CRMLA) + **NMLS** + AZ DIFI | discovery agent running |
-| Attorneys | reuse local **calbar / lawyer-directory** infra, else CA/AZ **State Bar** | discovery agent running |
-
-data.ca.gov CKAN was checked and does **NOT** host these — each is a per-agency
-download/scrape (that's what the discovery agents are speccing).
+## Sources — status: DISCOVERY COMPLETE + VERIFIED (2026-07-31). See `specs/`.
+| Role | Best source | Access | Verified volume |
+|------|-------------|--------|-----------------|
+| Contractors | CA **CSLB** master CSV | direct .ashx, no auth ✅ | ~290k statewide (52 cols, County+Class) |
+| Contractors AZ | **ROC** Commercial CSV | browser (CF-walled) ✅ | **47,235** commercial |
+| Escrow | CA **DFPI** Solr JSON | public token ✅ | **1,711** (~700–1k SoCal) |
+| Title | CA **DOI** APEX + AZ NAIC/DIFI | Playwright | small (insurers ~30–40 + UTCs) |
+| Lenders | CA **DFPI CFL** Solr JSON | public token ✅ | **2,565** Active SoCal CFL |
+| Lenders NMLS/AZ | ⛔ **NMLS ToS DROP** | — | AZ lenders route through NMLS → blocked |
+| Attorneys | reuse local `lawyer_professional_directory` | local DB, $0 ✅ | **55** CRE SoCal core (of 21.7k Active) + AZ Bar (Playwright) |
+
+data.ca.gov CKAN confirmed dead-end. **NMLS is a hard ToS red line** (no bulk scrape / no soliciting) — CA lenders come from DFPI instead; AZ lenders need a paid CSBS feed (gated spend) or drop.
 
 ## Honesty rules (non-negotiable)
 - Public **license** data = targeting only, **not** a send list. No email/phone unless the
diff --git a/specs/contractors.md b/specs/contractors.md
new file mode 100644
index 0000000..388c03d
--- /dev/null
+++ b/specs/contractors.md
@@ -0,0 +1,15 @@
+# Contractors / Builders — access spec  [discovery: COMPLETE + VERIFIED 2026-07-31]
+
+## 1. CA CSLB ✅ BEST — direct no-auth master CSV (feed-first, $0, one GET)
+- Endpoint (PROVEN 200 text/csv, no cookie/captcha): `https://www2.cslb.ca.gov/OnlineServices/DataPortal/DownLoadFile.ashx?fName=MasterLicenseData&type=C` (`type=E` = Excel). Skip the ASP.NET portal postback — hit the .ashx directly. Refreshed ~weekly ("as of 7/28/2026").
+- 52 cols (header verified): LicenseNo, BusinessName, FullBusinessName, MailingAddress, **City, State, County, ZIPCode**, BusinessPhone, BusinessType, IssueDate, ExpirationDate, **PrimaryStatus**, SecondaryStatus, **Classifications(s)**, WC fields… (no email/website).
+- Commercial filter: `Classifications(s)` contains B (General Building) / A (Engineering) / relevant C-codes (C-8,C-10,C-20,C-36,C-39,…). Geo: `County IN (Los Angeles, Orange, Riverside, San Bernardino, San Diego, Ventura)` (+Santa Barbara/Kern/Imperial for full 300mi). Status: PrimaryStatus CLEAR/Active.
+- ~290k statewide → ~120–160k SoCal commercial-relevant before status filter. **One curl → COPY into staging → filter in SQL. $0.**
+
+## 2. AZ ROC ✅ Commercial CSV (Steve's link — VERIFIED 47,235 records via browser)
+- File: `https://roc.az.gov/sites/default/files/ROC_Posting-List_Commercial_<YYYY-MM-DD>.csv` off `roc.az.gov/posting-list`. Whole roc.az.gov is Cloudflare-walled to curl (403) → **fetch via a real browser (Playwright cf_clearance cookie)**; verified 200 / **47,235 "Current Active Commercial Contractor Licenses"**.
+- Cols: #, License No, Business Name, Doing Business As, Class, Class Detail, Class Type, Address, City, State, Zip, Qualifying Party, Issued Date, Expiration Date, Status. (No county/phone/website → derive county from ZIP 85xxx=Maricopa.)
+- Already commercial-only (dedicated file) → just geo-filter to Maricopa/Scottsdale. Sibling files: All / Dual / Residential / Pending-Applications / New-Licenses / Disciplinary-Actions (same dir, daily).
+- Alt scriptable path (no browser): `POST azroc.my.site.com/AZRoc/s/sfsites/aura` action `ARCP_ContractorSearch.getRecords` (search-driven, paced). Bulk ToS-clean fallback: ROC Public Records Request.
+
+## Land in `rentv_licensed_targets` role='Contractor' source cslb / az_roc. Both are effectively feed-first ($0); AZ just needs the browser hop for the WAF.
← f92ecac auto-save: 2026-07-30T21:21:20 (1 files) — specs/lenders.md · back to Rentv Licensed Targets · pipeline: DFPI escrow+CFL scraper (3532 rows) + attorney reu ba624d2 →