source runbooks for toll-brothers + lennar live adapters

4ba331cd736427d6e64a3207ea5078ef1d254c67 · 2026-07-22 11:57:00 -0700 · Steve Abrams

Files touched

A docs/source-runbooks/lennar.md
A docs/source-runbooks/toll-brothers.md

Diff

commit 4ba331cd736427d6e64a3207ea5078ef1d254c67
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Wed Jul 22 11:57:00 2026 -0700

    source runbooks for toll-brothers + lennar live adapters
---
 docs/source-runbooks/lennar.md        | 21 +++++++++++++++++++++
 docs/source-runbooks/toll-brothers.md | 15 +++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/docs/source-runbooks/lennar.md b/docs/source-runbooks/lennar.md
new file mode 100644
index 0000000..f549559
--- /dev/null
+++ b/docs/source-runbooks/lennar.md
@@ -0,0 +1,21 @@
+# Source runbook — lennar-site
+
+**Method:** CRAWL of robots-permitted `/new-homes/` community pages; each page embeds a
+fully-hydrated Apollo state in `__NEXT_DATA__` with per-home `HomesiteType` records
+(price, beds, baths+half, sqft, status, address, lat/lon, plan ref) and a
+`CommunityType` rollup (name, city ref, stateCode, zipCode, monthlyHoaFee, coords).
+Recon 2026-07-22: cleanest per-home source of the top 10 (difficulty 2).
+
+- Adapter: `collectors/lennar` (v1.1.0) · registry key `lennar-site`
+- Discovery: `api/images/sitemapfe.xml` → community URLs (exactly
+  `/new-homes/{state}/{metro}/{city}/{community}`), sorted for determinism.
+- Batch control: `LENNAR_PAGE_LIMIT` (default 8). Full national sweep =
+  `LENNAR_PAGE_LIMIT=5000` (~4.4k pages ≈ 7h at 10 req/min). Change detection makes
+  re-sweeps cheap — unchanged pages stage nothing.
+- Status mapping: `UNDER_CONSTRUCTION` → UNDER_CONSTRUCTION; move-in-ready variants →
+  MOVE_IN_READY; `UNDEFINED` → null (raw kept in availabilityStatus — never guessed).
+  SOLD/CLOSED homesites are skipped entirely.
+- First controlled batch (2026-07-22): 8 pages → 8 communities + 172 homes, all priced,
+  0 rejections. Full sweep launched same day.
+- robots constraints: only fragment paths are disallowed (`/*included`, `/*faq` etc.) —
+  the adapter touches none of them.
diff --git a/docs/source-runbooks/toll-brothers.md b/docs/source-runbooks/toll-brothers.md
new file mode 100644
index 0000000..7f13991
--- /dev/null
+++ b/docs/source-runbooks/toll-brothers.md
@@ -0,0 +1,15 @@
+# Source runbook — toll-brothers-site
+
+**Method:** FEED (their own sitemap lists `/api/v1/search/homesearchV5.json` — a public,
+unauthenticated community feed). Recon 2026-07-22: no bot challenge, no Crawl-delay.
+
+- Adapter: `collectors/toll-brothers` (v1.0.0) · registry key `toll-brothers-site`
+- Wave 1 (LIVE, 2026-07-22): community ingest — 337 active communities published with
+  coords, county, metro, evidence per field. `isFuture` communities skipped.
+- Wave 2 (planned): per-home QMI ingest via the ~11k sitemap-listed
+  `/new-quick-move-in-homes/...` URLs (robots-allowed), volume-controlled.
+- robots constraints honored: never fetch `/siteplans/`, `/nse`, `/compare`,
+  `/new-homes`, `/sitesearch`.
+- Run: `pnpm --filter @spechomes/workers pipeline -- --adapter=toll-brothers-site`
+- Notes: feed carries no zip/street per community → those fields publish null
+  (never guessed); zip enrichment arrives with wave-2 QMI pages.

← 45f9ee2 lennar live adapter (172 real homes from 8 pages, Apollo sta · back to Homesonspec · auto-save: 2026-07-22T12:13:58 (1 files) — collectors/dream- 0e090e4 →