← back to Homesonspec
pm2 ecosystem config, 9 ADRs, data-rights policy, source runbook template, README
4dce3d29fb1eb164a6a792f9368ecbdb468c26df · 2026-07-22 11:42:56 -0700 · Steve Abrams
Files touched
A README.mdA docs/architecture/ADRS.mdA docs/data-rights/POLICY.mdA docs/source-runbooks/meridian-homes.mdA ecosystem.config.js
Diff
commit 4dce3d29fb1eb164a6a792f9368ecbdb468c26df
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Wed Jul 22 11:42:56 2026 -0700
pm2 ecosystem config, 9 ADRs, data-rights policy, source runbook template, README
---
README.md | 57 +++++++++++++++++++++++++++++++
docs/architecture/ADRS.md | 62 ++++++++++++++++++++++++++++++++++
docs/data-rights/POLICY.md | 39 +++++++++++++++++++++
docs/source-runbooks/meridian-homes.md | 18 ++++++++++
ecosystem.config.js | 26 ++++++++++++++
5 files changed, 202 insertions(+)
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..12ed3c5
--- /dev/null
+++ b/README.md
@@ -0,0 +1,57 @@
+# SpecHomes.com
+
+**Every new home. Every builder. One search. Verified from the source.**
+
+Nationwide search + comparison platform for newly constructed homes: move-in-ready and
+under-construction inventory, communities, floor plans, and incentives — permission-aware,
+evidence-backed, and verification-labeled.
+
+## Status
+
+Stage 1–2 complete: working marketplace over synthetic demonstration inventory
+(3 markets, 4 fictional builders, ~113 homes, all labeled DEMONSTRATION) plus a fully
+proven ingestion pipeline (fixture adapter → snapshots → staged → validated → review →
+published). Top-10 national builder acquisition in progress (source registry + recon).
+
+## Run it
+
+```sh
+pnpm install
+createdb spechomes && psql -d spechomes -c 'CREATE EXTENSION pg_trgm'
+cp .env.example .env
+pnpm db:migrate && pnpm db:seed
+pnpm pipeline -- --adapter=meridian-homes-fixtures # fixture pipeline e2e
+pnpm dev:web # consumer app → http://localhost:3100
+pnpm dev:admin # admin app → http://localhost:3101 (Basic Auth, BASIC_AUTH env)
+```
+
+Tests: `pnpm -r test` (unit) ·
+`pnpm --filter @spechomes/workers test:integration` (pipeline vs spechomes_test) ·
+`pnpm e2e` (Playwright, boots both apps).
+
+## Architecture
+
+pnpm monorepo — see `docs/architecture/ADRS.md` for the nine decision records.
+
+```
+apps/web consumer Next.js app (:3100) apps/admin protected admin (:3101)
+apps/workers pipeline stages + CLI + pg-boss daemon
+packages/ database (Prisma) · schemas (zod) · validation (rule engine) ·
+ search · publisher (ONLY writer of published tables) · shared · shared-ui
+collectors/ common adapter contract + per-builder adapters
+fixtures/ raw HTML pages + hand-authored expected extraction records
+docs/ product · data-rights (READ FIRST) · architecture · source-runbooks
+```
+
+**The invariant that matters:** crawlers never write published tables. Data flows
+source → RawSnapshot → StagedRecord (+ per-field SourceEvidence) → deterministic
+validators (ValidationEvent, verdict) → human review when flagged →
+`@spechomes/publisher` — the single bridge to consumer-facing inventory. Missing values
+are null, never guessed; every published fact traces to evidence.
+
+## Data rights
+
+`docs/data-rights/POLICY.md` is binding: facts only by default, no media without
+registry-recorded rights, no access-control circumvention, honest User-Agent, robots
+respected, removal requests honored. Verification labels derive from how data was
+collected and can never overstate it.
diff --git a/docs/architecture/ADRS.md b/docs/architecture/ADRS.md
new file mode 100644
index 0000000..7e5320e
--- /dev/null
+++ b/docs/architecture/ADRS.md
@@ -0,0 +1,62 @@
+# Architecture Decision Records
+
+## ADR-001 — Prisma 6 + plain lat/lon, PostGIS deferred
+Prisma 6.19.3 (fleet precedent: govarbitrage). Coordinates are `Decimal(9,6)` with btree
+indexes; bbox filters use `lat/lon BETWEEN`, distance sort uses haversine in `$queryRaw`.
+All geo math funnels through `packages/shared/src/geo.ts`, so adopting PostGIS later is
+one module + one migration. Avoids Prisma's `Unsupported("geography")` friction at a
+scale (hundreds of homes per metro) where it buys nothing. Only extension required:
+`pg_trgm` (location text resolution).
+
+## ADR-002 — pnpm workspaces, no turborepo
+Two Next.js apps + workers + small packages don't justify turbo's config surface. Shared
+packages are pure-TS source (`main: ./src/index.ts`) consumed via `transpilePackages`;
+workers consume TS directly through tsx. Caveat learned in build: intra-package relative
+imports must be extensionless (`./geo`, not `./geo.js`) or Turbopack resolves the module
+to zero exports.
+
+## ADR-003 — Separate admin app behind HTTP Basic Auth
+The spec requires a "protected administration application." A distinct Next.js app on
+:3101 keeps the review/ops surface out of the consumer bundle and puts one
+`middleware.ts` Basic-Auth gate (BASIC_AUTH env) in front of everything. Shared UI
+(badges, created-date chips) lives in `packages/shared-ui`.
+
+## ADR-004 — pg-boss over Redis/BullMQ; stages are pure functions
+Jobs live in Postgres (pg-boss) — no extra infra, and "what ran when" is queryable in
+the same database as the pipeline audit trail. Every pipeline stage
+(fetch/extract/validate/publish) is a plain async function; the queue only transports
+invocations. The CLI (`pnpm pipeline`) and integration tests call stages directly, so
+the queue is never required for correctness.
+
+## ADR-005 — Evidence-first data model; deterministic validators gate publish
+Every extracted field is a `FieldValue {value|null, raw, evidenceText, sourceUrl,
+confidence}`. `value: null` means "source did not state it" — values are never inferred.
+zod checks shape; the rule engine (`packages/validation`) checks truth-plausibility and
+emits `ValidationEvent` rows; verdict `publishable | needs_review | rejected` is folded
+deterministically (any failed error → rejected; any failed review → needs_review). The
+LLM normalization hook exists but can never bless a record into publishing.
+
+## ADR-006 — Organic-only ordering; objective filters (Fair Housing)
+No paid-placement concept exists anywhere in the v1 schema or code, so organic order
+cannot be contaminated. All search filters are objective attributes (price, beds,
+status, HOA, school district name, age-restriction flag). No subjective neighborhood
+dimensions exist in the schema at all — this is deliberate Fair Housing hygiene, not an
+omission.
+
+## ADR-007 — publisher package is the only writer of published tables
+`@spechomes/publisher.publishStagedRecord` is the single bridge from staged →
+published, and it throws on any staged status other than VALIDATED (validators passed)
+or APPROVED (human review). Collectors and extractors end at `StagedRecord`. The
+integration suite asserts this invariant directly.
+
+## ADR-008 — Verification label derives from collection method
+`SourceRegistry.collectionMethod` → label: FEED → VERIFIED_FROM_BUILDER, CRAWL →
+BUILDER_WEBSITE, MANUAL → BUILDER_SUBMITTED, FIXTURE/SYNTHETIC → DEMONSTRATION (+
+`isDemo: true`). A source can never claim a stronger verification than how its data was
+actually collected.
+
+## ADR-009 — Leaflet + OSM behind a props-only MapView contract
+$0, no keys. `MapView` accepts only {markers, center, zoom, onMoveEnd} so a commercial
+provider can swap in without touching callers. MapView is deliberately excluded from the
+shared-ui barrel (react-leaflet touches `window` at module scope) and imported by
+subpath behind `next/dynamic ssr:false`.
diff --git a/docs/data-rights/POLICY.md b/docs/data-rights/POLICY.md
new file mode 100644
index 0000000..534c7d6
--- /dev/null
+++ b/docs/data-rights/POLICY.md
@@ -0,0 +1,39 @@
+# SpecHomes Data Rights Policy
+
+A builder advertising a home for sale does not license its website for wholesale
+copying. Individual facts (address, price, sqft, beds, availability) receive thin
+copyright protection (Feist Publications, Inc. v. Rural Telephone Service Co., 499 U.S.
+340 (1991); 17 U.S.C. §§ 101–122), but photographs, renderings, floor-plan drawings,
+community maps, written descriptions, videos, and database arrangements can be
+protected — and site terms can restrict automated collection.
+
+## Source hierarchy (strongest first)
+1. Builder-provided feeds and APIs
+2. Direct builder partnerships
+3. Affiliate or syndication feeds
+4. Permissioned crawling
+5. Carefully limited collection of public factual data
+
+## Hard prohibitions
+- Never circumvent logins, CAPTCHAs, rate limits, or access controls. A bot challenge
+ means STOP and record the source as blocked.
+- Never copy builder photos, descriptions, floor-plan drawings, or renderings without
+ registry-recorded rights (`SourceRegistry.mediaRights`). Default is `NONE` — facts
+ only, link back to the source.
+- Never ignore source terms or removal requests.
+- Never present stale inventory as verified — freshness states + last-verified
+ timestamps are consumer-facing.
+- The LLM must never infer missing prices, addresses, availability, or incentives.
+ Missing = null, always.
+
+## Registry obligations
+Every source records: terms URL, permissions, attribution requirements, collection
+method, media rights, rate limits, and health — before any collection runs. Collection
+respects robots.txt and uses an honest User-Agent with a contact address. Publishing
+stops automatically when a source degrades (health gate in the pipeline).
+
+## Consumer-facing honesty
+Verification labels derive from the collection method (ADR-008). Demonstration data is
+always labeled DEMONSTRATION and `isDemo: true`, and is never presented as a production
+claim. Every listing carries a correction-report control, and correction reports land
+in an auditable review queue.
diff --git a/docs/source-runbooks/meridian-homes.md b/docs/source-runbooks/meridian-homes.md
new file mode 100644
index 0000000..0968aa5
--- /dev/null
+++ b/docs/source-runbooks/meridian-homes.md
@@ -0,0 +1,18 @@
+# Source runbook — meridian-homes-fixtures
+
+**What it is:** the reference FIXTURE adapter. Meridian Homes is a fictional builder;
+its "site" is 5 local HTML files in `fixtures/raw-pages/meridian-homes/`. No live
+fetching exists for this source.
+
+- Registry key: `meridian-homes-fixtures` · method FIXTURE · media rights NONE
+- Adapter: `collectors/meridian-homes` (extractor v1.0.0), contract-pinned by
+ `fixtures/expected-records/meridian-homes.json` and `src/extract.test.ts`
+- Run: `DATABASE_URL=… pnpm --filter @spechomes/workers pipeline -- --adapter=meridian-homes-fixtures`
+- Page order matters: the community page sorts first (01-…), so the publish stage
+ creates the Community row before homes validate FK integrity.
+- Known quirks encoded in fixtures: `03-home-124-silverleaf.html` has no price
+ (proves null-never-guessed); `05-incentives.html` has one incentive without an
+ expiration (proves the "expiration not provided" label path).
+
+Use this runbook as the template for real builder runbooks: registry facts, adapter
+path, run command, page-order requirements, and per-source quirks.
diff --git a/ecosystem.config.js b/ecosystem.config.js
new file mode 100644
index 0000000..30181a2
--- /dev/null
+++ b/ecosystem.config.js
@@ -0,0 +1,26 @@
+// pm2 process definitions. Production start assumes `pnpm build` has run.
+module.exports = {
+ apps: [
+ {
+ name: "spechomes-web",
+ cwd: __dirname,
+ script: "pnpm",
+ args: "--filter @spechomes/web start",
+ env: { NODE_ENV: "production", PORT: "3100" },
+ },
+ {
+ name: "spechomes-admin",
+ cwd: __dirname,
+ script: "pnpm",
+ args: "--filter @spechomes/admin start",
+ env: { NODE_ENV: "production", PORT: "3101" },
+ },
+ {
+ name: "spechomes-workers",
+ cwd: __dirname,
+ script: "pnpm",
+ args: "--filter @spechomes/workers start",
+ env: { NODE_ENV: "production" },
+ },
+ ],
+};
← 0ba8822 integration tests (pipeline e2e, only-writer invariant, revi
·
back to Homesonspec
·
polite LiveFetcher (rate-limited, never circumvents blocks) 1413161 →