TK-16: add Lane County OR (41039) free priced-deed feed — 4th arcgis_sales pass

d503d96d3a170bedbedc728fa4eae0f4c23e451a · 2026-08-10 14:42:23 -0700 · Steve Abrams

Lane County AddressParcelSales MapServer 'Sales (last 3 years)' layer (~6.7k
priced deeds w/ deed no/type + situs + rolling window). Join-layer prefixed keys
(SalesLayerCityJoin_SalesforGISLayerAll_) handled like Yakima's qualified keys.
Verified e2e vs local usre DB: 1833 parcels + 1958 sale events + 1833 GIS links +
parcel_source(41039). tsc clean (my files). Local master, NOT pushed; prod
deploy + full backfill to Kamatera stay Steve-gated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files touched

M SOURCES.md
M src/ingest/parcels/arcgis_sales.ts
M src/ingest/parcels/engine.ts
M src/jobs/hourly_loop.ts

Diff

commit d503d96d3a170bedbedc728fa4eae0f4c23e451a
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Mon Aug 10 14:42:23 2026 -0700

    TK-16: add Lane County OR (41039) free priced-deed feed — 4th arcgis_sales pass
    
    Lane County AddressParcelSales MapServer 'Sales (last 3 years)' layer (~6.7k
    priced deeds w/ deed no/type + situs + rolling window). Join-layer prefixed keys
    (SalesLayerCityJoin_SalesforGISLayerAll_) handled like Yakima's qualified keys.
    Verified e2e vs local usre DB: 1833 parcels + 1958 sale events + 1833 GIS links +
    parcel_source(41039). tsc clean (my files). Local master, NOT pushed; prod
    deploy + full backfill to Kamatera stay Steve-gated.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 SOURCES.md                         |  4 +++-
 src/ingest/parcels/arcgis_sales.ts | 22 ++++++++++++++++++++++
 src/ingest/parcels/engine.ts       |  1 +
 src/jobs/hourly_loop.ts            |  1 +
 4 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/SOURCES.md b/SOURCES.md
index 55aa55a..37540a6 100644
--- a/SOURCES.md
+++ b/SOURCES.md
@@ -108,10 +108,12 @@ incl. 8 gated/blocked states → `BROKER-COVERAGE.md`. Adapters: `src/ingest/bro
 
 ## Free priced-DEED feeds — config-driven ArcGIS (`src/ingest/parcels/arcgis_sales.ts`)
 
-Each entry is a keyless ArcGIS REST layer that publishes real recorded SALE PRICE; one `SOURCES` config maps a page of features → parcel row + `parcel_event(event_type='sale')` with grantor/grantee/doc + a `source_url` deep-link. Registered in `parcel_source`, paginated over time by the hourly loop (`src/jobs/hourly_loop.ts`, `runSources` = the config's `cursorKey`). Live configs: `oregon-rlis` (41051/67/05), `spokane` (53063), `alameda` (06001), `deschutes` (41017), `crook` (41013), `sonoma` (06097), `colorado` (statewide composite `08*`), `maricopa` (04013), `thurston` (53067), `palm-beach` (12099), `klamath` (41035), `marion` (41047), `yakima` (53077).
+Each entry is a keyless ArcGIS REST layer that publishes real recorded SALE PRICE; one `SOURCES` config maps a page of features → parcel row + `parcel_event(event_type='sale')` with grantor/grantee/doc + a `source_url` deep-link. Registered in `parcel_source`, paginated over time by the hourly loop (`src/jobs/hourly_loop.ts`, `runSources` = the config's `cursorKey`). Live configs: `oregon-rlis` (41051/67/05), `spokane` (53063), `alameda` (06001), `deschutes` (41017), `crook` (41013), `sonoma` (06097), `colorado` (statewide composite `08*`), `maricopa` (04013), `thurston` (53067), `palm-beach` (12099), `klamath` (41035), `marion` (41047), `yakima` (53077), `lane` (41039).
 
 - **Klamath County OR (41035)** — added 2026-07-27 (TK-16). ArcGIS Online FeatureServer `https://services.arcgis.com/H6Mh1bySxR4oHx6x/arcgis/rest/services/KC_ParcelSales/FeatureServer/0` (item `f6314b12b8cc4c35899157172601bf59`, "KC_ParcelSales" = Klamath County Parcel Sales). `where=SALE_PRICE>1000` → **38,796** priced sales. Rich fields: `SALE_PRICE` (int), `SALE_DATE` (epoch-ms), `OWNER_NAME` (→ grantee), `MIN_SITUS_ADDRESS`+`FIRST_CITY_NAME`, `SUM_FINSQFT`/`SUM_BEDRMS`/`SUM_BATH`, `MIN_YRBLT`, `SUM_Tot_Appr` (total appraised → `total_value`), `SALEBK` (→ doc_number, internal double-spaces collapsed by `clean()`), `FIRST_PROP_ID` (→ apn/source_id), `FIRST_PCLCD` (→ use_desc). Server caps pages at 1000. Verified end-to-end against the local `usre` DB (1,000 parcels + 1,000 sale events + links + source registration on first batch).
 
 - **Marion County OR / Salem (41047)** — added 2026-07-27 (TK-16, pass 2). ArcGIS Online FeatureServer `https://services1.arcgis.com/sYGZnQPdJ0azuLyn/arcgis/rest/services/TaxParcel_Assessment/FeatureServer/0` (Marion County official org). `where=SALEPRICE>1000` → **96,115** priced sales. Rich fields: `SALEPRICE` (int), `INSTDATE` (instrument/deed date, epoch-ms → `last_sale_date`), `INSTNUM` (→ doc_number), `INSTTYPE` (deed type WD/B&S/DEED → doc_type), `OWNERNAME` (→ grantee), `SITUS` (situs address, often blank on land parcels), `YEARBUILT`, `LIVINGAREA` (→ sqft), `RMVTOTAL` (real market total value → `total_value`), `PROPCLASS`+`STCLSDESC` (→ use_desc, prefers the description). `TAXLOT` (→ apn/source_id). Deed dates span decades so the future-date guard passes all rows. Verified end-to-end against the local `usre` DB (1,000 parcels + sale events + GIS links + `parcel_source` registration on first batch); `tsc` clean.
 
 - **Yakima County WA (53077)** — added 2026-07-28 (TK-16, pass 3). Yakima County Assessor ArcGIS MapServer `https://gis.yakimawa.gov/arcgis/rest/services/Assessor/AssessorParcels/MapServer/0`. `where=DBO.Parcels.GROSS_SALE_PRICE>1000` → **56,030** priced sales. NOTE: this SQL-Server-backed MapServer returns attribute keys **fully-qualified** (`DBO.Parcels.*`), so `outFields` and `toRecords` both use the qualified names, and `orderBy` = `DBO.Parcels.OBJECTID`. Rich fields: `GROSS_SALE_PRICE` (double → `last_sale_price`), `SALE_DATE` (string `M/D/YYYY`, handled by `anyDate()` → `last_sale_date`), `EXCISE_NUMBER` (→ doc_number), `GRANTOR_NAME` (→ grantor), owner from `ORG_NAME` else `FIRST_NAME`+`LAST_NAME` (→ grantee/owner_name), `SITUS_ADDR`/`SITUS_CITY`/`SITUS_ZIP` (situs "UNASSIGNED" mapped to null), `USE_CODE` (→ use_desc), `YEAR_BLT`, `MAIN_SQFT` (→ sqft), `BEDROOMS`/`FULL_BATH`, `MKT_LAND`+`MKT_IMPVT` (summed → `total_value`), `ASSESSOR_NO` (→ apn/source_id). Empty fields come back as `" "` (single space) and are trimmed to null by `s()`/`clean()`. Verified end-to-end against the local `usre` DB (1,000 parcels + sale events + GIS links + `parcel_source` registration on first batch); `tsc` clean.
+
+- **Lane County OR / Eugene (41039)** — added 2026-08-10 (TK-16, pass 4). Lane County "AddressParcelSales" ArcGIS MapServer, layer 1 "Sales (last 3 years)" `https://lcmaps.lanecounty.org/arcgis/rest/services/AT/AddressParcelSales/MapServer/1` (HTTPS only — the HTTP endpoint returns an Esri "Application Error" page). `where=SalesLayerCityJoin_SalesforGISLayerAll_sale_price>1000` → **~6,731** priced sales (a rolling last-3-years window, so it stays fresh rather than deep-historical). NOTE: this is a JOIN layer, so every attribute key is prefixed `SalesLayerCityJoin_SalesforGISLayerAll_` (bound to a local `P` const in `toRecords`; the un-prefixed `DeedDate` is the exception). Rich fields: `sale_price` (int → `last_sale_price`), `DeedDate` (epoch-ms → `last_sale_date`), `deed_transfer_no` (space-padded → doc_number, trimmed by `clean()`), `deed_type` (WD/PR/... → doc_type), `situs_address`/`city`/`zip`, `stat_class` (property class → use_desc), `maplot` (13-char taxlot → apn/source_id — MUST be the first `outField` since the per-parcel GIS deep-link is `where=<first>='<apn>'`). `reject_code` (ratio-study/arm's-length flag e.g. "Sale Adjusted") is kept, not filtered, and preserved in the row's raw-source JSON for provenance. Verified end-to-end against the local `usre` DB; `tsc` clean.
diff --git a/src/ingest/parcels/arcgis_sales.ts b/src/ingest/parcels/arcgis_sales.ts
index 78e5199..903d06f 100644
--- a/src/ingest/parcels/arcgis_sales.ts
+++ b/src/ingest/parcels/arcgis_sales.ts
@@ -323,6 +323,28 @@ const SOURCES: Record<string, Source> = {
         year: num(a['DBO.Parcels.YEAR_BLT']), totalValue: (land + impvt) || null, use: clean(a['DBO.Parcels.USE_CODE']),
         grantor: clean(a['DBO.Parcels.GRANTOR_NAME']), grantee: owner, docNum: s(a['DBO.Parcels.EXCISE_NUMBER']), raw: a } as Rec; }).filter(Boolean) as Rec[],
   },
+  lane: {
+    key: 'lane', label: 'Lane County OR (Eugene)', cursorKey: 'lane', geometry: false, orderBy: 'OBJECTID', pageSize: 2000,
+    // Lane County AddressParcelSales MapServer, "Sales (last 3 years)" layer — a JOIN layer, so every
+    // attribute key carries a long SalesLayerCityJoin_SalesforGISLayerAll_ prefix (same class of quirk
+    // as Yakima's DBO.Parcels.* qualified keys). Real recorded sale_price(int) + DeedDate(epoch-ms) +
+    // deed_transfer_no(doc, space-padded) + deed_type(WD/PR/...) + situs/city/zip + stat_class(use) +
+    // reject_code (ratio-study/arm's-length flag — kept, not filtered; preserved in raw for provenance).
+    layer: 'https://lcmaps.lanecounty.org/arcgis/rest/services/AT/AddressParcelSales/MapServer/1',
+    where: 'SalesLayerCityJoin_SalesforGISLayerAll_sale_price>1000',
+    // maplot (taxlot key) MUST be first — engine derives the per-parcel GIS deep-link as where=<first>='<apn>' and apn=maplot.
+    outFields: 'SalesLayerCityJoin_SalesforGISLayerAll_maplot,SalesLayerCityJoin_SalesforGISLayerAll_account,SalesLayerCityJoin_SalesforGISLayerAll_sale_price,DeedDate,SalesLayerCityJoin_SalesforGISLayerAll_deed_transfer_no,SalesLayerCityJoin_SalesforGISLayerAll_deed_type,SalesLayerCityJoin_SalesforGISLayerAll_reject_code,SalesLayerCityJoin_SalesforGISLayerAll_situs_address,SalesLayerCityJoin_SalesforGISLayerAll_city,SalesLayerCityJoin_SalesforGISLayerAll_zip,SalesLayerCityJoin_SalesforGISLayerAll_stat_class',
+    fieldMap: { source_id: 'SalesLayerCityJoin_SalesforGISLayerAll_maplot', address: 'SalesLayerCityJoin_SalesforGISLayerAll_situs_address',
+      city: 'SalesLayerCityJoin_SalesforGISLayerAll_city', zip: 'SalesLayerCityJoin_SalesforGISLayerAll_zip',
+      use_desc: 'SalesLayerCityJoin_SalesforGISLayerAll_stat_class', doc_type: 'SalesLayerCityJoin_SalesforGISLayerAll_deed_type',
+      last_sale_price: 'SalesLayerCityJoin_SalesforGISLayerAll_sale_price', last_sale_date: 'DeedDate' },
+    toRecords: (fs) => fs.map(f => { const a = f.attributes; const P = 'SalesLayerCityJoin_SalesforGISLayerAll_';
+      const apn = s(a[P + 'maplot']); if (!apn) return null;   // maplot only — keeps the deep-link where-clause valid
+      return { fips: '41039', apn, price: anyPrice(a[P + 'sale_price']), saleDate: anyDate(a.DeedDate),
+        address: clean(a[P + 'situs_address']), city: clean(a[P + 'city']), zip: s(a[P + 'zip']),
+        use: clean(a[P + 'stat_class']),
+        docNum: clean(a[P + 'deed_transfer_no']), docType: clean(a[P + 'deed_type']), raw: a } as Rec; }).filter(Boolean) as Rec[],
+  },
 };
 
 async function fetchPage(src: Source, offset: number): Promise<any[]> {
diff --git a/src/ingest/parcels/engine.ts b/src/ingest/parcels/engine.ts
index b782bf2..800045b 100644
--- a/src/ingest/parcels/engine.ts
+++ b/src/ingest/parcels/engine.ts
@@ -37,6 +37,7 @@ const ADAPTERS: Record<string, () => Promise<{ run: () => Promise<{ upserted: nu
   klamath: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('klamath') }; },
   marion: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('marion') }; },
   yakima: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('yakima') }; },
+  lane: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('lane') }; },
 };
 
 async function main() {
diff --git a/src/jobs/hourly_loop.ts b/src/jobs/hourly_loop.ts
index 2486f0e..98f90b9 100644
--- a/src/jobs/hourly_loop.ts
+++ b/src/jobs/hourly_loop.ts
@@ -73,6 +73,7 @@ const JOBS: Job[] = [
   { name: 'deeds-klam', script: 'src/ingest/parcels/engine.ts',     args: ['klamath'],     minIntervalHours: 4, runSources: ['klamath'] },
   { name: 'deeds-marion',script:'src/ingest/parcels/engine.ts',     args: ['marion'],      minIntervalHours: 3, runSources: ['marion'] },
   { name: 'deeds-yakima',script:'src/ingest/parcels/engine.ts',     args: ['yakima'],      minIntervalHours: 3, runSources: ['yakima'] },
+  { name: 'deeds-lane',  script: 'src/ingest/parcels/engine.ts',     args: ['lane'],        minIntervalHours: 4, runSources: ['lane'] },
   // TK-5: full San Diego County (1.09M) via SANDAG — zoning + assessed value; was in engine but missing from rotation.
   { name: 'parcels-sd-sandag', script: 'src/ingest/parcels/engine.ts', args: ['san-diego-sandag'], minIntervalHours: 2, runSources: ['san_diego_sandag'] },
   // TK-5: Santa Clara County CA (San Jose, 493k) — situs address; was in engine but missing from rotation.

← f1b3540 usre render: every agent shows firm tie + phone/firm-site/ag · back to Nationalrealestate · TK-16: add Jackson County OR (41029) priced-deed feed + curl 04c5f48 →