← back to Nationalrealestate

TK-16: add Jackson County OR (41029) priced-deed feed + curl fetch fallback — 5th arcgis_sales pass

04c5f482bcd9eb1f52946027f73a68e18f5a604e · 2026-08-10 15:02:41 -0700 · Steve Abrams

Jackson County Assessment LatestSales layer (58,400 priced deeds w/ grantor+grantee
names, doc no/type, situs, use, yr/sqft). Rich clean-key feed.

Also hardens the arcgis_sales fetcher: spatial.jacksoncountyor.gov emits a non-RFC
response header that undici (fetch) AND Node's http parser (even insecureHTTPParser)
reject; added a curl fallback (single-argv, no shell; --compressed) triggered only on
that specific 'Invalid header token' error — benefits any future non-compliant county
server. pageSize=1000 to match server maxRecordCount (avoids the la-county early-break).

Verified e2e vs local usre DB: 4000 parcels + 3997 sale events (paged past 1000,
cursor advancing), 8 deed types, parcel_source(41029) registered. tsc clean (my files).
Local master, NOT pushed; prod deploy + full backfill stay Steve-gated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files touched

• M SOURCES.md
• M src/ingest/parcels/arcgis_sales.ts
• M src/ingest/parcels/engine.ts
• M src/jobs/hourly_loop.ts

Diff

commit 04c5f482bcd9eb1f52946027f73a68e18f5a604e
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Mon Aug 10 15:02:41 2026 -0700

    TK-16: add Jackson County OR (41029) priced-deed feed + curl fetch fallback — 5th arcgis_sales pass
    
    Jackson County Assessment LatestSales layer (58,400 priced deeds w/ grantor+grantee
    names, doc no/type, situs, use, yr/sqft). Rich clean-key feed.
    
    Also hardens the arcgis_sales fetcher: spatial.jacksoncountyor.gov emits a non-RFC
    response header that undici (fetch) AND Node's http parser (even insecureHTTPParser)
    reject; added a curl fallback (single-argv, no shell; --compressed) triggered only on
    that specific 'Invalid header token' error — benefits any future non-compliant county
    server. pageSize=1000 to match server maxRecordCount (avoids the la-county early-break).
    
    Verified e2e vs local usre DB: 4000 parcels + 3997 sale events (paged past 1000,
    cursor advancing), 8 deed types, parcel_source(41029) registered. tsc clean (my files).
    Local master, NOT pushed; prod deploy + full backfill stay Steve-gated.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 SOURCES.md                         |  4 ++-
 src/ingest/parcels/arcgis_sales.ts | 53 +++++++++++++++++++++++++++++++++++---
 src/ingest/parcels/engine.ts       |  1 +
 src/jobs/hourly_loop.ts            |  1 +
 4 files changed, 54 insertions(+), 5 deletions(-)

diff --git a/SOURCES.md b/SOURCES.md
index 37540a6..f10c4cf 100644
--- a/SOURCES.md
+++ b/SOURCES.md
@@ -108,7 +108,7 @@ incl. 8 gated/blocked states → `BROKER-COVERAGE.md`. Adapters: `src/ingest/bro
 
 ## Free priced-DEED feeds — config-driven ArcGIS (`src/ingest/parcels/arcgis_sales.ts`)
 
-Each entry is a keyless ArcGIS REST layer that publishes real recorded SALE PRICE; one `SOURCES` config maps a page of features → parcel row + `parcel_event(event_type='sale')` with grantor/grantee/doc + a `source_url` deep-link. Registered in `parcel_source`, paginated over time by the hourly loop (`src/jobs/hourly_loop.ts`, `runSources` = the config's `cursorKey`). Live configs: `oregon-rlis` (41051/67/05), `spokane` (53063), `alameda` (06001), `deschutes` (41017), `crook` (41013), `sonoma` (06097), `colorado` (statewide composite `08*`), `maricopa` (04013), `thurston` (53067), `palm-beach` (12099), `klamath` (41035), `marion` (41047), `yakima` (53077), `lane` (41039).
+Each entry is a keyless ArcGIS REST layer that publishes real recorded SALE PRICE; one `SOURCES` config maps a page of features → parcel row + `parcel_event(event_type='sale')` with grantor/grantee/doc + a `source_url` deep-link. Registered in `parcel_source`, paginated over time by the hourly loop (`src/jobs/hourly_loop.ts`, `runSources` = the config's `cursorKey`). Live configs: `oregon-rlis` (41051/67/05), `spokane` (53063), `alameda` (06001), `deschutes` (41017), `crook` (41013), `sonoma` (06097), `colorado` (statewide composite `08*`), `maricopa` (04013), `thurston` (53067), `palm-beach` (12099), `klamath` (41035), `marion` (41047), `yakima` (53077), `lane` (41039), `jackson` (41029).
 
 - **Klamath County OR (41035)** — added 2026-07-27 (TK-16). ArcGIS Online FeatureServer `https://services.arcgis.com/H6Mh1bySxR4oHx6x/arcgis/rest/services/KC_ParcelSales/FeatureServer/0` (item `f6314b12b8cc4c35899157172601bf59`, "KC_ParcelSales" = Klamath County Parcel Sales). `where=SALE_PRICE>1000` → **38,796** priced sales. Rich fields: `SALE_PRICE` (int), `SALE_DATE` (epoch-ms), `OWNER_NAME` (→ grantee), `MIN_SITUS_ADDRESS`+`FIRST_CITY_NAME`, `SUM_FINSQFT`/`SUM_BEDRMS`/`SUM_BATH`, `MIN_YRBLT`, `SUM_Tot_Appr` (total appraised → `total_value`), `SALEBK` (→ doc_number, internal double-spaces collapsed by `clean()`), `FIRST_PROP_ID` (→ apn/source_id), `FIRST_PCLCD` (→ use_desc). Server caps pages at 1000. Verified end-to-end against the local `usre` DB (1,000 parcels + 1,000 sale events + links + source registration on first batch).
 
@@ -117,3 +117,5 @@ Each entry is a keyless ArcGIS REST layer that publishes real recorded SALE PRIC
 - **Yakima County WA (53077)** — added 2026-07-28 (TK-16, pass 3). Yakima County Assessor ArcGIS MapServer `https://gis.yakimawa.gov/arcgis/rest/services/Assessor/AssessorParcels/MapServer/0`. `where=DBO.Parcels.GROSS_SALE_PRICE>1000` → **56,030** priced sales. NOTE: this SQL-Server-backed MapServer returns attribute keys **fully-qualified** (`DBO.Parcels.*`), so `outFields` and `toRecords` both use the qualified names, and `orderBy` = `DBO.Parcels.OBJECTID`. Rich fields: `GROSS_SALE_PRICE` (double → `last_sale_price`), `SALE_DATE` (string `M/D/YYYY`, handled by `anyDate()` → `last_sale_date`), `EXCISE_NUMBER` (→ doc_number), `GRANTOR_NAME` (→ grantor), owner from `ORG_NAME` else `FIRST_NAME`+`LAST_NAME` (→ grantee/owner_name), `SITUS_ADDR`/`SITUS_CITY`/`SITUS_ZIP` (situs "UNASSIGNED" mapped to null), `USE_CODE` (→ use_desc), `YEAR_BLT`, `MAIN_SQFT` (→ sqft), `BEDROOMS`/`FULL_BATH`, `MKT_LAND`+`MKT_IMPVT` (summed → `total_value`), `ASSESSOR_NO` (→ apn/source_id). Empty fields come back as `" "` (single space) and are trimmed to null by `s()`/`clean()`. Verified end-to-end against the local `usre` DB (1,000 parcels + sale events + GIS links + `parcel_source` registration on first batch); `tsc` clean.
 
 - **Lane County OR / Eugene (41039)** — added 2026-08-10 (TK-16, pass 4). Lane County "AddressParcelSales" ArcGIS MapServer, layer 1 "Sales (last 3 years)" `https://lcmaps.lanecounty.org/arcgis/rest/services/AT/AddressParcelSales/MapServer/1` (HTTPS only — the HTTP endpoint returns an Esri "Application Error" page). `where=SalesLayerCityJoin_SalesforGISLayerAll_sale_price>1000` → **~6,731** priced sales (a rolling last-3-years window, so it stays fresh rather than deep-historical). NOTE: this is a JOIN layer, so every attribute key is prefixed `SalesLayerCityJoin_SalesforGISLayerAll_` (bound to a local `P` const in `toRecords`; the un-prefixed `DeedDate` is the exception). Rich fields: `sale_price` (int → `last_sale_price`), `DeedDate` (epoch-ms → `last_sale_date`), `deed_transfer_no` (space-padded → doc_number, trimmed by `clean()`), `deed_type` (WD/PR/... → doc_type), `situs_address`/`city`/`zip`, `stat_class` (property class → use_desc), `maplot` (13-char taxlot → apn/source_id — MUST be the first `outField` since the per-parcel GIS deep-link is `where=<first>='<apn>'`). `reject_code` (ratio-study/arm's-length flag e.g. "Sale Adjusted") is kept, not filtered, and preserved in the row's raw-source JSON for provenance. Verified end-to-end against the local `usre` DB; `tsc` clean.
+
+- **Jackson County OR / Medford (41029)** — added 2026-08-10 (TK-16, pass 5). Jackson County Assessment "LatestSales" FeatureServer `https://spatial.jacksoncountyor.gov/arcgis/rest/services/Assessment/LatestSales/FeatureServer/0` (HTTPS only — HTTP 302-redirects). `where=SalesPrice>1000` → **58,400** priced sales. An unusually rich, CLEAN (unqualified-key) layer: `SalesPrice` (double → `last_sale_price`), `SalesDate` (epoch-ms → `last_sale_date`), `DocumentNumber` (→ doc_number, e.g. `2025-18743`), `DocumentTypeDescription` (deed type e.g. "BARGAIN & SALE" → doc_type; null on some rows), `Grantor`+`Grantee` NAMES (both real), `SiteAddress`/`SiteCity`, `StatClassDescription` (→ use_desc), `YearBuilt`, `SquareFeet` (→ sqft), `maptaxlot` (→ apn/source_id — first `outField` for the deep-link). Also carries `DocumentURL` (recorded-instrument image link) preserved in raw. Verified end-to-end against the local `usre` DB; `tsc` clean.
diff --git a/src/ingest/parcels/arcgis_sales.ts b/src/ingest/parcels/arcgis_sales.ts
index 903d06f..7efcbfb 100644
--- a/src/ingest/parcels/arcgis_sales.ts
+++ b/src/ingest/parcels/arcgis_sales.ts
@@ -14,6 +14,8 @@
  *
  *   npm run ingest:parcels -- oregon-rlis | spokane | alameda
  */
+import { execFile } from 'node:child_process';
+import { promisify } from 'node:util';
 import { query } from '../../../db/pool.ts';
 import { openRun, closeRun } from '../run.ts';
 import { upsertParcels, normAddress, registerParcelSource, registerSourceFieldMap, type ParcelUpsertRow } from './upsert.ts';
@@ -345,8 +347,53 @@ const SOURCES: Record<string, Source> = {
         use: clean(a[P + 'stat_class']),
         docNum: clean(a[P + 'deed_transfer_no']), docType: clean(a[P + 'deed_type']), raw: a } as Rec; }).filter(Boolean) as Rec[],
   },
+  jackson: {
+    key: 'jackson', label: 'Jackson County OR (Medford)', cursorKey: 'jackson', geometry: false, orderBy: 'OBJECTID', pageSize: 1000, // server maxRecordCount=1000; pageSize>that early-breaks the loop after one page (same as la-county)
+    // Jackson County Assessment "LatestSales" FeatureServer — an unusually rich, clean (unqualified-key)
+    // priced-deed layer: SalesPrice(double)+SalesDate(epoch-ms)+DocumentNumber+DocumentTypeDescription
+    // (deed type e.g. BARGAIN & SALE)+Grantor+Grantee NAMES+SiteAddress/City+StatClassDescription(use)+
+    // YearBuilt+SquareFeet. maptaxlot is the parcel key. HTTPS only.
+    layer: 'https://spatial.jacksoncountyor.gov/arcgis/rest/services/Assessment/LatestSales/FeatureServer/0',
+    where: 'SalesPrice>1000',
+    // maptaxlot MUST be first — engine derives the per-parcel GIS deep-link as where=<first>='<apn>'.
+    outFields: 'maptaxlot,SalesPrice,SalesDate,DocumentNumber,DocumentTypeDescription,SiteAddress,SiteCity,StatClassDescription,Grantor,Grantee,YearBuilt,SquareFeet',
+    fieldMap: { source_id: 'maptaxlot', address: 'SiteAddress', city: 'SiteCity', use_desc: 'StatClassDescription',
+      sqft: 'SquareFeet', year_built: 'YearBuilt', doc_type: 'DocumentTypeDescription',
+      last_sale_price: 'SalesPrice', last_sale_date: 'SalesDate' },
+    toRecords: (fs) => fs.map(f => { const a = f.attributes; const apn = s(a.maptaxlot); if (!apn) return null;
+      return { fips: '41029', apn, price: anyPrice(a.SalesPrice), saleDate: anyDate(a.SalesDate),
+        address: clean(a.SiteAddress), city: clean(a.SiteCity), use: clean(a.StatClassDescription),
+        year: num(a.YearBuilt), sqft: num(a.SquareFeet),
+        grantor: clean(a.Grantor), grantee: clean(a.Grantee),
+        docNum: clean(a.DocumentNumber), docType: clean(a.DocumentTypeDescription), raw: a } as Rec; }).filter(Boolean) as Rec[],
+  },
 };
 
+const execFileP = promisify(execFile);
+// curl fallback for servers whose responses undici (global fetch) rejects. Some
+// county ArcGIS/IIS hosts (e.g. spatial.jacksoncountyor.gov) emit a non-RFC
+// response header that fetch refuses ("Invalid header token / does not match the
+// HTTP/1.1 protocol"); Node's http parser rejects it even with insecureHTTPParser,
+// but curl tolerates it. The URL is passed as a single argv (no shell) so the
+// where-clause chars can't inject. --compressed handles gzip.
+async function curlGet(url: string, timeoutMs: number): Promise<string> {
+  const { stdout } = await execFileP('curl', ['-s', '--compressed', '--max-time', String(Math.ceil(timeoutMs / 1000)), url],
+    { maxBuffer: 64 * 1024 * 1024 });
+  return stdout;
+}
+// fetch() first; only on the undici HTTP-compliance error, retry via curl.
+async function fetchJson(u: URL, timeoutMs: number): Promise<any> {
+  try {
+    const res = await fetch(u, { signal: AbortSignal.timeout(timeoutMs) });
+    if (!res.ok) throw new Error(`${res.status}: ${(await res.text()).slice(0, 140)}`);
+    return await res.json();
+  } catch (e: any) {
+    const msg = String(e?.cause?.message || e?.message || '');
+    if (/Invalid header token|does not match the HTTP/i.test(msg)) return JSON.parse(await curlGet(u.toString(), timeoutMs));
+    throw e;
+  }
+}
+
 async function fetchPage(src: Source, offset: number): Promise<any[]> {
   const u = new URL(src.layer + '/query');
   u.searchParams.set('where', src.where);
@@ -361,9 +408,7 @@ async function fetchPage(src: Source, offset: number): Promise<any[]> {
   let lastErr: any;
   for (let attempt = 0; attempt < 3; attempt++) {
     try {
-      const res = await fetch(u, { signal: AbortSignal.timeout(90_000) });
-      if (!res.ok) throw new Error(`${src.key} ${res.status}: ${(await res.text()).slice(0, 140)}`);
-      const j: any = await res.json();
+      const j: any = await fetchJson(u, 90_000);
       if (j.error) throw new Error(`${src.key} error: ${JSON.stringify(j.error).slice(0, 140)}`);
       return j.features || [];
     } catch (e: any) { lastErr = e; if (attempt < 2) await new Promise(r => setTimeout(r, 2000 * (attempt + 1))); }
@@ -373,7 +418,7 @@ async function fetchPage(src: Source, offset: number): Promise<any[]> {
 async function countTotal(src: Source): Promise<number> {
   const u = new URL(src.layer + '/query');
   u.searchParams.set('where', src.where); u.searchParams.set('returnCountOnly', 'true'); u.searchParams.set('f', 'json');
-  const j: any = await (await fetch(u, { signal: AbortSignal.timeout(30_000) })).json();
+  const j: any = await fetchJson(u, 30_000);
   return Number(j.count || 0);
 }
 
diff --git a/src/ingest/parcels/engine.ts b/src/ingest/parcels/engine.ts
index 800045b..27bec0e 100644
--- a/src/ingest/parcels/engine.ts
+++ b/src/ingest/parcels/engine.ts
@@ -38,6 +38,7 @@ const ADAPTERS: Record<string, () => Promise<{ run: () => Promise<{ upserted: nu
   marion: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('marion') }; },
   yakima: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('yakima') }; },
   lane: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('lane') }; },
+  jackson: async () => { const m = await import('./arcgis_sales.ts'); return { run: m.ingestArcgisSales('jackson') }; },
 };
 
 async function main() {
diff --git a/src/jobs/hourly_loop.ts b/src/jobs/hourly_loop.ts
index 98f90b9..68c92c9 100644
--- a/src/jobs/hourly_loop.ts
+++ b/src/jobs/hourly_loop.ts
@@ -74,6 +74,7 @@ const JOBS: Job[] = [
   { name: 'deeds-marion',script:'src/ingest/parcels/engine.ts',     args: ['marion'],      minIntervalHours: 3, runSources: ['marion'] },
   { name: 'deeds-yakima',script:'src/ingest/parcels/engine.ts',     args: ['yakima'],      minIntervalHours: 3, runSources: ['yakima'] },
   { name: 'deeds-lane',  script: 'src/ingest/parcels/engine.ts',     args: ['lane'],        minIntervalHours: 4, runSources: ['lane'] },
+  { name: 'deeds-jackson',script:'src/ingest/parcels/engine.ts',     args: ['jackson'],     minIntervalHours: 3, runSources: ['jackson'] },
   // TK-5: full San Diego County (1.09M) via SANDAG — zoning + assessed value; was in engine but missing from rotation.
   { name: 'parcels-sd-sandag', script: 'src/ingest/parcels/engine.ts', args: ['san-diego-sandag'], minIntervalHours: 2, runSources: ['san_diego_sandag'] },
   // TK-5: Santa Clara County CA (San Jose, 493k) — situs address; was in engine but missing from rotation.

← d503d96 TK-16: add Lane County OR (41039) free priced-deed feed — 4t  ·  back to Nationalrealestate  ·  TK-16: add Orange County CA (06059) parcel coverage + centro 9af3911 →