[object Object]

← back to Nationalrealestate

TK-10155 fix2: wire listings ingest + lifecycle sweep into existing refresh_all.ts cadence (monthly, non-fatal, guarded) — no new launchd job (daily upgrade drafted separately)

d8955b6655d8e3e0659d8391ef799121cd360d43 · 2026-08-02 23:45:16 -0700 · Steve

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files touched

Diff

commit d8955b6655d8e3e0659d8391ef799121cd360d43
Author: Steve <steve@designerwallcoverings.com>
Date:   Sun Aug 2 23:45:16 2026 -0700

    TK-10155 fix2: wire listings ingest + lifecycle sweep into existing refresh_all.ts cadence (monthly, non-fatal, guarded) — no new launchd job (daily upgrade drafted separately)
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 src/jobs/refresh_all.ts | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/src/jobs/refresh_all.ts b/src/jobs/refresh_all.ts
index 669d2bc..7115f41 100644
--- a/src/jobs/refresh_all.ts
+++ b/src/jobs/refresh_all.ts
@@ -13,7 +13,7 @@ import { fileURLToPath } from 'node:url';
 const __dirname = dirname(fileURLToPath(import.meta.url));
 const ROOT = join(__dirname, '..', '..');
 
-const JOBS: Array<[string, string]> = [
+const JOBS: Array<[string, string, string[]?]> = [
   ['zillow', 'src/ingest/zillow_research.ts'],
   ['redfin', 'src/ingest/redfin_tracker.ts'],
   ['fhfa', 'src/ingest/fhfa_hpi.ts'],
@@ -21,13 +21,22 @@ const JOBS: Array<[string, string]> = [
   ['derive', 'src/ingest/derive_metrics.ts'],
   ['score', 'src/score/opportunity.ts'],
   ['alerts', 'src/jobs/alerts_check.ts'],
+  // TK-10155: the listings ingest + lifecycle sweep run on THIS existing cadence (monthly,
+  // com.steve.usre-refresh Day 25) rather than a new launchd job (installing one is gated).
+  // (a) refresh the broker listings feed, then (b) sweep lifecycle states so gone-from-feed
+  // withdrawals + withdrawal-snapshots are captured. Both are appended and, like every job
+  // above, wrapped so a failure logs + continues — it must never break zillow/redfin/etc.
+  // NOTE: listings churn in DAYS, so a DAILY sweep is the real fix — drafted separately as a
+  // gated launchd job (fix 3). This monthly wiring is the reversible, no-new-job baseline.
+  ['listings', 'src/ingest/listings/engine.ts', ['all']],
+  ['lifecycle-sweep', 'src/jobs/listing-lifecycle-sweep.ts'],
 ];
 
 let failed = 0;
-for (const [name, script] of JOBS) {
+for (const [name, script, extraArgs] of JOBS) {
   console.log(`\n[refresh] ── ${name} ──`);
   try {
-    execFileSync('npx', ['tsx', script], { cwd: ROOT, stdio: 'inherit', timeout: 45 * 60_000 });
+    execFileSync('npx', ['tsx', script, ...(extraArgs ?? [])], { cwd: ROOT, stdio: 'inherit', timeout: 45 * 60_000 });
   } catch (e: any) {
     failed++;
     console.error(`[refresh] ${name} FAILED: ${e.message}`);

← fdb639e TK-10155 fix1: guard gone-from-feed withdrawal behind a per-  ·  back to Nationalrealestate  ·  TK-10155 fix4: backfill writes ONE snapshot per listing (bas 37575d8 →