firm email crawl: survive stray HTTP/2 session errors (uncaughtException guard) so one remote GOAWAY can't kill the background run (TK-10669)

de5e83f01d3bc47f6360b7b1e28cc006b36b76f0 · 2026-08-18 10:04:54 -0700 · Steve Abrams

Files touched

M src/enrich/firm_email_crawl.ts

Diff

commit de5e83f01d3bc47f6360b7b1e28cc006b36b76f0
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Tue Aug 18 10:04:54 2026 -0700

    firm email crawl: survive stray HTTP/2 session errors (uncaughtException guard) so one remote GOAWAY can't kill the background run (TK-10669)
---
 src/enrich/firm_email_crawl.ts | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/enrich/firm_email_crawl.ts b/src/enrich/firm_email_crawl.ts
index be94637..d434c38 100644
--- a/src/enrich/firm_email_crawl.ts
+++ b/src/enrich/firm_email_crawl.ts
@@ -18,6 +18,14 @@
 import 'dotenv/config';
 import { pool, query } from '../../db/pool.ts';
 
+// Node's fetch (undici) can negotiate HTTP/2; a remote GOAWAY surfaces as a stray
+// 'error' event on the Http2Session — OUTSIDE the promise chain, so a per-firm
+// try/catch can't see it and Node escalates it to a process-killing uncaughtException.
+// A background crawler must survive these; log and continue (only HTTP fetches +
+// idempotent inserts are in flight, so there's no corruptible state to protect).
+process.on('uncaughtException', (e) => console.log(`  [uncaught] ${(e as Error).message}`));
+process.on('unhandledRejection', (e) => console.log(`  [unhandledRejection] ${String(e)}`));
+
 const LIVE = process.argv.includes('--live');
 const LIMIT = Number(process.argv.find(a => a.startsWith('--limit='))?.split('=')[1] || 0) || 0;
 const CONCURRENCY = Number(process.argv.find(a => a.startsWith('--concurrency='))?.split('=')[1] || 8);

← 5a7ce2a firm email collection: crawl public contact emails off firm · back to Nationalrealestate · residential enrichment: --asset scoping on free SERP discove 65cab77 →