fix(workers): health classifier — DEGRADED on error RATE (>25%), not any nonzero errorCount

059c2688c7d55e33087365c9be529ca34e78ca0c · 2026-08-03 10:28:31 -0700 · Steve Abrams

A successful crawl that skips a fraction of pages (e.g. coming-soon Toll QMIs with
no priced payload) was mislabeled DEGRADED off a single error, hiding real failures
in noise. Now DEGRADED only when >25% of attempted records error. Audit ratios were
all ~4-7% (toll/century/beazer/highland/landsea) → all flip to HEALTHY on next crawl;
a genuinely failing source (low-published/high-error) still shows DEGRADED. [yoloforever c4]

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files touched

M apps/workers/src/verify.ts

Diff

commit 059c2688c7d55e33087365c9be529ca34e78ca0c
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Mon Aug 3 10:28:31 2026 -0700

    fix(workers): health classifier — DEGRADED on error RATE (>25%), not any nonzero errorCount
    
    A successful crawl that skips a fraction of pages (e.g. coming-soon Toll QMIs with
    no priced payload) was mislabeled DEGRADED off a single error, hiding real failures
    in noise. Now DEGRADED only when >25% of attempted records error. Audit ratios were
    all ~4-7% (toll/century/beazer/highland/landsea) → all flip to HEALTHY on next crawl;
    a genuinely failing source (low-published/high-error) still shows DEGRADED. [yoloforever c4]
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 apps/workers/src/verify.ts | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/apps/workers/src/verify.ts b/apps/workers/src/verify.ts
index 8230ae7e..ddeda44d 100644
--- a/apps/workers/src/verify.ts
+++ b/apps/workers/src/verify.ts
@@ -165,8 +165,15 @@ export async function recordSourceRun(sourceKey: string, outcome: RunOutcome, no
 
   const failures = outcome.ok ? 0 : source.consecutiveFailures + 1;
   const tripped = failures >= source.maxConsecutiveFailures;
+  // A successful crawl is DEGRADED only when errors are a MEANINGFUL SHARE of the
+  // attempted records — not merely nonzero. Many builders legitimately error on a
+  // fraction of pages (e.g. coming-soon Toll QMIs with no priced payload); flagging
+  // a 3,104-published / 142-skip run as DEGRADED is false noise that hides real
+  // failures. >25% of attempted records erroring = genuinely degraded. [yoloforever c4]
+  const attempted = (outcome.published ?? 0) + (outcome.errorCount ?? 0);
+  const errRate = attempted > 0 ? (outcome.errorCount ?? 0) / attempted : 0;
   const health = outcome.ok
-    ? outcome.errorCount && outcome.errorCount > 0
+    ? errRate > 0.25
       ? "DEGRADED"
       : "HEALTHY"
     : tripped

← d858b396 feat(toll): raise recurring QMI crawl cap 400->2500 (Steve-a · back to Homesonspec · ops: age-based retention prune for snapshot page-cache (7d, eb45f0f3 →