[object Object]

← back to Designer Wallcoverings

drain hardening: INT/TERM traps now exit (bare cleanup trap made drivers unkillable + lockless → two immortal drivers raced the queues); in-flight dedup rejections refund the attempt instead of terminal-failing

2409e85dca33d3e7bce587690fa3ff8389af40b7 · 2026-07-13 10:37:55 -0700 · Steve

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Files touched

Diff

commit 2409e85dca33d3e7bce587690fa3ff8389af40b7
Author: Steve <steve@designerwallcoverings.com>
Date:   Mon Jul 13 10:37:55 2026 -0700

    drain hardening: INT/TERM traps now exit (bare cleanup trap made drivers unkillable + lockless → two immortal drivers raced the queues); in-flight dedup rejections refund the attempt instead of terminal-failing
    
    Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 DW-Programming/ImportNewSkufromURL/scripts/drain-all-queues.sh   | 6 +++++-
 .../ImportNewSkufromURL/scripts/drain-import-queues.sh           | 5 ++++-
 .../ImportNewSkufromURL/scripts/import-queue-runner.js           | 9 +++++++++
 .../ImportNewSkufromURL/scripts/resurrect-infra-failed.js        | 2 +-
 4 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/DW-Programming/ImportNewSkufromURL/scripts/drain-all-queues.sh b/DW-Programming/ImportNewSkufromURL/scripts/drain-all-queues.sh
index 2079b70b..06181914 100755
--- a/DW-Programming/ImportNewSkufromURL/scripts/drain-all-queues.sh
+++ b/DW-Programming/ImportNewSkufromURL/scripts/drain-all-queues.sh
@@ -24,7 +24,11 @@ if ! mkdir "$LOCKDIR" 2>/dev/null; then
   echo "[drain-all] stale lock (holder $HOLDER dead) — taking over $(date)" >> "$LOG"
 fi
 echo $$ > "$LOCKDIR/pid"
-trap 'rm -rf "$LOCKDIR"' EXIT INT TERM
+# INT/TERM must EXIT explicitly — a bare cleanup trap swallows the signal and the
+# loop keeps running (lockless + unkillable except -9; bit us 2026-07-13 with two
+# immortal drivers racing the same queue files).
+trap 'rm -rf "$LOCKDIR"' EXIT
+trap 'rm -rf "$LOCKDIR"; trap - EXIT; exit 143' INT TERM
 
 count_pending() {
   node -e '
diff --git a/DW-Programming/ImportNewSkufromURL/scripts/drain-import-queues.sh b/DW-Programming/ImportNewSkufromURL/scripts/drain-import-queues.sh
index 96c2d3ad..634a3afc 100755
--- a/DW-Programming/ImportNewSkufromURL/scripts/drain-import-queues.sh
+++ b/DW-Programming/ImportNewSkufromURL/scripts/drain-import-queues.sh
@@ -21,7 +21,10 @@ if ! mkdir "$LOCKDIR" 2>/dev/null; then
   echo "[drain] stale lock (holder $HOLDER dead) — taking over $(date)"
 fi
 echo $$ > "$LOCKDIR/pid"
-trap 'rm -rf "$LOCKDIR"' EXIT INT TERM
+# INT/TERM must EXIT explicitly — a bare cleanup trap swallows the signal (see
+# drain-all-queues.sh, 2026-07-13 immortal-driver bug).
+trap 'rm -rf "$LOCKDIR"' EXIT
+trap 'rm -rf "$LOCKDIR"; trap - EXIT; exit 143' INT TERM
 
 BUDGET=${DRAIN_BUDGET:-400}
 for q in data/scraper-audit/reconcile/*-import-queue.jsonl; do
diff --git a/DW-Programming/ImportNewSkufromURL/scripts/import-queue-runner.js b/DW-Programming/ImportNewSkufromURL/scripts/import-queue-runner.js
index c0d38ac9..19188b75 100644
--- a/DW-Programming/ImportNewSkufromURL/scripts/import-queue-runner.js
+++ b/DW-Programming/ImportNewSkufromURL/scripts/import-queue-runner.js
@@ -245,6 +245,15 @@ if (!QUEUE || !fs.existsSync(QUEUE)) { console.error('usage: import-queue-runner
             continue; // same pass resumes with the next pending item
           }
           envFailStreak = 0;
+          // "already being processed / recently processed" = the app's in-flight
+          // dedup guard (or a concurrent runner) touched this URL moments ago —
+          // never the item's fault, always retryable next pass.
+          if (/already being processed|recently processed/i.test(note)) {
+            it.attempts--;
+            it.note = note.slice(0, 160) + ' (in-flight dedup — attempt refunded)';
+            console.log(`  ↻  ${it.sku}: in-flight dedup — left pending (attempt not charged)`);
+            save(items); await sleep(DELAY); continue;
+          }
           // TRANSIENT one-off drops (e.g. request timeout mid-recompile) must NOT
           // permanently fail on one shot — leave pending.
           const transient = /timeout/i.test(note);
diff --git a/DW-Programming/ImportNewSkufromURL/scripts/resurrect-infra-failed.js b/DW-Programming/ImportNewSkufromURL/scripts/resurrect-infra-failed.js
index 9be65ff6..36f1475c 100644
--- a/DW-Programming/ImportNewSkufromURL/scripts/resurrect-infra-failed.js
+++ b/DW-Programming/ImportNewSkufromURL/scripts/resurrect-infra-failed.js
@@ -14,7 +14,7 @@ const fs = require('fs');
 const path = require('path');
 
 const DIR = process.argv[2] || 'data/scraper-audit/reconcile';
-const INFRA_RE = /ECONNREFUSED|ECONNRESET|socket hang up|backend down|env failure|timeout|Browser launch failed|Executable doesn't exist|playwright install/i;
+const INFRA_RE = /ECONNREFUSED|ECONNRESET|socket hang up|backend down|env failure|timeout|Browser launch failed|Executable doesn't exist|playwright install|already being processed|recently processed/i;
 
 let total = 0;
 for (const f of fs.readdirSync(DIR).filter(f => f.endsWith('-import-queue.jsonl'))) {

← e107ff3e fix: launchd bare-node PATH bug in two DW scheduled jobs  ·  back to Designer Wallcoverings  ·  local-first extraction: richness gate (>=1 img, >=3 tags, sk 5bbdd74e →