← back to All Designerwallcoverings

crawl-microsites: serialize seedFromPm2 pm2 jlist via shared mkdir-lock (TK-10970)

94b66739c8b486b6d92eca43f84c64f5a1e65593 · 2026-08-30 14:33:34 -0700 · Steve Abrams

Share the ~/.pm2/cli.lock.d lock with keep-alive so the two scheduled pm2-CLI callers
never hit rpc.sock concurrently and stop auto-forking orphan God daemons (pm2-fracture,
TK-10932). FAIL-OPEN: crawl proceeds if lock unattainable. Local commit only — Kamatera
deploy stays Steve-gated per repo CLAUDE.md (never full-tree, never data/).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files touched

• M scripts/crawl-microsites.js

Diff

commit 94b66739c8b486b6d92eca43f84c64f5a1e65593
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Sun Aug 30 14:33:34 2026 -0700

    crawl-microsites: serialize seedFromPm2 pm2 jlist via shared mkdir-lock (TK-10970)
    
    Share the ~/.pm2/cli.lock.d lock with keep-alive so the two scheduled pm2-CLI callers
    never hit rpc.sock concurrently and stop auto-forking orphan God daemons (pm2-fracture,
    TK-10932). FAIL-OPEN: crawl proceeds if lock unattainable. Local commit only — Kamatera
    deploy stays Steve-gated per repo CLAUDE.md (never full-tree, never data/).
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 scripts/crawl-microsites.js | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/scripts/crawl-microsites.js b/scripts/crawl-microsites.js
index d4fe048..21461d5 100644
--- a/scripts/crawl-microsites.js
+++ b/scripts/crawl-microsites.js
@@ -76,7 +76,23 @@ function seedVendorPairsFromUnified() {
 function seedFromPm2() {
   try {
     const cp = require('child_process');
-    const procs = JSON.parse(cp.execSync('pm2 jlist', { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 }));
+    // Serialize this `pm2 jlist` against the fleet's other pm2-CLI callers (keep-alive 60s)
+    // via the shared atomic mkdir lock on ~/.pm2/cli.lock.d (TK-10970). Concurrent pm2 CLI
+    // access races on rpc.sock and auto-forks orphan "God" daemons (the pm2-fracture class,
+    // TK-10932). FAIL-OPEN: proceeds anyway if the lock can't be acquired in budget (status
+    // quo) — never blocks the crawl. mkdir is atomic; a stale lock (>20s) is stolen.
+    const nodefs = require('fs'), nodepath = require('path');
+    const PM2_LOCK = nodepath.join(process.env.PM2_HOME || nodepath.join(process.env.HOME, '.pm2'), 'cli.lock.d');
+    const withPm2Lock = (fn) => {
+      const sleep = ms => Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
+      const deadline = Date.now() + 5000; let held = false;
+      while (Date.now() < deadline) {
+        try { nodefs.mkdirSync(PM2_LOCK); held = true; break; }
+        catch (e) { if (e.code !== 'EEXIST') break; try { if (Date.now() - nodefs.statSync(PM2_LOCK).mtimeMs > 20000) { nodefs.rmdirSync(PM2_LOCK); continue; } } catch {} sleep(80); }
+      }
+      try { return fn(); } finally { if (held) { try { nodefs.rmdirSync(PM2_LOCK); } catch {} } }
+    };
+    const procs = JSON.parse(withPm2Lock(() => cp.execSync('pm2 jlist', { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 })));
     const lsof = cp.execSync('lsof -nP -iTCP -sTCP:LISTEN 2>/dev/null || true', { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 });
     const pidPort = new Map();
     for (const line of lsof.split('\n')) {

← c142012 build full internal catalog for every unified vendor  ·  back to All Designerwallcoverings  ·  crawl-microsites: coherent pm2-lock constants (deadline 10s 43684c2 →