workers: exp_backoff_restart_delay to cap PG-outage restart storms (DTD verdict A + contrarian guardrails) — TK-10239

033d4dbb65ea06e9f983e9afb6034ce368fef53a · 2026-08-05 08:36:12 -0700 · Steve Abrams

Files touched

M ecosystem.config.js

Diff

commit 033d4dbb65ea06e9f983e9afb6034ce368fef53a
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Wed Aug 5 08:36:12 2026 -0700

    workers: exp_backoff_restart_delay to cap PG-outage restart storms (DTD verdict A + contrarian guardrails) — TK-10239
---
 ecosystem.config.js | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ecosystem.config.js b/ecosystem.config.js
index 8ec241cc..e7c4c7b1 100644
--- a/ecosystem.config.js
+++ b/ecosystem.config.js
@@ -25,6 +25,12 @@ module.exports = {
       interpreter: "node",
       interpreter_args: "--env-file=" + __dirname + "/.env --import tsx",
       env: { NODE_ENV: "production" },
+      // This pg-boss worker process.exit(1)s on any Postgres error; pm2's default
+      // instant respawn turned the 2026-08-05 disk-full/PG outage into a ~12,564-restart
+      // storm (TK-10239). exp_backoff spaces crash-restarts (base 100ms, pm2 caps ~15s,
+      // resets once stable) so the next transient PG blip can't tight-loop. A single blip
+      // still recovers in ~base delay since the backoff resets on stability.
+      exp_backoff_restart_delay: 100,
     },
   ],
 };

← 53ff5542 prune: fail-open per-builder (|| true vs pipefail abort) + S · back to Homesonspec · auto-save: 2026-08-05T14:42:56 (1 files) — ops/drain-dr-hort b7da7ff9 →