[object Object]

← back to Dw Canary Watchdog

ops-digest: CNCP post only on health state-change (no per-tick spam on persistent FAILs)

bd93b050cbd944f8ccaaa35d7f246984565967a4 · 2026-06-15 23:50:47 -0700 · Steve Abrams

Files touched

Diff

commit bd93b050cbd944f8ccaaa35d7f246984565967a4
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Mon Jun 15 23:50:47 2026 -0700

    ops-digest: CNCP post only on health state-change (no per-tick spam on persistent FAILs)
---
 .gitignore     |  1 +
 README.md      |  7 ++++---
 data/digest.md | 16 ++++++++--------
 ops-digest.mjs | 14 +++++++++-----
 4 files changed, 22 insertions(+), 16 deletions(-)

diff --git a/.gitignore b/.gitignore
index a6e4cc2..2b58574 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,4 +4,5 @@ tmp/
 *.log
 .DS_Store
 data/*.json
+data/.last-post-sig
 !data/.gitkeep
diff --git a/README.md b/README.md
index b3a6393..f53908d 100644
--- a/README.md
+++ b/README.md
@@ -66,9 +66,10 @@ node ~/Projects/dw-canary-watchdog/ops-digest.mjs --daily    # force the CNCP ca
 node ~/Projects/dw-canary-watchdog/ops-digest.mjs --email    # also email the digest via George
 ```
 - Always writes `data/digest-latest.json` + `data/digest.md` (the scorecard).
-- **Posts to CNCP parking-lot only when non-green or `--daily`** — an hourly run won't spam
-  24 green cards/day. Suggested schedule: hourly bare (alerts), plus one `--daily --email`
-  at 8am for the morning summary.
+- **Posts to CNCP parking-lot only when the health STATE CHANGES** (a canary flips) or
+  `--daily` — persistent known FAILs do NOT re-post every tick (state signature cached in
+  `data/.last-post-sig`). Suggested schedule: hourly bare (fires only on a real flip), plus
+  one `--daily --email` at 8am for the morning summary.
 - Per-canary adapters read each canary's `data/latest.json` (differing shapes).
 - *Follow-up (council first-step):* a dedicated CNCP `/api/ops-status` **panel** that shows
   ONE live updating scorecard instead of parking-lot cards — touches cncp-starter, deferred.
diff --git a/data/digest.md b/data/digest.md
index ce0b834..b45532d 100644
--- a/data/digest.md
+++ b/data/digest.md
@@ -1,10 +1,10 @@
 ```
-DW Ops Health — 🔴 ATTENTION — 2026-06-16 06:15 PT
-🟢 uptime-probe         13↑ 0~ 0↓ of 20 sites  ·  0.2h ago
-🔴 scraper-canary       1 fail / 0 warn of 10 vendors (1 unknown)  ·  0.5h ago
-🟢 map-auditor          PASS · 0 fail / 0 warn  ·  0.5h ago
-🔴 backup-canary        1 fail / 6 warn · 324 repos (318 no-remote ok)  ·  0.5h ago
-🟢 leak-scanner         0 leaking / 6 clean of 6  ·  0.5h ago
-🟡 scraper-healthcheck  regressions in last run  ·  20.8h ago
-🟢 canary-watchdog      6/6 canaries running  ·  0.3h ago
+DW Ops Health — 🔴 ATTENTION — 2026-06-16 06:50 PT
+🟢 uptime-probe         13↑ 0~ 0↓ of 20 sites  ·  0h ago
+🔴 scraper-canary       1 fail / 0 warn of 10 vendors (1 unknown)  ·  1h ago
+🟢 map-auditor          PASS · 0 fail / 0 warn  ·  0.2h ago
+🔴 backup-canary        1 fail / 6 warn · 324 repos (318 no-remote ok)  ·  1.1h ago
+🟢 leak-scanner         0 leaking / 6 clean of 6  ·  1.1h ago
+🟡 scraper-healthcheck  regressions in last run  ·  21.4h ago
+🟢 canary-watchdog      6/6 canaries running  ·  0h ago
 ```
diff --git a/ops-digest.mjs b/ops-digest.mjs
index c45c9d9..f2e2753 100644
--- a/ops-digest.mjs
+++ b/ops-digest.mjs
@@ -74,13 +74,17 @@ async function emailGeorge(subject, html) {
     clearTimeout(t); const j = await r.json().catch(() => ({})); return !!j.success; } catch { return false; }
 }
 
-// Post to CNCP only when there's something to see (non-green) or once-daily (--daily),
-// so an hourly run doesn't append 24 green cards/day to the parking-lot.
-if (nFail || nWarn || DAILY) {
+// Post to CNCP only when the health STATE CHANGES (a canary's status flipped) or once-daily
+// (--daily). Persistent known FAILs must NOT re-post every tick — that's the noise we avoid.
+const sigFile = `${DATA}/.last-post-sig`;
+const sig = `${overall}|` + rows.filter(r => r.status !== 'OK').map(r => `${r.label}:${r.status}`).sort().join(',');
+let lastSig = ''; try { lastSig = fs.readFileSync(sigFile, 'utf8').trim(); } catch {}
+if (sig !== lastSig || DAILY) {
   const posted = await postCNCP(`[OPS HEALTH ${stamp.slice(0, 10)}] ${overall} — ${rows.map(r => dot(r.status) + r.label.replace(/-canary|-probe|-scanner|-healthcheck/, '')).join(' ')}`);
-  console.log(posted ? 'CNCP scorecard posted' : 'CNCP post failed (card may still have landed — server is slow)');
+  if (posted) { try { fs.writeFileSync(sigFile, sig); } catch {} }
+  console.log(posted ? `CNCP scorecard posted (state ${sig === lastSig ? 'forced --daily' : 'changed'})` : 'CNCP post failed (card may still have landed — server is slow)');
 } else {
-  console.log('all-green — scorecard written to data/digest.md, CNCP post skipped (use --daily to force)');
+  console.log(`state unchanged since last post — CNCP skipped (sig: ${sig.slice(0, 60)})`);
 }
 if (EMAIL) {
   const html = `<div style="font-family:-apple-system,monospace"><h3>DW Ops Health — ${overall}</h3><pre style="font-size:13px;line-height:1.5">${lines.join('\n')}</pre><div style="color:#888;font-size:11px">${stamp} PT · ${nFail} fail / ${nWarn} watch</div></div>`;

← 347a2b5 Add ops-digest: unified canary health scorecard + CNCP card  ·  back to Dw Canary Watchdog  ·  watchdog: fix dead George alert URL (HTTPS flip 2026-06-17) 9db5a97 →