← back to Beverlyhillsvideos
BHV IG token watchdog: emit PASS/WARN/FAIL + bridge heartbeat + receipted alerts (TK-11656)
3c74f16d77cc893dc1ad530947c77a674fa39506 · 2026-09-14 00:11:32 -0700 · Steve Abrams
Fixes 3 independent reasons the watchdog was invisible on the 7am fleet-health panel
(TK-11431 false-green class), all verified in live code:
1. WRONG LOCATION: also publish the heartbeat to
~/.claude/skills/bhv-ig-token-watchdog/data/latest.json, which fleet-health-rollup
actually globs (it never read social/poster/data/latest.json).
2. UNRECOGNIZED VOCABULARY: replace state='ALERT' (which the rollup's GOOD/WARNY/BADY
regexes all reject -> false green) with a top-level verdict AND status in PASS/WARN/FAIL.
token DEAD or live POST failures -> FAIL; expiring/missed-slot/UNMEASURED -> WARN
(an unmeasured input is never PASS); healthy+current -> PASS.
3. UNVERIFIED ALERT DELIVERY: post through _shared/cncp_post.sh (asserts 2xx, writes a
delivery receipt on the bridge row) instead of a hand-rolled curl -s that exits 0 on 4xx/5xx.
Adds a testability seam (env-injected fault, never set by the plist). No launchd change —
the existing com.steve.bhv-token-health job runs this script.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UVgEj4WxSqB62ieGpgrixR
Files touched
M social/poster/token-health-run.sh
Diff
commit 3c74f16d77cc893dc1ad530947c77a674fa39506
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Mon Sep 14 00:11:32 2026 -0700
BHV IG token watchdog: emit PASS/WARN/FAIL + bridge heartbeat + receipted alerts (TK-11656)
Fixes 3 independent reasons the watchdog was invisible on the 7am fleet-health panel
(TK-11431 false-green class), all verified in live code:
1. WRONG LOCATION: also publish the heartbeat to
~/.claude/skills/bhv-ig-token-watchdog/data/latest.json, which fleet-health-rollup
actually globs (it never read social/poster/data/latest.json).
2. UNRECOGNIZED VOCABULARY: replace state='ALERT' (which the rollup's GOOD/WARNY/BADY
regexes all reject -> false green) with a top-level verdict AND status in PASS/WARN/FAIL.
token DEAD or live POST failures -> FAIL; expiring/missed-slot/UNMEASURED -> WARN
(an unmeasured input is never PASS); healthy+current -> PASS.
3. UNVERIFIED ALERT DELIVERY: post through _shared/cncp_post.sh (asserts 2xx, writes a
delivery receipt on the bridge row) instead of a hand-rolled curl -s that exits 0 on 4xx/5xx.
Adds a testability seam (env-injected fault, never set by the plist). No launchd change —
the existing com.steve.bhv-token-health job runs this script.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UVgEj4WxSqB62ieGpgrixR
---
social/poster/token-health-run.sh | 94 +++++++++++++++++++++++++++++++++------
1 file changed, 80 insertions(+), 14 deletions(-)
diff --git a/social/poster/token-health-run.sh b/social/poster/token-health-run.sh
index c16f07b..e2c8b19 100755
--- a/social/poster/token-health-run.sh
+++ b/social/poster/token-health-run.sh
@@ -1,17 +1,48 @@
#!/bin/zsh
# BHV posting watchdog: (1) token health, (2) recent live-post failures, (3) SILENT
# missed-slot detection — if the 6h cadence stopped firing (Mac2 asleep/launchd dead),
-# no error line is ever written, so we detect it by post.log going stale. Alerts via CNCP.
-# Writes data/latest.json each run as a heartbeat the dw-canary-meta-watchdog can watch.
+# no error line is ever written, so we detect it by post.log going stale.
+#
+# TK-11656 (TK-11431 class) — this watcher WAS invisible on the 7am fleet-health panel
+# THREE ways, all fixed here:
+# (1) WRONG LOCATION. It wrote ONLY social/poster/data/latest.json. fleet-health-rollup
+# globs ~/.claude/skills/*/data/latest.json, so that file is NEVER read. Fixed by
+# ALSO publishing a bridge heartbeat under ~/.claude/skills/bhv-ig-token-watchdog/.
+# (2) UNRECOGNIZED VOCABULARY. It emitted state='ALERT', a word the rollup's
+# GOOD/WARNY/BADY regexes all reject -> null -> row stayed PASS (a dead token read
+# GREEN). Fixed by emitting a top-level verdict AND status in the PASS/WARN/FAIL
+# vocabulary the rollup actually reads.
+# (3) UNVERIFIED ALERT DELIVERY. It alerted via a hand-rolled `curl -s ... || true`,
+# which exits 0 on HTTP 4xx/5xx and left no receipt. Fixed by posting through
+# ~/.claude/skills/_shared/cncp_post.sh, which asserts a 2xx and writes a delivery
+# receipt the rollup maps to WARN on a failed arm.
cd "$(dirname "$0")/../.." || exit 1
LOG=social/poster/post.log
mkdir -p social/poster/data
-OUT=$(/opt/homebrew/bin/node social/poster/token-health.mjs 2>&1); RC=$?
-FAILS=$(tail -60 "$LOG" 2>/dev/null | grep -c "POST FAILED")
+# --- TESTABILITY SEAM (TK-11431 amendment 3) --------------------------------------------
+# test-negative.sh injects a fault via these env vars to prove the watchdog goes RED. The
+# launchd plist NEVER sets them, so a scheduled run always measures the real token.
+# BHV_WATCHDOG_TEST_OUT — override the token-health.mjs output line (e.g. "DEAD: ...")
+# BHV_WATCHDOG_TEST_RC — override its exit code
+# BHV_WATCHDOG_TEST_FAILS — override the recent-POST-FAILED count
+# BHV_WATCHDOG_TEST_STALE — override hours-since-last-run
+# BHV_WATCHDOG_BRIDGE_DIR — override where the bridge heartbeat is written (test scratch)
+# BHV_WATCHDOG_NO_ALERT=1 — skip the CNCP post (tests must not spam the real board)
+BRIDGE_DIR="${BHV_WATCHDOG_BRIDGE_DIR:-$HOME/.claude/skills/bhv-ig-token-watchdog}"
+
+if [[ -n "$BHV_WATCHDOG_TEST_OUT" ]]; then
+ OUT="$BHV_WATCHDOG_TEST_OUT"; RC="${BHV_WATCHDOG_TEST_RC:-1}"
+else
+ OUT=$(/opt/homebrew/bin/node social/poster/token-health.mjs 2>&1); RC=$?
+fi
+FAILS="${BHV_WATCHDOG_TEST_FAILS:-$(tail -60 "$LOG" 2>/dev/null | grep -c "POST FAILED")}"
# Age (hours) of the newest scheduled run header ("----- <ts> · slot=... -----").
-STALE_H=$(/opt/homebrew/bin/node -e '
+if [[ -n "$BHV_WATCHDOG_TEST_STALE" ]]; then
+ STALE_H="$BHV_WATCHDOG_TEST_STALE"
+else
+ STALE_H=$(/opt/homebrew/bin/node -e '
try{
const fs=require("fs");
const lines=fs.readFileSync("social/poster/post.log","utf8").trim().split("\n").filter(l=>l.startsWith("----- "));
@@ -21,17 +52,52 @@ try{
console.log(isNaN(t)?999:((Date.now()-t)/3.6e6).toFixed(1));
}catch{console.log(999);}
' 2>/dev/null)
+fi
# 6h cadence + grace: >8h since the last run means a slot was silently missed.
MISSED=$(/opt/homebrew/bin/node -e "console.log(parseFloat('${STALE_H:-999}')>8?1:0)" 2>/dev/null)
-STATE="HEALTHY"
-if [[ $RC -ne 0 || $FAILS -gt 0 || $MISSED -eq 1 ]]; then STATE="ALERT"; fi
-printf '{"ts":"%s","state":"%s","recentFailures":%s,"hoursSinceLastRun":%s}\n' \
- "$(date -u +%FT%TZ)" "$STATE" "${FAILS:-0}" "${STALE_H:-999}" > social/poster/data/latest.json
+# --- Classify the TOKEN state from token-health.mjs output ------------------------------
+# token-health.mjs prints one of: "HEALTHY: ...", "EXPIRING: ...", "DEAD: ...". If node
+# crashed before printing one of those, we did NOT actually measure the token.
+case "$OUT" in
+ DEAD*) TOKEN="DEAD" ;;
+ EXPIRING*) TOKEN="EXPIRING" ;;
+ HEALTHY*) TOKEN="HEALTHY" ;;
+ *) TOKEN="UNKNOWN" ;; # unmeasured (node missing/crashed/unparseable)
+esac
+
+# --- Map to the fleet-health-rollup PASS/WARN/FAIL vocabulary ---------------------------
+# FAIL — token DEAD (posting is dead), or live POST FAILED lines (posting is broken).
+# WARN — token EXPIRING soon, a cadence slot silently missed, OR UNKNOWN (an unmeasured
+# input is NEVER PASS — CLAUDE.md TK-11431 rule 1).
+# PASS — token alive + no recent failures + cadence current.
+VERDICT="PASS"
+if [[ "$TOKEN" == "DEAD" || "$FAILS" -gt 0 ]]; then
+ VERDICT="FAIL"
+elif [[ "$TOKEN" == "EXPIRING" || "$TOKEN" == "UNKNOWN" || "$MISSED" -eq 1 ]]; then
+ VERDICT="WARN"
+fi
+
+TS="$(date -u +%FT%TZ)"
+DETAIL="token=[$OUT] recentFailures=$FAILS hoursSinceLastRun=${STALE_H} missedSlot=$MISSED"
+
+# --- (1)(2) Publish heartbeats: bridge (globbed by the rollup) + legacy (unchanged path) ---
+write_latest() { # $1 = dir
+ mkdir -p "$1/data"
+ printf '{"ts":"%s","verdict":"%s","status":"%s","token":"%s","recentFailures":%s,"hoursSinceLastRun":%s,"missedSlot":%s,"detail":"%s"}\n' \
+ "$TS" "$VERDICT" "$VERDICT" "$TOKEN" "${FAILS:-0}" "${STALE_H:-999}" "${MISSED:-0}" \
+ "$(printf '%s' "$DETAIL" | sed 's/"/\\"/g')" > "$1/data/latest.json"
+}
+write_latest "$BRIDGE_DIR" # ~/.claude/skills/bhv-ig-token-watchdog — the row the rollup reads
+write_latest "social/poster" # legacy heartbeat kept for anything still reading it
-if [[ "$STATE" == "ALERT" ]]; then
- MSG="BHV IG watchdog: token=[$OUT] recentFailures=$FAILS hoursSinceLastPost=${STALE_H} missedSlot=$MISSED"
- curl -s -X POST http://127.0.0.1:3333/api/parking-lot -H 'Content-Type: application/json' \
- -d "{\"project\":\"beverlyhillsvideos\",\"title\":\"BHV posting needs attention\",\"note\":\"$MSG\"}" >/dev/null 2>&1 || true
- echo "$(date) ALERT: $MSG" >> social/poster/watchdog.log
+# --- (3) Alert through the shared sender (asserts 2xx + writes a delivery receipt) -------
+if [[ "$VERDICT" != "PASS" && "$BHV_WATCHDOG_NO_ALERT" != "1" ]]; then
+ MSG="BHV IG watchdog [$VERDICT]: $DETAIL"
+ # SKILL points the receipt at the bridge row so a failed arm surfaces on THAT skill's line.
+ SKILL="$BRIDGE_DIR" bash -c '
+ . "$HOME/.claude/skills/_shared/cncp_post.sh" 2>/dev/null || exit 0
+ cncp_post "alert://beverlyhillsvideos-ig-token" "$1"
+ ' _ "$MSG" || true
+ echo "$(date) $VERDICT: $MSG" >> social/poster/watchdog.log
fi
← 135ea6e auto-data-snapshot: 2026-09-13T20:22:40 (1 data files) — soc
·
back to Beverlyhillsvideos
·
auto-data-snapshot: 2026-09-14T00:22:42 (2 data files) — soc 9bea690 →