← back to Nas Setup

daemon-health --alert: add de-nag (fire only on worsening transition or 20h re-nag) so an hourly canary can't spam FAIL emails (TK-10547)

1a0f9852411e11ada6bda31e6175592836af1d1c · 2026-08-13 21:17:40 -0700 · Steve

Files touched

• M scripts/daemon-health.sh

Diff

commit 1a0f9852411e11ada6bda31e6175592836af1d1c
Author: Steve <steve@designerwallcoverings.com>
Date:   Thu Aug 13 21:17:40 2026 -0700

    daemon-health --alert: add de-nag (fire only on worsening transition or 20h re-nag) so an hourly canary can't spam FAIL emails (TK-10547)
---
 scripts/daemon-health.sh | 35 ++++++++++++++++++++++++-----------
 1 file changed, 24 insertions(+), 11 deletions(-)

diff --git a/scripts/daemon-health.sh b/scripts/daemon-health.sh
index 5d6a11d..acacce7 100755
--- a/scripts/daemon-health.sh
+++ b/scripts/daemon-health.sh
@@ -65,18 +65,31 @@ echo "== overall: $worst =="
 printf '%s\n' "${rows[@]}" | jq -s --arg v "$worst" \
   '{checked_at:(now|todate),verdict:$v,daemons:.}' > "$OUT"
 
-# --alert: on non-PASS, post CNCP card + email Steve. Runs in USER context, where george-send.sh
-# EXISTS (the root daemon's own email path is dead — HOME=/var/root). Default (no flag) = no send.
-if [ "$ALERT" = "1" ] && [ "$worst" != "PASS" ]; then
-  CNCP="${CNCP_URL:-http://localhost:3333}"
-  note="[BACKUP DAEMON HEALTH $(date +%F)] Henry mirror $worst — $(printf '%s\n' "${rows[@]}" | jq -r 'select(.verdict!="PASS")|"\(.daemon): \(.reason)"' | paste -sd'; ' -). The nightly root daemon can't write /Volumes/Henry (TCC/FDA). Fix: grant FDA to the daemon's launcher (TK-10547)."
-  curl -sS --max-time 10 "$CNCP/api/parking-lot" -H 'Content-Type: application/json' \
-    -d "$(jq -n --arg u "onprem://henry-mirror-daemon" --arg note "$note" '{url:$u,note:$note}')" >/dev/null 2>&1 || true
-  if [ -f "$HOME/.claude/skills/_shared/george-send.sh" ]; then
-    . "$HOME/.claude/skills/_shared/george-send.sh"
-    body="<div style=\"font-family:-apple-system,sans-serif;color:#222\"><h3 style=\"color:#b23b3b\">⚠ Henry backup daemon $worst</h3><div>$note</div></div>"
-    george_send steve-office "${BACKUP_HEALTH_TO:-steve@designerwallcoverings.com}" "⚠ Henry backup daemon $worst — $(date +%F)" "$body" >/dev/null 2>&1 || true
+# --alert with DE-NAG: only alert on a WORSENING transition (e.g. PASS->FAIL) or a daily re-nag
+# while still non-PASS — so an hourly canary can't spam ~24 FAIL emails/day. Matches the fleet
+# canary convention. Runs in USER context, where george-send.sh + ~/.claude.json creds EXIST
+# (the root daemon's own email path is dead — HOME=/var/root). Default (no --alert) = never sends.
+STATE="$DATA/daemon-health-alert-state"; RENAG_H="${RENAG_H:-20}"
+if [ "$ALERT" = "1" ]; then
+  last_v="PASS"; last_t=0
+  [ -f "$STATE" ] && { last_v=$(cut -d' ' -f1 "$STATE" 2>/dev/null || echo PASS); last_t=$(cut -d' ' -f2 "$STATE" 2>/dev/null || echo 0); }
+  fire=0
+  if [ "$worst" != "PASS" ]; then
+    [ "$(rank "$worst")" -gt "$(rank "$last_v")" ] && fire=1                       # worsened
+    [ $(( (now - last_t) / 3600 )) -ge "$RENAG_H" ] && fire=1                      # daily re-nag while broken
   fi
+  if [ "$fire" = "1" ]; then
+    CNCP="${CNCP_URL:-http://localhost:3333}"
+    note="[BACKUP DAEMON HEALTH $(date +%F)] Henry mirror $worst — $(printf '%s\n' "${rows[@]}" | jq -r 'select(.verdict!="PASS")|"\(.daemon): \(.reason)"' | paste -sd'; ' -). The nightly root daemon can't write /Volumes/Henry (TCC/FDA). Fix: grant FDA to the daemon's launcher (TK-10547)."
+    curl -sS --max-time 10 "$CNCP/api/parking-lot" -H 'Content-Type: application/json' \
+      -d "$(jq -n --arg u "onprem://henry-mirror-daemon" --arg note "$note" '{url:$u,note:$note}')" >/dev/null 2>&1 || true
+    if [ -f "$HOME/.claude/skills/_shared/george-send.sh" ]; then
+      . "$HOME/.claude/skills/_shared/george-send.sh"
+      body="<div style=\"font-family:-apple-system,sans-serif;color:#222\"><h3 style=\"color:#b23b3b\">⚠ Henry backup daemon $worst</h3><div>$note</div></div>"
+      george_send steve-office "${BACKUP_HEALTH_TO:-steve@designerwallcoverings.com}" "⚠ Henry backup daemon $worst — $(date +%F)" "$body" >/dev/null 2>&1 || true
+    fi
+  fi
+  echo "$worst $now" > "$STATE"   # remember state so we don't re-nag until RENAG_H passes / it worsens
 fi
 
 [ "$worst" = "FAIL" ] && exit 1 || exit 0

← b1e42e2 pull-dw-dump: write per-DB heartbeat latest-<db>.json so dw_  ·  back to Nas Setup  ·  Add com.steve.backup-daemon-health LaunchAgent (hourly user- a0216d5 →