← back to Wallco Ai
Add autonomous overnight seam-fix wave runner (DTD-gated, self-gating, never publishes)
f1efeda5aedba39cf8a4808e1c977942ae795680 · 2026-06-11 19:02:33 -0700 · Steve Abrams
scripts/overnight-seam-loop.sh drives seam-wave.sh wave-by-wave overnight.
5 stop conditions (07:00 PDT / manifest exhausted / kill switch / DTD PAUSE /
3 zero-PASS waves). Self-gates: waits (never runs a wave) until the manifest is
populated, the generator exists, and ComfyUI is idle — no collision with Wave-1.
PASS rate from seam-fix-queue.jsonl per-wave window; DTD panel (codex+qwen+rule)
votes CONTINUE/PAUSE, ties fail-safe to PAUSE. bash-3.2 compatible. Nothing
publishes — only is_published=FALSE candidates; never touches settlement/deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Files touched
A scripts/overnight-seam-loop.sh
Diff
commit f1efeda5aedba39cf8a4808e1c977942ae795680
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Thu Jun 11 19:02:33 2026 -0700
Add autonomous overnight seam-fix wave runner (DTD-gated, self-gating, never publishes)
scripts/overnight-seam-loop.sh drives seam-wave.sh wave-by-wave overnight.
5 stop conditions (07:00 PDT / manifest exhausted / kill switch / DTD PAUSE /
3 zero-PASS waves). Self-gates: waits (never runs a wave) until the manifest is
populated, the generator exists, and ComfyUI is idle — no collision with Wave-1.
PASS rate from seam-fix-queue.jsonl per-wave window; DTD panel (codex+qwen+rule)
votes CONTINUE/PAUSE, ties fail-safe to PAUSE. bash-3.2 compatible. Nothing
publishes — only is_published=FALSE candidates; never touches settlement/deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
scripts/overnight-seam-loop.sh | 299 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 299 insertions(+)
diff --git a/scripts/overnight-seam-loop.sh b/scripts/overnight-seam-loop.sh
new file mode 100755
index 0000000..7f6497f
--- /dev/null
+++ b/scripts/overnight-seam-loop.sh
@@ -0,0 +1,299 @@
+#!/usr/bin/env bash
+# overnight-seam-loop.sh — AUTONOMOUS overnight runner for the seam-fix wave scale-up.
+#
+# Steve sleeps; this drives scripts/seam-wave.sh wave-by-wave all night and uses
+# DTD (the 3-LLM debate panel) as the CONTINUE/PAUSE decider between waves, so he
+# wakes to a curated-ready batch of is_published=FALSE candidates.
+#
+# ── ABSOLUTE SAFETY ENVELOPE (this script NEVER violates, unattended all night) ──
+# • NOTHING PUBLISHES. seam-wave.sh only generates is_published=FALSE candidates.
+# This loop never sets is_published=TRUE, never touches the settlement gate,
+# never deploys, never pushes to live.
+# • Roots are sacred — handled by the generator (new id, parent_design_id=root,
+# INSERT-only). This loop only INVOKES seam-wave.sh; it never writes the DB.
+# • DATABASE_URL self-resolves inside the generator; this loop never echoes secrets.
+# • If a safety gate would be violated or anything is ambiguous → STOP + log;
+# never improvise around a gate.
+#
+# ── STOP CONDITIONS (checked every iteration; clean exit on ANY) ──
+# (a) local time >= 07:00 PDT 2026-06-12
+# (b) seam-wave.sh --status reports 0 remaining (manifest exhausted)
+# (c) /tmp/seam-overnight.STOP exists (Steve's kill switch)
+# (d) DTD votes PAUSE (majority of 3)
+# (e) 3 consecutive waves produce 0 new PASS candidates (quality floor)
+#
+# ── EACH ITERATION ──
+# 1. Run ONE wave: SEAM_GENERATOR=<gen> bash scripts/seam-wave.sh (gated runner).
+# 2. Compute the wave's PASS rate from data/seam-fix-queue.jsonl (records added
+# during the wave window): PASS-both-axes count / roots attempted.
+# 3. DTD decider: panel.sh framed forced-choice → parse codex+qwen votes, cast a
+# 3rd vote by rule (CONTINUE if pass rate >= 50%, else PAUSE). Majority rules.
+# 4. Log wave to $LOG + best-effort CNCP heartbeat.
+# 5. ~30s sleep between waves.
+#
+# Launch DETACHED (survives the launching session ending):
+# cd ~/Projects/wallco-ai
+# SEAM_GENERATOR=scripts/controlnet-anchored-reroll-2.py \
+# nohup setsid bash scripts/overnight-seam-loop.sh >>/tmp/seam-overnight.log 2>&1 &
+#
+# Dry self-test (no generation, exercises stop-flag + status parse):
+# DRY_RUN=1 bash scripts/overnight-seam-loop.sh
+
+set -u
+cd "$(dirname "$0")/.."
+REPO="$(pwd)"
+
+# ── config / paths ───────────────────────────────────────────────────────────────
+LOG="/tmp/seam-overnight.log"
+STOP_FLAG="/tmp/seam-overnight.STOP"
+WAVE_SH="scripts/seam-wave.sh"
+QUEUE="data/seam-fix-queue.jsonl"
+MANIFEST="data/seam-wave-roots.txt"
+DTD_PANEL="$HOME/.claude/skills/dtd/scripts/panel.sh"
+CNCP="http://127.0.0.1:3333/api/wins"
+
+# Generator: the generalized retry-generator (confirmed by the Wave-1 agent).
+# Override at launch with SEAM_GENERATOR=... ; default is the reroll-2 retry generator.
+SEAM_GENERATOR="${SEAM_GENERATOR:-scripts/controlnet-anchored-reroll-2.py}"
+WAVE_SIZE="${WAVE_SIZE:-20}"
+DRY_RUN="${DRY_RUN:-0}"
+
+# 07:00 PDT 2026-06-12 as an epoch (BSD date -j -f). Fallback: now + 12h.
+STOP_EPOCH="$(TZ="America/Los_Angeles" date -j -f "%Y-%m-%d %H:%M:%S" "2026-06-12 07:00:00" "+%s" 2>/dev/null || echo $(( $(date +%s) + 43200 )))"
+
+# ComfyUI (Mac1) — must be idle before a wave fires. LAN first, tailnet fallback.
+COMFY="http://192.168.1.133:8188"
+ZERO_PASS_STREAK=0
+WAVE_COUNT=0
+TOTAL_PASS=0
+
+# ── helpers ──────────────────────────────────────────────────────────────────────
+ts() { date "+%Y-%m-%dT%H:%M:%S%z"; }
+logl() { printf '[%s] %s\n' "$(ts)" "$*" | tee -a "$LOG"; }
+
+# best-effort CNCP heartbeat — never blocks, never fails the loop
+cncp() {
+ local title="$1" summary="$2" value="$3"
+ curl -s -m 5 -X POST "$CNCP" -H 'Content-Type: application/json' \
+ -d "$(node -e 'const a=process.argv;process.stdout.write(JSON.stringify({project:"wallco-ai",title:a[1],summary:a[2],valueToday:a[3]}))' "$title" "$summary" "$value" 2>/dev/null || echo '{}')" \
+ >/dev/null 2>&1 || true
+}
+
+# remaining-roots from seam-wave.sh --status (the runner is the source of truth)
+remaining_roots() {
+ SEAM_GENERATOR="$SEAM_GENERATOR" WAVE_SIZE="$WAVE_SIZE" bash "$WAVE_SH" --status 2>/dev/null \
+ | awk -F: '/remaining/{gsub(/[^0-9]/,"",$2); print $2; exit}'
+}
+
+# manifest has at least one real (uncommented) root id?
+manifest_ready() {
+ [ -f "$MANIFEST" ] && [ "$(grep -vcE '^[[:space:]]*(#|$)' "$MANIFEST" 2>/dev/null)" -gt 0 ]
+}
+
+# ComfyUI idle? (queue_running + queue_pending both empty). Returns 0 if idle.
+comfy_idle() {
+ local q
+ q="$(curl -s -m 5 "$COMFY/queue" 2>/dev/null)"
+ if [ -z "$q" ]; then
+ COMFY="http://100.94.103.98:8188" # tailnet fallback
+ q="$(curl -s -m 5 "$COMFY/queue" 2>/dev/null)"
+ fi
+ [ -z "$q" ] && return 1 # can't reach → treat as NOT idle (fail safe)
+ local running pending
+ running="$(printf '%s' "$q" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{process.stdout.write(String((JSON.parse(s).queue_running||[]).length))}catch(e){process.stdout.write("99")}})' 2>/dev/null || echo 99)"
+ pending="$(printf '%s' "$q" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{process.stdout.write(String((JSON.parse(s).queue_pending||[]).length))}catch(e){process.stdout.write("99")}})' 2>/dev/null || echo 99)"
+ [ "$running" = "0" ] && [ "$pending" = "0" ]
+}
+
+# Count PASS-both-axes records appended to the queue since epoch $1, plus the set
+# of DISTINCT roots attempted in that window. Echoes: "<pass> <attempted>"
+wave_pass_stats() {
+ local since="$1"
+ [ -f "$QUEUE" ] || { echo "0 0"; return; }
+ SINCE_EPOCH="$since" node -e '
+ const fs=require("fs");
+ const since=parseInt(process.env.SINCE_EPOCH||"0",10);
+ const lines=fs.readFileSync(process.argv[1],"utf8").split("\n").filter(Boolean);
+ const roots=new Set(); let pass=0;
+ for(const ln of lines){
+ let r; try{r=JSON.parse(ln)}catch(e){continue}
+ const t=Date.parse(r.ts||0)/1000;
+ if(!(t>=since)) continue; // only records from THIS wave window
+ if(r.root_id!=null) roots.add(String(r.root_id));
+ const ok = r.edge_seamless===true || r.edge_verdict==="PASS";
+ if(ok) pass++;
+ }
+ process.stdout.write(pass+" "+roots.size);
+ ' "$QUEUE" 2>/dev/null || echo "0 0"
+}
+
+# DTD forced-choice decider. Echoes CONTINUE or PAUSE. Args: wave passrate remaining mins
+dtd_decide() {
+ local wave="$1" passrate="$2" passes="$3" attempted="$4" remaining="$5" mins="$6"
+ local q="Seam-fix overnight wave ${wave} just finished: ${passes}/${attempted} PASS (${passrate}% pass rate), ${remaining} roots remaining, ${mins} min to the 07:00 PDT stop. The runner only produces is_published=FALSE candidates (nothing publishes). Should the autonomous runner CONTINUE to the next wave or PAUSE for Steve to review in the morning? Answer with one word on its own line. VERDICT: CONTINUE or PAUSE."
+
+ # 3rd vote by simple rule
+ local rule_vote="PAUSE"
+ [ "$passrate" -ge 50 ] 2>/dev/null && rule_vote="CONTINUE"
+
+ local out dir codex_vote qwen_vote
+ out="$(bash "$DTD_PANEL" "$q" 2>/dev/null)"
+ dir="$(printf '%s\n' "$out" | awk -F= '/^DIR=/{print $2; exit}')"
+
+ parse_vote() { # $1 = file → echoes CONTINUE | PAUSE | ABSTAIN
+ local f="$1"
+ [ -f "$f" ] || { echo ABSTAIN; return; }
+ # last explicit VERDICT: line wins; else last bare CONTINUE/PAUSE token
+ local v
+ v="$(grep -ioE 'VERDICT:[[:space:]]*(CONTINUE|PAUSE)' "$f" 2>/dev/null | tail -1 | grep -ioE 'CONTINUE|PAUSE' | tr a-z A-Z)"
+ [ -z "$v" ] && v="$(grep -ioE '\b(CONTINUE|PAUSE)\b' "$f" 2>/dev/null | tail -1 | tr a-z A-Z)"
+ [ -z "$v" ] && v="ABSTAIN"
+ echo "$v"
+ }
+
+ if [ -n "$dir" ] && [ -d "$dir" ]; then
+ codex_vote="$(parse_vote "$dir/codex.txt")"
+ qwen_vote="$(parse_vote "$dir/qwen.txt")"
+ else
+ codex_vote="ABSTAIN"; qwen_vote="ABSTAIN"
+ fi
+
+ # tally — abstentions don't count toward either side; rule vote always counts
+ local cont=0 pause=0
+ for v in "$codex_vote" "$qwen_vote" "$rule_vote"; do
+ [ "$v" = "CONTINUE" ] && cont=$((cont+1))
+ [ "$v" = "PAUSE" ] && pause=$((pause+1))
+ done
+ logl "DTD votes — codex=$codex_vote qwen=$qwen_vote rule=$rule_vote (passrate=${passrate}%) → CONTINUE=$cont PAUSE=$pause"
+
+ # majority rules; tie → fail-safe PAUSE (conservative, unattended)
+ if [ "$cont" -gt "$pause" ]; then echo CONTINUE; else echo PAUSE; fi
+}
+
+final_summary() {
+ local reason="$1" remaining
+ remaining="$(remaining_roots)"; [ -z "$remaining" ] && remaining="?"
+ logl "──────── OVERNIGHT RUN COMPLETE ────────"
+ logl "FINAL: waves_run=$WAVE_COUNT total_PASS_candidates_added=$TOTAL_PASS roots_remaining=$remaining stop_reason=$reason"
+ cncp "seam-fix overnight COMPLETE: $WAVE_COUNT waves, $TOTAL_PASS PASS added" \
+ "Autonomous overnight seam-fix runner stopped: $reason. $WAVE_COUNT waves run, $TOTAL_PASS PASS-both-axes candidates added (all is_published=FALSE, awaiting Steve's curation)." \
+ "$remaining roots remaining"
+}
+
+# ── startup banner ────────────────────────────────────────────────────────────────
+logl "════════ overnight-seam-loop START ════════"
+logl "repo=$REPO generator=$SEAM_GENERATOR wave_size=$WAVE_SIZE dry_run=$DRY_RUN"
+logl "stop at: $(TZ=America/Los_Angeles date -r "$STOP_EPOCH" '+%Y-%m-%d %H:%M %Z') (epoch $STOP_EPOCH) | kill switch: touch $STOP_FLAG"
+
+# ── main loop ──────────────────────────────────────────────────────────────────────
+while true; do
+ NOW="$(date +%s)"
+
+ # STOP (a) — 07:00 PDT
+ if [ "$NOW" -ge "$STOP_EPOCH" ]; then
+ final_summary "reached 07:00 PDT stop"; exit 0
+ fi
+ # STOP (c) — kill switch
+ if [ -f "$STOP_FLAG" ]; then
+ final_summary "kill switch $STOP_FLAG present"; exit 0
+ fi
+
+ MINS_LEFT=$(( (STOP_EPOCH - NOW) / 60 ))
+
+ # ── pre-wave gates: wait (don't run) until everything is ready ──
+ if ! manifest_ready; then
+ logl "WAIT: manifest $MANIFEST not yet populated by Wave-1 agent (0 root ids). Re-check in 5 min. (${MINS_LEFT} min to stop)"
+ sleep 300; continue
+ fi
+ if [ ! -f "$SEAM_GENERATOR" ]; then
+ logl "WAIT: generator $SEAM_GENERATOR not present yet. Re-check in 5 min."
+ sleep 300; continue
+ fi
+ if ! comfy_idle; then
+ logl "WAIT: ComfyUI ($COMFY) not idle/reachable — holding off (no collision). Re-check in 2 min."
+ sleep 120; continue
+ fi
+
+ REMAINING_BEFORE="$(remaining_roots)"; [ -z "$REMAINING_BEFORE" ] && REMAINING_BEFORE=0
+ # STOP (b) — manifest exhausted
+ if [ "$REMAINING_BEFORE" -le 0 ] 2>/dev/null; then
+ final_summary "manifest exhausted (0 remaining)"; exit 0
+ fi
+
+ WAVE_COUNT=$((WAVE_COUNT+1))
+ WAVE_START_EPOCH="$NOW"
+ logl "▶ WAVE $WAVE_COUNT starting — $REMAINING_BEFORE roots remaining, ${MINS_LEFT} min to stop"
+
+ if [ "$DRY_RUN" = "1" ]; then
+ logl "[DRY_RUN] would run: SEAM_GENERATOR=$SEAM_GENERATOR bash $WAVE_SH"
+ SEAM_GENERATOR="$SEAM_GENERATOR" WAVE_SIZE="$WAVE_SIZE" bash "$WAVE_SH" --dry-run 2>&1 | sed 's/^/[DRY] /' | tee -a "$LOG"
+ # In dry mode, fabricate a neutral stat so we exercise the DTD path once then stop.
+ PASSES=0; ATTEMPTED=1
+ else
+ # ── 1. run ONE wave (gated runner enforces GATE 1/2, advances state, ledgers) ──
+ SEAM_GENERATOR="$SEAM_GENERATOR" WAVE_SIZE="$WAVE_SIZE" bash "$WAVE_SH" 2>&1 | tee -a "$LOG"
+ # ── 2. compute PASS rate from queue records appended during THIS wave window ──
+ read -r PASSES ATTEMPTED <<EOF
+$(wave_pass_stats "$WAVE_START_EPOCH")
+EOF
+ fi
+ [ -z "${PASSES:-}" ] && PASSES=0
+ [ -z "${ATTEMPTED:-}" ] && ATTEMPTED=0
+
+ if [ "$ATTEMPTED" -gt 0 ]; then
+ PASSRATE=$(( PASSES * 100 / ATTEMPTED ))
+ else
+ PASSRATE=0
+ fi
+ TOTAL_PASS=$((TOTAL_PASS + PASSES))
+ REMAINING_AFTER="$(remaining_roots)"; [ -z "$REMAINING_AFTER" ] && REMAINING_AFTER="$REMAINING_BEFORE"
+ logl "WAVE $WAVE_COUNT result — ${PASSES}/${ATTEMPTED} PASS (${PASSRATE}%), roots remaining now: $REMAINING_AFTER"
+
+ # ── quality floor: STOP (e) — 3 consecutive zero-PASS waves ──
+ if [ "$PASSES" -eq 0 ]; then
+ ZERO_PASS_STREAK=$((ZERO_PASS_STREAK+1))
+ else
+ ZERO_PASS_STREAK=0
+ fi
+
+ # ── 4. log + best-effort CNCP heartbeat ──
+ cncp "seam-fix overnight wave $WAVE_COUNT: ${PASSES}/${ATTEMPTED} PASS" \
+ "Autonomous overnight seam-fix wave $WAVE_COUNT: ${PASSES}/${ATTEMPTED} PASS-both-axes (${PASSRATE}%). All candidates is_published=FALSE, awaiting Steve's curation." \
+ "$REMAINING_AFTER roots remaining"
+
+ if [ "$ZERO_PASS_STREAK" -ge 3 ]; then
+ final_summary "quality floor: 3 consecutive zero-PASS waves"; exit 0
+ fi
+
+ # re-check 07:00 + kill switch BEFORE spending money on a DTD call
+ NOW="$(date +%s)"
+ if [ "$NOW" -ge "$STOP_EPOCH" ]; then final_summary "reached 07:00 PDT stop"; exit 0; fi
+ if [ -f "$STOP_FLAG" ]; then final_summary "kill switch present"; exit 0; fi
+
+ # STOP (b) again — if this wave exhausted the manifest, no need to ask DTD
+ if [ "${REMAINING_AFTER:-0}" -le 0 ] 2>/dev/null; then
+ final_summary "manifest exhausted after wave $WAVE_COUNT"; exit 0
+ fi
+
+ # ── 3. DTD decider — CONTINUE or PAUSE ──
+ MINS_LEFT=$(( (STOP_EPOCH - NOW) / 60 ))
+ VERDICT="$(dtd_decide "$WAVE_COUNT" "$PASSRATE" "$PASSES" "$ATTEMPTED" "$REMAINING_AFTER" "$MINS_LEFT")"
+ logl "WAVE $WAVE_COUNT DTD verdict: $VERDICT"
+
+ if [ "$VERDICT" = "PAUSE" ]; then
+ cncp "seam-fix overnight PAUSED after wave $WAVE_COUNT" \
+ "DTD panel voted PAUSE after wave $WAVE_COUNT (${PASSRATE}% pass rate). Runner stopped for Steve to review." \
+ "$REMAINING_AFTER roots remaining"
+ final_summary "DTD voted PAUSE after wave $WAVE_COUNT"; exit 0
+ fi
+
+ # In dry mode, one iteration is enough to prove the path.
+ if [ "$DRY_RUN" = "1" ]; then
+ logl "[DRY_RUN] single iteration complete — exiting dry self-test."
+ final_summary "dry-run self-test complete"; exit 0
+ fi
+
+ # ── 5. brief sleep between waves ──
+ logl "WAVE $WAVE_COUNT → CONTINUE. Sleeping 30s before next wave."
+ sleep 30
+done
← d08b86e seam-wave: bash-3.2 compat — replace declare -A (unsupported
·
back to Wallco Ai
·
seam-wave: build manifest — 3 animal-forward FAIL roots (431 0438663 →