daily 4:45am + choose-best-downloaded-model-for-the-day (wrapper loads if on disk; downloads stay gated)

c96a2b6db11283180ea1c202299be512222920fe · 2026-08-17 16:12:59 -0700 · steve
Files touched

M review-prompt.md
M review.sh
Diff

commit c96a2b6db11283180ea1c202299be512222920fe
Author: steve <steve@designerwallcoverings.com>
Date:   Mon Aug 17 16:12:59 2026 -0700

    daily 4:45am + choose-best-downloaded-model-for-the-day (wrapper loads if on disk; downloads stay gated)
---
 review-prompt.md |  9 +++++----
 review.sh        | 25 +++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/review-prompt.md b/review-prompt.md
index 500cd60..86c7557 100644
--- a/review-prompt.md
+++ b/review-prompt.md
@@ -1,4 +1,4 @@
-You are the hourly LOCAL-MODEL LEADERBOARD WATCHER for Steve's exo Mac cluster. Your job: make sure the cluster is running the BEST open-weight model that actually FITS the hardware. READ-ONLY + RECOMMEND-ONLY — never download a model, never change the loaded model, never touch the cluster. Draft recommendations only.
+You are the DAILY (4:45am) LOCAL-MODEL CHOOSER for Steve's exo Mac cluster. Your job: pick the BEST open-weight model to use TODAY that actually FITS the hardware, from among the models ALREADY DOWNLOADED — and separately recommend (gated) any better model that would need a fresh download. You have NO shell. You NEVER download a model and you NEVER trigger a load yourself — you WRITE YOUR CHOICE to a file and the deterministic wrapper loads it (only if already downloaded). New downloads = a gated memo only.
 
 HARDWARE REALITY (the binding constraint — do not forget it):
 - exo cluster = 3 Macs: one 96 GB M3 Ultra (primary) + two 32 GB nodes.
@@ -10,7 +10,8 @@ STEPS:
 2. From the leaderboard, take the top OPEN-WEIGHT models, then FILTER to those that (a) fit ~90 GB single-node at 4–8 bit and (b) have an MLX-community build. That's the candidate set.
 3. Read what we currently have from the file `data/local-state.txt` (the wrapper already wrote it: the loaded exo model + downloaded models + hardware). You have NO shell — read that file with the Read tool. Current default = Qwen3.6-35B-A3B-8bit.
 4. COMPARE. If a candidate open model clearly beats our current default AND fits AND has an MLX build we don't yet have → that's a RECOMMENDATION.
-5. Write a dated one-line entry to data/reviews.jsonl (date, top-fitting-open-model, our-default, verdict: KEEP or UPGRADE-AVAILABLE). ALWAYS write this heartbeat even if nothing changed.
-6. ONLY if UPGRADE-AVAILABLE and it's a NEW finding (not already recommended in a prior review): write a recommendation memo to ~/.claude/yolo-queue/pending-approval/ named `local-model-upgrade-<date>.md` — what model, why it's better, its MLX id, download size, and the exact (gated) command to pull it. Do NOT pull it.
+5. CHOOSE TODAY'S MODEL. From the models ALREADY DOWNLOADED (in local-state.txt) that fit ~90 GB, pick the single BEST one for today (best leaderboard-ranked open model we actually have on disk). Write ONLY its exact exo model id to `data/chosen-model.txt` (one line, nothing else). The wrapper will load it if it isn't already the loaded model. If the best downloaded model IS already loaded, still write it (a no-op load is fine/skipped by the wrapper).
+6. Write a dated one-line entry to data/reviews.jsonl (date, chosen_model, top-fitting-open-model-overall, verdict: CHOSE-EXISTING or UPGRADE-AVAILABLE). ALWAYS write this heartbeat.
+7. ONLY if a BETTER open model exists that FITS ~90 GB but is NOT yet downloaded, AND it's a NEW finding (not already in a prior pending-approval memo): write a recommendation memo to ~/.claude/yolo-queue/pending-approval/ named `local-model-upgrade-<date>.md` — model, why better, exact MLX id (VERIFY it against exo's /v1/models list which is in scope via WebSearch/knowledge — note the id may differ from the HF card, e.g. gpt-oss uses `-MXFP4-Q8`), download size, and the exact GATED pull command. Do NOT pull it; the download stays Steve's to run.
 
-RULES: $0 where possible (WebSearch is free; openclaw is free/local). If you must note any paid call, show its cost. Never download (a pull is 35–120 GB — always Steve-gated). Never change the loaded model. Keep the whole run tight — this fires every hour, so if the leaderboard's top-fitting open model is unchanged from the last review, just write the heartbeat line and stop.
\ No newline at end of file
+RULES: $0 (WebSearch free). Never trigger a download. Never write anything except data/chosen-model.txt, data/reviews.jsonl, and (if warranted) one pending-approval memo. The wrapper — not you — loads the chosen model, and ONLY if it's already on disk. Keep the run tight: if today's best-downloaded choice and the overall best-fitting model are both unchanged from yesterday, still write chosen-model.txt + the heartbeat, skip the memo, and stop.
\ No newline at end of file
diff --git a/review.sh b/review.sh
index 27a59f3..0a70f0a 100755
--- a/review.sh
+++ b/review.sh
@@ -28,4 +28,29 @@ timeout 600 claude -p "$(cat "$DIR/review-prompt.md")" \
   --allowedTools "WebSearch,WebFetch,Read,Write" \
   >> "$DIR/data/run.log" 2>&1
 rc=$?
+
+# 3) DETERMINISTIC load of today's chosen model — ONLY if it's already downloaded.
+#    Never downloads: if the model dir is absent, we leave the current model and rely
+#    on the gated pending-approval memo. This is the wrapper's job, not the agent's.
+CHOSEN="$(head -1 "$DIR/data/chosen-model.txt" 2>/dev/null | tr -d '[:space:]')"
+if [ -n "$CHOSEN" ]; then
+  DNAME="$(printf '%s' "$CHOSEN" | sed 's#/#--#g')"
+  if [ -d "$HOME/.exo/models/$DNAME" ]; then
+    LOADED="$(curl -s -m8 http://127.0.0.1:52415/state 2>/dev/null | python3 -c "import json,sys
+try:
+  d=json.load(sys.stdin); i=d.get('instances',{})
+  print(list(i.values())[0]['MlxRingInstance']['shardAssignments']['modelId'] if i else '')
+except: print('')" 2>/dev/null)"
+    if [ "$CHOSEN" != "$LOADED" ]; then
+      echo "[$TS] loading TODAY's chosen (on disk): $CHOSEN (was: ${LOADED:-none})" >> "$DIR/data/run.log"
+      curl -s -m600 http://127.0.0.1:52415/v1/chat/completions -H 'Content-Type: application/json' \
+        -d "{\"model\":\"$CHOSEN\",\"messages\":[{\"role\":\"user\",\"content\":\"ok\"}],\"max_tokens\":1}" >/dev/null 2>&1
+      echo "[$TS] chosen model load requested" >> "$DIR/data/run.log"
+    else
+      echo "[$TS] chosen $CHOSEN already loaded — no change" >> "$DIR/data/run.log"
+    fi
+  else
+    echo "[$TS] chosen $CHOSEN NOT on disk — keeping current model; download is gated (see pending-approval)" >> "$DIR/data/run.log"
+  fi
+fi
 echo "[$TS] === review done (exit $rc) ===" >> "$DIR/data/run.log"
← 7070cc0 hourly local-model leaderboard watcher (no-Bash agent, recom · back to Local Model Leaderboard Watch · tighten fit ceiling to ~55GB weights (gpt-oss-120b 70.6GB fa dddd583 →