tighten fit ceiling to ~55GB weights (gpt-oss-120b 70.6GB failed placement 8/17)

dddd583952bcb7b9230a6302f8c3ff77a39ec9b1 · 2026-08-17 17:10:28 -0700 · steve
Files touched

M review-prompt.md
M review.sh
Diff

commit dddd583952bcb7b9230a6302f8c3ff77a39ec9b1
Author: steve <steve@designerwallcoverings.com>
Date:   Mon Aug 17 17:10:28 2026 -0700

    tighten fit ceiling to ~55GB weights (gpt-oss-120b 70.6GB failed placement 8/17)
---
 review-prompt.md | 4 ++--
 review.sh        | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/review-prompt.md b/review-prompt.md
index 86c7557..3ec282d 100644
--- a/review-prompt.md
+++ b/review-prompt.md
@@ -7,10 +7,10 @@ HARDWARE REALITY (the binding constraint — do not forget it):
 
 STEPS:
 1. Get the current arena.ai / LMArena leaderboard. It is JS-rendered, so WebFetch returns only the shell — rely on WebSearch for current rankings (query e.g. "LMArena arena.ai leaderboard top open-weight models <this month> Qwen DeepSeek Llama Mistral Gemma"), cross-checking 2 sources (swfte.com/lmarena, localaimaster, HF space lmarena-ai/arena-leaderboard).
-2. From the leaderboard, take the top OPEN-WEIGHT models, then FILTER to those that (a) fit ~90 GB single-node at 4–8 bit and (b) have an MLX-community build. That's the candidate set.
+2. From the leaderboard, take the top OPEN-WEIGHT models, then FILTER to those that (a) fit ~55 GB of WEIGHTS single-node at 4–8 bit (HARD LIMIT — learned 2026-08-17: gpt-oss-120b at 70.6GB FAILED exo placement with "No cycles found with sufficient memory". Raw model size is NOT enough — the 96GB M3 Ultra must also hold macOS + MLX activations + KV cache, especially at long context, so usable weight budget is ~55GB, not 90GB. The two 32GB nodes are too small to meaningfully share a big model. NEVER recommend a model whose weights exceed ~55GB.) and (b) have an MLX-community build. That's the candidate set.
 3. Read what we currently have from the file `data/local-state.txt` (the wrapper already wrote it: the loaded exo model + downloaded models + hardware). You have NO shell — read that file with the Read tool. Current default = Qwen3.6-35B-A3B-8bit.
 4. COMPARE. If a candidate open model clearly beats our current default AND fits AND has an MLX build we don't yet have → that's a RECOMMENDATION.
-5. CHOOSE TODAY'S MODEL. From the models ALREADY DOWNLOADED (in local-state.txt) that fit ~90 GB, pick the single BEST one for today (best leaderboard-ranked open model we actually have on disk). Write ONLY its exact exo model id to `data/chosen-model.txt` (one line, nothing else). The wrapper will load it if it isn't already the loaded model. If the best downloaded model IS already loaded, still write it (a no-op load is fine/skipped by the wrapper).
+5. CHOOSE TODAY'S MODEL. From the models ALREADY DOWNLOADED (in local-state.txt) that fit ~55GB of weights, pick the single BEST one for today (best leaderboard-ranked open model we actually have on disk). Write ONLY its exact exo model id to `data/chosen-model.txt` (one line, nothing else). The wrapper will load it if it isn't already the loaded model. If the best downloaded model IS already loaded, still write it (a no-op load is fine/skipped by the wrapper).
 6. Write a dated one-line entry to data/reviews.jsonl (date, chosen_model, top-fitting-open-model-overall, verdict: CHOSE-EXISTING or UPGRADE-AVAILABLE). ALWAYS write this heartbeat.
 7. ONLY if a BETTER open model exists that FITS ~90 GB but is NOT yet downloaded, AND it's a NEW finding (not already in a prior pending-approval memo): write a recommendation memo to ~/.claude/yolo-queue/pending-approval/ named `local-model-upgrade-<date>.md` — model, why better, exact MLX id (VERIFY it against exo's /v1/models list which is in scope via WebSearch/knowledge — note the id may differ from the HF card, e.g. gpt-oss uses `-MXFP4-Q8`), download size, and the exact GATED pull command. Do NOT pull it; the download stays Steve's to run.
 
diff --git a/review.sh b/review.sh
index 0a70f0a..65ade10 100755
--- a/review.sh
+++ b/review.sh
@@ -20,7 +20,7 @@ try:
 except Exception as e: print('state-unreachable')" 2>/dev/null
   echo "## Downloaded models (~/.exo/models):"
   ls -la ~/.exo/models 2>/dev/null | awk '{print $5, $NF}' | grep -viE '^\s*$|caches|\.$'
-  echo "## Hardware: 96GB M3 Ultra (primary) + 2x 32GB nodes; target = fits ~90GB single-node."
+  echo "## Hardware: 96GB M3 Ultra (primary) + 2x 32GB nodes; target = fits ~55GB weights single-node."
 } > "$DIR/data/local-state.txt" 2>&1
 
 # 2) claude reviews — NO Bash, NO Skill: research + read the state file + write report/recommendation
← c96a2b6 daily 4:45am + choose-best-downloaded-model-for-the-day (wra · back to Local Model Leaderboard Watch · purge remaining 90GB refs -> 55GB weights ceiling everywhere 0747d9f →