purge remaining 90GB refs -> 55GB weights ceiling everywhere

0747d9f5086dcab9c6c5f2ed05b6ee49ecfa3ecb · 2026-08-17 17:10:42 -0700 · steve
Files touched

M review-prompt.md
Diff

commit 0747d9f5086dcab9c6c5f2ed05b6ee49ecfa3ecb
Author: steve <steve@designerwallcoverings.com>
Date:   Mon Aug 17 17:10:42 2026 -0700

    purge remaining 90GB refs -> 55GB weights ceiling everywhere
---
 review-prompt.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/review-prompt.md b/review-prompt.md
index 3ec282d..568064d 100644
--- a/review-prompt.md
+++ b/review-prompt.md
@@ -2,8 +2,8 @@ You are the DAILY (4:45am) LOCAL-MODEL CHOOSER for Steve's exo Mac cluster. Your
 
 HARDWARE REALITY (the binding constraint — do not forget it):
 - exo cluster = 3 Macs: one 96 GB M3 Ultra (primary) + two 32 GB nodes.
-- Models ≤ ~90 GB run BEST single-node on the M3 Ultra (faster — no inter-node hops). exo only shards when a model exceeds 96 GB.
-- So "best local model for us" = the highest-ranked OPEN-WEIGHT model that fits ~90 GB at a good quant (4–8 bit) AND exists as an MLX build. Frontier MoEs (DeepSeek V4.x ~671B, Llama 4 Maverick ~400B) DO NOT FIT even the whole 160 GB cluster — exclude them, note them as "too big for this hardware."
+- A model's WEIGHTS must be ≤ ~55 GB to actually place single-node on the 96 GB M3 Ultra (the rest of the 96 GB goes to macOS + MLX activations + KV cache). The two 32 GB nodes are too small to meaningfully share a big model. Raw size fitting ≠ placement fitting.
+- So "best local model for us" = the highest-ranked OPEN-WEIGHT model whose WEIGHTS are ≤ ~55 GB at a good quant (4–8 bit) AND exists as an MLX build. Frontier MoEs (DeepSeek V4.x ~671B, Llama 4 Maverick ~400B) DO NOT FIT even the whole 160 GB cluster — exclude them, note them as "too big for this hardware."
 
 STEPS:
 1. Get the current arena.ai / LMArena leaderboard. It is JS-rendered, so WebFetch returns only the shell — rely on WebSearch for current rankings (query e.g. "LMArena arena.ai leaderboard top open-weight models <this month> Qwen DeepSeek Llama Mistral Gemma"), cross-checking 2 sources (swfte.com/lmarena, localaimaster, HF space lmarena-ai/arena-leaderboard).
@@ -12,6 +12,6 @@ STEPS:
 4. COMPARE. If a candidate open model clearly beats our current default AND fits AND has an MLX build we don't yet have → that's a RECOMMENDATION.
 5. CHOOSE TODAY'S MODEL. From the models ALREADY DOWNLOADED (in local-state.txt) that fit ~55GB of weights, pick the single BEST one for today (best leaderboard-ranked open model we actually have on disk). Write ONLY its exact exo model id to `data/chosen-model.txt` (one line, nothing else). The wrapper will load it if it isn't already the loaded model. If the best downloaded model IS already loaded, still write it (a no-op load is fine/skipped by the wrapper).
 6. Write a dated one-line entry to data/reviews.jsonl (date, chosen_model, top-fitting-open-model-overall, verdict: CHOSE-EXISTING or UPGRADE-AVAILABLE). ALWAYS write this heartbeat.
-7. ONLY if a BETTER open model exists that FITS ~90 GB but is NOT yet downloaded, AND it's a NEW finding (not already in a prior pending-approval memo): write a recommendation memo to ~/.claude/yolo-queue/pending-approval/ named `local-model-upgrade-<date>.md` — model, why better, exact MLX id (VERIFY it against exo's /v1/models list which is in scope via WebSearch/knowledge — note the id may differ from the HF card, e.g. gpt-oss uses `-MXFP4-Q8`), download size, and the exact GATED pull command. Do NOT pull it; the download stays Steve's to run.
+7. ONLY if a BETTER open model exists whose WEIGHTS are ≤ ~55 GB (verified-fittable) but is NOT yet downloaded, AND it's a NEW finding (not already in a prior pending-approval memo): write a recommendation memo to ~/.claude/yolo-queue/pending-approval/ named `local-model-upgrade-<date>.md` — model, why better, exact MLX id (VERIFY it against exo's /v1/models list which is in scope via WebSearch/knowledge — note the id may differ from the HF card, e.g. gpt-oss uses `-MXFP4-Q8`), download size, and the exact GATED pull command. Do NOT pull it; the download stays Steve's to run.
 
 RULES: $0 (WebSearch free). Never trigger a download. Never write anything except data/chosen-model.txt, data/reviews.jsonl, and (if warranted) one pending-approval memo. The wrapper — not you — loads the chosen model, and ONLY if it's already on disk. Keep the run tight: if today's best-downloaded choice and the overall best-fitting model are both unchanged from yesterday, still write chosen-model.txt + the heartbeat, skip the memo, and stop.
\ No newline at end of file
← dddd583 tighten fit ceiling to ~55GB weights (gpt-oss-120b 70.6GB fa · back to Local Model Leaderboard Watch · daily chooser 2026-08-18: chose Qwen3.6-35B-A3B-8bit; correc a52578d →