← back to Designer Wallcoverings
Local VLM auto-judge + finalize for room bake-off
8df04ff8633306316ab9c5945d2ddc27348d5581 · 2026-07-07 08:09:52 -0700 · Steve
- judge_rooms.py: comparative qwen2.5vl judge (Ollama, $0) picks per-SKU winner;
comparative ranking discriminates where absolute scoring rated everything ~9
- score_rooms.py: absolute-rubric scorer (kept for reference)
- apply_winners.py: finalize chosen room (manual override > auto-judge > fallback)
into room_setting_images + room_variants + chosen_room_engine (local, non-gated)
- compare-rooms.html: 🏆 auto-judge badges + reasons, winning cell highlighted
Early tally: kontext wins when rendered, sdxl loses on coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Files touched
M scripts/wallquest-refresh/carl-robinson-viewer/compare-rooms.htmlA scripts/wallquest-refresh/carl-robinson-viewer/room-engines/apply_winners.pyM scripts/wallquest-refresh/carl-robinson-viewer/room-engines/build_comparison.pyA scripts/wallquest-refresh/carl-robinson-viewer/room-engines/judge_rooms.pyA scripts/wallquest-refresh/carl-robinson-viewer/room-engines/score_rooms.py
Diff
commit 8df04ff8633306316ab9c5945d2ddc27348d5581
Author: Steve <steve@designerwallcoverings.com>
Date: Tue Jul 7 08:09:52 2026 -0700
Local VLM auto-judge + finalize for room bake-off
- judge_rooms.py: comparative qwen2.5vl judge (Ollama, $0) picks per-SKU winner;
comparative ranking discriminates where absolute scoring rated everything ~9
- score_rooms.py: absolute-rubric scorer (kept for reference)
- apply_winners.py: finalize chosen room (manual override > auto-judge > fallback)
into room_setting_images + room_variants + chosen_room_engine (local, non-gated)
- compare-rooms.html: 🏆 auto-judge badges + reasons, winning cell highlighted
Early tally: kontext wins when rendered, sdxl loses on coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
.../carl-robinson-viewer/compare-rooms.html | 28 ++++---
.../room-engines/apply_winners.py | 54 ++++++++++++++
.../room-engines/build_comparison.py | 23 +++++-
.../room-engines/judge_rooms.py | 87 ++++++++++++++++++++++
.../room-engines/score_rooms.py | 66 ++++++++++++++++
5 files changed, 244 insertions(+), 14 deletions(-)
diff --git a/scripts/wallquest-refresh/carl-robinson-viewer/compare-rooms.html b/scripts/wallquest-refresh/carl-robinson-viewer/compare-rooms.html
index 00c06b89..2575b35d 100644
--- a/scripts/wallquest-refresh/carl-robinson-viewer/compare-rooms.html
+++ b/scripts/wallquest-refresh/carl-robinson-viewer/compare-rooms.html
@@ -7,58 +7,62 @@
.hd{font-weight:600;margin-bottom:8px;color:#ddd}
.row{display:grid;grid-template-columns:repeat(4,1fr);gap:12px}
.cell{background:#191919;border:1px solid #2a2a2a;border-radius:8px;overflow:hidden}
+ .cell.win{border-color:#3a7;box-shadow:0 0 0 1px #3a7}
.lbl{font-size:11px;color:#9a9a9a;padding:5px 8px;border-bottom:1px solid #2a2a2a}
+ .cell.win .lbl{color:#7d9;background:#16241c}
.cell img{width:100%;display:block;aspect-ratio:1;object-fit:cover}
+ .rz{font-size:10px;color:#8a8;padding:4px 8px;font-style:italic}
.pending{aspect-ratio:1;display:flex;align-items:center;justify-content:center;color:#666;font-style:italic}
.pick{text-align:center;font-size:11px;color:#7c7;padding:5px;cursor:pointer;user-select:none}
.pick.on{background:#2a5;color:#fff;font-weight:600}
</style>
<header>
<h1>Carl Robinson → Phillipe Romano — Local Room Engine Bake-off</h1>
- <div class="prog">Kontext 0/67 · SDXL 1/67 rendered · all local, $0 · refresh to update</div>
+ <div class="prog">Kontext 2/67 · SDXL 6/67 rendered · all local, $0 · refresh to update</div>
+ <div class="prog">🏆 auto-judge (local qwen2.5vl) winner tally: gemini 4 · kontext 2</div>
</header>
<div class="card">
<div class="hd">BAM-100048 · Bamboo · Sour Cream</div>
<div class="row">
<div class="cell"><div class="lbl">Swatch</div><img src="img/BAM-100048.jpg" loading="lazy"><div class="pick" data-sku="BAM-100048" data-eng="swatch">pick swatch</div></div>
<div class="cell"><div class="lbl">Gemini (current)</div><img src="rooms/DWCR-50004-living.png" loading="lazy"><div class="pick" data-sku="BAM-100048" data-eng="gemini">pick gemini</div></div>
- <div class="cell"><div class="lbl">FLUX Kontext (B)</div><div class="pending">rendering…</div></div>
+ <div class="cell win"><div class="lbl">FLUX Kontext (B) 🏆</div><img src="rooms-local/BAM-100048-kontext.png" loading="lazy"><div class="rz">Full even wall coverage, exact texture match.</div><div class="pick" data-sku="BAM-100048" data-eng="kontext">pick kontext</div></div>
<div class="cell"><div class="lbl">SDXL+IPA (C)</div><img src="rooms-local/BAM-100048-sdxl.png" loading="lazy"><div class="pick" data-sku="BAM-100048" data-eng="sdxl">pick sdxl</div></div>
</div></div><div class="card">
<div class="hd">BAM-100049 · Bamboo · Sea Shell</div>
<div class="row">
<div class="cell"><div class="lbl">Swatch</div><img src="img/BAM-100049.jpg" loading="lazy"><div class="pick" data-sku="BAM-100049" data-eng="swatch">pick swatch</div></div>
<div class="cell"><div class="lbl">Gemini (current)</div><img src="rooms/DWCR-50005-living.png" loading="lazy"><div class="pick" data-sku="BAM-100049" data-eng="gemini">pick gemini</div></div>
- <div class="cell"><div class="lbl">FLUX Kontext (B)</div><div class="pending">rendering…</div></div>
- <div class="cell"><div class="lbl">SDXL+IPA (C)</div><div class="pending">rendering…</div></div>
+ <div class="cell win"><div class="lbl">FLUX Kontext (B) 🏆</div><img src="rooms-local/BAM-100049-kontext.png" loading="lazy"><div class="rz">Full even wall coverage, exact texture match, photorealism.</div><div class="pick" data-sku="BAM-100049" data-eng="kontext">pick kontext</div></div>
+ <div class="cell"><div class="lbl">SDXL+IPA (C)</div><img src="rooms-local/BAM-100049-sdxl.png" loading="lazy"><div class="pick" data-sku="BAM-100049" data-eng="sdxl">pick sdxl</div></div>
</div></div><div class="card">
<div class="hd">BAM-100050 · Bamboo · Mica</div>
<div class="row">
<div class="cell"><div class="lbl">Swatch</div><img src="img/BAM-100050.jpg" loading="lazy"><div class="pick" data-sku="BAM-100050" data-eng="swatch">pick swatch</div></div>
- <div class="cell"><div class="lbl">Gemini (current)</div><img src="rooms/DWCR-50024-living.png" loading="lazy"><div class="pick" data-sku="BAM-100050" data-eng="gemini">pick gemini</div></div>
+ <div class="cell win"><div class="lbl">Gemini (current) 🏆</div><img src="rooms/DWCR-50024-living.png" loading="lazy"><div class="rz">Full even wall coverage, exact texture match, photorealism.</div><div class="pick" data-sku="BAM-100050" data-eng="gemini">pick gemini</div></div>
<div class="cell"><div class="lbl">FLUX Kontext (B)</div><div class="pending">rendering…</div></div>
- <div class="cell"><div class="lbl">SDXL+IPA (C)</div><div class="pending">rendering…</div></div>
+ <div class="cell"><div class="lbl">SDXL+IPA (C)</div><img src="rooms-local/BAM-100050-sdxl.png" loading="lazy"><div class="pick" data-sku="BAM-100050" data-eng="sdxl">pick sdxl</div></div>
</div></div><div class="card">
<div class="hd">BAM-100051 · Bamboo · Silver Lining</div>
<div class="row">
<div class="cell"><div class="lbl">Swatch</div><img src="img/BAM-100051.jpg" loading="lazy"><div class="pick" data-sku="BAM-100051" data-eng="swatch">pick swatch</div></div>
- <div class="cell"><div class="lbl">Gemini (current)</div><img src="rooms/DWCR-50028-living.png" loading="lazy"><div class="pick" data-sku="BAM-100051" data-eng="gemini">pick gemini</div></div>
+ <div class="cell win"><div class="lbl">Gemini (current) 🏆</div><img src="rooms/DWCR-50028-living.png" loading="lazy"><div class="rz">Full even wall coverage, exact texture match, photorealism.</div><div class="pick" data-sku="BAM-100051" data-eng="gemini">pick gemini</div></div>
<div class="cell"><div class="lbl">FLUX Kontext (B)</div><div class="pending">rendering…</div></div>
- <div class="cell"><div class="lbl">SDXL+IPA (C)</div><div class="pending">rendering…</div></div>
+ <div class="cell"><div class="lbl">SDXL+IPA (C)</div><img src="rooms-local/BAM-100051-sdxl.png" loading="lazy"><div class="pick" data-sku="BAM-100051" data-eng="sdxl">pick sdxl</div></div>
</div></div><div class="card">
<div class="hd">BAM-100052 · Bamboo · Shaved Ice</div>
<div class="row">
<div class="cell"><div class="lbl">Swatch</div><img src="img/BAM-100052.jpg" loading="lazy"><div class="pick" data-sku="BAM-100052" data-eng="swatch">pick swatch</div></div>
- <div class="cell"><div class="lbl">Gemini (current)</div><img src="rooms/DWCR-50029-living.png" loading="lazy"><div class="pick" data-sku="BAM-100052" data-eng="gemini">pick gemini</div></div>
+ <div class="cell win"><div class="lbl">Gemini (current) 🏆</div><img src="rooms/DWCR-50029-living.png" loading="lazy"><div class="rz">Full even wall coverage, exact texture match, photorealism.</div><div class="pick" data-sku="BAM-100052" data-eng="gemini">pick gemini</div></div>
<div class="cell"><div class="lbl">FLUX Kontext (B)</div><div class="pending">rendering…</div></div>
- <div class="cell"><div class="lbl">SDXL+IPA (C)</div><div class="pending">rendering…</div></div>
+ <div class="cell"><div class="lbl">SDXL+IPA (C)</div><img src="rooms-local/BAM-100052-sdxl.png" loading="lazy"><div class="pick" data-sku="BAM-100052" data-eng="sdxl">pick sdxl</div></div>
</div></div><div class="card">
<div class="hd">CORK-100057 · Cork · Chalk</div>
<div class="row">
<div class="cell"><div class="lbl">Swatch</div><img src="img/DWCR-50001.jpeg" loading="lazy"><div class="pick" data-sku="CORK-100057" data-eng="swatch">pick swatch</div></div>
- <div class="cell"><div class="lbl">Gemini (current)</div><img src="rooms/DWCR-50001-living.png" loading="lazy"><div class="pick" data-sku="CORK-100057" data-eng="gemini">pick gemini</div></div>
+ <div class="cell win"><div class="lbl">Gemini (current) 🏆</div><img src="rooms/DWCR-50001-living.png" loading="lazy"><div class="rz">Full even wall coverage, exact texture match, photorealism.</div><div class="pick" data-sku="CORK-100057" data-eng="gemini">pick gemini</div></div>
<div class="cell"><div class="lbl">FLUX Kontext (B)</div><div class="pending">rendering…</div></div>
- <div class="cell"><div class="lbl">SDXL+IPA (C)</div><div class="pending">rendering…</div></div>
+ <div class="cell"><div class="lbl">SDXL+IPA (C)</div><img src="rooms-local/CORK-100057-sdxl.png" loading="lazy"><div class="pick" data-sku="CORK-100057" data-eng="sdxl">pick sdxl</div></div>
</div></div><div class="card">
<div class="hd">CORK-100058 · Cork · Water Lily</div>
<div class="row">
diff --git a/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/apply_winners.py b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/apply_winners.py
new file mode 100644
index 00000000..648141fe
--- /dev/null
+++ b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/apply_winners.py
@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+# Finalize: for each SKU set the CHOSEN room (manual override > auto-judge > gemini
+# fallback) and record ALL variants. Local DB only (reversible, non-gated) — the
+# Shopify image push is a separate gated step. Idempotent.
+# picks.json (optional) = {"GRS-100020":"kontext", ...} manual overrides win.
+import json, os, subprocess
+
+ROOMS_LOCAL_REL = 'rooms-local'
+VIEW = '/tmp/carl-robinson-viewer'
+CONN = os.environ.get('DATABASE_URL', 'postgresql://dw_admin@127.0.0.1:5432/dw_unified')
+judg = json.load(open('/tmp/cr-room-bakeoff/judgments.json')) if os.path.exists('/tmp/cr-room-bakeoff/judgments.json') else {}
+picks = json.load(open('/tmp/cr-room-bakeoff/picks.json')) if os.path.exists('/tmp/cr-room-bakeoff/picks.json') else {}
+
+def psql(sql):
+ return subprocess.run(['psql', CONN, '-tAc', sql], capture_output=True, text=True).stdout.strip()
+
+# ensure a room_variants column exists
+psql("ALTER TABLE carl_robinson_catalog ADD COLUMN IF NOT EXISTS room_variants jsonb;")
+psql("ALTER TABLE carl_robinson_catalog ADD COLUMN IF NOT EXISTS chosen_room_engine text;")
+
+# current gemini path per sku
+gem = {}
+for line in psql("SELECT dw_sku, COALESCE(room_setting_images::text,'[]') FROM carl_robinson_catalog ORDER BY dw_sku").splitlines():
+ if '|' not in line: continue
+ sku, arr = line.split('|', 1)
+ try: a = json.loads(arr); gem[sku] = a[0] if a else ''
+ except Exception: gem[sku] = ''
+
+rows = [l.split('|') for l in psql("SELECT dw_sku FROM carl_robinson_catalog ORDER BY dw_sku").splitlines() if l]
+applied = 0; tally = {}
+sqls = []
+for (sku,) in rows:
+ variants = {}
+ if gem.get(sku): variants['gemini'] = gem[sku]
+ for eng in ('kontext', 'sdxl'):
+ p = f'{ROOMS_LOCAL_REL}/{sku}-{eng}.png'
+ if os.path.exists(f'{VIEW}/{p}'): variants[eng] = p
+ if not variants: continue
+ # winner: manual override > auto-judge > prefer kontext > gemini
+ win = picks.get(sku) or judg.get(sku, {}).get('winner')
+ if win not in variants:
+ win = 'kontext' if 'kontext' in variants else ('gemini' if 'gemini' in variants else next(iter(variants)))
+ chosen = variants[win]
+ tally[win] = tally.get(win, 0) + 1
+ vj = json.dumps(variants).replace("'", "''")
+ sqls.append(f"UPDATE carl_robinson_catalog SET room_variants='{vj}'::jsonb, chosen_room_engine='{win}', "
+ f"room_setting_images='{json.dumps([chosen])}'::jsonb WHERE dw_sku='{sku}';")
+ applied += 1
+
+if sqls:
+ open('/tmp/cr-room-bakeoff/_apply.sql', 'w').write('\n'.join(sqls))
+ subprocess.run(['psql', CONN, '-q', '-f', '/tmp/cr-room-bakeoff/_apply.sql'])
+print(f"applied winners to {applied} SKUs. engine tally: {tally}")
+print("(room_setting_images now points to the chosen room; room_variants holds all; chosen_room_engine set)")
diff --git a/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/build_comparison.py b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/build_comparison.py
index 38795831..a3571e72 100644
--- a/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/build_comparison.py
+++ b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/build_comparison.py
@@ -6,6 +6,10 @@ VIEW = '/tmp/carl-robinson-viewer'
CONN = os.environ.get('DATABASE_URL', 'postgresql://dw_admin@127.0.0.1:5432/dw_unified')
rows = [l.rstrip('\n').split('|') for l in open('/tmp/cr-room-bakeoff/manifest.tsv') if l.strip()]
+# auto-judge results (local VLM comparative winner per SKU)
+JUDGE = '/tmp/cr-room-bakeoff/judgments.json'
+judg = json.load(open(JUDGE)) if os.path.exists(JUDGE) else {}
+
# gemini room path per sku (from the DB room_setting_images we stored earlier)
import subprocess
q = "SELECT dw_sku, COALESCE(room_setting_images::text,'[]') FROM carl_robinson_catalog ORDER BY dw_sku"
@@ -19,11 +23,17 @@ for line in out.splitlines():
def cell(path, label, sku, engine):
exists = path and os.path.exists(os.path.join(VIEW, path))
+ win = judg.get(sku, {}).get('winner')
+ is_win = (win == engine)
+ reason = judg.get(sku, {}).get('reasons', {}).get(engine, '') if isinstance(judg.get(sku, {}).get('reasons'), dict) else ''
+ crown = ' 🏆' if is_win else ''
+ cls = 'cell win' if is_win else 'cell'
if exists:
- inner = f'<img src="{path}" loading="lazy"><div class="pick" data-sku="{sku}" data-eng="{engine}">pick {engine}</div>'
+ rz = f'<div class="rz">{html.escape(reason)}</div>' if reason else ''
+ inner = f'<img src="{path}" loading="lazy">{rz}<div class="pick" data-sku="{sku}" data-eng="{engine}">pick {engine}</div>'
else:
inner = '<div class="pending">rendering…</div>'
- return f'<div class="cell"><div class="lbl">{label}</div>{inner}</div>'
+ return f'<div class="{cls}"><div class="lbl">{label}{crown}</div>{inner}</div>'
done_k = done_s = 0
cards = []
@@ -42,6 +52,11 @@ for sku, color, mat, img in rows:
</div></div>''')
total = len(rows)
+jt = {}
+for _s, _j in judg.items():
+ _w = _j.get('winner')
+ if _w: jt[_w] = jt.get(_w, 0) + 1
+tally_str = ' · '.join(f'{k} {v}' for k, v in sorted(jt.items(), key=lambda x: -x[1])) or 'judging…'
doc = f'''<!doctype html><meta charset=utf-8><title>CR Room Engine Bake-off</title>
<style>
body{{font:14px/1.4 -apple-system,system-ui,sans-serif;margin:0;background:#111;color:#eee}}
@@ -51,8 +66,11 @@ doc = f'''<!doctype html><meta charset=utf-8><title>CR Room Engine Bake-off</tit
.hd{{font-weight:600;margin-bottom:8px;color:#ddd}}
.row{{display:grid;grid-template-columns:repeat(4,1fr);gap:12px}}
.cell{{background:#191919;border:1px solid #2a2a2a;border-radius:8px;overflow:hidden}}
+ .cell.win{{border-color:#3a7;box-shadow:0 0 0 1px #3a7}}
.lbl{{font-size:11px;color:#9a9a9a;padding:5px 8px;border-bottom:1px solid #2a2a2a}}
+ .cell.win .lbl{{color:#7d9;background:#16241c}}
.cell img{{width:100%;display:block;aspect-ratio:1;object-fit:cover}}
+ .rz{{font-size:10px;color:#8a8;padding:4px 8px;font-style:italic}}
.pending{{aspect-ratio:1;display:flex;align-items:center;justify-content:center;color:#666;font-style:italic}}
.pick{{text-align:center;font-size:11px;color:#7c7;padding:5px;cursor:pointer;user-select:none}}
.pick.on{{background:#2a5;color:#fff;font-weight:600}}
@@ -60,6 +78,7 @@ doc = f'''<!doctype html><meta charset=utf-8><title>CR Room Engine Bake-off</tit
<header>
<h1>Carl Robinson → Phillipe Romano — Local Room Engine Bake-off</h1>
<div class="prog">Kontext {done_k}/{total} · SDXL {done_s}/{total} rendered · all local, $0 · refresh to update</div>
+ <div class="prog">🏆 auto-judge (local qwen2.5vl) winner tally: {tally_str}</div>
</header>
{''.join(cards)}
<script>
diff --git a/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/judge_rooms.py b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/judge_rooms.py
new file mode 100644
index 00000000..e92cd930
--- /dev/null
+++ b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/judge_rooms.py
@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+# Local VLM COMPARATIVE judge: for each SKU, show the swatch + all available candidate
+# rooms (gemini/kontext/sdxl) in ONE call and force a ranking + winner. Comparative
+# judgment discriminates where absolute scoring does not (VLM rates everything ~9).
+# Cached/resumable: re-judges a SKU only if the set of available engines changed.
+import base64, json, os, subprocess, urllib.request, time
+
+VIEW = '/tmp/carl-robinson-viewer'
+ROOMS_LOCAL = '/Users/macstudio3/Projects/Designer-Wallcoverings/scripts/wallquest-refresh/carl-robinson-viewer/rooms-local'
+OUTF = '/tmp/cr-room-bakeoff/judgments.json'
+CONN = os.environ.get('DATABASE_URL', 'postgresql://dw_admin@127.0.0.1:5432/dw_unified')
+MODEL = 'qwen2.5vl:7b'
+
+def b64(p): return base64.b64encode(open(p, 'rb').read()).decode()
+
+def judge(swatch, labeled):
+ # labeled = [(engine, path), ...] in display order
+ order = ", ".join(f"image {i+2} = {eng}" for i, (eng, _) in enumerate(labeled))
+ prompt = (f"You are a HARSH art director for a luxury wallcovering store. Image 1 is the product "
+ f"SWATCH. The other images are candidate room-settings of the SAME product ({order}). "
+ f"Pick the ONE best room to sell this exact wallpaper. Be critical and DISCRIMINATE — penalize: "
+ f"blank or patchy walls (texture missing in places), visible tiling seams or repeated-panel "
+ f"artifacts, wall texture that does NOT match the swatch weave, unrealistic or altered furniture. "
+ f"Reward full even wall coverage + exact texture match + photorealism. You MUST rank them from "
+ f"best to worst and choose a single winner. Respond ONLY JSON: "
+ f'{{"winner":"<engine>","ranking":["<engine>",...],"reasons":{{"<engine>":"<=8 words"}}}}')
+ imgs = [b64(swatch)] + [b64(p) for _, p in labeled]
+ body = json.dumps({"model": MODEL, "prompt": prompt, "images": imgs,
+ "format": "json", "stream": False, "options": {"temperature": 0}}).encode()
+ req = urllib.request.Request("http://localhost:11434/api/generate", body, {"Content-Type": "application/json"})
+ r = json.load(urllib.request.urlopen(req, timeout=300))
+ return json.loads(r["response"])
+
+def normalize(j, cand):
+ # map positional labels ("image 2", "image2", "2") -> engine names by order
+ pos = {str(i + 2): eng for i, (eng, _) in enumerate(cand)} # image2->cand[0]...
+ names = {eng for eng, _ in cand}
+ def fix(x):
+ if not isinstance(x, str): return x
+ s = x.strip().lower().replace('image', '').strip()
+ if x in names: return x
+ return pos.get(s, x)
+ j['winner'] = fix(j.get('winner'))
+ j['ranking'] = [fix(r) for r in j.get('ranking', [])]
+ if isinstance(j.get('reasons'), dict):
+ j['reasons'] = {fix(k): v for k, v in j['reasons'].items()}
+ return j
+
+
+def run():
+ gem = {}
+ q = "SELECT dw_sku, COALESCE(room_setting_images::text,'[]') FROM carl_robinson_catalog ORDER BY dw_sku"
+ for line in subprocess.run(['psql', CONN, '-tAc', q], capture_output=True, text=True).stdout.splitlines():
+ if '|' not in line: continue
+ sku, arr = line.split('|', 1)
+ try: a = json.loads(arr); gem[sku] = a[0] if a else ''
+ except Exception: gem[sku] = ''
+ rows = [l.rstrip('\n').split('|') for l in open('/tmp/cr-room-bakeoff/manifest.tsv') if l.strip()]
+ data = json.load(open(OUTF)) if os.path.exists(OUTF) else {}
+ judged = 0
+ for sku, color, mat, img in rows:
+ swatch = f'{VIEW}/{img}'
+ if not os.path.exists(swatch): continue
+ cand = []
+ if gem.get(sku) and os.path.exists(f'{VIEW}/{gem[sku]}'): cand.append(('gemini', f'{VIEW}/{gem[sku]}'))
+ for eng in ('kontext', 'sdxl'):
+ p = f'{ROOMS_LOCAL}/{sku}-{eng}.png'
+ if os.path.exists(p): cand.append((eng, p))
+ if len(cand) < 2: continue # need >=2 to compare
+ engset = sorted(e for e, _ in cand)
+ if data.get(sku, {}).get('_engines') == engset: continue # already judged this set
+ try:
+ j = normalize(judge(swatch, cand), cand); j['_engines'] = engset
+ data[sku] = j; judged += 1
+ print(f"{sku}: winner={j.get('winner')} rank={j.get('ranking')}", flush=True)
+ json.dump(data, open(OUTF, 'w'), indent=1)
+ except Exception as e:
+ print(f"{sku} ERR {str(e)[:90]}", flush=True)
+ tally = {}
+ for sku, j in data.items():
+ w = j.get('winner')
+ if w: tally[w] = tally.get(w, 0) + 1
+ print(f"\nJUDGED {judged} SKUs. winner tally: {tally}")
+
+
+if __name__ == '__main__':
+ run()
diff --git a/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/score_rooms.py b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/score_rooms.py
new file mode 100644
index 00000000..04a03d11
--- /dev/null
+++ b/scripts/wallquest-refresh/carl-robinson-viewer/room-engines/score_rooms.py
@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+# Local VLM auto-scorer: grades each rendered room (gemini/kontext/sdxl) vs its swatch
+# on a 1-10 rubric via qwen2.5vl (Ollama, $0), picks the per-SKU winner. Cached +
+# resumable: only scores (sku,engine) pairs not already in scores.json, re-run as
+# renders land. Writes /tmp/cr-room-bakeoff/scores.json.
+import base64, json, os, subprocess, urllib.request, time
+
+VIEW = '/tmp/carl-robinson-viewer'
+ROOMS_LOCAL = '/Users/macstudio3/Projects/Designer-Wallcoverings/scripts/wallquest-refresh/carl-robinson-viewer/rooms-local'
+SCORES = '/tmp/cr-room-bakeoff/scores.json'
+CONN = os.environ.get('DATABASE_URL', 'postgresql://dw_admin@127.0.0.1:5432/dw_unified')
+MODEL = 'qwen2.5vl:7b'
+
+RUBRIC = ("Image 1 is a wallpaper SWATCH. Image 2 is an interior room where that wallpaper should "
+ "cover the large accent wall. Rate image 2 as a product room-setting, 1-10 each: "
+ "coverage (WHOLE wall papered, no blank patches), seamless (no tiling seams/repeated-panel "
+ "artifacts), texture_match (wall texture matches the swatch weave), realism (real photo, furniture "
+ "intact). Respond ONLY JSON: {\"coverage\":n,\"seamless\":n,\"texture_match\":n,\"realism\":n,\"overall\":n,\"note\":\"<8 words\"}")
+
+def b64(p): return base64.b64encode(open(p, 'rb').read()).decode()
+
+def score(swatch, room):
+ body = json.dumps({"model": MODEL, "prompt": RUBRIC, "images": [b64(swatch), b64(room)],
+ "format": "json", "stream": False, "options": {"temperature": 0}}).encode()
+ req = urllib.request.Request("http://localhost:11434/api/generate", body, {"Content-Type": "application/json"})
+ r = json.load(urllib.request.urlopen(req, timeout=240))
+ return json.loads(r["response"])
+
+# gemini room path per sku from DB
+gem = {}
+q = "SELECT dw_sku, COALESCE(room_setting_images::text,'[]') FROM carl_robinson_catalog ORDER BY dw_sku"
+for line in subprocess.run(['psql', CONN, '-tAc', q], capture_output=True, text=True).stdout.splitlines():
+ if '|' not in line: continue
+ sku, arr = line.split('|', 1)
+ try: a = json.loads(arr); gem[sku] = a[0] if a else ''
+ except Exception: gem[sku] = ''
+
+rows = [l.rstrip('\n').split('|') for l in open('/tmp/cr-room-bakeoff/manifest.tsv') if l.strip()]
+data = json.load(open(SCORES)) if os.path.exists(SCORES) else {}
+
+scored = 0
+for sku, color, mat, img in rows:
+ swatch = f'{VIEW}/{img}'
+ if not os.path.exists(swatch): continue
+ engines = {'gemini': f'{VIEW}/{gem.get(sku,"")}' if gem.get(sku) else None,
+ 'kontext': f'{ROOMS_LOCAL}/{sku}-kontext.png',
+ 'sdxl': f'{ROOMS_LOCAL}/{sku}-sdxl.png'}
+ rec = data.setdefault(sku, {})
+ for eng, path in engines.items():
+ if not path or not os.path.exists(path): continue
+ if eng in rec and 'overall' in rec[eng]: continue # cached
+ try:
+ s = score(swatch, path); rec[eng] = s; scored += 1
+ print(f'{sku} {eng}: overall={s.get("overall")} {s.get("note","")}', flush=True)
+ json.dump(data, open(SCORES, 'w'), indent=1) # persist after each
+ except Exception as e:
+ print(f'{sku} {eng} ERR {str(e)[:80]}', flush=True)
+ # winner = highest overall among scored engines
+ graded = {e: v['overall'] for e, v in rec.items() if isinstance(v, dict) and 'overall' in v}
+ if graded:
+ rec['winner'] = max(graded, key=graded.get)
+json.dump(data, open(SCORES, 'w'), indent=1)
+wins = {}
+for sku, rec in data.items():
+ w = rec.get('winner'); wins[w] = wins.get(w, 0) + 1 if w else wins.get(w, 0)
+print(f'\nSCORED {scored} new. winner tally: {wins}')
← 0302bcad Local room-engine bake-off: FLUX Kontext + SDXL/IP-Adapter +
·
back to Designer Wallcoverings
·
Reusable local room-render pipeline (any DW catalog) 16b10021 →