← back to Dw Photo Capture
TK-12090: decouple exo VLM ring from HAS_MACVISION, make it the reasoning-path default
8452fcc63bc5deaf63ea645989621a25ae17504b · 2026-09-24 12:49:06 -0700 · Steve Abrams
/api/recognize and /api/identify could never reach the exo ring on Kamatera
(Linux): the ring was only wired into engine==='macvision', and
engineAvailable('macvision') is gated on HAS_MACVISION = IS_DARWIN &&
bin/ocr exists — always false on prod. VISION_URL config + /healthz were
correctly set but no route could actually consume it (confirmed live:
step-3 verify returned provider=gcv, not exo/gemini).
Adds 'exo' as its own engine, decoupled from platform: engineAvailable('exo')
= HAS_EXO (the shared exo-vision.mjs lib is present on this host), a static
config check mirroring HAS_MACVISION's own fs.existsSync pattern. Live
reachability (ring up/down, Gemini fallback, honest not_measured) stays
inside exo-vision.mjs's visionChat() where it already lived correctly.
visionJSON() now routes 'exo' to ollamaVision alongside 'macvision'. The two
reasoning-call sites (/api/identify, /api/recognize) prefer exo when
configured and no explicit {engine:...} was requested by the client -
DEFAULT_ENGINE (and /api/ocr's literal-OCR routing) is untouched, so this
never affects OCR engine selection. provider/model/cost_usd in both
responses already read the real per-call values from the lib, so a Gemini
fallback inside the ring will correctly still report provider:"gemini", not
a false "exo".
Verified: node --check passes; exo-vision.mjs confirmed present on Kamatera
prod at the real deploy path (/root/public-projects/_shared/lib/exo-vision.mjs).
Not yet deployed to prod.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vkV6GBLJ9rN1VQ8oM8HJc
Files touched
Diff
commit 8452fcc63bc5deaf63ea645989621a25ae17504b
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Thu Sep 24 12:49:06 2026 -0700
TK-12090: decouple exo VLM ring from HAS_MACVISION, make it the reasoning-path default
/api/recognize and /api/identify could never reach the exo ring on Kamatera
(Linux): the ring was only wired into engine==='macvision', and
engineAvailable('macvision') is gated on HAS_MACVISION = IS_DARWIN &&
bin/ocr exists — always false on prod. VISION_URL config + /healthz were
correctly set but no route could actually consume it (confirmed live:
step-3 verify returned provider=gcv, not exo/gemini).
Adds 'exo' as its own engine, decoupled from platform: engineAvailable('exo')
= HAS_EXO (the shared exo-vision.mjs lib is present on this host), a static
config check mirroring HAS_MACVISION's own fs.existsSync pattern. Live
reachability (ring up/down, Gemini fallback, honest not_measured) stays
inside exo-vision.mjs's visionChat() where it already lived correctly.
visionJSON() now routes 'exo' to ollamaVision alongside 'macvision'. The two
reasoning-call sites (/api/identify, /api/recognize) prefer exo when
configured and no explicit {engine:...} was requested by the client -
DEFAULT_ENGINE (and /api/ocr's literal-OCR routing) is untouched, so this
never affects OCR engine selection. provider/model/cost_usd in both
responses already read the real per-call values from the lib, so a Gemini
fallback inside the ring will correctly still report provider:"gemini", not
a false "exo".
Verified: node --check passes; exo-vision.mjs confirmed present on Kamatera
prod at the real deploy path (/root/public-projects/_shared/lib/exo-vision.mjs).
Not yet deployed to prod.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vkV6GBLJ9rN1VQ8oM8HJc
---
server.js | 48 +++++++++++++++++++++++++++++++++++-------------
1 file changed, 35 insertions(+), 13 deletions(-)
diff --git a/server.js b/server.js
index 66e3f30..c7dffaf 100644
--- a/server.js
+++ b/server.js
@@ -677,9 +677,9 @@ async function ollamaVision(b64, prompt, timeoutMs, model) {
} catch (e) { return { error: e.message, not_measured: true, provider: null, cost_usd: 0 }; }
}
-// ── Pluggable vision engines: macvision (Apple Vision + local Ollama) · gemini · gcv ──
-// Steve 2026-07-06: so the scanner runs on Kamatera (Linux), where the macOS `bin/ocr`
-// Vision binary and a local Ollama don't exist. Each engine is selectable per request
+// ── Pluggable vision engines: macvision (Apple Vision OCR) · exo (VLM ring, reasoning) ·
+// gemini · gcv ── Steve 2026-07-06: so the scanner runs on Kamatera (Linux), where the
+// macOS `bin/ocr` Vision binary doesn't exist. Each engine is selectable per request
// ({engine:"gemini"} or ?engine=gcv); engine:"all" fans out to every AVAILABLE engine and
// returns each result for comparison. Default = macvision on macOS, gemini elsewhere.
const GEMINI_API_KEY = process.env.GEMINI_API_KEY || '';
@@ -687,15 +687,26 @@ const GEMINI_VISION_MODEL = process.env.GEMINI_VISION_MODEL || 'gemini-2.5-flash
const GCV_API_KEY = process.env.GCV_API_KEY || process.env.GOOGLE_VISION_API_KEY || '';
const IS_DARWIN = process.platform === 'darwin';
const HAS_MACVISION = IS_DARWIN && fs.existsSync(path.join(ROOT, 'bin/ocr'));
+// TK-12090: 'exo' is a DISTINCT engine from 'macvision' — it maps to the reasoning/identify
+// VLM ring (ollamaVision → exoLib, below) and is cross-platform (unlike bin/ocr, which is
+// macOS-only and stays the literal-OCR meaning of 'macvision'). "Available" here means
+// CONFIGURED (the shared lib is present on this host), matching HAS_MACVISION's own static
+// fs.existsSync check — live reachability (ring up/down, Gemini fallback, honest
+// not_measured) is handled per-call inside exo-vision.mjs's visionChat(), not here.
+const HAS_EXO = fs.existsSync(EXO_VISION_LIB);
// Prefer gemini whenever a key is present: macvision's identify path routes to the
// retired Ollama (qwen2.5vl), so gemini is the only working OCR engine. Override with
// OCR_ENGINE=macvision to force the local path back. (Steve chose gemini, 2026-09-18.)
const DEFAULT_ENGINE = (process.env.OCR_ENGINE || (GEMINI_API_KEY ? 'gemini' : (HAS_MACVISION ? 'macvision' : 'gemini'))).toLowerCase();
-const ALL_ENGINES = ['macvision', 'gemini', 'gcv'];
+const ALL_ENGINES = ['macvision', 'exo', 'gemini', 'gcv'];
// Rough per-scan cost so the caller always sees the $ (Steve's "always show costs" rule).
-const ENGINE_COST_USD = { macvision: 0, gemini: 0.0006, gcv: 0.0015 };
+// exo is $0 when the ring serves it; if it silently falls back to Gemini inside the lib,
+// the REAL cost comes back on r.cost_usd from ollamaVision (this table is only the display
+// default before a call has run — see /api/ocr and /api/identify's cost_usd fields).
+const ENGINE_COST_USD = { macvision: 0, exo: 0, gemini: 0.0006, gcv: 0.0015 };
function engineAvailable(e) {
if (e === 'macvision') return HAS_MACVISION;
+ if (e === 'exo') return HAS_EXO;
if (e === 'gemini') return !!GEMINI_API_KEY;
if (e === 'gcv') return !!GCV_API_KEY;
return false;
@@ -877,12 +888,15 @@ function runOcr(engine, buf) {
});
}
-// Reasoning vision (brand/pattern identify) for one engine. macvision => local Ollama;
-// gemini/gcv => Gemini (GCV is OCR-only, so identify always routes to Gemini there).
-// Returns { response:<json-text>, model, error } to match the Ollama shape callers parse.
+// Reasoning vision (brand/pattern identify) for one engine. exo => the VLM ring (primary
+// reasoning path, TK-12090); macvision => local Ollama (legacy macOS-only alias, kept for
+// back-compat); gemini/gcv => Gemini (GCV is OCR-only, so identify always routes to Gemini
+// there). Returns { response:<json-text>, model, provider, cost_usd, error } — provider/
+// cost_usd come straight from exo-vision.mjs's visionChat() so the real serving engine
+// (exo vs its Gemini fallback) and real $ are always visible, never assumed.
function visionJSON(engine, b64, prompt, opts) {
opts = opts || {};
- if (engine === 'macvision') return ollamaVision(b64, prompt, opts.timeoutMs, opts.model);
+ if (engine === 'exo' || engine === 'macvision') return ollamaVision(b64, prompt, opts.timeoutMs, opts.model);
return geminiVision(b64, prompt, opts.timeoutMs || 30000);
}
@@ -1468,8 +1482,12 @@ const appHandler = (req, res) => {
const b64 = p.dataUrl.replace(/^data:image\/\w+;base64,/, '');
const prompt = 'This is a wallcovering or fabric SAMPLE label. Identify the BRAND from its logo or wordmark and note the typesetting. Reply ONLY as compact JSON: {"brand":"<manufacturer/brand or empty if unsure>","confidence":<0-1>,"logo":"<short logo/wordmark description>","typeface":"<e.g. serif wordmark / sans caps / script>","code":"<any SKU or model number visible, else empty>"}';
// engine-pluggable ({engine:...}); macvision keeps the model toggle ({model:.. }/{fast:true} → moondream).
- const engine = resolveEngines(p.engine)[0] || DEFAULT_ENGINE;
- const useModel = engine === 'macvision' ? pickVisionModel(p.fast ? 'fast' : p.model) : GEMINI_VISION_MODEL;
+ // TK-12090: prefer exo (the $0 VLM ring) for reasoning/identify calls specifically —
+ // NOT a change to the global DEFAULT_ENGINE, which /api/ocr (literal-OCR, exo doesn't
+ // do char-level transcription) still resolves independently and unaffected. An explicit
+ // client {engine:...} always wins (p.engine short-circuits before this ever applies).
+ const engine = resolveEngines(p.engine || (HAS_EXO ? 'exo' : undefined))[0] || DEFAULT_ENGINE;
+ const useModel = (engine === 'macvision' || engine === 'exo') ? pickVisionModel(p.fast ? 'fast' : p.model) : GEMINI_VISION_MODEL;
const r = await visionJSON(engine, b64, prompt, { timeoutMs: p.fast ? 25000 : undefined, model: useModel });
let out = {}; try { out = JSON.parse(r.response || '{}'); } catch (e) { out = {}; }
const brand = (out.brand || '').toString().trim();
@@ -1501,7 +1519,11 @@ const appHandler = (req, res) => {
if (typeof p.dataUrl !== 'string' || !p.dataUrl) return send(res, 400, { err: 'dataUrl required (string)' });
const b64 = p.dataUrl.replace(/^data:image\/\w+;base64,/, '');
const prompt = 'You are looking at a wallcovering or fabric SWATCH — the material itself, NOT a printed label. Describe the PATTERN so it can be matched against a catalog. Reply ONLY as compact JSON: {"description":"<one short sentence>","motif":"<main motif e.g. floral, damask, geometric, grasscloth, stripe, botanical, abstract, ikat, toile>","style":"<e.g. traditional, modern, transitional, scandinavian, art deco, contemporary>","material":"<e.g. grasscloth, non-woven, silk, vinyl, paper, leather>","colors":["<color name>","<color name>"],"background":"<background color name>","scale":"<small | medium | large>","code":"<any SKU or model number printed on it, else empty>"}';
- const engine = resolveEngines(p.engine)[0] || DEFAULT_ENGINE;
+ // TK-12090: prefer exo (the $0 VLM ring) for reasoning/identify calls specifically —
+ // NOT a change to the global DEFAULT_ENGINE, which /api/ocr (literal-OCR, exo doesn't
+ // do char-level transcription) still resolves independently and unaffected. An explicit
+ // client {engine:...} always wins (p.engine short-circuits before this ever applies).
+ const engine = resolveEngines(p.engine || (HAS_EXO ? 'exo' : undefined))[0] || DEFAULT_ENGINE;
const r = await visionJSON(engine, b64, prompt, { timeoutMs: 90000 }); // allow for a cold model load
let a = {}; try { a = JSON.parse(r.response || '{}'); } catch (e) { a = {}; }
const terms = similarTerms(a);
@@ -1526,7 +1548,7 @@ const appHandler = (req, res) => {
}
} catch (e) { visual = { error: e.message }; }
}
- return send(res, 200, { ok: !r.error, model: (engine === 'macvision' ? (r.model || OLLAMA_VISION_MODEL) : GEMINI_VISION_MODEL), engine, provider: r.provider || engine, cost_usd: r.cost_usd,
+ return send(res, 200, { ok: !r.error, model: ((engine === 'macvision' || engine === 'exo') ? (r.model || OLLAMA_VISION_MODEL) : GEMINI_VISION_MODEL), engine, provider: r.provider || engine, cost_usd: r.cost_usd,
recognized: {
description: a.description || null, motif: a.motif || null, style: a.style || null,
material: a.material || null, colors: Array.isArray(a.colors) ? a.colors.slice(0, 6) : [],
← 8364615 TK-12126: never report Gemini OCR cost for a call that error
·
back to Dw Photo Capture
·
TK-12162 Pro Booth concept: simple-probooth.html (3-slider c dc3cc71 →