← back to Enrich Local Hybrid
Route local vision leg through shared exo-vision lib, drop direct Ollama call (TK-12090 Lane E)
63ea825122f5c9fe966a0c51b2caf55f984b7245 · 2026-09-23 14:52:45 -0700 · Steve Abrams
enrich-local.js's ollamaVL() hit a hardcoded Mac tailnet Ollama host
directly; Ollama is retired fleet-wide. Now calls
~/Projects/_shared/lib/exo-vision.mjs visionChat() (exo ring primary $0,
Gemini fallback cost-ledgered). Ollama's grammar-constrained format:schema
has no lib equivalent, so the schema is now spelled out in-prompt and
parsed leniently. Output aiData shape is unchanged (verified against
enrich-ai-tags.patched.js's field reads).
Found in the process: Kamatera's live full-monte/.env already points
ENRICH_OLLAMA_URL at 100.82.17.107:11434 (a 2026-06-30 Mac1->Mac2 tailnet
failover) with ENRICH_PROVIDER=local already live — so production has
been silently 100% Gemini-billed since Ollama died on that host, with the
"local hybrid" fallback swallowing every failure. Verified Kamatera can
reach that same host's exo ring at :52415 (HTTP 200). DEPLOY.md documents
the fix + the required VISION_URL env cutover; the actual Kamatera
redeploy is left gated for Steve (live prod script + env change).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KXUyzc9vybUz39rhnNJdwY
Files touched
M DEPLOY.mdM enrich-local.jsM enrich-palette.pyM test-local.js
Diff
commit 63ea825122f5c9fe966a0c51b2caf55f984b7245
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Wed Sep 23 14:52:45 2026 -0700
Route local vision leg through shared exo-vision lib, drop direct Ollama call (TK-12090 Lane E)
enrich-local.js's ollamaVL() hit a hardcoded Mac tailnet Ollama host
directly; Ollama is retired fleet-wide. Now calls
~/Projects/_shared/lib/exo-vision.mjs visionChat() (exo ring primary $0,
Gemini fallback cost-ledgered). Ollama's grammar-constrained format:schema
has no lib equivalent, so the schema is now spelled out in-prompt and
parsed leniently. Output aiData shape is unchanged (verified against
enrich-ai-tags.patched.js's field reads).
Found in the process: Kamatera's live full-monte/.env already points
ENRICH_OLLAMA_URL at 100.82.17.107:11434 (a 2026-06-30 Mac1->Mac2 tailnet
failover) with ENRICH_PROVIDER=local already live — so production has
been silently 100% Gemini-billed since Ollama died on that host, with the
"local hybrid" fallback swallowing every failure. Verified Kamatera can
reach that same host's exo ring at :52415 (HTTP 200). DEPLOY.md documents
the fix + the required VISION_URL env cutover; the actual Kamatera
redeploy is left gated for Steve (live prod script + env change).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KXUyzc9vybUz39rhnNJdwY
---
DEPLOY.md | 82 +++++++++++++++++++++++++++++++++++++-----------
enrich-local.js | 94 ++++++++++++++++++++++++++++++++++---------------------
enrich-palette.py | 5 +--
test-local.js | 5 ++-
4 files changed, 128 insertions(+), 58 deletions(-)
diff --git a/DEPLOY.md b/DEPLOY.md
index 0f54b19..159098d 100644
--- a/DEPLOY.md
+++ b/DEPLOY.md
@@ -1,29 +1,66 @@
# enrich-ai-tags → local hybrid (build + you-deploy)
**What:** swaps DW image enrichment from paid Gemini to a HYBRID local pipeline —
-exact hex+% from real pixels (Pillow, ground-truth, beats Gemini) + qwen2.5vl on
-Mac1 (via Tailscale from Kamatera) for color names / styles / patterns / material /
-imageType / dims / description. Same `aiData` output contract; ~$0.
+exact hex+% from real pixels (Pillow, ground-truth, beats Gemini) + a vision LLM
+(exo ring, Gemini fallback) for color names / styles / patterns / material /
+imageType / dims / description. Same `aiData` output contract; ~$0 when the ring
+is live, cost-ledgered Gemini calls otherwise.
**Safety:** `ENRICH_PROVIDER` defaults to `gemini`, so dropping the files changes
-nothing until you set the env. Any local failure auto-falls-back to Gemini. Verified:
-Kamatera (100.107.67.67) reaches Mac1 Ollama (100.94.103.98:11434, qwen2.5vl present).
+nothing until you set the env. Any local failure auto-falls-back to Gemini.
+
+## ⚠️ LIVE PRODUCTION IS CURRENTLY SILENTLY GEMINI-ONLY (found 2026-09-23, TK-12090)
+
+`ENRICH_PROVIDER=local` is **already set** in `/root/DW-Agents/full-monte/.env` on
+Kamatera, and the patched `enrich-ai-tags.js` **is already deployed** there
+(`localAnalyze` present) — this pipeline has been running in production every
+Phase-3 cron since it was deployed. But `ENRICH_OLLAMA_URL` in that same `.env`
+points at `http://100.82.17.107:11434` (Mac2's tailnet IP — a failover applied
+2026-06-30 when Mac1 went offline, noted inline in that file). **Ollama on port
+11434 is retired** (zombie — the port accepts the connection, zero models are
+loaded, every `/api/generate` call fails). `analyze()`'s `catch` swallows that
+failure and falls through to Gemini on *every single image*, so the "local
+hybrid, ~$0" pipeline has actually been 100% Gemini-billed since Ollama was
+retired, with nothing alerting on it (the `console.log` fallback line is the
+only trace, and nothing reads Phase-3 logs for it).
+
+Verified 2026-09-23 (TK-12090): Kamatera → `100.94.103.98:11434` (the original
+Mac1 tailnet address) is now **completely unroutable** — that device is no
+longer even a member of the tailnet (not in `tailscale status` at all, 100%
+ping loss). Kamatera → `100.82.17.107:52415` (this Mac2/macstudio3's exo ring,
+over the SAME tailnet route the failover already uses) returns **HTTP 200** —
+the exo ring IS reachable from Kamatera today, just not on the port the live
+`.env` names.
+
+**The fix below (this repo, already applied) plus a redeploy + one `.env` line
+change on Kamatera closes the gap.** The redeploy is a live-production script
+replacement + env edit on a revenue-adjacent pipeline — left for Steve to run
+or explicitly approve, per standing gate rules; not executed by this ticket.
## Files
-- `enrich-palette.py` — Pillow ground-truth palette (exact hex + %)
-- `enrich-local.js` — hybrid analyzer, exports localAnalyze() (geminiAnalyze shape)
+- `enrich-palette.py` — Pillow ground-truth palette (exact hex + %), unchanged
+- `enrich-local.js` — hybrid analyzer, exports localAnalyze() (geminiAnalyze shape).
+ TK-12090 Lane E: now routes vision through `~/Projects/_shared/lib/exo-vision.mjs`
+ instead of a direct Ollama call.
- `enrich-ai-tags.patched.js` — prod script + analyze() dispatcher (off by default)
## Deploy (run from Mac2)
```bash
# 1. Back up the live prod script
-ssh my-server 'cp /root/DW-Agents/vendor-scrapers/enrich-ai-tags.js /root/DW-Agents/vendor-scrapers/enrich-ai-tags.js.bak-gemini-$(date +%Y%m%d)'
+ssh my-server 'cp /root/DW-Agents/vendor-scrapers/enrich-ai-tags.js /root/DW-Agents/vendor-scrapers/enrich-ai-tags.js.bak-pre-exo-$(date +%Y%m%d)'
-# 2. Copy the 3 files up; swap the patched main into place
+# 2. Copy the 3 files up (enrich-local.js now needs ~/Projects/_shared/lib/exo-vision.mjs
+# reachable at the SAME relative path on Kamatera — ../_shared/lib/exo-vision.mjs from
+# vendor-scrapers/ — deploy that lib dir too if it isn't already on the box)
scp enrich-local.js enrich-palette.py enrich-ai-tags.patched.js my-server:/root/DW-Agents/vendor-scrapers/
+ssh my-server 'mkdir -p /root/DW-Agents/_shared/lib' # if not already present
+scp ../_shared/lib/exo-vision.mjs my-server:/root/DW-Agents/_shared/lib/
ssh my-server 'cd /root/DW-Agents/vendor-scrapers && mv enrich-ai-tags.patched.js enrich-ai-tags.js'
-# 3. DRY-RUN with local provider (NO DB writes) — proves Kamatera→Mac1 + quality
+# 3. Fix the .env — replace the dead ENRICH_OLLAMA_URL:11434 line with VISION_URL:52415
+ssh my-server "sed -i 's#^ENRICH_OLLAMA_URL=.*#VISION_URL=http://100.82.17.107:52415#' /root/DW-Agents/full-monte/.env"
+
+# 4. DRY-RUN with local provider (NO DB writes) — proves Kamatera→exo-ring + quality
ssh my-server 'cd /root/DW-Agents/vendor-scrapers && ENRICH_PROVIDER=local node enrich-ai-tags.js thibaut --dry-run --limit 3'
```
@@ -31,19 +68,26 @@ ssh my-server 'cd /root/DW-Agents/vendor-scrapers && ENRICH_PROVIDER=local node
The nightly enrichment is NOT `node enrich-ai-tags.js` directly — it's three Phase-3
crons (tier 1/2/3 @ 9/10/11 UTC ≈ 2-4am PT) running `full-monte-batch.js --phase 3`,
which SPAWNS `enrich-ai-tags.js` with `env: {...process.env}`. All three source
-`/root/DW-Agents/full-monte/.env`, so the entire go-live is ONE line in that file:
-```bash
-ssh my-server 'grep -q "^ENRICH_PROVIDER=" /root/DW-Agents/full-monte/.env || echo "ENRICH_PROVIDER=local" >> /root/DW-Agents/full-monte/.env'
-```
-Takes effect next Phase-3 run (cron re-sources .env each run; no restart). Covers all tiers.
+`/root/DW-Agents/full-monte/.env`, so once `VISION_URL` is set per step 3 above,
+`ENRICH_PROVIDER=local` (already set) takes effect next Phase-3 run — cron re-sources
+`.env` each run, no restart needed. Covers all tiers.
**Rollback (instant):**
```bash
ssh my-server "sed -i '/^ENRICH_PROVIDER=/d' /root/DW-Agents/full-monte/.env"
```
-Or restore `enrich-ai-tags.js.bak-gemini-*`. Per-image Gemini fallback also auto-covers any local outage.
+Or restore `enrich-ai-tags.js.bak-pre-exo-*`. Per-image Gemini fallback also auto-covers
+any local outage (exo ring unreachable, or up with no live vision instance).
-## Env knobs
+## Env knobs (canonical exo-vision-lib names — shared with every Lane in TK-12090)
- `ENRICH_PROVIDER` = local | hybrid | gemini (default gemini)
-- `ENRICH_OLLAMA_URL` = http://100.94.103.98:11434 (Mac1 tailnet; default)
-- `ENRICH_VL_MODEL` = qwen2.5vl:7b (default)
+- `VISION_URL` = exo ring base. **On Kamatera this must be a tailnet/LAN address that
+ can reach a Mac in the ring** — verified working 2026-09-23:
+ `http://100.82.17.107:52415` (this Mac2/macstudio3). The shared lib's own default
+ (`127.0.0.1:52415`) is only correct for local Mac testing via `test-local.js`.
+- `VISION_MODEL` = vision model id (default from the shared lib:
+ `mlx-community/Qwen3-VL-4B-Instruct-4bit`). `ENRICH_VL_MODEL` accepted as a legacy alias.
+- `VISION_FALLBACK` = gemini (default) | none — vision-only; text has no fallback in the
+ shared lib (this pipeline is vision-only, so that doesn't apply here).
+- `GEMINI_API_KEY` — read from env or `~/Projects/secrets-manager/.env` by the shared lib;
+ every Gemini fallback call is logged to `~/.claude/cost-ledger.jsonl`.
diff --git a/enrich-local.js b/enrich-local.js
index 5d49240..5c2da30 100644
--- a/enrich-local.js
+++ b/enrich-local.js
@@ -1,63 +1,85 @@
// enrich-local.js — HYBRID local enrichment, drop-in companion to enrich-ai-tags.js.
// Ground-truth hex + percentages from real pixels (Pillow, more accurate than any VLM),
-// qwen2.5vl for the semantic fields (color names, styles, patterns, material, imageType,
+// a vision LLM for the semantic fields (color names, styles, patterns, material, imageType,
// dims, description, usability). Returns the SAME aiData shape as geminiAnalyze().
//
-// Env:
-// ENRICH_OLLAMA_URL Ollama base (default Mac1 via tailnet for Kamatera)
-// ENRICH_VL_MODEL vision model (default qwen2.5vl:7b)
-// ENRICH_PY python3 path (default 'python3')
+// TK-12090 Lane E (2026-09-23): this used to hit a direct Ollama /api/generate on a Mac
+// tailnet host. Ollama is RETIRED (a zombie — the port accepts connections but ZERO models
+// are loaded, so every generate call failed and silently fell through to the Gemini
+// fallback below, at full per-image cost, since the day Ollama died). Now routed through
+// ~/Projects/_shared/lib/exo-vision.mjs: exo ring primary ($0 local), Gemini fallback
+// (cost-ledgered) if the ring is unreachable or has no live vision instance right now.
+//
+// Env (canonical exo-vision-lib names — shared with every other Lane in TK-12090):
+// VISION_URL exo ring base (default 127.0.0.1:52415 — correct for local Mac
+// testing via test-local.js). On the KAMATERA deploy this MUST be
+// overridden to a tailnet/LAN address that can reach a Mac in the
+// ring — see DEPLOY.md "Env knobs" for the verified reachable host.
+// VISION_MODEL vision model id (default from the shared lib:
+// mlx-community/Qwen3-VL-4B-Instruct-4bit). ENRICH_VL_MODEL accepted
+// as a legacy alias.
+// VISION_FALLBACK gemini (default) | none
+// ENRICH_PY python3 path (default 'python3')
const { spawnSync } = require('child_process');
-const http = require('http');
const path = require('path');
const fs = require('fs');
const os = require('os');
+const { pathToFileURL } = require('url');
-const OLLAMA_URL = process.env.ENRICH_OLLAMA_URL || 'http://100.94.103.98:11434'; // Mac1 tailnet
-const VL_MODEL = process.env.ENRICH_VL_MODEL || 'qwen2.5vl:7b';
-const PY = process.env.ENRICH_PY || 'python3';
+const EXO_VISION_LIB = process.env.EXO_VISION_LIB
+ || path.join(__dirname, '..', '_shared', 'lib', 'exo-vision.mjs');
+const PY = process.env.ENRICH_PY || 'python3';
const PALETTE_PY = path.join(__dirname, 'enrich-palette.py');
const MAX_COLORS = 6;
+let _libPromise = null;
+function exoLib() {
+ if (!_libPromise) {
+ _libPromise = import(pathToFileURL(EXO_VISION_LIB).href)
+ .catch(e => { _libPromise = null; throw new Error(`exo-vision lib unavailable (${EXO_VISION_LIB}): ${e.message}`); });
+ }
+ return _libPromise;
+}
+
function samplePalette(imgPath, k = MAX_COLORS) {
const r = spawnSync(PY, [PALETTE_PY, imgPath, String(k)], { encoding: 'utf8', timeout: 25000, maxBuffer: 1 << 20 });
if (r.status !== 0 || !r.stdout) throw new Error('palette failed: ' + (r.stderr || (r.error && r.error.message) || 'no output'));
return JSON.parse(r.stdout); // [{hex, percentage}]
}
-function ollamaVL(imageB64, palette) {
+function extractJson(text) {
+ const m = String(text || '').match(/\{[\s\S]*\}/);
+ if (!m) throw new Error(`VL: no JSON object found in response: ${String(text || '').slice(0, 200)}`);
+ return JSON.parse(m[0]);
+}
+
+// Was ollamaVL() using Ollama's grammar-constrained `format: <schema>` — the shared lib
+// has no equivalent (it's a plain chat-completions passthrough to exo/Gemini), so the
+// schema is now spelled out in the prompt and the response is parsed leniently.
+async function visionAnalyze(imageB64, palette) {
const prompt =
`This wallcovering/fabric image was pixel-sampled into these EXACT colors (hex + area%): ` +
`${JSON.stringify(palette)} . In the SAME ORDER, give a designer color name for each hex. ` +
`Also: backgroundIndex (0-based index of the base/background color in that list), styles, ` +
`patterns, material, imageType (scan_swatch|scan_flatbed|photo_full|photo_crop|render), ` +
`physicalWidthInches (number), physicalHeightInches (number), usable (false only if blank/` +
- `corrupt/not a product), rejectionReason (short, "" if usable), description (one sentence).`;
- const schema = { type: 'object', properties: {
- colorNames: { type: 'array', items: { type: 'string' } },
- backgroundIndex: { type: 'integer' },
- styles: { type: 'array', items: { type: 'string' } },
- patterns: { type: 'array', items: { type: 'string' } },
- material: { type: 'string' }, imageType: { type: 'string' },
- physicalWidthInches: { type: 'number' }, physicalHeightInches: { type: 'number' },
- usable: { type: 'boolean' }, rejectionReason: { type: 'string' }, description: { type: 'string' },
- }, required: ['colorNames','backgroundIndex','styles','patterns','material','imageType','physicalWidthInches','description','usable'] };
- // keep_alive holds the 7B model resident on Mac1 between calls so only the FIRST
- // request pays the cold-load (which over the Kamatera→Mac1 tailnet exceeded the old
- // 120s ceiling and timed out). 300s timeout absorbs that first cold load.
- const body = JSON.stringify({ model: VL_MODEL, prompt, images: [imageB64], stream: false, format: schema, keep_alive: '15m', options: { temperature: 0.1 } });
- return new Promise((resolve, reject) => {
- const u = new URL(OLLAMA_URL + '/api/generate');
- const req = http.request(
- { hostname: u.hostname, port: u.port || 11434, path: u.pathname, method: 'POST',
- headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) }, timeout: 300000 },
- res => { let raw = ''; res.on('data', c => raw += c); res.on('end', () => {
- try { resolve(JSON.parse(JSON.parse(raw).response)); } catch (e) { reject(new Error('VL parse: ' + e.message)); }
- }); });
- req.on('error', reject);
- req.on('timeout', () => req.destroy(new Error('VL timeout')));
- req.write(body); req.end();
+ `corrupt/not a product), rejectionReason (short, "" if usable), description (one sentence).\n\n` +
+ `Respond with ONLY a single raw JSON object — no code fences, no commentary — with exactly ` +
+ `these keys: colorNames (array of strings, same order/length as the color list above), ` +
+ `backgroundIndex (integer), styles (array of strings), patterns (array of strings), ` +
+ `material (string), imageType (string), physicalWidthInches (number), physicalHeightInches ` +
+ `(number), usable (boolean), rejectionReason (string), description (string).`;
+
+ const lib = await exoLib();
+ const r = await lib.visionChat({
+ prompt,
+ image: { b64: imageB64, mime: 'image/jpeg' },
+ model: process.env.VISION_MODEL || process.env.ENRICH_VL_MODEL || undefined,
+ timeoutMs: 120000,
+ maxTokens: 800,
});
+ if (!r.ok) throw new Error(r.error || 'vision call not measured (exo ring + Gemini fallback both unavailable)');
+ return extractJson(r.text);
}
// Drop junk values ("None"/"N/A"/empty), de-dup, cap to 4 — keeps tag sets clean.
@@ -87,7 +109,7 @@ async function localAnalyze(imageUrl, mode, fetchBuffer) {
const sampled = samplePalette(tmp, MAX_COLORS); // {palette, image_b64 (normalized JPEG)}
const palette = sampled.palette || [];
if (!sampled.image_b64) throw new Error('palette: no normalized image');
- const vl = await ollamaVL(sampled.image_b64, palette); // normalized → ollama always loads it
+ const vl = await visionAnalyze(sampled.image_b64, palette);
if (vl.usable === false) {
return { image_rejected: true, rejection_reason: vl.rejectionReason || 'not a usable product image' };
}
diff --git a/enrich-palette.py b/enrich-palette.py
index 20e748b..e505c0b 100644
--- a/enrich-palette.py
+++ b/enrich-palette.py
@@ -2,8 +2,9 @@
"""Ground-truth palette + a NORMALIZED image for the VL.
- palette: exact hex + area-% from real pixels (more accurate than any VLM), near-dup merged.
- image_b64: the image re-encoded as a clean RGB JPEG, long-edge-capped. Normalizing through
- Pillow guarantees ollama/llama-server can load it (catalog images are often CMYK JPEG / WebP /
- odd encodings that Pillow reads but the VL image loader rejects → "Failed to load image").
+ Pillow guarantees the vision model (exo ring / Gemini) can load it (catalog images are often
+ CMYK JPEG / WebP / odd encodings that Pillow reads but the VL image loader rejects →
+ "Failed to load image").
Usage: enrich-palette.py <image> [k] → prints {"palette":[...], "image_b64":"..."}"""
import sys, json, io, base64, os
from PIL import Image
diff --git a/test-local.js b/test-local.js
index 2b759ad..5e832c4 100644
--- a/test-local.js
+++ b/test-local.js
@@ -1,5 +1,8 @@
const fs = require('fs');
-process.env.ENRICH_OLLAMA_URL = process.env.ENRICH_OLLAMA_URL || 'http://127.0.0.1:11434'; // Mac2 local for test
+// TK-12090 Lane E: was forcing Ollama at 127.0.0.1:11434 (retired). Force the exo ring on
+// this Mac for a local test run instead — matches the shared lib's own default, spelled
+// out here so the test is explicit about which host it's hitting.
+process.env.VISION_URL = process.env.VISION_URL || 'http://127.0.0.1:52415';
const { localAnalyze } = require('./enrich-local');
// fetchBuffer stub: read a local file path as if it were a fetched image
const fetchBuffer = async (p) => fs.readFileSync(p);
← 0efb0b0 reconcile-image-less: guard mfr_sku col + scope to monitor's
·
back to Enrich Local Hybrid
·
enrich-local: fast-fail exo (15s) + consecutive-failure circ 5193103 →