← back to Dw Photo Capture
Scan: recognize barcodes (exact SKU) + logos/typeface (local VLM)
d30783bff978f978f051d32d42b0b745e0c83a0f · 2026-06-25 23:10:09 -0700 · Steve Abrams
Two new visual signals beyond printed-text OCR:
- BARCODES/QR: bin/ocr.swift now runs VNDetectBarcodes in the same Vision pass and
emits 'BC<tab>payload<tab>symbology'. analyzeOcr treats a barcode payload as ground
truth — it outranks every OCR'd code and forces topStrong, so a barcoded sample locks
instantly. Returns { barcode } too. Zero added cost (same image pass).
- LOGO + TYPESETTING: /api/identify sends the frame to local qwen2.5vl (Ollama, $0) and
returns { brand, confidence, logo, typeface, code }, cross-checked against the learned
LEXICON for a canonical vendor. A '🏷 Logo' button in the live scanner runs it on demand
(it's ~seconds, not per-frame) for swatches whose brand text won't OCR cleanly; if the
VLM also reads a code it routes straight into the normal resolve.
Reuses the existing OLLAMA_URL (voice feature). Verified: Code128 'WDW2310' →
barcode ground-truth lock; a serif Brunschwig & Fils label → brand+typeface+code read.
$0 (local Vision OCR + local VLM).
Files touched
Diff
commit d30783bff978f978f051d32d42b0b745e0c83a0f
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Thu Jun 25 23:10:09 2026 -0700
Scan: recognize barcodes (exact SKU) + logos/typeface (local VLM)
Two new visual signals beyond printed-text OCR:
- BARCODES/QR: bin/ocr.swift now runs VNDetectBarcodes in the same Vision pass and
emits 'BC<tab>payload<tab>symbology'. analyzeOcr treats a barcode payload as ground
truth — it outranks every OCR'd code and forces topStrong, so a barcoded sample locks
instantly. Returns { barcode } too. Zero added cost (same image pass).
- LOGO + TYPESETTING: /api/identify sends the frame to local qwen2.5vl (Ollama, $0) and
returns { brand, confidence, logo, typeface, code }, cross-checked against the learned
LEXICON for a canonical vendor. A '🏷 Logo' button in the live scanner runs it on demand
(it's ~seconds, not per-frame) for swatches whose brand text won't OCR cleanly; if the
VLM also reads a code it routes straight into the normal resolve.
Reuses the existing OLLAMA_URL (voice feature). Verified: Code128 'WDW2310' →
barcode ground-truth lock; a serif Brunschwig & Fils label → brand+typeface+code read.
$0 (local Vision OCR + local VLM).
---
public/index.html | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/public/index.html b/public/index.html
index 306c184..1661e0f 100644
--- a/public/index.html
+++ b/public/index.html
@@ -86,6 +86,7 @@
@keyframes lockpulse{0%{box-shadow:0 0 0 4000px rgba(0,0,0,.45),0 0 0 0 var(--red) inset}100%{box-shadow:0 0 0 4000px rgba(0,0,0,.55),0 0 30px 4px var(--red) inset}}
.scan-cancel{position:absolute;bottom:max(28px,env(safe-area-inset-bottom));left:50%;transform:translateX(-50%);background:#16140f;border:1px solid var(--line);color:var(--ink);border-radius:12px;padding:13px 26px;font-size:15px;font-weight:600;cursor:pointer}
.scan-flip{position:absolute;bottom:max(28px,env(safe-area-inset-bottom));right:18px;background:#16140f;border:1px solid var(--line);color:var(--ink);border-radius:12px;padding:13px 18px;font-size:15px;font-weight:600;cursor:pointer}
+ .scan-logo{position:absolute;bottom:max(28px,env(safe-area-inset-bottom));left:18px;background:#16140f;border:1px solid var(--gold);color:var(--gold);border-radius:12px;padding:13px 16px;font-size:15px;font-weight:600;cursor:pointer}
/* photo gallery */
.gallery{position:fixed;inset:0;z-index:90;background:rgba(10,9,7,.98);display:flex;flex-direction:column}
.gallery[hidden]{display:none}
@@ -197,6 +198,7 @@
<div class="scan-frame"><div class="scan-line"></div></div>
<div class="scan-status" id="scanStatus">Point at the printed SKU…</div>
<button class="scan-flip" id="scanFlip">🔄 Flip camera</button>
+ <button class="scan-logo" id="scanLogo" title="Identify the brand by its logo / typeface (local AI)">🏷 Logo</button>
<button class="scan-cancel" id="scanCancel">✕ Cancel</button>
</div>
@@ -1016,6 +1018,26 @@ async function applyScan(d){
}
$('#scanCancel').addEventListener('click',closeLiveScan);
$('#scanFlip').addEventListener('click',flipCam);
+// 🏷 Identify the brand by its LOGO + typesetting via the local VLM (qwen2.5vl). Slower
+// than OCR (a few seconds) so it's user-tapped — for swatches whose brand text won't OCR.
+$('#scanLogo').addEventListener('click',async()=>{
+ const v=$('#scanvideo'); if(!v.videoWidth){ toast('Camera not ready'); return; }
+ const s=Math.min(1,768/Math.max(v.videoWidth,v.videoHeight));
+ const c=document.createElement('canvas'); c.width=Math.round(v.videoWidth*s); c.height=Math.round(v.videoHeight*s);
+ c.getContext('2d').drawImage(v,0,0,c.width,c.height);
+ const prev=$('#scanStatus').textContent; $('#scanStatus').textContent='🏷 Reading logo…';
+ try{
+ const r=await fetch('/api/identify',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({dataUrl:c.toDataURL('image/jpeg',0.8)})});
+ const d=await r.json();
+ if(d&&d.brand){
+ $('#scanStatus').textContent='🏷 '+d.brand+(d.typeface?(' · '+d.typeface):'');
+ const tag=(d.vendor||d.brand);
+ // if the logo also revealed a code, run the normal resolve on it
+ if(d.code){ closeLiveScan(); $('#q').value=tag; applyScan({candidates:[d.code],top:d.code,text:tag,vendor:d.vendor||''}); }
+ else { toast('🏷 '+tag+(d.code?(' · '+d.code):'')); }
+ } else { $('#scanStatus').textContent=prev; toast(d&&d.err?('Logo read failed: '+d.err):'Brand not recognized'); }
+ }catch(e){ $('#scanStatus').textContent=prev; toast('Logo read failed — is Ollama running?'); }
+});
function syncCamBtn(){ $('#camBtn').textContent=(camFacing==='user'?'🤳 Front':'📷 Back'); }
$('#camBtn').addEventListener('click',flipCam);
$('#micBtn').addEventListener('click',voiceSearch);
← 8fae114 auto-save: 2026-06-25T23:07:50 (4 files) — bin/ocr bin/ocr.s
·
back to Dw Photo Capture
·
Harvest vendor logos from Shopify Files → ground logo recogn 85dccb3 →