`API.md`

43 lines
# dw-photo-capture — API notes

Local sample scanner. All recognition is on-device, $0 (macOS Vision OCR + local Ollama VLM).

## `POST /api/ocr`
Body: `{ dataUrl }` (a base64 image data URL). Returns ranked SKU candidates:
`{ ok, text, vendor, candidates[], top, topStrong, barcode }`.
- Barcodes/QR (read in the same Vision pass) are ground truth — they outrank OCR codes and force `topStrong`.
- `vendor` is detected from a brand name printed on the swatch (180 learned vendor profiles).
- `topStrong` = safe to lock on the first read (known prefix, brand+font-dominant, or a barcode).

## `POST /api/identify`
VLM logo/typeface recognition — the assist for when OCR can't read a stylized wordmark.
Body: `{ dataUrl, model?, fast? }`. Returns
`{ ok, model, brand, confidence, logo, typeface, code, vendor, match_via }`.

### Model toggle (speed vs accuracy)
By default `/api/identify` uses **qwen2.5vl:7b** (~15-20s, most accurate). To trade accuracy for speed,
pick a faster model per request:

| Request body | Model used | Notes |
|---|---|---|
| *(none)* | `qwen2.5vl:7b` | default — most accurate |
| `{ "fast": true }` | `moondream:latest` | ~2x faster, lower accuracy (weaker at structured JSON) |
| `{ "model": "moondream" }` | `moondream:latest` | explicit |
| `{ "model": "qwen2.5vl:7b" }` | `qwen2.5vl:7b` | explicit |
| `{ "model": "llava:13b" }` | `llava:13b` | |

Only **allow-listed** models are honored (`VISION_MODELS` in server.js); anything off-list falls back to the
default — a caller can't request an arbitrary model to pull/run. The chosen model is echoed back in `model`.

`vendor` is resolved from the VLM's brand guess by: (1) lexicon regex, (2) fuzzy name match, then
(3) **fingerprint tiebreaker** (only if 1+2 are inconclusive) — comparing the live read to the validated
reference-logo fingerprints. `match_via` reports which path resolved it (`name` | `fingerprint`).

## `GET /api/vendor-profiles` · `POST /api/learn` · `/learn`
What the scanner has learned about each vendor's sample labeling (prefixes, examples, logo fingerprint).
`/api/learn` refines a vendor's profile from a scan that resolved to a real product. `/learn` is the viewer.

## Default vision model via env
`OLLAMA_VISION_MODEL` overrides the default for the whole process (still must be an installed model).
`OLLAMA_URL` (shared with the voice feature) points at the Ollama host.