[object Object]

← back to Dw Photo Capture

harden: prefix support gate so one bad scan can't corrupt learned prefixes (cycle 14)

a3b00cc22c432ecf35d2c1739b8246a270020d97 · 2026-06-26 06:20:58 -0700 · Steve Abrams

Brainstorm idea #1 (learning-loop robustness). profileToLex used ALL prefix keys
regardless of support — so a single /api/learn scan (count 1) immediately injected a
full-weight prefix into the ranking lexicon. For a scan-CREATED vendor, one bad scan
could bias future ranking. Now a prefix needs >=2 supporting scans before it enters the
lexicon; catalog-built prefixes (builder already keeps n>=3) are unaffected, and the
prefix data is still stored (just gated until confirmed — auto-activates on a 2nd scan).

Real-data impact verified: of 291 prefixes, exactly 1 is gated (WC count-1, a test
artifact on Phillip Jeffries which still resolves via DWJP); 0 vendors lose matching.
Exposed profileToLex to the test harness + 5 gate tests (count-1 gated→null, count-2
included, mixed confirmed-matches/unconfirmed-ignored, numeric_share still matches).
41→46 tests, lint clean, restart 200. $0 (local).

Files touched

Diff

commit a3b00cc22c432ecf35d2c1739b8246a270020d97
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Fri Jun 26 06:20:58 2026 -0700

    harden: prefix support gate so one bad scan can't corrupt learned prefixes (cycle 14)
    
    Brainstorm idea #1 (learning-loop robustness). profileToLex used ALL prefix keys
    regardless of support — so a single /api/learn scan (count 1) immediately injected a
    full-weight prefix into the ranking lexicon. For a scan-CREATED vendor, one bad scan
    could bias future ranking. Now a prefix needs >=2 supporting scans before it enters the
    lexicon; catalog-built prefixes (builder already keeps n>=3) are unaffected, and the
    prefix data is still stored (just gated until confirmed — auto-activates on a 2nd scan).
    
    Real-data impact verified: of 291 prefixes, exactly 1 is gated (WC count-1, a test
    artifact on Phillip Jeffries which still resolves via DWJP); 0 vendors lose matching.
    Exposed profileToLex to the test harness + 5 gate tests (count-1 gated→null, count-2
    included, mixed confirmed-matches/unconfirmed-ignored, numeric_share still matches).
    41→46 tests, lint clean, restart 200. $0 (local).
---
 OVERNIGHT_LEDGER.md |  1 +
 server.js           |  5 ++++-
 test-analyze-ocr.js | 23 +++++++++++++++++++++--
 3 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/OVERNIGHT_LEDGER.md b/OVERNIGHT_LEDGER.md
index d2d8e4a..751442c 100644
--- a/OVERNIGHT_LEDGER.md
+++ b/OVERNIGHT_LEDGER.md
@@ -56,3 +56,4 @@ the single highest-value safe one, the officer executes it, and the loop resched
 | 11 | investigate→pivot | A/B proved Swift low-light pre-pass MOOT (Vision reads 4-level contrast raw); pivoted to UX: tentative "hold steady · CODE" cue when a non-strong read needs a 2nd frame | OCR reads all contrast tiers raw; inline JS+lint+32 tests green; page 200 | $0 |
 | 12 | brainstorm (qwen3:14b $0) → investigate+pick #5 | A/B proved OCR-confidence-threshold MOOT (Vision = 1.000 or no-read); shipped fingerprintVendor edge tests (empty/short→null, punct-normalized, REJECTED logo never matches) | 32→36 tests pass; lint clean | $0 |
 | 13 | DEBUG disk-full + disk-light refine | Data volume was 100% full (ENOSPC blocked all Bash); diagnosed: home=138G (Projects 83G, Ollama 45G/16 models), ~250G in sudo-gated areas (Steve-gated, not auto-deleted); shipped a disk-light pure-test refinement (5 fuzzyVendor edge guards) | df recovered 124Mi→4.7Gi (purgeable reclaimed); 36→41 tests; lint clean; no server restart/scratch | $0 |
+| 14 | brainstorm #1 (learning-loop) | profileToLex prefix support gate (>=2 scans) — one bad scan can no longer inject a wrong prefix into the ranking lexicon; catalog prefixes (builder-vetted n>=3) unaffected; +5 profileToLex tests | df 27G healthy; real-data impact = 1 unconfirmed prefix gated (WC test artifact), 0 vendors lose matching; 41→46 tests; lint clean; restart 200 | $0 |
diff --git a/server.js b/server.js
index a83b94d..e49c785 100644
--- a/server.js
+++ b/server.js
@@ -360,7 +360,10 @@ function saveVendorProfiles() {
 // Compile one learned profile into the seed's {name, re, pfx} shape.
 function profileToLex(name, p) {
   let re = null; try { re = p.alias_re ? new RegExp(p.alias_re) : null; } catch (e) { re = null; }
-  const prefs = Object.keys(p.prefixes || {}).filter(Boolean).sort((a, b) => b.length - a.length); // WDW before WD
+  // Require >=2 supporting scans before a prefix biases ranking, so ONE bad scan can't inject
+  // a wrong prefix into the lexicon. Catalog-built prefixes are already vetted (builder keeps
+  // n>=3), so this only gates thin learned ones until a 2nd scan confirms them.
+  const prefs = Object.keys(p.prefixes || {}).filter(k => k && (p.prefixes[k] || 0) >= 2).sort((a, b) => b.length - a.length); // WDW before WD
   // A vendor can be BOTH prefixed AND numeric-dominant (Schumacher: DWLK house code + bare
   // numbers on the swatch) — express both so the scanner matches either form it sees.
   const alts = prefs.map(escapeRe);
diff --git a/test-analyze-ocr.js b/test-analyze-ocr.js
index a547c17..38e1604 100644
--- a/test-analyze-ocr.js
+++ b/test-analyze-ocr.js
@@ -41,9 +41,11 @@ const moduleSrc = [
   sliceFn('_lev'),
   sliceFn('fuzzyVendor'),
   sliceFn('fingerprintVendor'),
-  'return { parseOcrRows, analyzeOcr, fuzzyVendor, fingerprintVendor, VENDOR_LEX };',
+  sliceConst('escapeRe'),
+  sliceFn('profileToLex'),
+  'return { parseOcrRows, analyzeOcr, fuzzyVendor, fingerprintVendor, profileToLex, VENDOR_LEX };',
 ].join('\n\n');
-const { parseOcrRows, analyzeOcr, fuzzyVendor, fingerprintVendor } =
+const { parseOcrRows, analyzeOcr, fuzzyVendor, fingerprintVendor, profileToLex } =
   new Function('VENDOR_PROFILES', moduleSrc)(VENDOR_PROFILES);
 
 // --- tiny assert harness ---
@@ -171,5 +173,22 @@ const bc = (payload, sym = 'Code128') => `BC\t${payload}\t${sym}`;
   } else { ok(true, 'fp edge: (no rejected logo in data to assert against)'); }
 }
 
+// ── profileToLex prefix support gate (cycle 14: one bad scan can't inject a prefix) ──
+{
+  // a single-scan prefix (count 1) is gated out — no pfx, no alias → null profile
+  eq(profileToLex('TestCo', { prefixes: { XX: 1 }, alias_re: null, numeric_share: 0 }), null,
+     'profileToLex: a count-1 (unconfirmed) prefix is gated → null');
+  // a confirmed prefix (count ≥2) is included and matches its codes
+  const ok2 = profileToLex('TestCo', { prefixes: { XX: 2 }, alias_re: null, numeric_share: 0 });
+  ok(ok2 && ok2.pfx.test('XX1234'), 'profileToLex: count-2 prefix is included');
+  // mixed: the confirmed prefix matches, the unconfirmed one does NOT (one bad scan ignored)
+  const mix = profileToLex('TestCo', { prefixes: { YY: 5, XX: 1 }, alias_re: null, numeric_share: 0 });
+  ok(mix && mix.pfx.test('YY9'), 'profileToLex: confirmed YY matches');
+  ok(mix && !mix.pfx.test('XX9'), 'profileToLex: unconfirmed XX (count 1) does NOT match');
+  // a numeric-dominant vendor still matches bare numbers even with no confirmed prefix
+  const num = profileToLex('NumCo', { prefixes: {}, alias_re: null, numeric_share: 0.9 });
+  ok(num && num.pfx.test('5255'), 'profileToLex: numeric_share≥0.3 matches bare numbers');
+}
+
 console.log(`\nanalyzeOcr/vendor tests: ${pass} passed, ${fail} failed`);
 process.exit(fail ? 1 : 0);

← 3eaa90d auto-save: 2026-06-26T06:12:56 (1 files) — data/build.json  ·  back to Dw Photo Capture  ·  auto-save: 2026-06-26T06:43:07 (1 files) — data/build.json bf76d61 →