[object Object]

← back to Designerwallcoverings

TK-10933: sniff image type from magic bytes instead of trusting Content-Type

5c48cd7f8732ce7ddc065f2fcd1425bced85e612 · 2026-09-10 11:21:19 -0700 · Steve Abrams

Every provider call was failing with a hard 400 while the ledger recorded only
the soft status 'vision_unparseable'. The real signal was one field deeper, in
provider_category: provider_error:INVALID_ARGUMENT.

Cause: fetchB64 passed the HTTP Content-Type header straight through as the
inline_data mime type. The images are hosted on an S3 bucket that serves valid
JPEGs (magic ffd8ffe0) as 'binary/octet-stream', and the vision API accepts only
image/jpeg|png|webp|heic|heif. Proven with identical bytes: binary/octet-stream
-> 400 'Unsupported MIME type', image/jpeg -> 200 OK. A live batch confirmed it:
58/58 attempts failed and zero verdicts were written.

This was independent of the quota exhaustion tracked on the same ticket, and
would have failed the same way once credits were restored.

sniffMime() now detects JPEG/PNG/WEBP/HEIC from magic bytes and falls back to the
header only when it already names a supported image type, else image/jpeg.
Checked: real-JPEG+octet-stream -> image/jpeg, PNG -> image/png, WEBP ->
image/webp, unknown+image/png header -> image/png, unknown+junk header ->
image/jpeg. Breaker, diagnostics and watchdog suites all PASS.

Also included: an explicit env var now takes precedence over the dotenv file when
resolving the model credential, so one run can be pointed at a different project
without editing a shared file every other process reads.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Files touched

Diff

commit 5c48cd7f8732ce7ddc065f2fcd1425bced85e612
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Thu Sep 10 11:21:19 2026 -0700

    TK-10933: sniff image type from magic bytes instead of trusting Content-Type
    
    Every provider call was failing with a hard 400 while the ledger recorded only
    the soft status 'vision_unparseable'. The real signal was one field deeper, in
    provider_category: provider_error:INVALID_ARGUMENT.
    
    Cause: fetchB64 passed the HTTP Content-Type header straight through as the
    inline_data mime type. The images are hosted on an S3 bucket that serves valid
    JPEGs (magic ffd8ffe0) as 'binary/octet-stream', and the vision API accepts only
    image/jpeg|png|webp|heic|heif. Proven with identical bytes: binary/octet-stream
    -> 400 'Unsupported MIME type', image/jpeg -> 200 OK. A live batch confirmed it:
    58/58 attempts failed and zero verdicts were written.
    
    This was independent of the quota exhaustion tracked on the same ticket, and
    would have failed the same way once credits were restored.
    
    sniffMime() now detects JPEG/PNG/WEBP/HEIC from magic bytes and falls back to the
    header only when it already names a supported image type, else image/jpeg.
    Checked: real-JPEG+octet-stream -> image/jpeg, PNG -> image/png, WEBP ->
    image/webp, unknown+image/png header -> image/png, unknown+junk header ->
    image/jpeg. Breaker, diagnostics and watchdog suites all PASS.
    
    Also included: an explicit env var now takes precedence over the dotenv file when
    resolving the model credential, so one run can be pointed at a different project
    without editing a shared file every other process reads.
    
    Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
 scripts/stroheim-onboard/settlement-gate.mjs | 31 +++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)

diff --git a/scripts/stroheim-onboard/settlement-gate.mjs b/scripts/stroheim-onboard/settlement-gate.mjs
index b5f6ce6..05645fe 100644
--- a/scripts/stroheim-onboard/settlement-gate.mjs
+++ b/scripts/stroheim-onboard/settlement-gate.mjs
@@ -43,8 +43,13 @@ const SOFT_429_ITEM_LIMIT = Math.max(1, parseInt(process.env.STROHEIM_SETTLEMENT
 const SOFT_429_COOLDOWN_MS = Math.max(0, parseInt(process.env.STROHEIM_SETTLEMENT_429_COOLDOWN_MS || '900000', 10));
 const RUN_ID = process.env.STROHEIM_SETTLEMENT_RUN_ID || `${new Date().toISOString()}-pid${process.pid}`;
 
-const KEY = TEST_MODE ? 'test-only' : (fs.readFileSync(path.join(process.env.HOME, 'Projects/secrets-manager/.env'), 'utf8')
-  .match(/^GEMINI_API_KEY=(.+)$/m) || [])[1]?.trim();
+// Key resolution: an explicit env GEMINI_API_KEY wins, else the canonical secrets file.
+// The env override exists so a single run can be pointed at a different project's key
+// without mutating ~/Projects/secrets-manager/.env (which every other process reads, and
+// which a mid-run crash would leave corrupted). Unset env => byte-identical prior behavior.
+const KEY = TEST_MODE ? 'test-only' : (process.env.GEMINI_API_KEY?.trim()
+  || (fs.readFileSync(path.join(process.env.HOME, 'Projects/secrets-manager/.env'), 'utf8')
+  .match(/^GEMINI_API_KEY=(.+)$/m) || [])[1]?.trim());
 if (!KEY) { console.error('no GEMINI_API_KEY in ~/Projects/secrets-manager/.env'); process.exit(1); }
 const URL = `https://generativelanguage.googleapis.com/v1beta/models/${MODEL}:generateContent?key=${KEY}`;
 
@@ -163,6 +168,26 @@ Acceptable (carveout): Does it clearly contain tree trunks, clearly-drawn branch
 Respond with ONLY this JSON, booleans true/false (never null), plus a 3-6 word evidence string each:
 {"a1":bool,"a2":bool,"a3":bool,"b":bool,"acceptable":bool,"evidence":{"a1":"","a2":"","a3":"","b":"","acceptable":""}}`;
 
+// Gemini accepts only image/jpeg|png|webp|heic|heif and rejects anything else with
+// INVALID_ARGUMENT ("Unsupported MIME type"). The Content-Type header cannot be trusted:
+// the Fabricut S3 bucket that hosts the Stroheim images serves valid JPEGs as
+// 'binary/octet-stream', which made 100% of provider calls fail while the ledger recorded
+// them only as 'vision_unparseable'. Sniff the real type from the magic bytes and fall back
+// to the header only when it is already a supported image type. (TK-10933)
+const GEMINI_MIMES = new Set(['image/jpeg', 'image/png', 'image/webp', 'image/heic', 'image/heif']);
+function sniffMime(buf, headerCt) {
+  const b = buf;
+  if (b.length > 3 && b[0] === 0xff && b[1] === 0xd8 && b[2] === 0xff) return 'image/jpeg';
+  if (b.length > 8 && b[0] === 0x89 && b[1] === 0x50 && b[2] === 0x4e && b[3] === 0x47) return 'image/png';
+  if (b.length > 12 && b.toString('ascii', 0, 4) === 'RIFF' && b.toString('ascii', 8, 12) === 'WEBP') return 'image/webp';
+  if (b.length > 12 && b.toString('ascii', 4, 8) === 'ftyp') {
+    const brand = b.toString('ascii', 8, 12);
+    if (brand.startsWith('hei') || brand.startsWith('mif')) return 'image/heic';
+  }
+  const bare = String(headerCt || '').split(';')[0].trim().toLowerCase();
+  return GEMINI_MIMES.has(bare) ? bare : 'image/jpeg';
+}
+
 async function fetchB64(url, outerSignal) {
   if (TEST_MODE) return { mime: 'image/jpeg', data: 'dGVzdA==' };
   for (let t = 0; t < 4; t++) {
@@ -171,7 +196,7 @@ async function fetchB64(url, outerSignal) {
       if (!r.ok) throw new Error(`img ${r.status}`);
       const ct = r.headers.get('content-type') || 'image/jpeg';
       const buf = Buffer.from(await r.arrayBuffer());
-      return { mime: ct.split(';')[0], data: buf.toString('base64') };
+      return { mime: sniffMime(buf, ct), data: buf.toString('base64') };
     } catch (e) {
       if (outerSignal.aborted || t === 3) throw e;
       await sleep(800 * (t + 1), outerSignal);

← 094abee TK-11357 action C: 15-min zero-price-orderable OUTCOME tripw  ·  back to Designerwallcoverings  ·  TK-10895: storefront verify needs a CDN grace period 5405216 →