← back to Wallco Ai
Stage 2: composition-detector + smart-fix retry gate
5575ce78263d1da4c238b5b40e6305f0a5c73768 · 2026-05-24 08:17:51 -0700 · Steve Abrams
Adds lib/composition-detector.js — sister module to lib/ghost-detector
that distinguishes a true multi-motif repeating wallpaper field from a
single centered hero with wrap-half corners. Both can be mathematically
tileable; only the first is what we want.
lib API:
analyzeComposition(input) → single Gemini-2.0-flash vision call
analyzeCompositionStable(input) → 3-vote majority (matches ghost-detector pattern)
gateComposition(input, {minInstances=3, stable=false}) → {ok, result, reason}
Result schema:
{ is_repeating_field: bool,
instance_count: int, // partial/wrap motifs at edges count
hero_centered: bool,
confidence: 0..1,
what_you_see: short sentence }
is_repeating_field is recomputed locally as (count>=3 && !hero_centered)
— don't trust Gemini's boolean math; trust its raw observations.
Wired into smart-fix retry loop: after the existing opacity-verify pass,
the candidate must also pass composition gate (minInstances=3). If it
fails (Gemini still produced a single-hero output despite the v2 prompt),
the loop retries with the next phrasing variant. Detector failures fall
open — opacity-pass alone still completes the request, so the detector
can never block a working pipeline.
Live tested on 4 fixtures:
- 39328 (broken): instance_count=1, hero_centered=true → caught ✅
- 39339 (smartfix clean): count=4, hero=false → passes ✅
- 39341 (smartfix borderline): count=3, hero=false → passes
(this is by design — single-hero detection is conservative; raise
WALLCO_COMP_GATE minInstances to be stricter)
- 39345 (bgswap clean): count=7, hero=false → passes ✅
A/B parity: ?legacy=1 query skips the gate so the old prompt path can
be benchmarked apples-to-apples. Env override: WALLCO_COMP_GATE=0
disables the gate entirely.
Response + fix-events log now include `composition` field and
`prompt_mode` ('legacy' or 'repeat-v2') so Stage 3 A/B has the
metadata it needs.
Files touched
A lib/composition-detector.jsM server.js
Diff
commit 5575ce78263d1da4c238b5b40e6305f0a5c73768
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Sun May 24 08:17:51 2026 -0700
Stage 2: composition-detector + smart-fix retry gate
Adds lib/composition-detector.js — sister module to lib/ghost-detector
that distinguishes a true multi-motif repeating wallpaper field from a
single centered hero with wrap-half corners. Both can be mathematically
tileable; only the first is what we want.
lib API:
analyzeComposition(input) → single Gemini-2.0-flash vision call
analyzeCompositionStable(input) → 3-vote majority (matches ghost-detector pattern)
gateComposition(input, {minInstances=3, stable=false}) → {ok, result, reason}
Result schema:
{ is_repeating_field: bool,
instance_count: int, // partial/wrap motifs at edges count
hero_centered: bool,
confidence: 0..1,
what_you_see: short sentence }
is_repeating_field is recomputed locally as (count>=3 && !hero_centered)
— don't trust Gemini's boolean math; trust its raw observations.
Wired into smart-fix retry loop: after the existing opacity-verify pass,
the candidate must also pass composition gate (minInstances=3). If it
fails (Gemini still produced a single-hero output despite the v2 prompt),
the loop retries with the next phrasing variant. Detector failures fall
open — opacity-pass alone still completes the request, so the detector
can never block a working pipeline.
Live tested on 4 fixtures:
- 39328 (broken): instance_count=1, hero_centered=true → caught ✅
- 39339 (smartfix clean): count=4, hero=false → passes ✅
- 39341 (smartfix borderline): count=3, hero=false → passes
(this is by design — single-hero detection is conservative; raise
WALLCO_COMP_GATE minInstances to be stricter)
- 39345 (bgswap clean): count=7, hero=false → passes ✅
A/B parity: ?legacy=1 query skips the gate so the old prompt path can
be benchmarked apples-to-apples. Env override: WALLCO_COMP_GATE=0
disables the gate entirely.
Response + fix-events log now include `composition` field and
`prompt_mode` ('legacy' or 'repeat-v2') so Stage 3 A/B has the
metadata it needs.
---
lib/composition-detector.js | 173 ++++++++++++++++++++++++++++++++++++++++++++
server.js | 35 ++++++++-
2 files changed, 205 insertions(+), 3 deletions(-)
diff --git a/lib/composition-detector.js b/lib/composition-detector.js
new file mode 100644
index 0000000..13e72e1
--- /dev/null
+++ b/lib/composition-detector.js
@@ -0,0 +1,173 @@
+// composition-detector — wallpaper-tile composition QA via Gemini Vision.
+//
+// Detects whether a generated wallpaper tile is:
+// (a) a true multi-motif REPEATING FIELD (what we want), or
+// (b) a single CENTERED HERO with wrap-half artefacts at the corners
+//
+// Both can be mathematically tileable. The composition-detector is the
+// gate that distinguishes them so the smart-fix / generate pipelines can
+// retry when Gemini slips into single-hero mode despite the multi-instance
+// prompt language from lib/repeat-prompt.
+//
+// Sister module to lib/ghost-detector — same shape, different defect.
+// Ghost-detector finds opacity defects. Composition-detector finds
+// "wallpaper-or-portrait" defects.
+//
+// Cost: one gemini-2.0-flash vision call per check (~$0.0005). Sometimes
+// callers will also want a stable variant — 3 votes + majority — modeled
+// after analyzeGhostLayerStable. Provided as analyzeCompositionStable.
+
+const fs = require('fs');
+const path = require('path');
+
+const GEMINI_MODEL = process.env.COMPOSITION_DETECTOR_MODEL || 'gemini-2.0-flash';
+const GEMINI_BASE = 'https://generativelanguage.googleapis.com/v1beta/models';
+
+const PROMPT = `You are inspecting a WALLPAPER TILE image (a single tile of a repeating pattern).
+
+Distinguish between two compositions:
+
+WALLPAPER REPEAT (the intended kind):
+ - Multiple instances of the same motif spread across the tile
+ - Each instance roughly the same scale and orientation
+ - The tile reads as an "all-over pattern" — your eye sees a field, not a focal point
+ - Motifs typically continue across left/right and top/bottom edges (partial motifs near edges that wrap to the opposite edge)
+
+CENTERED HERO (the defect):
+ - ONE large motif occupying the centre of the tile
+ - The corners show empty ground OR partial fragments of the same motif from a wrap-around shift
+ - The tile reads as a "portrait" or "medallion" — one focal point, not an all-over pattern
+ - If you mentally tile it 2×2, you would see one big motif per tile with cropped halves at the seams
+
+Count the visible instances of the dominant motif. A partial/cropped motif near an edge counts as 1 if you can see it's the same motif. Don't count background florals or small filler — count the primary repeating motif only.
+
+Return JSON ONLY (no markdown):
+{"is_repeating_field": <bool>, "instance_count": <int>, "hero_centered": <bool>, "confidence": <0..1>, "what_you_see": "<one short sentence>"}
+
+Decision rule for hero_centered: TRUE if there's ONE clearly-dominant motif at or near the centre that's larger than the others (or all alone), with corners showing wrap-fragments or empty ground.
+
+Decision rule for is_repeating_field: TRUE if instance_count >= 3 AND hero_centered === false. FALSE otherwise.`;
+
+function readKey() {
+ if (process.env.GEMINI_API_KEY) return process.env.GEMINI_API_KEY;
+ try {
+ const env = fs.readFileSync(path.join(__dirname, '..', '.env'), 'utf8');
+ const m = env.match(/^GEMINI_API_KEY=(.+)$/m);
+ if (m) return m[1].trim();
+ } catch {}
+ throw new Error('GEMINI_API_KEY missing');
+}
+
+// Accept either a path string or a base64 string. If it's neither, treat as buffer.
+function toBase64(input) {
+ if (Buffer.isBuffer(input)) return { b64: input.toString('base64'), mime: 'image/png' };
+ if (typeof input !== 'string') throw new Error('composition-detector: input must be path|b64|buffer');
+ // Heuristic: starts with '/' or contains '.' near end → treat as path
+ if (input.startsWith('/') || /\.(png|jpg|jpeg)$/i.test(input)) {
+ const bytes = fs.readFileSync(input);
+ const mime = /\.png$/i.test(input) ? 'image/png' : 'image/jpeg';
+ return { b64: bytes.toString('base64'), mime };
+ }
+ // Otherwise assume it's already base64
+ return { b64: input, mime: 'image/png' };
+}
+
+async function analyzeComposition(input) {
+ const KEY = readKey();
+ const { b64, mime } = toBase64(input);
+ const url = `${GEMINI_BASE}/${GEMINI_MODEL}:generateContent?key=${KEY}`;
+ const body = {
+ contents: [{ parts: [
+ { text: PROMPT },
+ { inline_data: { mime_type: mime, data: b64 } },
+ ]}],
+ generationConfig: {
+ temperature: 0.0,
+ topP: 1.0,
+ topK: 1,
+ candidateCount: 1,
+ response_mime_type: 'application/json',
+ },
+ };
+ const r = await fetch(url, {
+ method: 'POST',
+ headers: { 'Content-Type': 'application/json' },
+ body: JSON.stringify(body),
+ });
+ if (!r.ok) {
+ const txt = await r.text().catch(() => '');
+ throw new Error(`gemini ${r.status}: ${txt.slice(0, 200)}`);
+ }
+ const j = await r.json();
+ const text = j?.candidates?.[0]?.content?.parts?.[0]?.text || '';
+ let parsed;
+ try { parsed = JSON.parse(text); }
+ catch { throw new Error(`gemini returned non-JSON: ${text.slice(0, 200)}`); }
+ // Coerce field types; tolerate missing fields with sane defaults
+ const instance_count = Number.isFinite(parsed.instance_count) ? Math.max(0, parseInt(parsed.instance_count, 10)) : 0;
+ const hero_centered = parsed.hero_centered === true;
+ // Recompute is_repeating_field locally — don't trust Gemini's boolean math
+ const is_repeating_field = instance_count >= 3 && !hero_centered;
+ return {
+ is_repeating_field,
+ instance_count,
+ hero_centered,
+ confidence: Number(parsed.confidence) || 0,
+ what_you_see: String(parsed.what_you_see || '').slice(0, 250),
+ cost_estimate: 0.0005,
+ };
+}
+
+// Stable variant — 3 parallel votes, majority on the binary is_repeating_field flag.
+// Returns the winner with median instance_count, mode hero_centered, mean confidence.
+async function analyzeCompositionStable(input) {
+ const N = 3;
+ const settled = await Promise.allSettled(
+ Array.from({ length: N }, () => analyzeComposition(input))
+ );
+ const votes = settled.filter(s => s.status === 'fulfilled').map(s => s.value);
+ const errored = N - votes.length;
+ if (votes.length === 0) {
+ const e = settled.find(s => s.status === 'rejected');
+ throw new Error(`all ${N} composition votes errored: ${e?.reason?.message || 'unknown'}`);
+ }
+ const trueVotes = votes.filter(v => v.is_repeating_field).length;
+ const falseVotes = votes.length - trueVotes;
+ const is_repeating_field = trueVotes > falseVotes;
+ const unanimous = trueVotes === votes.length || falseVotes === votes.length;
+ // Median instance_count
+ const counts = votes.map(v => v.instance_count).sort((a, b) => a - b);
+ const median = counts[Math.floor(counts.length / 2)];
+ // Mode hero_centered
+ const heroTrue = votes.filter(v => v.hero_centered).length;
+ const hero_centered = heroTrue > votes.length / 2;
+ const avgConf = votes.reduce((s, v) => s + v.confidence, 0) / votes.length;
+ // Reason = whichever winning-side vote has highest confidence
+ const winners = votes.filter(v => v.is_repeating_field === is_repeating_field);
+ const reasonVote = winners.sort((a, b) => b.confidence - a.confidence)[0];
+ return {
+ is_repeating_field,
+ instance_count: median,
+ hero_centered,
+ confidence: avgConf,
+ consensus: Math.max(trueVotes, falseVotes) / votes.length, // 1.0 unanimous, 0.67 split
+ unanimous,
+ votes: votes.map(v => ({ rep: v.is_repeating_field, n: v.instance_count, hero: v.hero_centered })),
+ errored,
+ what_you_see: reasonVote.what_you_see,
+ cost_estimate: 0.0005 * votes.length,
+ };
+}
+
+// Gate variant — for the smart-fix / generate retry loop. Returns
+// { ok, reason } — ok is false when the image is a centered-hero or has too
+// few instances. Caller decides whether to retry or accept.
+async function gateComposition(input, opts = {}) {
+ const minInstances = opts.minInstances ?? 3;
+ const stable = opts.stable === true;
+ const r = stable ? await analyzeCompositionStable(input) : await analyzeComposition(input);
+ const ok = !r.hero_centered && r.instance_count >= minInstances;
+ return { ok, result: r, reason: ok ? 'pass' : (r.hero_centered ? `centered-hero (n=${r.instance_count})` : `too-few-instances (n=${r.instance_count}<${minInstances})`) };
+}
+
+module.exports = { analyzeComposition, analyzeCompositionStable, gateComposition, GEMINI_MODEL };
diff --git a/server.js b/server.js
index b1a1961..60bf8fe 100644
--- a/server.js
+++ b/server.js
@@ -7485,6 +7485,14 @@ app.post('/api/design/:id/smart-fix', express.json({ limit: '4kb' }), async (req
}
}
+ // Composition gate (Stage 2, 2026-05-24) — catches the "single centered hero
+ // with wrap-half corners" defect that opacity-only verification missed. The
+ // gate uses lib/composition-detector → Gemini vision. Skipped on ?legacy=1
+ // for A/B parity with the legacy prompt.
+ const { gateComposition } = require('./lib/composition-detector');
+ const COMP_GATE_ENABLED = !USE_LEGACY && process.env.WALLCO_COMP_GATE !== '0';
+ let lastComposition = null;
+
let data = null, lastVerify = null, attempts = 0;
const MAX_ATTEMPTS = 3;
while (attempts < MAX_ATTEMPTS) {
@@ -7510,7 +7518,24 @@ app.post('/api/design/:id/smart-fix', express.json({ limit: '4kb' }), async (req
} catch (e) {
lastVerify = { has_translucent_layers: false, confidence: 0, what_you_see: 'verify failed: ' + e.message };
}
- if (!lastVerify.has_translucent_layers || lastVerify.confidence < 0.6) {
+ const opacityOk = !lastVerify.has_translucent_layers || lastVerify.confidence < 0.6;
+ // Composition gate — only run if opacity passed (don't pay for vision twice on
+ // an already-failed candidate)
+ let compositionOk = true;
+ if (opacityOk && COMP_GATE_ENABLED) {
+ try {
+ const g = await gateComposition(candidate, { minInstances: 3 });
+ lastComposition = g.result;
+ compositionOk = g.ok;
+ if (!g.ok) console.warn(`[smart-fix] composition fail att${attempts}: ${g.reason} — retrying`);
+ } catch (e) {
+ // Composition detector failure should NOT block the response — fall
+ // back to accepting opacity-pass candidates so we don't regress.
+ console.warn('[smart-fix] composition-detector failed:', e.message.slice(0, 160));
+ compositionOk = true;
+ }
+ }
+ if (opacityOk && compositionOk) {
data = candidate;
break;
}
@@ -7636,10 +7661,14 @@ RETURNING id`;
motif: (detect.motif_description || '').slice(0, 300),
ground_hex: detect.ground_hex,
attempts: attempts + 1,
- verify: lastVerify });
+ prompt_mode: USE_LEGACY ? 'legacy' : 'repeat-v2',
+ verify: lastVerify,
+ composition: lastComposition });
return res.json({ ok: true, new_id: newId, src_id: id, elapsed_s: elapsed,
motif: detect.motif_description, ground_hex: detect.ground_hex,
- attempts: attempts + 1, verify: lastVerify });
+ attempts: attempts + 1,
+ prompt_mode: USE_LEGACY ? 'legacy' : 'repeat-v2',
+ verify: lastVerify, composition: lastComposition });
} catch (e) {
console.error('[smart-fix]', e);
return res.status(500).json({ error: e.message });
← 9e5767b Stage 1: canonical lib/repeat-prompt.js + wire into smart-fi
·
back to Wallco Ai
·
feat(scripts): learn-and-purge-flagged.js — headless equival b362f0f →