`data/yolo-overnight/settlement-gate-20260525-1214.md`

139 lines
# Settlement Publish-Gate Test & FP Audit
**Date:** 2026-05-25 12:14 PT
**Sample:** Last 100 published rows in `spoon_all_designs` (and broader checks across all 212 published rows)
**Trigger source:** `/root/public-projects/wallco-ai/sql/settlement_publish_trigger.sql`

---

## TL;DR

| | |
|---|---|
| **Trigger installed in prod DB?** | ❌ **NO.** Only `trg_spoon_all_designs_assign_dig` is attached. The settlement_publish_check / settlement_insert_check triggers from `sql/settlement_publish_trigger.sql` were never applied. |
| **Gate logic correct in isolation (TEMP table)?** | ✅ YES — blocks the Part A + Part B combination, allows monochrome escape, honors override flag, fires on UPDATE flip too. |
| **Literal task prompt `'banana, palm, leaves'` blocked?** | ❌ **NO.** That prompt is too thin to satisfy Part A (needs `directional` + `open_space` tokens). See TEST 1 below — gate let it through. |
| **Current FP rate on last 100 published rows?** | **0%** would block, **0%** would soft-flag. Gate is dormant against current corpus. |
| **Latent FP risk?** | ⚠️ **HIGH.** 88/212 (41.5%) of all published rows contain a Part B word (banana/bird/butterfly) — every one of them is in a *negation* context (`"no bananas, no birds, …"`). The text-based gate can't tell a negation from an affirmation; a future prompt that adds one more directional+open_space pairing on top of a "no bananas" stamp would falsely block. Row 38933 is one `silhouette` token away from FPing today. |

---

## 1. Trigger Status in Production

```sql
SELECT tgname, tgrelid::regclass FROM pg_trigger
 WHERE tgrelid::regclass::text LIKE 'spoon%' AND NOT tgisinternal;
```
| tgname | tgrelid |
|---|---|
| `trg_spoon_all_designs_assign_dig` | `spoon_all_designs` |

The settlement triggers defined in `sql/settlement_publish_trigger.sql` (lines 104–116) **are not installed** in `dw_unified`. The JS-side comment in `scripts/inspiration_generator.js:253` ("same fallback the existing settlement_publish_check trigger uses") is stale — there is no DB-level gate at the moment.

---

## 2. Trigger Logic Verification (TEMP Table Simulation)

Approach: installed the function from the SQL file as `pg_temp.settlement_publish_check_test`, attached it to a TEMP table mirroring the subset of `spoon_all_designs` columns the function reads (`id`, `prompt`, `is_published`, `notes`). All inserts wrapped in `BEGIN; … ROLLBACK;` — zero production side effects.

| # | Test prompt | Published? | Expected | Result |
|---|---|---|---|---|
| 1 | `banana, palm, leaves` (the literal task prompt) | TRUE | ALLOW (no Part A tokens) | ✅ ALLOWED — gate did not block. Only `palm`/`leaves` (Part A leaves), no directional, no open_space. |
| 2 | `tossed banana leaves with open space, directional fanning palm fronds, scattered birds` | TRUE | BLOCK | ✅ BLOCKED: `Settlement violation: prompt satisfies Part A AND Part B (banana)` |
| 3 | same as #2 but `is_published=FALSE` | FALSE | ALLOW | ✅ ALLOWED (gate is publish-only) |
| 4 | `monochrome tossed banana leaves with open space, directional birds silhouette` | TRUE | ALLOW (single-color escape) | ✅ ALLOWED |
| 5 | `tossed banana leaves with open space, directional fanning palm fronds, scattered birds on tree branch` | TRUE | BLOCK (acceptable cue is soft-only) | ✅ BLOCKED — `branch` did NOT override Part B hit. Acceptable cue only soft-flags when Part B is *missing*; it has no effect once Part B fires. |
| 6 | same as #2 with `SET LOCAL settlement.allow_override = 'true'` | TRUE | ALLOW + stamp notes | ✅ ALLOWED, `notes` stamped `settlement.allow_override=true at 2026-05-25 12:11:20…` |
| 7 | UPDATE flip false→true on Part A+B row (fresh tx, no override leak) | UPDATE | BLOCK | ✅ BLOCKED: `Settlement violation … [test id=90007]` |

**Conclusion:** Logic is correct. The literal task prompt `'banana, palm, leaves'` would NOT trigger the gate as written, because Part A requires a directional + open_space pair that the bare prompt lacks. To genuinely test the block path, you need a prompt like the TEST 2 string.

---

## 3. False-Positive Audit — Last 100 Published Rows

Sample = `SELECT * FROM spoon_all_designs WHERE is_published=TRUE ORDER BY id DESC LIMIT 100`.

### Top-line metrics

| Metric | Count | % |
|---|---|---|
| Rows sampled | 100 | 100% |
| Would be BLOCKED by the gate | **0** | 0.00% |
| Would be SOFT-FLAGGED (Part A only, no acceptable cue) | **0** | 0.00% |
| Contain any Part B word (banana/bird/…) | 74 | 74% |
| Contain any Part A leaves word (palm/frond/leaves/…) | 75 | 75% |

### Token-frequency in last 100

| token | hits |
|---|---|
| palm | 74 |
| frond | 74 |
| banana | 74 |
| monstera | 74 |
| leaves | 74 |
| bird | 74 |
| leaf | 17 |
| foliage | 43 |
| tossed | 0 |
| scatter | 8 |
| open_space | 0 |
| parrot | 0 |
| toucan | 0 |
| butterfly | 0 |
| jungle | 0 |

The suspiciously identical 74 across `palm/frond/banana/monstera/leaves/bird` is the smoking gun → these tokens all live inside a single negation stamp glued onto the bottom of every generated prompt.

---

## 4. Why 0% Block Rate Despite 74% Part B Hits: Negation-Stamp Pattern

Two variants of a "do not include tropical motifs" negation stamp are appearing in published prompts:

**Variant A — "Settlement compliance" prefix (31 / 212 = 14.6% of all published rows):**
> `Settlement compliance — no bananas, no banana leaves, no banana pods, no palm fronds, no monstera, no philodendron, no grapes, no birds, no butterflies. Tall trees only.`

**Variant B — "No banana" form (the remaining 57 stamped rows):**
> `No banana, no banana leaves, no banana pods, no palm, no palm fronds, no monstera, no philodendron, no tropical foliage. No grapes, no birds, no butterflies.`

**Aggregate exposure across all 212 published rows:**

| Metric | Count |
|---|---|
| Total published | 212 |
| Contain any negation stamp (either variant) | **88 (41.5%)** |
| Contain a Part B word | 88 |
| Part B word that is **negated** | **88 (100%)** |
| Negation-stamped rows that ALSO have a directional word | 4 |
| Negation-stamped rows that ALSO have directional + open_space | **0** (today) |
| Total would-block across full corpus | **0** |

**Saved by a thread.** Row 38933 (the California oak savanna mural) contains:
- `5 to 8 mature valley oaks **scattered** across rolling golden hills` → satisfies `directional` (matches `scatter`)
- `3 to 4 flat **overlapping** bands of ground tone` → satisfies `directional` again
- `Generous **breathing room**` → satisfies `open_space`
- `Settlement compliance — no bananas` → satisfies Part B (`banana`)
- **`Single-layer stencil silhouette only`** → satisfies `has_single_color` (via `silhouette`)

The `silhouette` token alone is what keeps the gate's `(NOT has_single_color)` clause false, sparing this row from a false block. If a future tall-tree mural prompt drops `silhouette`, `single-ink`, `monochrome`, etc., the gate will FP on a perfectly compliant aspen/oak design.

---

## 5. Recommendations

1. **Decide whether to install the trigger.** The SQL file exists but the trigger is not deployed. Apply with `psql dw_unified -f /root/public-projects/wallco-ai/sql/settlement_publish_trigger.sql` if you want defense-in-depth at the DB layer.
2. **Add negation-aware matching before installing.** Strip or skip sentences beginning with `^|\.\s+no |^|\.\s+No |settlement compliance` from `pt` before evaluating Part B. Otherwise every "no bananas, no birds" stamp is a ticking FP grenade — 88 rows are pre-loaded with the Part B trigger today.
3. **Tighten `directional` regex.** `scatter` appears in `\m(directional|…|scatter|…)\M` AND in `\m(open space|…|scatter|…)\M`. A single `scatter` token satisfies both Part A halves, which is too loose — currently 4 negation-stamped rows already meet half the Part A condition this way. Consider removing `scatter`/`tossed` from one of the two groups.
4. **Acceptable cue should also gate Part B hits.** Today `tree branch` only soft-flags when Part B is missing. The 2026-05-13 incident note in the SQL header says the gate exists to backstop hallucinated banana leaves; a clean prompt with `tree branch` + `no bananas` stamp shouldn't be the casualty. Either let `has_acceptable` override Part B, or require Part B match be in a non-negated clause.
5. **Validate post-gen vision check before relying on this gate.** This trigger only inspects the *prompt*. The original incident (design 3544) was a clean prompt with a dirty image. A text gate cannot catch that — `settlement-checker.ts` / the Gemini vision pass remains the canonical defense.

---

## 6. Reproducibility

- TEMP-table simulation script ran as inline `psql` heredoc against `dw_unified` (read-only on production; all writes inside `BEGIN; … ROLLBACK;`).
- FP analysis queries are SELECT-only against `spoon_all_designs`.
- No production rows were modified, no triggers were installed or dropped.
- No commits required — this is a read-only audit; no code changes to push.