← back to Dw Domain Fleet
deploy gate: assert catalog sites actually RENDER products, not just answer 200 (TK-11463)
35148f67e1586564f970eb0fbb72929b1a527500 · 2026-09-11 11:24:55 -0700 · Steve Abrams
The 8 catalog-serving sites served a 200 with a completely empty grid for two
days and deploy-fleet.sh printed 'ok' on every one of them, every deploy, for
the whole outage — a zero-product store answers 200 exactly like a healthy one.
dw-uptime-probe enumerates none of these 8 domains, and its blank-grid assertion
keys off Shopify /products.json which these Express sites don't serve, so its
fallback (body.length < 1000) passed them at ~21KB too. Both checks reported
success for something they did not measure.
scripts/assert-grids.js measures rendered product cards, carries the pool size
beside the observed count, derives its site list from sites/*.json so a new
catalog site can't be forgotten, and lands every site in one of three states —
never two: PASS / FAIL / NOT_MEASURED, where an unreachable or contract-changed
site is never green.
Ships with its negative test (scripts/test-assert-grids.sh): four injected
faults (empty pool, empty grid, dead origin, missing health contract) each
proven to go red, one healthy fixture proven to go green, plus a seam-inertness
check that --base is ignored without --test so a scheduled run can never
silently measure a fixture. 6/6 passing. Live fleet: PASS 8/8.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5t2AhAGwcTJDdHdmfpr8Z
Files touched
A scripts/assert-grids.jsM scripts/deploy-fleet.shA scripts/fixtures/grid-fixture.jsA scripts/test-assert-grids.sh
Diff
commit 35148f67e1586564f970eb0fbb72929b1a527500
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Fri Sep 11 11:24:55 2026 -0700
deploy gate: assert catalog sites actually RENDER products, not just answer 200 (TK-11463)
The 8 catalog-serving sites served a 200 with a completely empty grid for two
days and deploy-fleet.sh printed 'ok' on every one of them, every deploy, for
the whole outage — a zero-product store answers 200 exactly like a healthy one.
dw-uptime-probe enumerates none of these 8 domains, and its blank-grid assertion
keys off Shopify /products.json which these Express sites don't serve, so its
fallback (body.length < 1000) passed them at ~21KB too. Both checks reported
success for something they did not measure.
scripts/assert-grids.js measures rendered product cards, carries the pool size
beside the observed count, derives its site list from sites/*.json so a new
catalog site can't be forgotten, and lands every site in one of three states —
never two: PASS / FAIL / NOT_MEASURED, where an unreachable or contract-changed
site is never green.
Ships with its negative test (scripts/test-assert-grids.sh): four injected
faults (empty pool, empty grid, dead origin, missing health contract) each
proven to go red, one healthy fixture proven to go green, plus a seam-inertness
check that --base is ignored without --test so a scheduled run can never
silently measure a fixture. 6/6 passing. Live fleet: PASS 8/8.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5t2AhAGwcTJDdHdmfpr8Z
---
scripts/assert-grids.js | 178 +++++++++++++++++++++++++++++++++++++++
scripts/deploy-fleet.sh | 20 +++++
scripts/fixtures/grid-fixture.js | 31 +++++++
scripts/test-assert-grids.sh | 78 +++++++++++++++++
4 files changed, 307 insertions(+)
diff --git a/scripts/assert-grids.js b/scripts/assert-grids.js
new file mode 100755
index 0000000..aae6af4
--- /dev/null
+++ b/scripts/assert-grids.js
@@ -0,0 +1,178 @@
+#!/usr/bin/env node
+/**
+ * assert-grids.js — prove every catalog-serving fleet site actually RENDERS products.
+ *
+ * WHY THIS EXISTS (TK-11463). For two days the 8 catalog-serving sites served a
+ * 200 with a completely empty product grid: shared/catalog.js's display_variant
+ * junk rule went from rejecting 2,166/15,000 products to rejecting 100% of them
+ * (a silent tag-meaning drift), so CLEAN === 0 and every grid was empty by
+ * construction. Nothing caught it:
+ * - deploy-fleet.sh's smoke test asserts only "https://<d>/ answers 200". A
+ * zero-product store answers 200. It reported "ok" through the whole outage.
+ * - dw-uptime-probe doesn't enumerate ANY of these 8 domains, and its
+ * blank-grid assertion keys off Shopify /products.json, which these Express
+ * sites don't serve; its fallback is body.length < 1000 and they serve ~21KB.
+ * That is the false-green class from CLAUDE.md: a check reported success for
+ * something it did not measure. This script measures the thing that matters —
+ * rendered product cards — and refuses to report PASS on an input it could not
+ * read.
+ *
+ * THREE STATES, NEVER TWO (CLAUDE.md amendment 1). Absence of bad news is not
+ * good news, so every site lands in exactly one of:
+ * PASS cards >= MIN_CARDS AND health.serving > 0 (measured good)
+ * FAIL cards === 0, or health.serving === 0 (measured bad)
+ * NOT_MEASURED health/catalog unreachable, non-2xx, unparseable, or missing
+ * the `serving` field (never green)
+ * `cards 0 of pool 0` and `cards 0 of pool 11242` are different failures and are
+ * reported differently; a site we could not reach is never silently a pass.
+ *
+ * SCOPE is derived from sites/*.json (monetize !== true && adsense !== true), not
+ * a hand-maintained list, so a newly added catalog site is covered the day it
+ * ships and cannot be forgotten.
+ *
+ * Usage:
+ * node scripts/assert-grids.js # probe live https://<domain>
+ * node scripts/assert-grids.js --json # machine-readable (canary/rollup)
+ * node scripts/assert-grids.js --min 12 # raise the card floor
+ * node scripts/assert-grids.js --test --base http://127.0.0.1:PORT
+ *
+ * --base is the TESTABILITY SEAM and is INERT without --test (CLAUDE.md
+ * amendment 3: ship the seam, then guard it so a scheduled job can never
+ * silently measure a fixture and become the next false green). deploy-fleet.sh
+ * must never pass --test.
+ *
+ * Exit 0 only when every site is PASS. Any FAIL or NOT_MEASURED exits non-zero.
+ */
+const fs = require('fs');
+const path = require('path');
+const http = require('http');
+const https = require('https');
+
+const argv = process.argv.slice(2);
+const has = (f) => argv.includes(f);
+const val = (f, d) => { const i = argv.indexOf(f); return i >= 0 && argv[i + 1] ? argv[i + 1] : d; };
+
+const AS_JSON = has('--json');
+const TEST_MODE = has('--test');
+const MIN_CARDS = Number(val('--min', 1));
+const TIMEOUT = Number(val('--timeout', 20000));
+// Seam is inert unless --test is explicitly passed.
+const BASE = TEST_MODE ? val('--base', '') : '';
+
+function fetchText(url) {
+ return new Promise((resolve) => {
+ const lib = url.startsWith('https:') ? https : http;
+ const req = lib.get(url, {
+ headers: { 'User-Agent': 'dw-domain-fleet/assert-grids' },
+ timeout: TIMEOUT,
+ }, (res) => {
+ let body = '';
+ res.on('data', (c) => (body += c));
+ res.on('end', () => resolve({ status: res.statusCode, body }));
+ });
+ // A transport error is NOT a 200 and must not be swallowed into a pass.
+ req.on('timeout', () => { req.destroy(); resolve({ status: 0, body: '', err: 'timeout' }); });
+ req.on('error', (e) => resolve({ status: 0, body: '', err: e.message }));
+ });
+}
+
+/** Server-rendered cards come from shared/render.js card(): `<div class="card" data-handle=...`. */
+function countCards(html) {
+ return (html.match(/<div class="card"/g) || []).length;
+}
+
+function catalogSites() {
+ const dir = path.join(__dirname, '..', 'sites');
+ return fs.readdirSync(dir)
+ .filter((f) => f.endsWith('.json'))
+ .map((f) => {
+ const cfg = JSON.parse(fs.readFileSync(path.join(dir, f), 'utf8'));
+ return { slug: f.replace(/\.json$/, ''), domain: cfg.domain, monetize: cfg.monetize === true || cfg.adsense === true };
+ })
+ .filter((s) => !s.monetize && s.domain)
+ .sort((a, b) => a.slug.localeCompare(b.slug));
+}
+
+async function probe(site) {
+ const origin = BASE || ('https://' + site.domain);
+ const r = { site: site.slug, domain: site.domain, origin, cards: null, serving: null, state: null, why: '' };
+
+ const health = await fetchText(origin + '/health');
+ if (health.status !== 200) {
+ r.state = 'NOT_MEASURED';
+ r.why = `/health ${health.err ? health.err : 'HTTP ' + health.status} — pool size unread, cannot claim healthy`;
+ return r;
+ }
+ let hj;
+ try { hj = JSON.parse(health.body); }
+ catch { r.state = 'NOT_MEASURED'; r.why = '/health returned unparseable body — pool size unread'; return r; }
+ if (typeof hj.serving !== 'number') {
+ // Distinguish "no such field" from "field says zero" — the whole point of amendment 1.
+ r.state = 'NOT_MEASURED';
+ r.why = '/health has no numeric `serving` field (contract changed?) — pool size unread';
+ return r;
+ }
+ r.serving = hj.serving;
+
+ const cat = await fetchText(origin + '/catalog');
+ if (cat.status !== 200) {
+ r.state = 'NOT_MEASURED';
+ r.why = `/catalog ${cat.err ? cat.err : 'HTTP ' + cat.status} — rendered grid unread`;
+ return r;
+ }
+ r.cards = countCards(cat.body);
+
+ if (r.serving === 0) {
+ r.state = 'FAIL';
+ r.why = `pool is EMPTY (serving 0) — every grid on this site is empty by construction`;
+ return r;
+ }
+ if (r.cards === 0) {
+ r.state = 'FAIL';
+ r.why = `/catalog rendered 0 cards of a ${r.serving}-product pool — grid is broken downstream of the pool`;
+ return r;
+ }
+ if (r.cards < MIN_CARDS) {
+ r.state = 'FAIL';
+ r.why = `/catalog rendered ${r.cards} cards of a ${r.serving}-product pool, below floor ${MIN_CARDS}`;
+ return r;
+ }
+ r.state = 'PASS';
+ r.why = `${r.cards} cards rendered of a ${r.serving}-product pool`;
+ return r;
+}
+
+(async () => {
+ const sites = catalogSites();
+ if (!sites.length) {
+ // Zero sites enumerated is itself an unmeasured input, not a clean run.
+ const out = { verdict: 'WARN', status: 'WARN', reason: 'no catalog-serving sites enumerated from sites/*.json', sites: [] };
+ console.log(AS_JSON ? JSON.stringify(out, null, 2) : 'NOT_MEASURED: no catalog-serving sites enumerated');
+ process.exit(2);
+ }
+
+ const results = [];
+ for (const s of sites) results.push(await probe(s));
+
+ const fails = results.filter((r) => r.state === 'FAIL');
+ const unmeasured = results.filter((r) => r.state === 'NOT_MEASURED');
+ const verdict = fails.length ? 'FAIL' : (unmeasured.length ? 'WARN' : 'PASS');
+
+ if (AS_JSON) {
+ console.log(JSON.stringify({
+ verdict, status: verdict, ts: new Date().toISOString(),
+ population: sites.length, passed: results.length - fails.length - unmeasured.length,
+ failed: fails.length, not_measured: unmeasured.length,
+ min_cards: MIN_CARDS, test_mode: TEST_MODE, origin_override: BASE || null,
+ sites: results,
+ }, null, 2));
+ } else {
+ console.log(`==> grid assertion · ${sites.length} catalog-serving sites · floor ${MIN_CARDS} card(s)`);
+ for (const r of results) {
+ const mark = r.state === 'PASS' ? ' ok ' : (r.state === 'FAIL' ? ' FAIL' : ' ????');
+ console.log(`${mark} ${r.domain} — ${r.why}`);
+ }
+ console.log(`==> ${verdict} · ${results.length - fails.length - unmeasured.length} pass, ${fails.length} fail, ${unmeasured.length} not-measured`);
+ }
+ process.exit(verdict === 'PASS' ? 0 : (verdict === 'FAIL' ? 1 : 2));
+})();
diff --git a/scripts/deploy-fleet.sh b/scripts/deploy-fleet.sh
index dbe346d..754520b 100755
--- a/scripts/deploy-fleet.sh
+++ b/scripts/deploy-fleet.sh
@@ -44,5 +44,25 @@ for d in $(ls "$LOCAL/data/nginx/" | sed 's/.conf//'); do
*) echo " FAIL $d ($CODE)"; FAIL=$((FAIL+1)) ;;
esac
done
+# ---- GRID GATE (TK-11463) ----------------------------------------------------
+# The smoke test above proves each site ANSWERS. It does not prove each site has
+# any PRODUCTS. For two days the 8 catalog-serving sites served 200 with a
+# completely empty grid (shared/catalog.js's display_variant rule silently went
+# from rejecting 14% of the catalog to rejecting 100% of it) and this smoke test
+# printed "ok" on every one of them, every deploy, through the whole outage —
+# because a zero-product store answers 200 just like a healthy one.
+# assert-grids.js measures the thing that actually matters: rendered product
+# cards, with the pool size beside it, and it refuses to report PASS on any site
+# it could not read. It ships with scripts/test-assert-grids.sh, which injects
+# four faults and proves the gate goes red on each. Never pass --test here.
+echo "==> grid gate (do the catalog sites actually render products?)"
+if node "$LOCAL/scripts/assert-grids.js" --min 12; then
+ echo " grid gate ok"
+else
+ GRID_RC=$?
+ echo " GRID GATE FAILED (rc=$GRID_RC) — a catalog site is serving an empty or unreadable grid"
+ FAIL=$((FAIL+1))
+fi
+
echo "==> deploy complete · $FAIL failures"
exit $FAIL
diff --git a/scripts/fixtures/grid-fixture.js b/scripts/fixtures/grid-fixture.js
new file mode 100755
index 0000000..a41acff
--- /dev/null
+++ b/scripts/fixtures/grid-fixture.js
@@ -0,0 +1,31 @@
+#!/usr/bin/env node
+/**
+ * grid-fixture.js — a deliberately broken (or deliberately healthy) stand-in for
+ * a fleet site, so scripts/test-assert-grids.sh can prove the deploy grid gate
+ * actually goes red. TEST-ONLY. Never referenced by server.js or any deploy path.
+ *
+ * MODE=empty-pool | /health serving:0, /catalog with 0 cards (the TK-11463 outage)
+ * MODE=empty-grid | /health serving:11242, /catalog with 0 cards
+ * MODE=no-contract | /health 200 but no `serving` field
+ * MODE=healthy | /health serving:11242, /catalog with 60 cards
+ */
+const http = require('http');
+const MODE = process.env.MODE || 'healthy';
+const PORT = Number(process.env.PORT || 0);
+
+const CARD = '<div class="card" data-handle="x" data-sku="DW-1" data-price="10.00"></div>';
+const shell = (n) => `<!doctype html><html><body><div class="grid" id="grid">${CARD.repeat(n)}</div></body></html>`;
+
+const cfg = {
+ 'empty-pool': { health: { ok: true, site: 'fixture', serving: 0, niche: 0 }, cards: 0 },
+ 'empty-grid': { health: { ok: true, site: 'fixture', serving: 11242, niche: 11242 }, cards: 0 },
+ 'no-contract': { health: { ok: true, site: 'fixture' }, cards: 60 },
+ 'healthy': { health: { ok: true, site: 'fixture', serving: 11242, niche: 11242 }, cards: 60 },
+}[MODE];
+if (!cfg) { console.error('unknown MODE ' + MODE); process.exit(2); }
+
+http.createServer((req, res) => {
+ if (req.url.startsWith('/health')) { res.writeHead(200, { 'Content-Type': 'application/json' }); return res.end(JSON.stringify(cfg.health)); }
+ if (req.url.startsWith('/catalog')) { res.writeHead(200, { 'Content-Type': 'text/html' }); return res.end(shell(cfg.cards)); }
+ res.writeHead(404); res.end();
+}).listen(PORT, '127.0.0.1', () => process.stderr.write('fixture ' + MODE + ' on ' + PORT + '\n'));
diff --git a/scripts/test-assert-grids.sh b/scripts/test-assert-grids.sh
new file mode 100755
index 0000000..51bf2fb
--- /dev/null
+++ b/scripts/test-assert-grids.sh
@@ -0,0 +1,78 @@
+#!/usr/bin/env bash
+# test-assert-grids.sh — NEGATIVE TEST for the deploy grid gate.
+#
+# CLAUDE.md amendment 3: "A check ships with a negative test proving it goes RED
+# on an injected fault, or it does not ship. A positive-only test on a detector
+# proves nothing — it confirms the happy path and leaves the entire purpose of
+# the component unverified."
+#
+# So this breaks assert-grids.js on purpose, four ways, and asserts it goes red
+# each time — then proves it goes green on a healthy fixture, so we know red
+# isn't just its resting state.
+#
+# 1. EMPTY POOL /health {serving:0} -> FAIL (the TK-11463 outage)
+# 2. EMPTY GRID pool>0 but 0 cards rendered -> FAIL (render broke downstream)
+# 3. DEAD ORIGIN nothing listening -> WARN (NOT_MEASURED, never PASS)
+# 4. NO CONTRACT /health without `serving` -> WARN (NOT_MEASURED, never PASS)
+# 5. HEALTHY pool>0 and cards>0 -> PASS
+#
+# Run: bash scripts/test-assert-grids.sh
+set -uo pipefail
+cd "$(dirname "$0")/.."
+
+PASSED=0; FAILED=0
+fixture_pid=""
+cleanup() { [ -n "$fixture_pid" ] && kill "$fixture_pid" 2>/dev/null; }
+trap cleanup EXIT
+
+# $1=name $2=fixture mode $3=expected exit code $4=expected verdict substring
+run_case() {
+ local name="$1" mode="$2" want_code="$3" want_verdict="$4"
+ local port out code
+ port=$(node -e 'const s=require("net").createServer();s.listen(0,()=>{console.log(s.address().port);s.close()})')
+
+ if [ "$mode" != "dead" ]; then
+ MODE="$mode" PORT="$port" node scripts/fixtures/grid-fixture.js & fixture_pid=$!
+ # wait for listen
+ for _ in $(seq 1 40); do
+ curl -fsS -m 1 "http://127.0.0.1:$port/health" >/dev/null 2>&1 && break
+ # the no-contract fixture still answers 200, the dead one never does
+ sleep 0.1
+ done
+ fi
+
+ out=$(node scripts/assert-grids.js --json --test --base "http://127.0.0.1:$port" --timeout 3000 2>&1); code=$?
+ [ -n "$fixture_pid" ] && { kill "$fixture_pid" 2>/dev/null; wait "$fixture_pid" 2>/dev/null; fixture_pid=""; }
+
+ local got_verdict; got_verdict=$(printf '%s' "$out" | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{console.log(JSON.parse(s).verdict)}catch{console.log("UNPARSEABLE")}})')
+
+ if [ "$code" = "$want_code" ] && [ "$got_verdict" = "$want_verdict" ]; then
+ echo " ok $name -> exit $code, verdict $got_verdict (expected $want_code/$want_verdict)"
+ PASSED=$((PASSED+1))
+ else
+ echo " FAIL $name -> exit $code, verdict $got_verdict (EXPECTED $want_code/$want_verdict)"
+ printf '%s\n' "$out" | head -20
+ FAILED=$((FAILED+1))
+ fi
+}
+
+echo "==> negative tests: the gate MUST go red on an injected fault"
+run_case "1. empty pool (serving:0) " empty-pool 1 FAIL
+run_case "2. empty grid (0 cards, pool>0)" empty-grid 1 FAIL
+run_case "3. dead origin (nothing listens)" dead 2 WARN
+run_case "4. /health with no serving field" no-contract 2 WARN
+echo "==> positive test: and MUST go green when genuinely healthy"
+run_case "5. healthy (pool>0, cards>0) " healthy 0 PASS
+
+echo "==> $PASSED passed, $FAILED failed"
+# Guard the guard: the seam must be INERT without --test, or a scheduled job
+# could silently measure a fixture. Without --test, --base is ignored and the
+# script probes the real domains — so it must NOT report the fixture's verdict.
+echo "==> seam-inertness check (--base without --test must be ignored)"
+if node scripts/assert-grids.js --json --base http://127.0.0.1:1 --timeout 3000 2>&1 | grep -q '"origin_override": null'; then
+ echo " ok --base is inert without --test"; PASSED=$((PASSED+1))
+else
+ echo " FAIL --base leaked into a non-test run"; FAILED=$((FAILED+1))
+fi
+
+exit $((FAILED > 0 ? 1 : 0))
← 3f245e2 auto-data-snapshot: 2026-09-11T11:23:31 (1 data files) — .pl
·
back to Dw Domain Fleet
·
pull-catalog: flatten to the SELLABLE variant, not blindly v caaf32b →