← back to Sanderson Onboard
TK-11471: per-RUN weight proof for the SDG publish cadence
550735110ae64b2dfa1b66fc8c06808bd85c28e4 · 2026-09-11 11:28:34 -0700 · Steve Abrams
TK-11414 fixed create_sdg.mjs (8d09eed) so the SDG cadence stops shipping zero-weight
products live, but "the fix is committed" is not "the fix works in a real run" — and the
cadence is the thing that put 385 zero-weight ACTIVE variants live overnight.
- verify_cadence_weights.mjs: READ-ONLY. Resolves the inventoryItem ids of exactly the
products a given cadence run created (pilot/cadence.log run headers x create-audit.jsonl)
and asserts weight>0 on each against live Shopify, so an offender is attributable to the
RUN. dw-active-weight-canary cannot do this — it counts catalog-wide rows with no
provenance, so a clean catalog with a still-broken producer looks identical to a fixed one.
Fails CLOSED: an unreadable variant is UNMEASURED => UNKNOWN/WARN, never PASS. Owns its own
--heartbeat and writes a verdict on EVERY exit path including an uncaught throw, so a dead
verifier reports WARN instead of leaving no row on the morning panel.
Carries post_fix, and a PASS on a pre-fix run says in its own detail that it is evidence of
the backfill healing the run after the fact, NOT of the fix holding.
- verify_cadence_weights.test.mjs: the negative test (7/7 PASS). Drives the pure classify()
with injected faults — zero-weight sellable, zero-weight SAMPLE (samples count; the canary
FAILs on them too), null measurement, unreadable id, empty read, negative, NaN — and asserts
each goes RED. Live history cannot supply a red case because the TK-11414 backfill already
healed every pre-fix run, so faults are injected instead.
- cadence_resume.sh: runs the verify as step 5, so every daily run self-proves. `|| VRC=$?`
because the script runs under `set -e` and a bare call would abort the cadence the first
time verification reported FAIL; verified non-fatal on exit 1 and 2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FJHxAzaEMMxado57mFjiCk
Files touched
M scripts/cadence_resume.shA scripts/verify_cadence_weights.mjsA scripts/verify_cadence_weights.test.mjs
Diff
commit 550735110ae64b2dfa1b66fc8c06808bd85c28e4
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Fri Sep 11 11:28:34 2026 -0700
TK-11471: per-RUN weight proof for the SDG publish cadence
TK-11414 fixed create_sdg.mjs (8d09eed) so the SDG cadence stops shipping zero-weight
products live, but "the fix is committed" is not "the fix works in a real run" — and the
cadence is the thing that put 385 zero-weight ACTIVE variants live overnight.
- verify_cadence_weights.mjs: READ-ONLY. Resolves the inventoryItem ids of exactly the
products a given cadence run created (pilot/cadence.log run headers x create-audit.jsonl)
and asserts weight>0 on each against live Shopify, so an offender is attributable to the
RUN. dw-active-weight-canary cannot do this — it counts catalog-wide rows with no
provenance, so a clean catalog with a still-broken producer looks identical to a fixed one.
Fails CLOSED: an unreadable variant is UNMEASURED => UNKNOWN/WARN, never PASS. Owns its own
--heartbeat and writes a verdict on EVERY exit path including an uncaught throw, so a dead
verifier reports WARN instead of leaving no row on the morning panel.
Carries post_fix, and a PASS on a pre-fix run says in its own detail that it is evidence of
the backfill healing the run after the fact, NOT of the fix holding.
- verify_cadence_weights.test.mjs: the negative test (7/7 PASS). Drives the pure classify()
with injected faults — zero-weight sellable, zero-weight SAMPLE (samples count; the canary
FAILs on them too), null measurement, unreadable id, empty read, negative, NaN — and asserts
each goes RED. Live history cannot supply a red case because the TK-11414 backfill already
healed every pre-fix run, so faults are injected instead.
- cadence_resume.sh: runs the verify as step 5, so every daily run self-proves. `|| VRC=$?`
because the script runs under `set -e` and a bare call would abort the cadence the first
time verification reported FAIL; verified non-fatal on exit 1 and 2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FJHxAzaEMMxado57mFjiCk
---
scripts/cadence_resume.sh | 15 +++
scripts/verify_cadence_weights.mjs | 181 ++++++++++++++++++++++++++++++++
scripts/verify_cadence_weights.test.mjs | 62 +++++++++++
3 files changed, 258 insertions(+)
diff --git a/scripts/cadence_resume.sh b/scripts/cadence_resume.sh
index 0898bcd..813f795 100644
--- a/scripts/cadence_resume.sh
+++ b/scripts/cadence_resume.sh
@@ -31,4 +31,19 @@ node scripts/cadence_healthcheck.mjs --threshold=$THRESHOLD >> $LOG 2>&1 || {
echo "!! compliance below $THRESHOLD — unloading cadence to halt" >> $LOG
launchctl unload ~/Library/LaunchAgents/com.steve.sdg-publish-cadence.plist 2>/dev/null || true
}
+# 5) TK-11471 WEIGHT PROOF (read-only): assert every variant THIS run created is live with
+# weight>0, and write the fleet-health heartbeat. The SDG cadence is what put 385 zero-weight
+# variants live overnight (TK-11414); create_sdg.mjs was fixed in 8d09eed, and this is the step
+# that proves the fix actually held in a real run instead of assuming it did.
+# The verifier writes its OWN heartbeat (--heartbeat), including on its own failure, so a
+# verifier that dies reports WARN rather than leaving no row at all. Its exit code is CAPTURED
+# and logged, never propagated through `set -e` — verification must never abort the cadence.
+# NOTE `|| VRC=$?`: this script runs under `set -e`, so a bare call would ABORT the cadence the
+# first time the verifier reports FAIL(1)/UNKNOWN(2). A command in a `||` list is exempt from
+# `set -e`, which is what keeps a failing verification non-fatal while still capturing its code.
+VRC=0
+node scripts/verify_cadence_weights.mjs \
+ --heartbeat="$HOME/.claude/skills/sdg-cadence-weight-verify/data/latest.json" >> $LOG 2>&1 || VRC=$?
+echo "--- TK-11471 weight verify rc=$VRC (0=PASS 1=FAIL 2=UNKNOWN/WARN) ---" >> $LOG
+
echo "=== run complete rc=$RC ===" >> $LOG
diff --git a/scripts/verify_cadence_weights.mjs b/scripts/verify_cadence_weights.mjs
new file mode 100644
index 0000000..b242e2c
--- /dev/null
+++ b/scripts/verify_cadence_weights.mjs
@@ -0,0 +1,181 @@
+// verify_cadence_weights.mjs — TK-11471 (TK-11414 follow-on).
+// READ-ONLY proof that the SDG publish cadence no longer ships zero-weight products live.
+//
+// WHY THIS EXISTS: TK-11414 fixed create_sdg.mjs (commit 8d09eed) so the create payload carries
+// weight and goLive self-heals any zero-weight variant before activating. But "the fix is committed"
+// is NOT "the fix works in a real cadence run" — the cadence is the thing that put 385 zero-weight
+// variants live overnight in the first place. This asserts the invariant on EXACTLY the products a
+// given cadence run created, by reading their inventoryItem ids straight out of create-audit.jsonl
+// and asking LIVE Shopify what weight each one actually carries.
+//
+// It is deliberately NOT "count the zero-weight rows in the catalog" — that is the canary's job and
+// it cannot attribute an offender to a run. This measures the RUN.
+//
+// NEGATIVE TEST BUILT IN (CLAUDE.md TK-11431 amendment 3): point it at a PRE-fix run
+// (--date=YYYY-MM-DD before 2026-09-11) and it MUST come back FAIL. A verifier that only ever
+// goes green on the happy path proves nothing. See --self-test.
+//
+// Usage:
+// node scripts/verify_cadence_weights.mjs # the most recent cadence run
+// node scripts/verify_cadence_weights.mjs --date=2026-09-12
+// node scripts/verify_cadence_weights.mjs --self-test # assert a pre-fix run goes RED
+// READ-ONLY: issues GraphQL *queries* only. No mutation, no write, $0.
+import fs from 'node:fs';
+
+const env = fs.readFileSync('/Users/macstudio3/Projects/secrets-manager/.env', 'utf8');
+const STORE = (env.match(/^SHOPIFY_STORE=(.+)$/m) || [])[1].trim();
+const TOKEN = (env.match(/^SHOPIFY_FULL_ACCESS_TOKEN=(.+)$/m) || [])[1].trim();
+const GQL = `https://${STORE}/admin/api/2024-10/graphql.json`;
+const H = { 'X-Shopify-Access-Token': TOKEN, 'Content-Type': 'application/json' };
+const AUDIT = new URL('../pilot/create-audit.jsonl', import.meta.url).pathname;
+
+const arg = k => (process.argv.find(a => a.startsWith(`--${k}=`)) || '').split('=')[1] || null;
+const SELF_TEST = process.argv.includes('--self-test');
+const sleep = ms => new Promise(r => setTimeout(r, ms));
+
+// ── audit -> the iids a run created ────────────────────────────────────────────────────────────
+// create-audit.jsonl has no timestamp field, so a run is identified by file mtime ordering is NOT
+// safe. Instead we bucket by the audit line order against cadence.log run headers: every line
+// appended between two "SDG cadence run" headers belongs to that run. We reconstruct that by
+// reading cadence.log for the per-run CREATED sku lists.
+function runsFromLog() {
+ const log = fs.readFileSync(new URL('../pilot/cadence.log', import.meta.url).pathname, 'utf8').split('\n');
+ const runs = []; let cur = null;
+ for (const line of log) {
+ const h = line.match(/^=== (\d{4}-\d{2}-\d{2})T[\d:]+Z SDG cadence run/);
+ if (h) { cur = { date: h[1], skus: [] }; runs.push(cur); continue; }
+ const c = line.match(/^\s*✓ (\S+) → \d+ ACTIVE/);
+ if (c && cur) cur.skus.push(c[1]);
+ }
+ return runs.filter(r => r.skus.length);
+}
+
+function iidsForSkus(skus) {
+ const want = new Set(skus);
+ const out = []; // {dw, sku, iid, isSample}
+ for (const line of fs.readFileSync(AUDIT, 'utf8').split('\n')) {
+ if (!line.trim()) continue;
+ let r; try { r = JSON.parse(line); } catch { continue; }
+ if (!want.has(r.dw) || r.action !== 'CREATED') continue;
+ for (const v of (r.variants || [])) {
+ if (v.iid) out.push({ dw: r.dw, sku: v.sku, iid: v.iid, isSample: /-Sample$/i.test(v.sku || '') });
+ }
+ }
+ return out;
+}
+
+async function gql(query, variables) {
+ const res = await fetch(GQL, { method: 'POST', headers: H, body: JSON.stringify({ query, variables }) });
+ if (!res.ok) throw new Error(`HTTP ${res.status}`);
+ const j = await res.json();
+ if (j.errors) throw new Error(JSON.stringify(j.errors).slice(0, 300));
+ return j.data;
+}
+
+// Batched read of inventoryItem weights. `nodes` takes up to 250 ids; we chunk at 100 to stay light.
+const Q = `query($ids:[ID!]!){ nodes(ids:$ids){ ... on InventoryItem { id measurement { weight { value unit } } } } }`;
+
+async function weightsFor(iids) {
+ const map = new Map();
+ for (let i = 0; i < iids.length; i += 100) {
+ const chunk = iids.slice(i, i + 100).map(n => `gid://shopify/InventoryItem/${n}`);
+ const d = await gql(Q, { ids: chunk });
+ for (const n of (d.nodes || [])) {
+ if (!n || !n.id) continue; // a null node = id not found -> stays UNMEASURED
+ const w = n.measurement?.weight;
+ const val = w?.value == null ? null : Number(w.value);
+ const unit = String(w?.unit || '').toLowerCase();
+ const lb = val == null ? null : (unit === 'kilograms' ? val * 2.20462 : unit === 'grams' ? val / 453.59237 : val);
+ map.set(Number(n.id.split('/').pop()), lb);
+ }
+ await sleep(250);
+ }
+ return map;
+}
+
+// PURE classifier (no I/O) — the offline negative test drives THIS directly, because every
+// pre-fix cadence run has since been healed by the TK-11414 backfill, so a live "point it at a
+// known-bad run" self-test is structurally unavailable. A positive-only test on a detector proves
+// nothing, so the red path is proven against injected faults instead (CLAUDE.md TK-11431 am.3).
+export function classify(rows, weightMap, date = 'fixture') {
+ const zero = [], unmeasured = [];
+ for (const r of rows) {
+ if (!weightMap.has(r.iid)) { unmeasured.push(r); continue; } // never silently treated as OK
+ const lb = weightMap.get(r.iid);
+ if (lb == null || !Number.isFinite(lb) || lb <= 0) zero.push({ ...r, lb });
+ }
+ // An UNMEASURED input is never PASS (CLAUDE.md TK-11431 amendment 1).
+ const verdict = zero.length ? 'FAIL' : unmeasured.length ? 'UNKNOWN' : 'PASS';
+ const status = zero.length ? 'FAIL' : unmeasured.length ? 'WARN' : 'PASS';
+ return {
+ date, variants_measured: rows.length - unmeasured.length, variants_total: rows.length,
+ zero_weight: zero.length,
+ zero_weight_sample: zero.filter(z => z.isSample).length,
+ zero_weight_sellable: zero.filter(z => !z.isSample).length,
+ unmeasured: unmeasured.length, verdict, status,
+ detail: zero.length
+ ? `${zero.length} of ${rows.length} variants created by the ${date} cadence run are LIVE at zero/null weight (${zero.filter(z=>z.isSample).length} sample, ${zero.filter(z=>!z.isSample).length} sellable) — the SDG weight fix did NOT hold for this run.`
+ : unmeasured.length
+ ? `${unmeasured.length} of ${rows.length} variants could not be read from live Shopify — invariant NOT asserted (never reported as PASS).`
+ : `all ${rows.length} variants created by the ${date} cadence run carry positive weight — SDG weight fix held.`,
+ sample_offenders: zero.slice(0, 10).map(z => ({ sku: z.sku, iid: z.iid, lb: z.lb })),
+ };
+}
+
+async function verifyRun(run) {
+ const rows = iidsForSkus(run.skus);
+ if (!rows.length) return { date: run.date, verdict: 'UNKNOWN', status: 'WARN',
+ detail: `no inventoryItem ids in create-audit.jsonl for the ${run.date} run (${run.skus.length} skus in log) — cannot assert, NOT a pass` };
+
+ const w = await weightsFor(rows.map(r => r.iid));
+ return { products: run.skus.length, ...classify(rows, w, run.date) };
+}
+
+const FIX_DATE = '2026-09-11'; // create_sdg.mjs weight fix committed 2026-09-11T16:17Z, AFTER that day's 11:30Z run
+
+// HEARTBEAT — the verifier is the SINGLE writer of its own verdict. Doing this in shell was tried
+// and silently wrote nothing (env var never reached the child), which is a false-NOTHING: the
+// morning panel would show no row at all rather than a warning. A component must be able to report
+// its own failure, so every exit path below — including an uncaught throw — writes a verdict.
+const HEARTBEAT = (process.argv.find(a => a.startsWith('--heartbeat=')) || '').split('=')[1] || null;
+function beat(o) {
+ if (!HEARTBEAT) return;
+ try {
+ fs.mkdirSync(HEARTBEAT.replace(/\/[^/]+$/, ''), { recursive: true });
+ fs.writeFileSync(HEARTBEAT, JSON.stringify({ ...o, scanned_at: new Date().toISOString(), skill: 'sdg-cadence-weight-verify' }, null, 2));
+ } catch (e) { console.error('heartbeat write failed: ' + e.message); }
+}
+
+const INVOKED_DIRECTLY = process.argv[1] && import.meta.url.endsWith(process.argv[1].split('/').pop());
+if (INVOKED_DIRECTLY) await (async () => {
+ const runs = runsFromLog();
+ if (!runs.length) { beat({ verdict: 'UNKNOWN', status: 'WARN', detail: 'no cadence runs found in pilot/cadence.log — the SDG per-run weight invariant was NOT asserted' }); console.error('no cadence runs found in pilot/cadence.log'); process.exit(2); }
+
+ if (SELF_TEST) {
+ // NEGATIVE TEST: the newest PRE-fix run must come back FAIL. If it does not, this verifier is
+ // measuring the wrong thing and must not be trusted on a post-fix run.
+ const pre = runs.filter(r => r.date <= FIX_DATE).pop();
+ if (!pre) { console.error('SELF-TEST INCONCLUSIVE: no pre-fix run in the log'); process.exit(2); }
+ const r = await verifyRun(pre);
+ console.log(JSON.stringify(r, null, 2));
+ if (r.verdict === 'FAIL') { console.log(`\nSELF-TEST PASS — verifier goes RED on the known-bad ${pre.date} pre-fix run.`); process.exit(0); }
+ console.error(`\nSELF-TEST FAIL — pre-fix run ${pre.date} did not come back FAIL (got ${r.verdict}). Either those rows were backfilled after the fact, or this verifier does not actually measure weight. Do NOT trust a green post-fix result until this is explained.`);
+ process.exit(1);
+ }
+
+ const want = arg('date');
+ const run = want ? runs.find(r => r.date === want) : runs[runs.length - 1];
+ if (!run) { beat({ verdict: 'UNKNOWN', status: 'WARN', detail: `no cadence run for date=${want} — invariant NOT asserted` }); console.error(`no cadence run for date=${want}`); process.exit(2); }
+ const r = await verifyRun(run);
+ r.post_fix = run.date > FIX_DATE;
+ // A PASS on a PRE-fix run proves nothing about the fix — those variants were healed after the
+ // fact by the TK-11414 backfill. Say so in the verdict rather than letting it read as evidence.
+ if (r.verdict === 'PASS' && !r.post_fix) r.detail += ' NOTE: post_fix=false — this run predates the create_sdg.mjs weight fix (8d09eed), so this PASS reflects the TK-11414 backfill healing it after the fact, NOT the fix holding.';
+ beat(r);
+ console.log(JSON.stringify(r, null, 2));
+ process.exit(r.verdict === 'PASS' ? 0 : r.verdict === 'FAIL' ? 1 : 2);
+})().catch(e => {
+ beat({ verdict: 'UNKNOWN', status: 'WARN', detail: 'weight verifier threw: ' + (e && e.message ? e.message : String(e)) + ' — the SDG per-run weight invariant was NOT asserted for this run' });
+ console.error('verify_cadence_weights FAILED: ' + (e && e.stack ? e.stack : e));
+ process.exit(2);
+});
diff --git a/scripts/verify_cadence_weights.test.mjs b/scripts/verify_cadence_weights.test.mjs
new file mode 100644
index 0000000..f37b241
--- /dev/null
+++ b/scripts/verify_cadence_weights.test.mjs
@@ -0,0 +1,62 @@
+// verify_cadence_weights.test.mjs — TK-11471 NEGATIVE TEST.
+// A positive-only test on a detector proves nothing (CLAUDE.md TK-11431 amendment 3): the whole
+// purpose of this verifier is to go RED when the SDG cadence ships a zero-weight product live, so
+// the red path is what must be proven. Offline, zero network, zero DB.
+//
+// Why not "point it at a known-bad pre-fix run": every pre-fix cadence run has SINCE been healed by
+// the TK-11414 backfill, so live history can no longer produce a red. Faults are injected instead.
+import { classify } from './verify_cadence_weights.mjs';
+import assert from 'node:assert';
+
+let pass = 0, fail = 0;
+const t = (name, fn) => { try { fn(); console.log('PASS ' + name); pass++; }
+ catch (e) { console.log('FAIL ' + name + ' — ' + e.message); fail++; } };
+
+const sellable = (iid) => ({ dw: 'DWXX-1', sku: 'DWXX-1', iid, isSample: false });
+const sample = (iid) => ({ dw: 'DWXX-1', sku: 'DWXX-1-Sample', iid, isSample: true });
+
+t('injected ZERO-weight SELLABLE => FAIL (the 190-sellable leak)', () => {
+ const r = classify([sellable(1), sample(2)], new Map([[1, 0], [2, 0.25]]));
+ assert.strictEqual(r.verdict, 'FAIL');
+ assert.strictEqual(r.zero_weight_sellable, 1);
+ assert.strictEqual(r.zero_weight_sample, 0);
+});
+
+t('injected ZERO-weight SAMPLE => FAIL (samples count — the canary FAILs on them too)', () => {
+ const r = classify([sellable(1), sample(2)], new Map([[1, 3], [2, 0]]));
+ assert.strictEqual(r.verdict, 'FAIL');
+ assert.strictEqual(r.zero_weight_sample, 1);
+});
+
+t('NULL weight (measurement absent) => FAIL, never silently OK', () => {
+ const r = classify([sellable(1)], new Map([[1, null]]));
+ assert.strictEqual(r.verdict, 'FAIL');
+});
+
+t('UNMEASURED variant (id not returned by Shopify) => UNKNOWN/WARN, NEVER PASS', () => {
+ const r = classify([sellable(1), sellable(2)], new Map([[1, 3]])); // iid 2 missing
+ assert.strictEqual(r.verdict, 'UNKNOWN');
+ assert.strictEqual(r.status, 'WARN');
+ assert.strictEqual(r.unmeasured, 1);
+});
+
+t('EMPTY weight map (whole read failed) => UNKNOWN, never a false green', () => {
+ const r = classify([sellable(1), sample(2)], new Map());
+ assert.notStrictEqual(r.verdict, 'PASS');
+ assert.strictEqual(r.unmeasured, 2);
+});
+
+t('all variants positively weighted => PASS', () => {
+ const r = classify([sellable(1), sample(2)], new Map([[1, 3], [2, 0.25]]));
+ assert.strictEqual(r.verdict, 'PASS');
+ assert.strictEqual(r.status, 'PASS');
+ assert.strictEqual(r.zero_weight, 0);
+});
+
+t('negative / NaN weight => FAIL (not treated as a number that happens to be falsy-safe)', () => {
+ assert.strictEqual(classify([sellable(1)], new Map([[1, -1]])).verdict, 'FAIL');
+ assert.strictEqual(classify([sellable(1)], new Map([[1, NaN]])).verdict, 'FAIL');
+});
+
+console.log(`\n${fail ? 'TESTS FAILED' : 'ALL TESTS PASS'} — ${pass} passed, ${fail} failed`);
+process.exit(fail ? 1 : 0);
← 8d09eed TK-11414: stop the SDG weight leak at the source (create_sdg
·
back to Sanderson Onboard
·
TK-11471: verify the SDG weight heal, and stop the verifier d9e8fa8 →