← back to Designer Wallcoverings
worker-guard: pre-push dedup guard module + 9-test harness + gated deploy doc for shopify-queue-worker (built/tested, NOT deployed)
6cc85e550901c549d292e91610ad0e7404a70582 · 2026-06-12 09:09:33 -0700 · SteveStudio2
Files touched
A shopify/scripts/worker-guard/.gitignoreA shopify/scripts/worker-guard/DEPLOY.mdA shopify/scripts/worker-guard/dedup-guard.jsA shopify/scripts/worker-guard/test-dedup-guard.js
Diff
commit 6cc85e550901c549d292e91610ad0e7404a70582
Author: SteveStudio2 <steve@designerwallcoverings.com>
Date: Fri Jun 12 09:09:33 2026 -0700
worker-guard: pre-push dedup guard module + 9-test harness + gated deploy doc for shopify-queue-worker (built/tested, NOT deployed)
---
shopify/scripts/worker-guard/.gitignore | 3 +
shopify/scripts/worker-guard/DEPLOY.md | 46 +++++++++++++
shopify/scripts/worker-guard/dedup-guard.js | 84 ++++++++++++++++++++++++
shopify/scripts/worker-guard/test-dedup-guard.js | 65 ++++++++++++++++++
4 files changed, 198 insertions(+)
diff --git a/shopify/scripts/worker-guard/.gitignore b/shopify/scripts/worker-guard/.gitignore
new file mode 100644
index 00000000..b98293d1
--- /dev/null
+++ b/shopify/scripts/worker-guard/.gitignore
@@ -0,0 +1,3 @@
+worker.js.prod
+worker.js.patched
+_synckcheck.js
diff --git a/shopify/scripts/worker-guard/DEPLOY.md b/shopify/scripts/worker-guard/DEPLOY.md
new file mode 100644
index 00000000..95827eca
--- /dev/null
+++ b/shopify/scripts/worker-guard/DEPLOY.md
@@ -0,0 +1,46 @@
+# shopify-queue-worker pre-push dedup guard — build complete, DEPLOY GATED
+
+**Status:** Built + tested locally. **NOT deployed.** Needs Steve's explicit go (edits the central prod write path).
+**DTD:** unanimous A (build locally w/ dry mode → diff → gated deploy), 2026-06-12.
+
+## What it does
+Before the worker executes a product **CREATE** job (`productSet`/`productCreate` with no `input.id`),
+it asks Shopify whether a **non-archived** product already carries the create's base SKU. If so →
+backfill `vendor_catalog.shopify_product_id` + mark the job done (`{dedup_existing,product_id}`) +
+`pg_notify` + **skip the create**. Archived-only or no match → proceeds to create normally.
+The worker's existing dedup only catches double-enqueues (5s `request_hash`); this closes the
+real gap (SKU already LIVE on Shopify) for every writer in one place.
+
+## Files
+- `dedup-guard.js` — the guard module (dependency-injected `shopifyRequest`; unit-testable).
+- `test-dedup-guard.js` — 9 tests, all pass (extraction: create/update/metafield/sku-less; live
+ lookup; guard on/dry/new). Run: `node test-dedup-guard.js`.
+- `worker.js.prod` — read-only snapshot of current prod worker (gitignored).
+- `worker.js.patched` — prod + the 2-anchor insertion (gitignored). See the diff below.
+
+## The diff (2 anchors, fail-OPEN)
+1. top: `const { dedupGuardForCreate } = require('./dedup-guard'); const DEDUP_GUARD_MODE = process.env.DEDUP_GUARD || 'on';`
+2. before the generic `shopifyRequest(job.method,...)` execute block in `processJob`: a try/catch that
+ calls the guard. **Any guard error falls through to the normal create** (never blocks a legit push).
+
+Regenerate locally to review:
+```
+cd shopify/scripts/worker-guard
+git diff --no-index worker.js.prod worker.js.patched
+```
+
+## Safe rollout (when Steve approves)
+1. `scp dedup-guard.js root@45.61.58.125:/root/DW-Agents/shopify-queue-worker/dedup-guard.js`
+2. Back up + patch worker.js on prod (apply the 2 anchors; keep `worker.js.bak`).
+3. **Observation window first:** start the worker with `DEDUP_GUARD=dry` — it logs
+ `[dedup-guard:DRY] … WOULD skip create` for any hit but changes nothing. Watch
+ `logs/` for a few real create jobs to confirm decisions look right.
+4. Flip to `DEDUP_GUARD=on` (default) once dry output looks correct → guard actively skips+backfills.
+5. Rollback at any point: `DEDUP_GUARD=off` (no-op) or restore `worker.js.bak` + restart.
+
+## Review notes / open questions for Steve
+- Backfill targets the unified `vendor_catalog` (same table + `dw_sku` key the worker's own
+ post-success backfill uses at line ~751). If a vendor's gate reads its **per-vendor** catalog
+ (e.g. `schumacher_catalog`) instead, we may also want to backfill there — flag if so.
+- One extra Shopify read per CREATE job (creates are low-volume + variant-ceiling-gated, so cost is
+ negligible). Updates/metafields/sku-less creates are untouched (guard returns null fast).
diff --git a/shopify/scripts/worker-guard/dedup-guard.js b/shopify/scripts/worker-guard/dedup-guard.js
new file mode 100644
index 00000000..3d5a998b
--- /dev/null
+++ b/shopify/scripts/worker-guard/dedup-guard.js
@@ -0,0 +1,84 @@
+'use strict';
+/**
+ * PRE-PUSH DEDUP GUARD for the shopify-queue-worker (Steve 2026-06-12, DTD-picked structural fix).
+ *
+ * The worker's existing dedup only catches double-ENQUEUES (same request_hash within 5s). It does
+ * NOT check whether the SKU about to be CREATED already has a LIVE listing on Shopify. So any
+ * link drift — a product created elsewhere whose shopify_product_id never reached prod, a late
+ * sync, a re-import — turns the next create job into a LIVE duplicate.
+ *
+ * This guard runs immediately before the worker executes a product-CREATE job. It asks Shopify
+ * whether a NON-archived product already carries the create's base variant SKU. If so it tells the
+ * worker to SKIP the create and backfill the catalog link instead. Archived-only matches fall
+ * through to a fresh create (the SKU has no live listing). Idempotent; never clobbers a live product.
+ *
+ * Deliberately dependency-injected (shopifyRequest + pool passed in) so it is unit-testable
+ * against real Shopify WITHOUT running the worker, and adds zero new HTTP/DB plumbing to worker.js.
+ *
+ * Modes (env DEDUP_GUARD): 'on' (default — skip+backfill), 'dry' (log decision, do NOT skip —
+ * observation window), 'off' (no-op).
+ */
+
+const baseSku = s => String(s || '').toUpperCase().replace(/-(SAMPLE|ROLL|YARD)$/i, '').trim();
+
+// Pull the base variant SKU out of a create job's payload. Returns null when the job is NOT a
+// guardable product-create (updates, non-product writes, metafield ops, sku-less productCreates).
+function extractCreateSku(job) {
+ if (!job || job.method !== 'POST' || !job.endpoint || !job.endpoint.includes('/graphql.json')) return null;
+ const p = job.payload || {};
+ const q = String(p.query || '');
+ const isCreate = /\bproductSet\b/.test(q) || /\bproductCreate\b/.test(q);
+ if (!isCreate) return null;
+ const vars = p.variables || {};
+ // collect every "input"-shaped object (input, input0, input1, … for batched productCreate)
+ const inputs = Object.keys(vars).filter(k => /^input\d*$/.test(k)).map(k => vars[k]).filter(Boolean);
+ if (!inputs.length && vars.input) inputs.push(vars.input);
+ const skus = [];
+ let anyUpdate = false;
+ for (const inp of inputs) {
+ if (inp && inp.id) { anyUpdate = true; continue; } // has product id → UPDATE, not a create
+ const variants = Array.isArray(inp && inp.variants) ? inp.variants : [];
+ for (const v of variants) {
+ const sku = v && (v.sku || (v.inventoryItem && v.inventoryItem.sku));
+ if (sku) skus.push(String(sku));
+ }
+ }
+ if (!skus.length) return null; // sku-less create (graphic-agent default variant) — nothing to match yet
+ const base = baseSku(skus.find(s => !/-sample$/i.test(s)) || skus[0]);
+ if (!base) return null;
+ return { base, skus, hadUpdate: anyUpdate };
+}
+
+// Ask Shopify whether a NON-archived product already carries this base SKU. Returns
+// { productId, status } for the best live match (ACTIVE preferred, then DRAFT), or null when
+// only archived copies exist (or none) — in which case a fresh create is correct.
+async function findLiveProductBySku(base, shopifyRequest, token) {
+ const res = await shopifyRequest('POST', '/graphql.json', {
+ query: `query { products(first: 15, query: "sku:${base}") { edges { node { id status variants(first: 25) { edges { node { sku } } } } } } }`
+ }, token);
+ const edges = res && res.body && res.body.data && res.body.data.products ? res.body.data.products.edges : [];
+ const key = baseSku(base);
+ const matches = (edges || []).map(e => e.node).filter(n => (n.variants && n.variants.edges || []).some(v => baseSku(v.node.sku) === key));
+ if (!matches.length) return null;
+ const live = matches.find(n => n.status === 'ACTIVE') || matches.find(n => n.status === 'DRAFT');
+ if (!live) return null; // archived-only → allow create
+ return { productId: String(live.id).replace(/.*\//, ''), status: live.status };
+}
+
+/**
+ * Main entry. Returns one of:
+ * null → not a guardable create, or no live duplicate → proceed normally
+ * { skip:true, productId, base, status, dry:false } → live dup found, worker should skip+backfill
+ * { skip:false, productId, base, status, dry:true } → dry mode: live dup found but DON'T skip (observe)
+ */
+async function dedupGuardForCreate(job, shopifyRequest, token, mode = 'on') {
+ if (mode === 'off') return null;
+ const ext = extractCreateSku(job);
+ if (!ext) return null;
+ const hit = await findLiveProductBySku(ext.base, shopifyRequest, token);
+ if (!hit) return null;
+ const dry = mode === 'dry';
+ return { skip: !dry, dry, productId: hit.productId, status: hit.status, base: ext.base };
+}
+
+module.exports = { extractCreateSku, findLiveProductBySku, dedupGuardForCreate, baseSku };
diff --git a/shopify/scripts/worker-guard/test-dedup-guard.js b/shopify/scripts/worker-guard/test-dedup-guard.js
new file mode 100644
index 00000000..d7abaa62
--- /dev/null
+++ b/shopify/scripts/worker-guard/test-dedup-guard.js
@@ -0,0 +1,65 @@
+'use strict';
+/**
+ * Standalone test for dedup-guard.js — proves SKU extraction + live Shopify lookup WITHOUT
+ * running the worker. Uses the same Shopify token the worker uses (from secrets-manager .env).
+ * node test-dedup-guard.js
+ */
+const https = require('https'); const fs = require('fs'); const os = require('os');
+const { extractCreateSku, findLiveProductBySku, dedupGuardForCreate } = require('./dedup-guard');
+
+const TOKEN = (fs.readFileSync(os.homedir() + '/Projects/secrets-manager/.env', 'utf8').match(/^SHOPIFY_ADMIN_TOKEN=(.*)$/m) || [])[1].replace(/['"]/g, '').trim();
+const STORE = 'designer-laboratory-sandbox.myshopify.com', API = '2024-10';
+const sleep = ms => new Promise(r => setTimeout(r, ms));
+// minimal stand-in for the worker's shopifyRequest(method, endpoint, payload, token)
+function shopifyRequest(method, endpoint, payload, token) {
+ return new Promise(res => {
+ const data = JSON.stringify(payload);
+ const req = https.request({ host: STORE, path: `/admin/api/${API}${endpoint}`, method, headers: { 'X-Shopify-Access-Token': token, 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data) } },
+ r => { let d = ''; r.on('data', c => d += c); r.on('end', () => { try { res({ status: r.statusCode, body: JSON.parse(d) }); } catch { res({ status: r.statusCode, body: {} }); } }); });
+ req.on('error', () => res({ status: 0, body: {} })); req.write(data); req.end();
+ });
+}
+
+let pass = 0, fail = 0;
+const eq = (name, got, want) => { const ok = JSON.stringify(got) === JSON.stringify(want); console.log(`${ok ? '✓' : '✗'} ${name} got=${JSON.stringify(got)}`); ok ? pass++ : fail++; };
+
+// --- 1. extraction unit tests (no network) ---
+const createJob = { method: 'POST', endpoint: '/graphql.json', payload: { query: 'mutation ProductPush($input: ProductSetInput!){ productSet(synchronous:true, input:$input){ product{id} userErrors{field message} } }',
+ variables: { input: { title: 'X', variants: [ { sku: 'DWHD-508056-Sample', price: '4.25' }, { sku: 'DWHD-508056', price: '72.11' } ] } } } };
+const updateJob = { method: 'POST', endpoint: '/graphql.json', payload: { query: 'mutation{ productSet(input:$input){product{id}} }',
+ variables: { input: { id: 'gid://shopify/Product/123', variants: [ { sku: 'DWHD-508056', price: '72.11' } ] } } } };
+const metafieldJob = { method: 'POST', endpoint: '/graphql.json', payload: { query: 'mutation{ metafieldsSet(metafields:$m){userErrors{field}} }', variables: { m: [] } } };
+const skulessCreate = { method: 'POST', endpoint: '/graphql.json', payload: { query: 'mutation{ productCreate(input:$input){product{id}} }', variables: { input: { title: 'no variants' } } } };
+
+eq('extract: create → base DWHD-508056', extractCreateSku(createJob)?.base, 'DWHD-508056');
+eq('extract: update (has input.id) → null', extractCreateSku(updateJob), null);
+eq('extract: metafield op → null', extractCreateSku(metafieldJob), null);
+eq('extract: sku-less create → null', extractCreateSku(skulessCreate), null);
+
+// --- 2. live Shopify lookup + full guard (network) ---
+(async () => {
+ if (!TOKEN) { console.error('no token'); process.exit(1); }
+ // known-live SKU (kept Kravet 2024 listing) should be found → guard says SKIP
+ const liveBase = 'DWKK-100001';
+ const live = await findLiveProductBySku(liveBase, shopifyRequest, TOKEN); await sleep(300);
+ eq(`lookup: ${liveBase} found live`, !!(live && live.productId), true);
+
+ // fake SKU should NOT be found → guard allows create
+ const none = await findLiveProductBySku('DWKK-999999-NOPE', shopifyRequest, TOKEN); await sleep(300);
+ eq('lookup: fake sku → null', none, null);
+
+ // full guard on a create job whose base already exists live → skip:true
+ const g1 = await dedupGuardForCreate({ ...createJob, payload: { ...createJob.payload, variables: { input: { variants: [ { sku: 'DWKK-100001-Sample' }, { sku: 'DWKK-100001' } ] } } } }, shopifyRequest, TOKEN, 'on'); await sleep(300);
+ eq('guard ON: existing-live create → skip:true', !!(g1 && g1.skip), true);
+
+ // dry mode → skip:false but still reports the hit
+ const g2 = await dedupGuardForCreate({ ...createJob, payload: { ...createJob.payload, variables: { input: { variants: [ { sku: 'DWKK-100001-Sample' }, { sku: 'DWKK-100001' } ] } } } }, shopifyRequest, TOKEN, 'dry'); await sleep(300);
+ eq('guard DRY: existing-live create → skip:false, dry:true', g2 && g2.skip === false && g2.dry === true, true);
+
+ // genuinely-new sku create → null (proceed)
+ const g3 = await dedupGuardForCreate({ ...createJob, payload: { ...createJob.payload, variables: { input: { variants: [ { sku: 'DWZZ-NEWSKU-0001-Sample' }, { sku: 'DWZZ-NEWSKU-0001' } ] } } } }, shopifyRequest, TOKEN, 'on'); await sleep(300);
+ eq('guard ON: brand-new sku → null (proceed)', g3, null);
+
+ console.log(`\n${fail === 0 ? 'ALL PASS' : 'FAILURES'}: ${pass} passed, ${fail} failed`);
+ process.exit(fail === 0 ? 0 : 1);
+})();
← 79f2fa59 cadence-import: pre-push dedup guard — query Shopify by base
·
back to Designer Wallcoverings
·
vendors: wire Newwall (6 confirmed lines, cost=trade_net*0.8 7c8eaf1b →