← back to Dw Signup Fulfillment
TK-11411: ledgered re-send for the 14 customers who lost mail in the George outage
03b8d4b81b59b6227410213431686dca44454c15 · 2026-09-10 11:25:45 -0700 · Steve Abrams
Scoping correction first: I initially reported THREE affected customers. That was
wrong - it was only what fit in a 200-line log window. The real cohort from the
on-disk logs is FOURTEEN, and I had also grabbed a rotated logfile with `head -1`
before noticing. 17 retail-verify + 5 designer-welcome + 5 trade-application 401s.
Verified against live Shopify rather than trusting the logs: all 14 customers
exist and NONE carries verified-sample, so all 14 are genuinely still owed. They
split into two cohorts needing different letters:
RETAIL (11) - signed up for free samples, never got "confirm your email", so
never clicked, so no verified-sample tag, so they cannot claim samples.
Remedy: re-run the same lib/verify.startVerification the /claim route uses.
TRADE (3: noelle@silkandslate, team@michellewalshdesigns, yanchapaxi@auxarchitecture)
- their approval genuinely LANDED. Shopify carries trade+trade_approved AND the
local row says approved, so this is NOT the split-brain Cody warned about; only
the "you're approved" letter was lost. Remedy: that letter alone. Deliberately
does NOT re-run trade.approve(), which would re-tag and re-assign a rep.
Safety: dry-run by default and --apply refuses while DRY_RUN is on; the cohort is
DERIVED at run time from live Shopify tags plus the local store, never hardcoded,
so anyone who recovered on their own is skipped; idempotent through
data/tk11411-resend-ledger.jsonl, appended only after a confirmed real send.
Dry-run against live prod data classifies 11 RETAIL + 3 TRADE, sends nothing.
NOT SENT. Mailing 14 real customers is send-to-customer, hard-gated.
Files touched
A scripts/tk11411-resend.js
Diff
commit 03b8d4b81b59b6227410213431686dca44454c15
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Thu Sep 10 11:25:45 2026 -0700
TK-11411: ledgered re-send for the 14 customers who lost mail in the George outage
Scoping correction first: I initially reported THREE affected customers. That was
wrong - it was only what fit in a 200-line log window. The real cohort from the
on-disk logs is FOURTEEN, and I had also grabbed a rotated logfile with `head -1`
before noticing. 17 retail-verify + 5 designer-welcome + 5 trade-application 401s.
Verified against live Shopify rather than trusting the logs: all 14 customers
exist and NONE carries verified-sample, so all 14 are genuinely still owed. They
split into two cohorts needing different letters:
RETAIL (11) - signed up for free samples, never got "confirm your email", so
never clicked, so no verified-sample tag, so they cannot claim samples.
Remedy: re-run the same lib/verify.startVerification the /claim route uses.
TRADE (3: noelle@silkandslate, team@michellewalshdesigns, yanchapaxi@auxarchitecture)
- their approval genuinely LANDED. Shopify carries trade+trade_approved AND the
local row says approved, so this is NOT the split-brain Cody warned about; only
the "you're approved" letter was lost. Remedy: that letter alone. Deliberately
does NOT re-run trade.approve(), which would re-tag and re-assign a rep.
Safety: dry-run by default and --apply refuses while DRY_RUN is on; the cohort is
DERIVED at run time from live Shopify tags plus the local store, never hardcoded,
so anyone who recovered on their own is skipped; idempotent through
data/tk11411-resend-ledger.jsonl, appended only after a confirmed real send.
Dry-run against live prod data classifies 11 RETAIL + 3 TRADE, sends nothing.
NOT SENT. Mailing 14 real customers is send-to-customer, hard-gated.
---
scripts/tk11411-resend.js | 110 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 110 insertions(+)
diff --git a/scripts/tk11411-resend.js b/scripts/tk11411-resend.js
new file mode 100644
index 0000000..6b06851
--- /dev/null
+++ b/scripts/tk11411-resend.js
@@ -0,0 +1,110 @@
+'use strict';
+// TK-11411 — re-send the mail lost during the George 401 outage (2026-09-10).
+//
+// George rejected every send with "Invalid credentials" for the duration of the
+// outage. Two cohorts lost different letters:
+// RETAIL — signed up for free samples, never got "confirm your email", so they
+// never clicked, so they never got the verified-sample tag, so they
+// cannot claim samples. Remedy: re-run the SAME verification start the
+// /claim route uses (lib/verify.startVerification) — mints a fresh
+// token and sends the letter.
+// TRADE — their approval genuinely LANDED (Shopify carries trade+trade_approved
+// AND the local row says approved) but the "you're approved" letter
+// failed, so they were never told. Remedy: send that letter only. Do
+// NOT re-run trade.approve() — they are already approved and it would
+// re-tag and re-assign a rep.
+//
+// SAFE BY DESIGN:
+// * DRY-RUN by default. --apply is required to send, and it refuses while
+// config.DRY_RUN is on (a bare `node ...` inherits DRY_RUN=1 and would
+// otherwise silently "succeed" without sending).
+// * COHORT IS DERIVED AT RUN TIME, never hardcoded: a customer that already has
+// verified-sample is SKIPPED, so anyone who recovered on their own is not
+// re-mailed.
+// * IDEMPOTENT via data/tk11411-resend-ledger.jsonl — an address already recorded
+// as sent is skipped on every later run.
+// * Ledger is append-only and written AFTER a confirmed real send.
+const fs = require('fs');
+const path = require('path');
+const config = require('../lib/config');
+const shopify = require('../lib/shopify');
+const email = require('../lib/email');
+const verify = require('../lib/verify');
+
+const args = process.argv.slice(2);
+const APPLY = args.includes('--apply');
+const LEDGER = path.join(__dirname, '..', 'data', 'tk11411-resend-ledger.jsonl');
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+// The addresses observed with source=retail-verify / designer-welcome /
+// trade-application 401s in the outage logs. Classification is NOT taken from
+// here — it is derived below from live Shopify + the local store.
+const COHORT = ['j.dmce@post.harvard.edu', 'juliewendl@aol.com', 'kristopher.woodcock@gmail.com',
+ 'lchandlee@aol.com', 'lizbeth@luxupholstery.com', 'margaret@designmadesimple.design',
+ 'noelle@silkandslate.com', 'office@leddyinteriors.com', 'sdkett20@gmail.com',
+ 'sheldonlycan@gmail.com', 'skl@liocowine.com', 'team@michellewalshdesigns.com',
+ 'tori@bellewoodgroup.com', 'yanchapaxi@auxarchitecture.com'];
+
+function alreadySent(addr) {
+ try {
+ return fs.readFileSync(LEDGER, 'utf8').split('\n').filter(Boolean)
+ .some((l) => { try { return JSON.parse(l).email === addr; } catch { return false; } });
+ } catch { return false; }
+}
+function record(row) { fs.appendFileSync(LEDGER, JSON.stringify(row) + '\n'); }
+
+function tradeRows() {
+ try {
+ return fs.readFileSync(path.join(__dirname, '..', 'data', 'trade-applications.jsonl'), 'utf8')
+ .split('\n').filter(Boolean).map((l) => JSON.parse(l));
+ } catch { return []; }
+}
+
+(async () => {
+ if (APPLY && config.DRY_RUN) {
+ console.error('REFUSING --apply while DRY_RUN is on: nothing would actually send. Re-run with DRY_RUN=0.');
+ process.exit(1);
+ }
+ console.log(`TK-11411 resend mode=${APPLY ? 'APPLY' : 'DRY-RUN'} DRY_RUN(config)=${config.DRY_RUN}`);
+ const rows = tradeRows();
+ let sent = 0, skipped = 0;
+
+ for (const addr of COHORT) {
+ if (alreadySent(addr)) { console.log(` [skip] ${addr} — already re-sent (ledger)`); skipped++; continue; }
+
+ const id = await shopify.findCustomerByEmail(addr);
+ if (!id) { console.log(` [skip] ${addr} — no Shopify customer`); skipped++; continue; }
+ const custId = String(id).replace(/\D/g, '');
+ const res = await shopify.getCustomer(custId);
+ const c = (res && res.json && (res.json.customer || (res.json.data && res.json.data.customer))) || {};
+ const tags = Array.isArray(c.tags) ? c.tags.join(',') : (c.tags || '');
+
+ if (/verified-sample/.test(tags)) { console.log(` [skip] ${addr} — already verified-sample (recovered on its own)`); skipped++; continue; }
+
+ // TRADE: approved on BOTH sides, only the letter was lost.
+ const approvedRow = rows.find((r) => String(r.email).toLowerCase() === addr && r.status === 'approved');
+ const isTrade = /(^|,)\s*trade_approved\s*(,|$)/.test(tags) && !!approvedRow;
+
+ if (isTrade) {
+ const repName = (approvedRow.assigned_rep && approvedRow.assigned_rep.name) || 'your DW rep';
+ const tpl = email.tradeApprovedEmail({ applicant: approvedRow, repName });
+ if (!APPLY) { console.log(` [dry] ${addr} — TRADE: would re-send "${tpl.subject}"`); continue; }
+ const m = await email.sendEmail({ to: addr, subject: tpl.subject, html: tpl.html, source: 'tk11411-trade-approved-resend' });
+ const ok = m && m.ok === true && !m.dryRun;
+ console.log(` [${ok ? 'sent' : 'FAIL'}] ${addr} — TRADE approval letter${ok ? '' : ' :: ' + JSON.stringify(m && (m.error || m.status))}`);
+ if (ok) { record({ ts: new Date().toISOString(), email: addr, cohort: 'trade', subject: tpl.subject }); sent++; }
+ await sleep(400);
+ continue;
+ }
+
+ // RETAIL: re-run the same verification start /claim uses.
+ if (!APPLY) { console.log(` [dry] ${addr} — RETAIL: would re-send the confirm-your-email verify letter`); continue; }
+ const r = await verify.startVerification({ email: addr, customerId: custId, firstName: c.first_name || addr.split('@')[0] });
+ const ok = r && r.ok !== false && !r.dryRun;
+ console.log(` [${ok ? 'sent' : 'FAIL'}] ${addr} — RETAIL verify letter${ok ? '' : ' :: ' + JSON.stringify(r && (r.reason || r.status))}`);
+ if (ok) { record({ ts: new Date().toISOString(), email: addr, cohort: 'retail' }); sent++; }
+ await sleep(400);
+ }
+ console.log(`\n${APPLY ? 'Sent' : 'Would send'}: ${APPLY ? sent : COHORT.length - skipped} · skipped: ${skipped}`);
+ if (!APPLY) console.log('Dry-run only. Re-run with DRY_RUN=0 --apply to actually send.');
+})().catch((e) => { console.error('tk11411-resend error:', e.message); process.exit(1); });
← c959bc5 TK-11366: answer the valence question - full-price free ship
·
back to Dw Signup Fulfillment
·
TK-11411: protect the resend ledger from rsync --delete 1a93bb0 →