[object Object]

← back to Dw Signup Fulfillment

TK-11366: ROOT CAUSE FOUND - checkout's default shipping option flipped ~2026-08-18

07e8ba575679a1a44002964ac3d019d31fea63ba · 2026-09-10 09:23:12 -0700 · Steve Abrams

Ran the counterfactual population my order-only cohorts could not see (Cody's original
objection): abandoned checkouts, which record what the shopper was HANDED, not what they
chose.

US sample-only abandoned checkouts recording $0.00 shipping:
  Jun 1-Aug 17: 24/26 (92%%)
  Aug 18-Sep 10: 2/56 (4%%)
  September: 0 of 17
Paid amounts cluster on UPS carrier rates: $23.35 x17, $23.13 x16.

Decisive because /cart/shipping_rates.json still returns 'Free Shipping (No Tracking)'
$0.00 listed FIRST. The rate is OFFERED but no longer DEFAULTED. Offered != defaulted,
and that one distinction explains every observation at once.

Measured impact: 51 orders since Aug 18 that qualified for $0.00 paid anyway
($1,120.10, avg $21.96); 54 of 56 abandoned checkouts carry a paid line; abandonment
volume more than doubled 26 -> 56.

Corrects me: I claimed Kelly's $24.95 was a self-selected upgrade. Not supported - her
checkout is in the post-flip population. Cody was right in substance.

Also corrects the record: shopifyFunctions is APP-SCOPED, so my earlier 'no rate-hiding
function exists' proved nothing; a function owned by another app is invisible to us.

Why the default flipped is NOT determined. Inspecting/changing the checkout default is
customer-facing money config and stays gated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Files touched

Diff

commit 07e8ba575679a1a44002964ac3d019d31fea63ba
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Thu Sep 10 09:23:12 2026 -0700

    TK-11366: ROOT CAUSE FOUND - checkout's default shipping option flipped ~2026-08-18
    
    Ran the counterfactual population my order-only cohorts could not see (Cody's original
    objection): abandoned checkouts, which record what the shopper was HANDED, not what they
    chose.
    
    US sample-only abandoned checkouts recording $0.00 shipping:
      Jun 1-Aug 17: 24/26 (92%%)
      Aug 18-Sep 10: 2/56 (4%%)
      September: 0 of 17
    Paid amounts cluster on UPS carrier rates: $23.35 x17, $23.13 x16.
    
    Decisive because /cart/shipping_rates.json still returns 'Free Shipping (No Tracking)'
    $0.00 listed FIRST. The rate is OFFERED but no longer DEFAULTED. Offered != defaulted,
    and that one distinction explains every observation at once.
    
    Measured impact: 51 orders since Aug 18 that qualified for $0.00 paid anyway
    ($1,120.10, avg $21.96); 54 of 56 abandoned checkouts carry a paid line; abandonment
    volume more than doubled 26 -> 56.
    
    Corrects me: I claimed Kelly's $24.95 was a self-selected upgrade. Not supported - her
    checkout is in the post-flip population. Cody was right in substance.
    
    Also corrects the record: shopifyFunctions is APP-SCOPED, so my earlier 'no rate-hiding
    function exists' proved nothing; a function owned by another app is invisible to us.
    
    Why the default flipped is NOT determined. Inspecting/changing the checkout default is
    customer-facing money config and stays gated.
    
    Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
 verification/tk10836/ROOT-CAUSE-FOUND.md         | 70 ++++++++++++++++++++++++
 verification/tk10836/verify-abandoned-default.js | 59 ++++++++++++++++++++
 verification/tk10836/verify-functions-recheck.js | 51 +++++++++++++++++
 3 files changed, 180 insertions(+)

diff --git a/verification/tk10836/ROOT-CAUSE-FOUND.md b/verification/tk10836/ROOT-CAUSE-FOUND.md
new file mode 100644
index 0000000..ea287d3
--- /dev/null
+++ b/verification/tk10836/ROOT-CAUSE-FOUND.md
@@ -0,0 +1,70 @@
+# TK-11366 — ROOT CAUSE FOUND: checkout's DEFAULT shipping option flipped ~2026-08-18
+
+Read-only. No login, no checkout created, no writes. $0. 2026-09-10.
+
+## The test
+An **abandoned checkout** records whichever shipping line was selected when the shopper walked away.
+Someone who abandons early never actively picks a rate — so the recorded line approximates the
+**DEFAULT**. Orders only show what people ended up with; abandoned checkouts show what they were
+*handed*. That is the population my earlier cohorts structurally could not see (Cody's point).
+
+## The result — unambiguous
+US sample-only **abandoned checkouts**, shipping = total − items − tax:
+
+| window | n | recorded $0.00 | recorded PAID |
+|---|---|---|---|
+| Jun 1 – Aug 17 | 26 | **24 (92%)** | 2 |
+| Aug 18 – Sep 10 | 56 | **2 (4%)** | **54** |
+
+| month | n | $0.00 |
+|---|---|---|
+| 2026-06 | 6 | 83% |
+| 2026-07 | 15 | 93% |
+| 2026-08 | 44 | **16%** |
+| **2026-09** | **17** | **0%** |
+
+**In September, ZERO of 17 abandoned sample checkouts carry the free rate.**
+Paid amounts cluster on the UPS carrier rates: **$23.35 ×17, $23.13 ×16**, $21.78 ×5, $24.95 ×4.
+
+## Why this is decisive
+`GET /cart/shipping_rates.json` **still returns "Free Shipping (No Tracking)" $0.00, listed FIRST**
+(measured today, Kelly's exact address). So the rate is *offered* — it is simply **no longer the
+default selection in checkout**. Offered ≠ defaulted. That single distinction explains every
+observation:
+
+- Same delivery profile, same active rates, no Admin-API-visible config diff. ✔
+- Orders: 97% free → 43% free. ✔
+- Abandoned: 92% free → 4% free. ✔
+- Single-swatch orders paying $23.13 — irrational as a *choice*, automatic as a *default*. ✔
+- Nothing correlates with Shop Pay, weight, cart size, profile, or the >10-unit rule. ✔
+
+## I WAS WRONG, AND SO WAS MY CORRECTION OF THE REVERSAL
+I argued the $24.95 on Kelly's checkout was a **self-selected upgrade**. That is **not supported**.
+Her checkout is one of the Aug-18+ population where the default is a paid rate. She most likely
+never chose it — she was *handed* it. Cody was right in substance about her; the reversal's concern
+was right; my "availability proves it's benign" framing was wrong. What survives from my work is
+narrower: the free rate *exists and is offered*. It is **not** what customers are landing on.
+
+## Impact (measured)
+- **51 orders** since Aug 18 that qualified for $0.00 shipping paid anyway — **$1,120.10 charged,
+  avg $21.96**, on samples advertised as free.
+- **54 of 56** abandoned sample checkouts since Aug 18 carry a paid shipping line.
+- **Abandonment volume more than doubled** in the same window: 26 → 56. People are bailing when a
+  "free samples" cart quotes ~$23 shipping. That is the larger, invisible cost.
+- Markup makes it worse: UPS Ground base ≈$9.25 is charged at **$23.13** (+$13.88).
+
+## What I could NOT determine
+**Why** the default flipped. `deliveryCustomizations` returns 0 for the full-access token, but note
+**`shopifyFunctions` is APP-SCOPED** — the full token sees 0 functions while the fulfillment token
+sees its own "DW Free Samples" function. So a delivery-customization function owned by *another app*
+would be invisible to me, and I cannot rule one out. Candidates, unranked and unproven:
+1. A newly installed/updated app with a delivery-customization function reordering options.
+2. Shopify's newer checkout delivery UI ordering by **delivery estimate** — the free rate is the only
+   cheap option with **no** estimate (`delivery_date: null`), UPS/FedEx carry real dates.
+3. A checkout-profile change — the published profile was **updated 2026-08-30** (inside the window,
+   though after the ~Aug-18 onset).
+
+## Next step (GATED — customer-facing shipping/checkout config)
+Look at the live checkout's Delivery options and at Settings → Shipping/Checkout for a recently
+installed app or a changed default. **The likely fix — forcing "Free Shipping (No Tracking)" to be
+the default for sample carts — is a customer-facing money change and is NOT something I will make.**
diff --git a/verification/tk10836/verify-abandoned-default.js b/verification/tk10836/verify-abandoned-default.js
new file mode 100644
index 0000000..0728d61
--- /dev/null
+++ b/verification/tk10836/verify-abandoned-default.js
@@ -0,0 +1,59 @@
+#!/usr/bin/env node
+// TK-11366 DECISIVE TEST of the preselection hypothesis.
+// An ABANDONED checkout records whatever shipping line was selected when the shopper walked away.
+// A shopper who abandons early never actively picks a rate -> the recorded line approximates the
+// DEFAULT. So: if the default flipped away from $0.00 around 2026-08-18, abandoned checkouts should
+// show the same before/after swing as orders. If the default never changed (customers are actively
+// upgrading), abandoned checkouts should stay overwhelmingly $0.00 in BOTH windows.
+// READ-ONLY.
+const fs=require('fs'),os=require('os');
+for (const line of fs.readFileSync(os.homedir()+'/Projects/secrets-manager/.env','utf8').split('\n')) {
+  const m=line.match(/^\s*([A-Z0-9_]+)\s*=\s*(.*)$/); if(!m) continue;
+  let v=m[2].trim().replace(/^['"]|['"]$/g,''); if(!process.env[m[1]]) process.env[m[1]]=v;
+}
+const STORE=process.env.SHOPIFY_STORE_DOMAIN, TOKEN=process.env.SHOPIFY_FULL_ACCESS_TOKEN, API='2024-10';
+async function gql(q,v={}){const r=await fetch(`https://${STORE}/admin/api/${API}/graphql.json`,{method:'POST',
+  headers:{'X-Shopify-Access-Token':TOKEN,'Content-Type':'application/json'},body:JSON.stringify({query:q,variables:v})});
+  const j=await r.json(); if(j.errors)console.error('ERR',JSON.stringify(j.errors).slice(0,400)); return j.data;}
+const Q=`query($c:String){ abandonedCheckouts(first:40, after:$c, query:"created_at:>2026-06-01", sortKey:CREATED_AT){
+  pageInfo{hasNextPage endCursor}
+  nodes{ id createdAt
+    subtotalPriceSet{shopMoney{amount}} totalPriceSet{shopMoney{amount}} totalTaxSet{shopMoney{amount}}
+    shippingAddress{ countryCodeV2 }
+    lineItems(first:20){ nodes{ title quantity variant{ sku } } } } } }`;
+(async()=>{
+  let c=null, rows=[];
+  do{ const d=await gql(Q,{c}); if(!d) break;
+    for(const a of d.abandonedCheckouts.nodes){
+      const lis=a.lineItems.nodes; if(!lis.length) continue;
+      if(!lis.every(li=>/(^|-)sample/i.test(li.variant?.sku||'')||/sample|memo|swatch/i.test(li.title||''))) continue;
+      if(a.shippingAddress?.countryCodeV2!=='US') continue;
+      const items=Number(a.subtotalPriceSet?.shopMoney?.amount||0);
+      const total=Number(a.totalPriceSet?.shopMoney?.amount||0);
+      const tax=Number(a.totalTaxSet?.shopMoney?.amount||0);
+      const ship=+(total-items-tax).toFixed(2);
+      rows.push({date:a.createdAt.slice(0,10), items, ship, units:lis.reduce((s,l)=>s+l.quantity,0)});
+    } c=d.abandonedCheckouts.pageInfo.hasNextPage?d.abandonedCheckouts.pageInfo.endCursor:null; }while(c);
+  console.log(`US sample-only ABANDONED checkouts since 2026-06-01: n=${rows.length}\n`);
+  const bucket=(lo,hi)=>{
+    const s=rows.filter(r=>r.date>=lo&&r.date<=hi);
+    const withShip=s.filter(r=>r.ship>0.001);
+    return {n:s.length, zero:s.length-withShip.length, paid:withShip.length,
+            pct: s.length? (100*(s.length-withShip.length)/s.length).toFixed(0):'-'};
+  };
+  console.log('### shipping line recorded on ABANDONED sample checkouts (proxy for the DEFAULT) ###');
+  for(const [lab,lo,hi] of [['Jun 1 - Aug 17','2026-06-01','2026-08-17'],['Aug 18 - Sep 10','2026-08-18','2026-09-30']]){
+    const b=bucket(lo,hi);
+    console.log(`  ${lab}: n=${b.n}  ship\$0.00=${b.zero} (${b.pct}%)  ship>\$0=${b.paid}`);
+  }
+  console.log('\n### monthly ###');
+  const months=[...new Set(rows.map(r=>r.date.slice(0,7)))].sort();
+  for(const m of months){
+    const s=rows.filter(r=>r.date.startsWith(m));
+    const z=s.filter(r=>r.ship<=0.001).length;
+    console.log(`  ${m}: n=${String(s.length).padStart(3)}  \$0.00=${String(z).padStart(3)} (${s.length?(100*z/s.length).toFixed(0):0}%)`);
+  }
+  const amts={}; for(const r of rows.filter(r=>r.ship>0.001&&r.date>='2026-08-18')) amts[r.ship]=(amts[r.ship]||0)+1;
+  console.log('\n  paid shipping amounts on abandoned checkouts since Aug 18:', JSON.stringify(amts));
+  fs.writeFileSync('verification/tk10836/abandoned-default.json',JSON.stringify(rows,null,2));
+})();
diff --git a/verification/tk10836/verify-functions-recheck.js b/verification/tk10836/verify-functions-recheck.js
new file mode 100644
index 0000000..c5bf5ce
--- /dev/null
+++ b/verification/tk10836/verify-functions-recheck.js
@@ -0,0 +1,51 @@
+#!/usr/bin/env node
+// TK-11366: my earlier shopifyFunctions/deliveryCustomizations query returned EMPTY, but a
+// "DW Free Samples" discount FUNCTION demonstrably exists (TK-11361). So that empty result may
+// have been a silent failure -> my "no rate-hiding customization" conclusion is UNVERIFIED.
+// Re-check with a positive control: if I cannot see the discount function I KNOW exists,
+// I cannot trust an empty deliveryCustomizations result either.
+const fs=require('fs'),os=require('os');
+for (const line of fs.readFileSync(os.homedir()+'/Projects/secrets-manager/.env','utf8').split('\n')) {
+  const m=line.match(/^\s*([A-Z0-9_]+)\s*=\s*(.*)$/); if(!m) continue;
+  let v=m[2].trim().replace(/^['"]|['"]$/g,''); if(!process.env[m[1]]) process.env[m[1]]=v;
+}
+const STORE=process.env.SHOPIFY_STORE_DOMAIN, API='2024-10';
+async function gql(q,token,v={}){
+  const r=await fetch(`https://${STORE}/admin/api/${API}/graphql.json`,{method:'POST',
+    headers:{'X-Shopify-Access-Token':token,'Content-Type':'application/json'},body:JSON.stringify({query:q,variables:v})});
+  const j=await r.json();
+  return {data:j.data, errors:j.errors};
+}
+const TOKENS={FULL:process.env.SHOPIFY_FULL_ACCESS_TOKEN, FULFILLMENT:process.env.SHOPIFY_FULFILLMENT_TOKEN};
+(async()=>{
+ for(const [name,tok] of Object.entries(TOKENS)){
+  if(!tok) continue;
+  console.log(`\n================ TOKEN ${name} (…${tok.slice(-4)}) ================`);
+
+  // POSITIVE CONTROL: automatic app discounts (we KNOW >=1 function-backed discount exists)
+  let r=await gql(`query{ automaticDiscountNodes(first:25){ nodes{ id automaticDiscount{
+      __typename
+      ... on DiscountAutomaticApp{ title status appDiscountType{ functionId title } }
+      ... on DiscountAutomaticBasic{ title status } } } } }`,tok);
+  if(r.errors) console.log('  automaticDiscountNodes ERRORS:', JSON.stringify(r.errors).slice(0,300));
+  const nodes=r.data?.automaticDiscountNodes?.nodes||[];
+  console.log(`  CONTROL automaticDiscountNodes: ${nodes.length} node(s)`);
+  for(const n of nodes){
+    const a=n.automaticDiscount;
+    console.log(`    - [${a.__typename}] "${a.title}" status=${a.status} fn=${a.appDiscountType?.functionId||'-'} id=${n.id.split('/').pop()}`);
+  }
+
+  for(const [label,q] of [
+    ['deliveryCustomizations',`query{ deliveryCustomizations(first:25){ nodes{ id title enabled functionId } } }`],
+    ['shopifyFunctions',`query{ shopifyFunctions(first:50){ nodes{ id title apiType app{ title } } } }`],
+    ['cartTransforms',`query{ cartTransforms(first:10){ nodes{ id functionId } } }`],
+  ]){
+    r=await gql(q,tok);
+    if(r.errors){ console.log(`  ${label}: ERROR -> ${JSON.stringify(r.errors).slice(0,220)}`); continue; }
+    const key=Object.keys(r.data||{})[0];
+    const ns=r.data?.[key]?.nodes||[];
+    console.log(`  ${label}: ${ns.length} node(s)`);
+    for(const n of ns) console.log(`    - ${JSON.stringify(n).slice(0,200)}`);
+  }
+ }
+})();

← d0135eb auto-data-snapshot: 2026-09-10T09:22:46 (1 data files) — ver  ·  back to Dw Signup Fulfillment  ·  TK-11366: DTD verdict - A-prime (close the $45 gap first, th 52e57ab →