Repurpose dw-pricing-resume loop → Schumacher API cadence (browser-free, 12h)

0ed8df7d58880651af602ea9989660f7ad76cad5 · 2026-06-11 11:18:56 -0700 · SteveStudio2

Loop now runs the authenticated API recrawl instead of the dead Browserbase scrape:
(1) --all fills new-arrival prices + flags vendor-404 discontinued, (2) --refresh 400
rolling-refreshes the oldest-fetched rows (keeps prices current, re-detects newly-disco).
Added --refresh N mode + a token-expiry guard that skips the tick loudly when the JWT
dies. Catalog-only writes; Shopify archiving stays human-gated (not in the loop).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Files touched

M shopify/scripts/dw-pricing-resume.sh
M shopify/scripts/schu-api-recrawl.js

Diff

commit 0ed8df7d58880651af602ea9989660f7ad76cad5
Author: SteveStudio2 <stevestudio2@SteveStudio2s-Mac-Studio.local>
Date:   Thu Jun 11 11:18:56 2026 -0700

    Repurpose dw-pricing-resume loop → Schumacher API cadence (browser-free, 12h)
    
    Loop now runs the authenticated API recrawl instead of the dead Browserbase scrape:
    (1) --all fills new-arrival prices + flags vendor-404 discontinued, (2) --refresh 400
    rolling-refreshes the oldest-fetched rows (keeps prices current, re-detects newly-disco).
    Added --refresh N mode + a token-expiry guard that skips the tick loudly when the JWT
    dies. Catalog-only writes; Shopify archiving stays human-gated (not in the loop).
    
    Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
 shopify/scripts/dw-pricing-resume.sh | 29 ++++++++++++++++++++---------
 shopify/scripts/schu-api-recrawl.js  |  8 ++++++++
 2 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/shopify/scripts/dw-pricing-resume.sh b/shopify/scripts/dw-pricing-resume.sh
index 2b820408..a31ca57f 100755
--- a/shopify/scripts/dw-pricing-resume.sh
+++ b/shopify/scripts/dw-pricing-resume.sh
@@ -1,7 +1,14 @@
 #!/usr/bin/env bash
-# DW pricing auto-resume — runs INDEPENDENT of any Claude Code session.
-# Loaded by launchd (com.steve.dw-pricing-resume): RunAtLoad + every 30 min.
+# DW Schumacher price cadence — runs INDEPENDENT of any Claude Code session.
+# Loaded by launchd (com.steve.dw-pricing-resume): RunAtLoad + daily.
 # Survives reboot/logout-of-Claude. Single-flight lock so ticks never overlap.
+#
+# Uses the authenticated Schumacher product API (browser-free) via the trade-session
+# token in .schu-token (~92d). Each tick: (1) fetch any NEW unpriced wallpaper SKUs
+# (auto-catch new arrivals), (2) rolling-refresh the 400 oldest-fetched rows so prices
+# stay current. Catalog writes only — NO Shopify push, no customer-facing writes.
+# When the token expires the recrawl exits non-zero (AUTH_401) and this logs it for a
+# manual browser token refresh (see ~/.claude/skills/browserbase/SCHUMACHER-NOTES.md).
 set -u
 export PATH=/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin
 DIR="$HOME/Projects/Designer-Wallcoverings/shopify/scripts"
@@ -14,12 +21,16 @@ if ! mkdir "$LOCK" 2>/dev/null; then echo "$(date '+%F %T') still running — sk
 trap 'rmdir "$LOCK" 2>/dev/null' EXIT
 
 cd "$DIR" || exit 1
-echo "$(date '+%F %T') ===== resume tick =====" >> "$LOG"
+echo "$(date '+%F %T') ===== schumacher cadence tick =====" >> "$LOG"
 
-# Schumacher cost via Browserbase (cloud browser — does NOT need local Chrome).
-# Writes only schumacher_catalog.trade_price for rows missing price; no product
-# creation, no customer-facing writes. Small batch to bound Browserbase cost.
-# NOTE: login submit still needs a fix (browserbase/SCHUMACHER-NOTES.md) — until
-# then this logs a failed login and exits cheaply; it starts producing once fixed.
-node schu-browserbase-scrape.js --limit 40 --write >> "$LOG" 2>&1
+# token expiry guard — bail early + loudly if the JWT is dead so we don't spin
+if ! node -e 'const fs=require("fs"),p=require("path");const t=fs.readFileSync(p.join("'"$DIR"'",".schu-token"),"utf8").trim();const e=JSON.parse(Buffer.from(t.split(".")[1],"base64").toString()).exp;process.exit(e>Date.now()/1000?0:1);' 2>/dev/null; then
+  echo "$(date '+%F %T') TOKEN EXPIRED — refresh .schu-token from a logged-in browser (skipping tick)" >> "$LOG"
+  exit 0
+fi
+
+# 1) new arrivals (fills NULL prices, marks vendor-404 discontinued)
+node schu-api-recrawl.js --all --commit >> "$LOG" 2>&1
+# 2) rolling refresh of the 400 oldest-fetched live rows (keeps prices fresh)
+node schu-api-recrawl.js --refresh 400 --commit >> "$LOG" 2>&1
 echo "$(date '+%F %T') tick done" >> "$LOG"
diff --git a/shopify/scripts/schu-api-recrawl.js b/shopify/scripts/schu-api-recrawl.js
index 91ab7f2e..5582f136 100644
--- a/shopify/scripts/schu-api-recrawl.js
+++ b/shopify/scripts/schu-api-recrawl.js
@@ -26,6 +26,10 @@ const COMMIT = args.includes('--commit');
 const ALL = args.includes('--all');
 const lIdx = args.indexOf('--limit');
 const LIMIT = TEST ? 3 : (lIdx >= 0 ? parseInt(args[lIdx + 1], 10) : (ALL ? 100000 : 25));
+// --refresh N: re-fetch the N already-priced rows with the OLDEST price_fetched_at
+// (rolling price refresh for the cadence loop), instead of only filling NULL prices.
+const rfIdx = args.indexOf('--refresh');
+const REFRESH = rfIdx >= 0 ? (parseInt(args[rfIdx + 1], 10) || 300) : 0;
 const sleep = ms => new Promise(r => setTimeout(r, ms));
 
 function psql(sql) { return execFileSync('psql', ['-At', '-F', '\t', '-d', 'dw_unified', '-c', sql], { encoding: 'utf8', maxBuffer: 1 << 28 }).trim(); }
@@ -95,6 +99,10 @@ function parse(j) {
   let skus;
   if (TEST) {
     skus = ['R11358658', ...psql(`SELECT mfr_sku FROM schumacher_catalog WHERE product_type IN ('Wallcovering','Commercial Wallcovering') AND retail_price IS NULL ORDER BY mfr_sku LIMIT 2;`).split('\n').filter(Boolean)];
+  } else if (REFRESH) {
+    // rolling refresh: re-price the N oldest-fetched live rows (keeps prices current)
+    skus = psql(`SELECT mfr_sku FROM schumacher_catalog WHERE product_type IN ('Wallcovering','Commercial Wallcovering') AND retail_price IS NOT NULL AND COALESCE(excluded,false)=false AND mfr_sku IS NOT NULL AND mfr_sku<>'' ORDER BY price_fetched_at ASC NULLS FIRST LIMIT ${REFRESH};`).split('\n').filter(Boolean);
+    console.log(`REFRESH mode: re-pricing ${skus.length} oldest-fetched rows`);
   } else {
     // resume-safe: only rows still missing price AND not already marked discontinued/excluded
     skus = psql(`SELECT mfr_sku FROM schumacher_catalog WHERE product_type IN ('Wallcovering','Commercial Wallcovering') AND retail_price IS NULL AND COALESCE(excluded,false)=false AND mfr_sku IS NOT NULL AND mfr_sku<>'' ORDER BY mfr_sku LIMIT ${LIMIT};`).split('\n').filter(Boolean);

← 47bef875 price-coverage: add Shopify roll-price outcome layer (correc · back to Designer Wallcoverings · romo roll pricing: RFC-4180 CSV parser (quoted fields w/ com 8eb73344 →