gateway: prefer staging catalog over tiny live-shopify view when it dwarfs it (>=3x)

99403537111aa34ed357efab7dbb15e003da5ca7 · 2026-07-23 15:44:23 -0700 · Steve Abrams

DTD verdict B (2026-07-23). A vendor with >=1 non-archived shopify row was always
routed to the all.dw live view, shadowing a far larger <slug>_catalog staging table
(~12k rows stranded: malibu 3 live vs 3,718 staged; dedar 1 vs 436; sancar 42 vs
3,418). The internal viewer's job is to browse the FULL scraped line, so when the
catalog has >=3x the non-archived-live count, show it instead. 15 vendors flip (all
>=3.3x, verified 100% image coverage); the 3x guard leaves near-tie mainline vendors
and vendors whose staging is smaller/staler than live (fentucci, phillipe-romano,
lee-jofa, thibaut) on the live view. resolveTable now caches an exact row count.

Files touched

M server.js

Diff

commit 99403537111aa34ed357efab7dbb15e003da5ca7
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Thu Jul 23 15:44:23 2026 -0700

    gateway: prefer staging catalog over tiny live-shopify view when it dwarfs it (>=3x)
    
    DTD verdict B (2026-07-23). A vendor with >=1 non-archived shopify row was always
    routed to the all.dw live view, shadowing a far larger <slug>_catalog staging table
    (~12k rows stranded: malibu 3 live vs 3,718 staged; dedar 1 vs 436; sancar 42 vs
    3,418). The internal viewer's job is to browse the FULL scraped line, so when the
    catalog has >=3x the non-archived-live count, show it instead. 15 vendors flip (all
    >=3.3x, verified 100% image coverage); the 3x guard leaves near-tie mainline vendors
    and vendors whose staging is smaller/staler than live (fentucci, phillipe-romano,
    lee-jofa, thibaut) on the live view. resolveTable now caches an exact row count.
---
 server.js | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/server.js b/server.js
index 6935b00..6c5de09 100644
--- a/server.js
+++ b/server.js
@@ -90,7 +90,12 @@ async function resolveTable(slug) {
     for (const [role, cands] of Object.entries(CANDIDATES)) {
       cols[role] = cands.find(c => names.includes(c)) || null;
     }
-    info = { table, cols };
+    // Exact staging row count — used by the dispatcher to decide whether this
+    // catalog dwarfs the vendor's live-shopify subset (see the 3x rule below).
+    // table is regex-validated ^[a-z0-9_]+$ above, so safe to interpolate; count
+    // is cached with `info` for the process lifetime (same staleness as cols).
+    const cnt = await pool.query(`select count(*)::int as n from "${table}"`);
+    info = { table, cols, rows: cnt.rows[0].n };
   }
   tableCache.set(slug, info);
   return info;
@@ -389,11 +394,22 @@ const server = http.createServer(async (req, res) => {
     const vinfo = VENDOR_MAP.get(slug);
     const info = await resolveTable(slug);
 
-    // 2. Vendor with LIVE shopify rows → all.dw schema/UI, pre-filtered. The standard.
-    if (vinfo && vinfo.live > 0) return proxyAllDw(req, res, slug, vinfo.names, vinfo.live);
+    // A staging catalog "dwarfs" the live-shopify subset when it has >=3x the rows.
+    // That's the signal only a fraction of a scraped line was onboarded, so the
+    // internal curation viewer should show the full staging table, not the tiny live
+    // slice (e.g. malibu 3 live vs 3,718 staged). The 3x guard (DTD verdict B,
+    // 2026-07-23) keeps mainline vendors whose live set IS the real catalog on the
+    // live view (kravet 1.7x, thibaut 1.2x) and never flips a vendor whose staging
+    // table is smaller/staler than live (fentucci 118 vs 3,479; phillipe_romano
+    // 3,503 vs 24,173) — those would regress under a naive staging-first rule.
+    const catalogDwarfsLive = !!info && info.rows >= 3 * ((vinfo && vinfo.live) || 0);
+
+    // 2. Vendor with LIVE shopify rows → all.dw schema/UI, pre-filtered. The standard —
+    //    UNLESS a staging catalog dwarfs it, in which case fall through to that catalog.
+    if (vinfo && vinfo.live > 0 && !catalogDwarfsLive) return proxyAllDw(req, res, slug, vinfo.names, vinfo.live);
 
     // 3. A *_catalog table (scraped/live lines, incl. vendors whose shopify line is
-    //    fully archived like Arte) → the generic dw_unified catalog viewer.
+    //    fully archived like Arte, OR whose staging dwarfs live) → generic catalog viewer.
     if (info) {
       // fall through to the generic viewer below
     } else if (vinfo && vinfo.total > 0) {

← 5ac5875 gateway: alias lilycolor.internal.dw → japan-viewer pre-filt · back to Dw Internal Gateway · Add 'Re-check all sites' control + /api/status health probe 1061465 →