← back to Sanderson Onboard
TK-11256: fix gallery-image cross-contamination in Sanderson extractor
4eb1f7dd57dff41a322dad01012424240c234fa1 · 2026-09-04 11:46:47 -0700 · Steve Abrams
Root cause of dw-image-identity-canary FAIL (Sanderson foreign_products
6 -> 24, foreign_imgs 6 -> 41): the gallery extractor swept EVERY
static/media/catalog/product/*.jpg URL anywhere on the raw PDP HTML
(IMG.findall(h) over the whole page) into that product's gallery_images.
Sanderson's PDP always embeds a 'shop other colourways' swatch carousel
(other SKUs' hero images at 64w) plus cross-sell 'shop the room' panels
(other products' images at 1080w) on every product page, so those foreign
images got written into the wrong SKU's gallery on the next re-scrape.
Verified live against 3 real PDPs (SAW0094-01, SAW0094-03, SAW0224-01,
SAW0079-02): the product's OWN real gallery/detail shots consistently
render at srcset max-width 2560 (often with the SKU literally embedded in
the filename, e.g. ..._SAW0224_01_SANDERSON_Carpet_Garden_WALLPAPER...),
while every foreign image tops out at 1080w or 64w -- a clean, reproducible
separation. New own_gallery() scopes extraction to >=1500w srcset entries
only; falls back to the single primary image (never the old whole-page
sweep) if no such image is found on a PDP shape variant.
Code-only -- no scrape run, no dw_unified/Shopify writes. The 24 already-
live contaminated products need a gallery-image correction on Shopify --
gated, see pending-approval memo.
Files touched
Diff
commit 4eb1f7dd57dff41a322dad01012424240c234fa1
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Fri Sep 4 11:46:47 2026 -0700
TK-11256: fix gallery-image cross-contamination in Sanderson extractor
Root cause of dw-image-identity-canary FAIL (Sanderson foreign_products
6 -> 24, foreign_imgs 6 -> 41): the gallery extractor swept EVERY
static/media/catalog/product/*.jpg URL anywhere on the raw PDP HTML
(IMG.findall(h) over the whole page) into that product's gallery_images.
Sanderson's PDP always embeds a 'shop other colourways' swatch carousel
(other SKUs' hero images at 64w) plus cross-sell 'shop the room' panels
(other products' images at 1080w) on every product page, so those foreign
images got written into the wrong SKU's gallery on the next re-scrape.
Verified live against 3 real PDPs (SAW0094-01, SAW0094-03, SAW0224-01,
SAW0079-02): the product's OWN real gallery/detail shots consistently
render at srcset max-width 2560 (often with the SKU literally embedded in
the filename, e.g. ..._SAW0224_01_SANDERSON_Carpet_Garden_WALLPAPER...),
while every foreign image tops out at 1080w or 64w -- a clean, reproducible
separation. New own_gallery() scopes extraction to >=1500w srcset entries
only; falls back to the single primary image (never the old whole-page
sweep) if no such image is found on a PDP shape variant.
Code-only -- no scrape run, no dw_unified/Shopify writes. The 24 already-
live contaminated products need a gallery-image correction on Shopify --
gated, see pending-approval memo.
---
scripts/extract.py | 42 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/scripts/extract.py b/scripts/extract.py
index 685f4d0..61f75af 100644
--- a/scripts/extract.py
+++ b/scripts/extract.py
@@ -59,6 +59,39 @@ rows = []
JSONLD = re.compile(r'<script[^>]*application/ld\+json[^>]*>(.*?)</script>', re.S)
IMG = re.compile(r'(static/media/catalog/product/[A-Za-z0-9/_.\-]+\.jpg)')
+# FIX (TK-11256, 2026-09-04): the old gallery extraction (IMG.findall(h) over the WHOLE
+# page) swept up EVERY image referenced anywhere on the PDP -- including the "shop other
+# colourways" swatch carousel and cross-sell "shop the room" panels, which show OTHER
+# products' images at real (non-thumbnail) resolution. That's the dw-image-identity-canary
+# foreign_colorway/foreign_pattern contamination (Sanderson foreign_products 6->24).
+# Verified live (2026-09-04): Sanderson's own gallery/detail shots consistently render at
+# srcset max-width 2560, while "other colourway" swatches cap at 64w and cross-sell/
+# "shop the room" images cap at 1080w -- a clean, reproducible separation on every PDP
+# checked (SAW0094-01/-03, SAW0224-01). Scope gallery_images to that >=1500w band only.
+GALLERY_MIN_W = 1500
+SRCSET = re.compile(r'srcset="([^"]*static/media/catalog/product/[A-Za-z0-9/_.\-]+\.jpg[^"]*)"')
+FNAME = re.compile(r'([A-Za-z0-9_]+\.jpg)')
+PATHFNAME = re.compile(r'(static/media/catalog/product/[A-Za-z0-9/]+/([A-Za-z0-9_]+\.jpg))')
+
+def own_gallery(h):
+ best_w = {}
+ for m in SRCSET.finditer(h):
+ srcset = m.group(1)
+ widths = [int(w) for w in re.findall(r'(\d+)w', srcset)]
+ if not widths:
+ continue
+ w = max(widths)
+ for fm in FNAME.finditer(srcset):
+ fname = fm.group(1)
+ best_w[fname] = max(best_w.get(fname, 0), w)
+ keep = {f for f, w in best_w.items() if w >= GALLERY_MIN_W}
+ if not keep:
+ return []
+ fname_to_path = {}
+ for pm in PATHFNAME.finditer(h):
+ fname_to_path.setdefault(pm.group(2), pm.group(1))
+ return sorted(fname_to_path[f] for f in keep if f in fname_to_path)
+
for i, path in enumerate(urls):
url = BASE + path
prod = None
@@ -88,7 +121,14 @@ for i, path in enumerate(urls):
image = imgs[0] if imgs else None
offers = prod.get("offers") or {}
price = offers.get("price"); cur = offers.get("priceCurrency")
- gallery = sorted(set(BASE + "/" + m for m in IMG.findall(h)))
+ gallery_paths = own_gallery(h)
+ if gallery_paths:
+ gallery = sorted(set(BASE + "/" + m for m in gallery_paths))
+ else:
+ # fallback: no >=1500w own-gallery images found on this PDP shape -- better to
+ # carry only the product's own primary image than to fall back to the old
+ # whole-page sweep (which is exactly the contamination bug this fixes).
+ gallery = [image] if image else []
settle = "review" if path in settlement else "clear"
rows.append({
"mfr_sku": sku, "pattern_name": pattern, "color_name": color,
← a721805 auto-data-snapshot: 2026-09-04T05:15:13 (5 data files) — pil
·
back to Sanderson Onboard
·
auto-data-snapshot: 2026-09-05T04:56:24 (4 data files) — pil d3f540c →