[object Object]

← back to Designer Wallcoverings

Sangetsu: sitemap discovery (879 wallpaper pattern pages, regional .co.th seed) + dossier session progress

5ac4e2095ce0406cfb744d9969e7829fbd2b07b4 · 2026-06-29 15:39:50 -0700 · steve@designerwallcoverings.com

Files touched

Diff

commit 5ac4e2095ce0406cfb744d9969e7829fbd2b07b4
Author: steve@designerwallcoverings.com <steve@designerwallcoverings.com>
Date:   Mon Jun 29 15:39:50 2026 -0700

    Sangetsu: sitemap discovery (879 wallpaper pattern pages, regional .co.th seed) + dossier session progress
---
 onboarding/sangetsu-lilycolor/.gitignore           |  2 +-
 onboarding/sangetsu-lilycolor/DOSSIER.md           | 21 ++++++++
 .../sangetsu-sitemap-scraper.cjs                   | 58 ++++++++++++++++++++++
 3 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/onboarding/sangetsu-lilycolor/.gitignore b/onboarding/sangetsu-lilycolor/.gitignore
index 6065974d..4a9d0dbd 100644
--- a/onboarding/sangetsu-lilycolor/.gitignore
+++ b/onboarding/sangetsu-lilycolor/.gitignore
@@ -1 +1 @@
-staging/*.jsonl
+staging/
diff --git a/onboarding/sangetsu-lilycolor/DOSSIER.md b/onboarding/sangetsu-lilycolor/DOSSIER.md
index d4f2606e..a54c1388 100644
--- a/onboarding/sangetsu-lilycolor/DOSSIER.md
+++ b/onboarding/sangetsu-lilycolor/DOSSIER.md
@@ -87,3 +87,24 @@ Sangetsu second.
 - Assigning real `vendor_registry` rows + DW prefixes.
 - Any cost confirmation / pricing.
 - Any Shopify create / publish (stays "Staged for New" → draft → active only on approval).
+
+## 6. Session progress (2026-06-29, post-approval — Steve said "1 then 3")
+
+**Lilycolor (built):** `lilycolor-feed-scraper.cjs` staged **3,107 products** offline →
+`staging/lilycolor-staging.jsonl` (gitignored). Normalized: mfr_sku, prefix (LV 1003 / LL 882 /
+LW 628 / LYT 167 / LH 114 / XR 90 / LB 69 / …), width, composition, fire grade (F☆☆☆☆ / 準不燃),
+images, product_type. Nothing online.
+- **Nuance found:** Lilycolor splits SAMPLE and MAIN into SEPARATE Shopify products (handle
+  `…___sample`, product_type 壁紙サンプル) — sample↔main is a **product-level join**, not a variant.
+  Handle this at enrichment when building DW's sample+product variant pair.
+- **Next (gated):** JA→EN translate pass, dedup by prefix vs dw_sku_registry+shopify_products, cost.
+
+**Sangetsu (discovery solved):** `sangetsu-sitemap-scraper.cjs discover` enumerates the catalog from
+`sangetsu-goodrich.co.th/wp-sitemap-posts-product-1.xml` → **879 pattern pages** (filtered to
+wallpaper-wallcovering). US site blocks sitemap/wp-json; the **regional .co.th sitemap is the seed**
+(shared global catalog). Each page = a pattern with N colorway SKUs (explodes to thousands of SKUs).
+- **Next (gated):** implement `parsePage()` (H1 pattern + `.product_desc` overview + `<select>` colorway
+  SKUs); rendered fallback (dev-browser/Browserbase) only if SKUs are JS-injected.
+
+**Customer:** Andrew Glick reply DRAFTED via George (`info@` account, draftId r-7044…) with top-3
+Sanshin/Virtuoso/Savannah — awaiting Steve review/send.
diff --git a/onboarding/sangetsu-lilycolor/sangetsu-sitemap-scraper.cjs b/onboarding/sangetsu-lilycolor/sangetsu-sitemap-scraper.cjs
new file mode 100644
index 00000000..db8f71a9
--- /dev/null
+++ b/onboarding/sangetsu-lilycolor/sangetsu-sitemap-scraper.cjs
@@ -0,0 +1,58 @@
+#!/usr/bin/env node
+/**
+ * Sangetsu (Goodrich) scraper — DISCOVERY + per-page parse scaffold (OFFLINE/STAGING).
+ *
+ * Discovery (WORKS): the US site blocks wp-json/sitemap, but the regional .co.th site
+ * exposes wp-sitemap-posts-product-1.xml (~879 pattern pages). The Sangetsu/Goodrich
+ * catalog is shared across regions, so this is a valid seed for the global product set.
+ *
+ * Per-page parse (STUB): each "product" page is a PATTERN holding N colorway SKUs in a
+ * <select> ("Choose an option AZ52790 ..."). The overview/spec text is in the static HTML.
+ * If the colorway SKUs turn out to be JS-injected, fall back to dev-browser/Browserbase.
+ *
+ * HARD: writes ONLY to local staging. No Shopify, no dw_unified, no publish.
+ * Usage: node sangetsu-sitemap-scraper.cjs discover    # enumerate + filter wallpaper URLs
+ */
+const https = require('https');
+const fs = require('fs');
+const path = require('path');
+
+const SITEMAP = 'https://www.sangetsu-goodrich.co.th/wp-sitemap-posts-product-1.xml';
+const OUT_DIR = path.join(__dirname, 'staging');
+const URLS_OUT = path.join(OUT_DIR, 'sangetsu-wallpaper-urls.txt');
+
+function get(url) {
+  return new Promise((resolve, reject) => {
+    https.get(url, { headers: { 'User-Agent': 'Mozilla/5.0 (DW-onboarding-recon)' } }, (res) => {
+      let body = '';
+      res.on('data', (c) => (body += c));
+      res.on('end', () => resolve(body));
+    }).on('error', reject);
+  });
+}
+
+async function discover() {
+  const xml = await get(SITEMAP);
+  const all = [...xml.matchAll(/<loc>([^<]+)<\/loc>/g)].map((m) => m[1]);
+  // keep only wallpaper/wallcovering pattern pages (drop carpet / flooring / etc.)
+  const wp = all.filter((u) => /\/products\/wallpaper-wallcovering\//.test(u));
+  if (!fs.existsSync(OUT_DIR)) fs.mkdirSync(OUT_DIR, { recursive: true });
+  fs.writeFileSync(URLS_OUT, wp.join('\n') + '\n');
+  const grass = wp.filter((u) => /grass|sanshin|virtuoso|savannah|sass-a-grass|woven/i.test(u));
+  console.log(JSON.stringify({
+    totalProductPages: all.length,
+    wallpaperPages: wp.length,
+    out: URLS_OUT,
+    grasscloth_line: grass,
+  }, null, 2));
+}
+
+// TODO (gated build step): parsePage(url) -> { pattern, overview, specs, colorways:[{sku,name,img}] }
+//   selectors observed: H1 = pattern name; ".product_desc" = overview; <select> option values = colorway SKUs.
+//   Each colorway SKU -> one DW staging record (dedup vs dw_sku_registry by SKU; settlement-gate naturals).
+
+(async () => {
+  const cmd = process.argv[2] || 'discover';
+  if (cmd === 'discover') return discover();
+  console.error('unknown command:', cmd); process.exit(1);
+})().catch((e) => { console.error('FAILED:', e.message); process.exit(1); });

← 110f255d Lilycolor: feed-first scraper staged 3,107 products offline  ·  back to Designer Wallcoverings  ·  appointments: GO-LIVE — theme 5.9.0 published, page live + i 3206484f →