add fentucci-scraper-manager skill (in-repo copy; live at ~/.claude/skills/) — canonical DWFN scheme, refresh runbook, scrape gotchas, hard gates

ba65537e943af31c462352eba7bcf9218f3d8a07 · 2026-07-22 18:14:52 -0700 · Steve Abrams
Files touched

A docs/fentucci-scraper-manager-SKILL.md
Diff

commit ba65537e943af31c462352eba7bcf9218f3d8a07
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Wed Jul 22 18:14:52 2026 -0700

    add fentucci-scraper-manager skill (in-repo copy; live at ~/.claude/skills/) — canonical DWFN scheme, refresh runbook, scrape gotchas, hard gates
---
 docs/fentucci-scraper-manager-SKILL.md | 87 ++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

diff --git a/docs/fentucci-scraper-manager-SKILL.md b/docs/fentucci-scraper-manager-SKILL.md
new file mode 100644
index 0000000..032a7d3
--- /dev/null
+++ b/docs/fentucci-scraper-manager-SKILL.md
@@ -0,0 +1,87 @@
+---
+name: fentucci-scraper-manager
+description: Fentucci Naturals (Tokiwa USA private label) import manager — use when scraping/refreshing tokiwausa.com, refreshing tokiwa_catalog, or operating the Fentucci Naturals Tokiwa line; real name Tokiwa NEVER customer-facing; short alias /fentucci exists.
+---
+
+# Fentucci Naturals — Tokiwa USA private label
+
+**Project:** `~/Projects/fentucci-naturals` (git). **Staging table:** local `dw_unified.tokiwa_catalog`
+(`psql "host=/tmp dbname=dw_unified"`), 504 rows / 64 patterns as of 2026-07-22.
+**Internal viewer:** http://127.0.0.1:9981 (pm2 `fentucci-viewer`, basic-auth admin/DW2024!,
+unauth `/healthz`→200, images served locally from `./images/`).
+
+## HARD GATES (never violate)
+
+- **"Tokiwa" / "Tokiwa USA" is NEVER customer-facing** — no title, description, tag,
+  collection, alt text, image filename, or URL on any public surface. The internal
+  basic-auth viewer MAY show mfr_sku (internal-line-viewer precedent) — that's the
+  only place the real line shows.
+- **Word "Wallpaper" banned** — always "Wallcovering(s)".
+- **PostgreSQL first, then Shopify.** tokiwa_catalog is the staging source of truth.
+- **Shopify import is Steve-gated** — memo at
+  `~/.claude/yolo-queue/pending-approval/fentucci-tokiwa-shopify-import.md`.
+  `scripts/create-drafts.py` exists but must NOT run without approval. Drafts only,
+  day-capped (1k variants/day store rule), resumable.
+- **NEVER "Unknown" in a title**; fall back mfr SKU → color → skip.
+- **Settlement screen before any publish** (already run at enrich; 0 review flags —
+  but re-screen on any re-scrape; see metal-leaf gotcha below).
+
+## Canonical scheme
+
+- **DW SKU:** `DWFN-100001+` (sequential, one per colorway; UNIQUE on both dw_sku and mfr_sku).
+- **One product per colorway** — pattern (e.g. Portofino) × color (e.g. Flax) = one product.
+- **Vendor string:** exactly `Fentucci Naturals`.
+- **Pattern names:** Italian coastal towns (Portofino, Ischia, Amalfi, …) — mapped
+  per mfr pattern_series in `data/rename-map.json`; FULL rename, no Tokiwa series
+  names survive anywhere public.
+- **Pricing:** QUOTE-ONLY (no cost list yet) + `{DW_SKU}-Sample` variant at $4.25,
+  no inventory tracking. Tags carry `Needs-Price` / `Needs-Width` until specs land.
+- **Images: ALWAYS LOCAL** — `images/<mfr_sku>.jpg` in the repo (504/504 coverage),
+  gitignored; never hotlink tokiwausa.com on any public surface.
+
+## Refresh runbook (in order)
+
+```sh
+cd ~/Projects/fentucci-naturals
+node scripts/scrape.js          # tokiwausa.com category pages → data/raw + products.json
+node scripts/normalize.js       # SKU/series/color parse, rename-map, flags → normalize-flags.json
+node scripts/download-images.js # per-colorway images → images/ (then node scripts/image-audit.js)
+node scripts/stage-db.js        # UPSERT into dw_unified.tokiwa_catalog (INSERT/UPSERT only)
+python3 scripts/enrich.py       # Mac1 qwen3:14b per-PATTERN copy + colorway/tags/specs + settlement screen ($0 local)
+python3 scripts/build-drafts.py # draft payloads → data/shopify-drafts.jsonl (NO push)
+# create-drafts.py = the GATED Shopify push — Steve approval required
+```
+
+Viewer picks up DB changes automatically (15-min reload) or `pm2 restart fentucci-viewer`.
+
+## Gotchas (learned the hard way, 2026-07)
+
+1. **`warmupData` is EMPTY** — tokiwausa.com looks like it hydrates from a JS payload
+   but the real catalog data is in the SSR `<img alt="...">` tags. Parse alt text,
+   don't chase the empty warmup JSON.
+2. **/japanese-silk pairing** — the silk category serves bare `1.jpg..52.jpg` image
+   names; pair them positionally with the rich-text labels on the page to recover
+   SKU/color. Verify counts match before staging.
+3. **Metal-leaf "leaf" is GILDING, not foliage** — the settlement screen must treat
+   "Metal Leaf"/"leaf" in this line as a surface material (gold/silver leaf), not a
+   botanical element. enrich.py's screen is gilding-aware; don't "fix" it into a
+   false positive.
+4. **`RR02-01_Fxx` tails** — some mfr skus carry `_Fxx` suffixes; those go to
+   `source_alt`, the canonical mfr_sku is the base (`RR02-01`).
+5. **Image-src ≠ item-number** (RIGO-class gotcha) — never derive a colorway image
+   from the item number; use the actual per-colorway `src`, then audit
+   `distinct(image_local)` vs `count(*)` PER PATTERN (WallQuest image-collapse class).
+
+## Viewer spec
+
+Built to `~/.claude/skills/internal-line-viewer/SKILL.md` (crezana-internal/astek
+reference): left rail collapsed facet tables (Material/Pattern/Series/Category/
+Color/Color name/Tags/Settlement/Status/Price/Image) with counts + type-ahead +
+click-to-filter, card-field toggles down to image-only wall, sort select + density
+slider (localStorage), created date+time chip, Memo/Stock/Price purchasing actions
+(shared `lib/vendor-requests.js`; no vendor email on file → logs + call).
+
+## Aliases
+
+`/fentucci` currently routes to the shared 108-vendor engine — this dedicated skill
+supersedes it for anything Tokiwa/Fentucci-specific.
← 934b282 internal line viewer :9981 (fentucci-viewer pm2) — full inte · back to Fentucci Naturals · FIX 1: re-town 27 pattern-name collisions vs live Fentucci f 15039eb →