`README.md`

50 lines
# jf-fabrics-recrawl

Fixes the **2,580 net-new `jf_fabrics` rows missing `all_images`** (Steve's standing
*"always pull all data and images"* completeness gate, 2026-06-10).

## Root cause
`/root/DW-Agents/vendor-scrapers/scrape-jffabrics.js` only ever captured a single
primary `image_url` (650px product-image / og:image) and one spec field (`material`).
It never populated `all_images`, `fire_rating`, `width`, `repeat_v`, `match_type`,
`finish`, or `features`. So every freshly scraped JF colorway lands incomplete.

## The fix (this repo — LOCAL, validated, NOT yet deployed)
- **`extract-jffabrics.js`** — drop-in fixed extractor. Mirrors the proven
  2026-06-10 osborne/hollywood pattern: `all_images = JSON.stringify(<distinct
  full-res images for THIS sku>)` + a full sweep of the "More Details" spec block.
  - Upgrades `image_url` to the full-res `data-zoom` (2000×2000) variant.
  - Keys every image to **this** sku's filename stem (`10000-48` → `10000_48`) so
    sibling-colorway thumbnails (`_61`, `_68`…) in the same book are **excluded** —
    honors the never-duplicate / no-shared-image rule.
  - Rejects share-link URLs (pinterest/facebook ?media=…) and theme/favicon junk.
  - Maps specs → columns: Flammability Codes→`fire_rating`, Content→`material`,
    Width→`width`, Repeat→`repeat_v`, Match→`match_type`, Finish→`finish`;
    washable/origin/paste etc. → `features`.
- **`recrawl-jffabrics.js`** — additive re-crawl driver (mirror of
  `recrawl-osborne.js`). Pulls distinct product_urls from `jf_fabrics_catalog`,
  re-extracts, and `upsertProduct`s `ON CONFLICT (mfr_sku) DO UPDATE` (no new
  rows, no dups). Min-byte guard + 3× retry on throttled short pages.

### Validated locally (2026-06-10)
Ran the extractor against live JF pages (via Kamatera fetch):
- `10000-48` → `all_images=[…10000_48W8771-2000x2000.jpg]`, `fire_rating=ASTM E84`,
  `match_type=Straight Match`, `features=Country/Washable/Paste`. ✅
- `10000-61` (same book) → only its own `10000_61` image, **no bleed**. ✅

## Deploy + run = GATED (prod write to Kamatera dw_unified)
Do NOT run autonomously. See
`~/.claude/yolo-queue/pending-approval/jf_fabrics-recrawl.md` for the exact
ready-to-run commands and the /dtd verdict.

Short version (after Steve approves):
```sh
scp ~/Projects/jf-fabrics-recrawl/extract-jffabrics.js \
    ~/Projects/jf-fabrics-recrawl/recrawl-jffabrics.js my-server:/root/DW-Agents/vendor-scrapers/
ssh my-server 'cd /root/DW-Agents/vendor-scrapers && node recrawl-jffabrics.js --net-new --all-images --limit 20 --dry-run'  # smoke
ssh my-server 'cd /root/DW-Agents/vendor-scrapers && node recrawl-jffabrics.js --net-new --all-images'                       # full
```

## NOT in scope here
No Shopify publish. The net-new push is separately gated for an attended window.