`pilot/morris-image-source-finding.md`

111 lines
# Morris image sourcing — corrected finding (2026-09-02, vp-dw-commerce, TK-10881)

**Premise in the cadence memo was WRONG.** "102/1162 pattern-matchable to william_morris_catalog"
does NOT yield usable images:
- `william_morris_catalog` = 339 rows / 309 imaged, but a DIFFERENT product universe
  (SKUs `saw####-##`, patterns like Florrie / Alice In Wonderland).
- Current Morris feed = 1,162 rows, SKUs `DJA1A7102` etc., feed `pattern` field is
  actually pattern+color combined ("Arbutus Blue/Pink"), `color` is null.
- **Exact pattern+color match feed→wmc = 0.** Base-pattern fuzzy overlap ~12 real (235 is a
  loose upper bound that over-counts Acanthus/Willow variants). Colorways do NOT align, so DB
  images would put WRONG-COLORWAY pictures on products = quality defect. **DB path rejected.**

**Correct source = sanderson.design (same as SHZ).**
- SHZ feed rows carry `image` = `/product/<c1>/<c2>/<MFRSKU>_<hash>.jpg`; full URL base
  **`https://www.sanderson.design/static/media/catalog`** — VERIFIED 200 image/jpeg on
  DKH1C2205 + HOT04409.
- The `_<hash>` suffix is per-image and NOT guessable → Morris cannot be URL-constructed; it
  must be **RE-HARVESTED** from the Morris pages on sanderson.design (the harvest that imaged
  SHZ simply wasn't run for Morris — morris_feed_harvest.jsonl `image` is empty on all 1,162).

**Next action (long-running, own window):** run the SHZ harvester pointed at the Morris brand
to capture per-row image paths, HTTP-200-validate, stage into `pilot/manifest-morris.json`.
Reversible (manifest staging only). Settlement gate MUST run before any publish (Morris & Co is
botanical/floral-heavy). Preview of the (failed) DB match is at `pilot/morris-image-match-preview.jsonl` (0 rows).

## UPDATE 2 (2026-09-02) — three paths ruled out; Morris image sourcing is a NEW method, not a quick fix
- **DB (william_morris_catalog):** 0 color-correct matches. REJECTED.
- **SHZ `.design` feed harvester (harvest_sdg_feed.mjs):** Morris SKUs `DJA1A7102` do NOT match its
  enum regex `^[A-Z]{2}[WF][0-9]{4}`; and `www.morrisandco.com` has no `/us/api/n/find` feed
  (probe --brand morris_probe returned 0 pattern codes). REJECTED.
- **Existing morris_feed_harvest.jsonl (1,162 rows):** sourced from the SDG **trade portal**
  (logged-in) raw dump (data/morris_*_raw.txt = "Dashboard/Quick Order/..." UI) — carries prices
  but ZERO `/product/*.jpg` image paths. That is why image=null on all 1,162.
- **Actual next method (real R&D, own ticket):** scrape morrisandco.com public product pages for
  per-colorway image paths (different platform than sanderson.design), OR pull from the SDG DAM.
  Requires discovering Morris & Co's public product/image endpoint. Reversible/manifest-only when built.
- **Priority note:** NOT on the cadence critical path — the armed cadence drains ~4,620 SHZ at
  200/day regardless; Morris simply contributes 0 until this method ships. So this is a
  deliberate follow-on ticket, not an autonomous-overnight item (two unattended windows stalled).
- Probe artifact pilot/morris_probe_feed_harvest.jsonl (0 rows) can be deleted.

## UPDATE 3 (2026-09-02) — morrisandco.com probe: NOT the SHZ platform (final quick-path ruled out)
- `www.morrisandco.com/us/api/n/find` and `/api/n/find` → **404** (no SDG feed API).
- Homepage carries **0** `static/media/catalog/product/*.jpg` paths.
- CONCLUSION: Morris & Co runs a DIFFERENT platform than sanderson.design. Its per-colorway
  image endpoint must be discovered (different CMS/API, or a headless render, or the SDG DAM).
  This is genuine R&D — an ATTENDED ticket, not an unattended harvest. Confirmed NOT cadence-critical.

## UPDATE 4 (2026-09-02, live discovery) — REAL Morris & Co domain FOUND; all decoys ruled out
Decisively mapped the Morris & Co image-source problem end to end:
- **REAL domain = `www.wmorrisandco.com`** (title "Morris & Co. | Founded by William Morris").
  Note the **`w` prefix** — this is why every prior attempt failed (all pointed at wrong hosts).
- Ruled out WITH EVIDENCE:
  - `morrisandco.com` = a Shrewsbury PROPERTY DEVELOPER (WordPress, dragon-boat news). Wrong company.
  - `morrisandco.design` = EXISTS but **expired SSL cert** (ERR_CERT_DATE_INVALID) — parked/dead, browser refuses it.
  - `sanderson.design` / `harlequin.design` feeds = Sanderson (SAW/SAF) / Harlequin (HAW) — **NOT Morris**.
  - **DATA-INTEGRITY FLAG:** `dw_unified.william_morris_catalog` holds `saw####` codes with SANDERSON
    patterns (Dandelion Clocks, Swallows, Finches, Alice In Wonderland) — it appears to be
    **mislabeled Sanderson data, not Morris & Co**. Worth its own audit ticket.
- `wmorrisandco.com` does NOT use the SHZ `/us/api/n/find` feed (404) — DIFFERENT platform.
  NEXT STEP (focused, attended): Playwright network-trace a wmorrisandco.com wallpaper listing to
  capture its real product/image API (it's a JS SPA), OR authenticated trade-portal image scrape.
- Still NOT cadence-critical; SHZ drains regardless. This is a scoped follow-on, not autonomous.

## ✅ UPDATE 5 (2026-09-02) — SOURCE CRACKED. Morris & Co images found + join key identified.
**Source = www.wmorrisandco.com — a Magento 2 store** (theme "Janeclayton", customer/section/load).
Product images at:
`https://www.wmorrisandco.com/media/catalog/product/cache/<hash>/<c1>/<c2>/<slug>-morris-and-co-<color>-wallpaper-<CODE>-image01.jpg`
- **Images are plain-HTTP 200** (no browser/login needed to fetch a known image URL).
- **JOIN KEY (the breakthrough):** the image filename embeds the design CODE = the trade-feed SKU tail.
  Proven: image `arbutus-morris-and-co-blue-pink-wallpaper-1a7102-image01.jpg` ↔ trade SKU **DJA1A7102**
  ("Arbutus Blue/Pink"). So the 1,162 trade rows map to wmorrisandco images by (a) code-tail and
  (b) pattern+color slug (`<pattern>-morris-and-co-<color>`).
- **HARVEST METHOD:** enumerate Morris wallpaper via **Magento category pagination** (server-rendered
  product grid carries the /media/catalog/product image URLs) — NOT catalogsearch (its query filter is
  JS/AJAX; plain HTML returns a default grid). Feed-first Magento pattern. $0 plain HTTP.
- **BUILD (next, scoped, ~method-proven):** scripts/harvest_morris_wmorrisandco.mjs — paginate Morris
  wallpaper categories, extract {slug,color,code,image}, match to the 1,162 trade rows by code-tail then
  pattern+color, HTTP-200-validate, stage pilot/manifest-morris.json. Reversible/manifest-only. Settlement
  gate before publish. Still NOT cadence-critical (SHZ drains regardless).

## UPDATE 6 (2026-09-02) — enumeration constraint characterized; discovery COMPLETE
- `sitemap.xml` = **403** (bot-blocked); category grid (`/uk/wallpaper`) is **JS-rendered** (plain HTML
  has no products). So HARVEST ENUMERATION must be **browser-driven (Playwright)**: per-pattern
  `catalogsearch/result/?q=<pattern>` renders the product cards → capture `/media/catalog/product/...`
  image URLs from the DOM/network (PROVEN on Arbutus = 6 colorways, codes present).
- IMAGE FETCH stays **plain-HTTP 200** (no browser needed once the URL is known).
- FINAL BUILD SHAPE: Playwright loop over the ~987 distinct Morris patterns (from morris_feed_harvest)
  → per pattern capture images → match to trade rows by code-tail + pattern/color slug → plain-HTTP
  validate → stage manifest-morris.json. Long-running (browser loop) → attended run, not unattended
  (two unattended windows stalled). Reversible/manifest-only. Settlement gate before publish.
- DISCOVERY IS DONE. Remaining = a mechanical (if slow) browser-harvest build.

## UPDATE 5 (2026-09-02) — SOLVED. Morris images RE-HARVESTED & STAGED (99.1%)
- **Platform confirmed:** `www.wmorrisandco.com` = Magento 2 (theme "Janeclayton"), Cloudflare-JS-gated,
  NO Nucleus `/us/api/n/find` feed, NO public GraphQL. (`morrisandco.sandersondesigngroup.com` 302→ here.)
- **Method that worked:** openclaw REAL Chrome clears Cloudflare; in-page `fetch()` (carries the CF
  clearance cookie) pulls the Magento **sitemap** + each PDP's **og:image** on `/media/catalog/product`.
  The media CDN is UN-challenged → every image 200-validated by plain curl ($0). SERIAL (1 browser).
- **SKU→URL join:** sitemap trailing-digit / full-code (1,079) + live per-pattern search-DOM URL pool +
  exact **MOW####-## full-code-suffix** recovery (the archive codes live in the URL slug tail).
- **Wrong-colorway defect GUARDED** by a title-parse QA gate (page "<Pattern> in <Color>" vs portal
  pattern+color, stemmed). It caught & rejected real bad joins (Fruit→Chrysanthemum, Seaweed→404,
  Blackthorn→Artichoke). Rejected joins never enter the manifest.
- **RESULT: 1,152 / 1,162 (99.1%)** images staged to `pilot/manifest-morris.json`, all HTTP-200 image/*
  (img_bad=0), 469 wallcovering + 683 fabric. 10 still-missing (US-discontinued / not-on-site / 404 on US),
  documented in `pilot/morris_still_missing.json` + `pilot/morris-image-COVERAGE-REPORT.md`.
- **REVERSIBLE / staging-only** (no Shopify/DB/launchd). **SETTLEMENT GATE MANDATORY before any publish**
  (`manifest-morris.SETTLEMENT-REQUIRED.json`) — Morris is botanical/floral-heavy. Scripts:
  `harvest_morris_images.mjs`, `pool_morris_search.mjs`, `finalize_morris_manifest.mjs`.