`PLAN.md`

52 lines
# EUR- recrawl — capture logged-in (trade) prices, then price the line

**Ticket:** TK-10068 · **Origin:** Steve — "recrawl. we pay their price when logged in" + "source cost data first" (do NOT discontinue).

## Goal
The EUR- line (2,099 active, sample-only $4.25, no cost) can't be priced from the stale catalogs. Steve's call: recrawl the vendor **trade portals** with DW's logged-in account to capture the **price DW actually pays**, then price every EUR- product from that. Nothing gets discontinued.

## Targets
`targets.csv` — 2,099 active EUR- products with their extracted `mfr_code` (the portal lookup key):
Osborne & Little 838 · Designers Guild 792 · Nina Campbell 306 · Christian Lacroix Europe 163.

## Credential coverage (parent-brand logins cover all 4)
- **OSBORNE_USERNAME / OSBORNE_PASSWORD** → Osborne & Little **+ Nina Campbell** (Nina is an O&L brand).
- **DESIGNERSGUILD_PORTAL_USER / _PASS / _LOGIN_URL** → Designers Guild **+ Christian Lacroix** (Lacroix is a DG brand).
(Both already in the secrets master.)

## Feasibility (probed 2026-07-30)
- **Designers Guild portal = Cloudflare-challenge-walled** (curl → HTTP 403 + cf-challenge). Osborne `/trade` = 301 redirect. → the crawl needs a **real browser (Playwright)** that passes Cloudflare naturally. This is DW logging into **its own supplier accounts** — legitimate access, NOT bot-wall evasion. HARD LINE: pass the challenge as a real browser; never script-bypass/evade it.

## Approach (real-browser, polite)
1. Playwright: log into each parent portal once (persist the authed session in `.auth/`, gitignored).
2. For each `mfr_code`, navigate to its product/search page, read the **logged-in price** (trade/net).
3. Write `eur_prices.csv` (mfr_code → trade_price, currency, captured_at). Rate-limited + resumable; on-disk cache so a re-run never re-hits captured products.
4. Compute DW retail from the trade price (cost/0.65/0.85 unless Steve sets otherwise) → stage price updates in a dry-run CSV.

## GATED actions (Steve-run / Steve-authorized)
- **The authenticated crawl itself** — drives DW supplier credentials against Cloudflare-walled portals at scale. Steve authorizes; the Cloudflare/login step may need his hands (interactive challenge / 2FA).
- **The downstream Shopify price writes** — customer-facing, ~up-to-2,099 products → sellable rolls. Draft dry-run first, Steve runs.

## Status
Scaffold + target list built (safe/local). Authenticated crawler = next, gated on Steve's go for the credentialed Cloudflare login. Zero live writes. Cost so far: $0 (local).

## Cody gate — FIX FIRST (verified 2026-07-30, before any crawl)
1. **PRICE BASIS UNKNOWN (critical).** In osborne_catalog where both set, `price_trade` avg $240 > `price_retail` avg $173 (52/55 rows) — backwards from normal wholesale. So it's unclear what "the price we pay logged in" maps to (net cost? trade list? = price_retail already?). Applying cost/0.65/0.85 to the wrong number mis-prices ALL 2,099. **Blocker: Steve must confirm what the logged-in portal number represents + what DW should charge from it (formula).** Best resolved against one known DW invoice OR one authenticated portal lookup.
2. **729 SKUs (35%) share an mfr_code** with sibling colorways. One portal price stamped on all → mis-prices any colorway priced differently. Crawler must capture per-colorway OR verify same-price-per-pattern before stamping.
3. **Normalization does NOT recover more matches** (369=369 for O&L) — the 348 catalog misses are real gaps, so the crawl IS necessary for them (not avoidable by a better join). Catalog tops out ~369 priceable + 105 matched-no-price.
4. **Cloudflare feasibility unproven** — "Playwright passes CF naturally" is an assumption; CF Bot Management fingerprints automation. Needs a real authenticated-session test, not the curl inference.

**Loop state: BLOCKED on Steve for the price basis (#1) — cannot build the pricing step until the logged-in number's meaning + DW's charge formula are pinned.**

## UPDATE 2026-07-30 — price basis RESOLVED + login delegated
- **Price basis SOLVED (verified live):** retail = trade_cost / 0.65 / 0.85 (= ×1.810). Confirmed against canary EUR-71216: live roll $590.05 = trade $326.00 × 1.810, Sample $4.25 preserved. (Prior work TK-00113, re-verified here.) Cody FIX-FIRST #1 is closed.
- **Live write path SOLVED:** bespoke productVariantsBulkCreate (API 2024-10, inventoryPolicy CONTINUE, tracked=false) adds a sellable roll to an already-active sample-only product without disturbing the Sample. Canary proven.
- **Login delegated:** sibling agent `claude-eur` owns the Playwright trade-LOGIN flow (Osborne + DG Cloudflare) and will drop an authed storage-state at `.auth/osborne.json` + note where per-SKU trade price lives. This terminal (vp-dw-commerce) owns: targets.csv → crawl w/ that session → parse trade price → ×1.810 → PG stage → gated productVariantsBulkCreate write. Standing down on login (avoid clobber/lockout).
- **Remaining before writes:** (a) authed session handoff; (b) per-SKU trade-price parse; (c) 729 shared-mfr_code colorway handling (capture per-colorway); (d) dry-run then Steve-gated write.

## REONBOARD WRITE — ready + gated (Steve runs) + CRITICAL below-cost finding
- **CRITICAL:** many EUR- products already have a live roll variant priced BELOW COST (sample: 10/13 loss-making — EUR-80210 $120 vs $186 cost; EUR-80217 $157 vs $284). DW loses money on every one sold. The reonboard FIXES this to retail = trade x1.810.
- reonboard.csv = 1,410 priceable (381 exact + 595 pattern-single + 434 absent-cw-safe); 689 held (368 disco + 165 disco-colorway + 154 panels + 2 ambiguous).
- write_reonboard.py (GATED, DRY_RUN default, dry-run validated): per product -> UPDATE existing roll price OR CREATE roll (Size:Roll, CONTINUE, tracked=false, Sample preserved). Never lowers below cost; skips already-correct.
- RUN (Steve): `DRY_RUN=0 LIMIT=1 python3 write_reonboard.py` (smoke), verify in admin, then `DRY_RUN=0 python3 write_reonboard.py`. $0 API.