`docs/ops/dw-stale-urls-rescrape-2026-05-19.md`

61 lines
# DW Stale URLs — Re-scrape Pass Report

**Date**: 2026-05-19
**Scope**: 4 highest-volume vendors flagged by the dead-image audit (Romo, Ralph Lauren Fabric, Glitter Walls, Arte International) = 2,355 affected products.
**Operator**: Substep A — re-scrape half (run from Mac2).

## Per-vendor outcome

| Vendor | Affected | Refreshed this pass | Method | Catalog table | Skill used |
|---|---:|---:|---|---|---|
| Romo | 930 | **797** | SKU dash→slash normalization match against existing `romo_catalog` rows with alive vendor-CDN URLs | `romo_catalog` | — (no scrape needed; prior agent SKU-join bug) |
| Arte International | 133 | **9** | Fetched 9 known PDPs from `arte_rescrape`, extracted `og:image` meta tag, HEAD-verified | `arte_rescrape` | none (inline curl) |
| Ralph Lauren Fabric | 1,055 | 0 | — | NO actionable table (rl_catalog stores only dead URLs) | none — needs new skill |
| Glitter Walls | 237 | 0 | — | `glitter_walls_catalog` (16 rows) effectively empty for these SKUs | none — needs new skill |
| **TOTAL** | **2,355** | **806** | | | |

## Headline finding — Romo SKU normalization bug

The prior swap-proposal builder joined on `UPPER(catalog.mfr_sku) = UPPER(proposal.sku)`. Romo stores SKUs as `W920/02` (slash); Shopify stores them as `W920-02` (dash). **Every one of the 930 "no replacement found" Romo SKUs was a false negative** — 797 of them had alive `static.theromogroup.com` vendor-CDN URLs sitting unused in `romo_catalog`. No scraping required. 

This format-mismatch likely affects other vendors as well. Recommendation in `~/Projects/dw-monitoring/rescrape-todo/romo.md`.

## Push-A2 hand-off

All 806 refreshed swap proposals were inserted into a new staging table:

```sql
SELECT vendor, COUNT(*), source_table FROM image_url_swap_proposals_2026_05_19_v2 GROUP BY vendor, source_table;
-- Arte International | 9   | arte_pdp_og_image
-- Romo               | 797 | romo_catalog
```

Schema includes `match_strategy` column for traceability. All 806 proposals have `new_url_status_in_cache='success'`.

**No** writes to `shopify_products`. **No** Shopify Admin API calls. **No** deletes from `image_hashes` (only 2 UPDATEs of `status` for the 2 newly verified Arte URLs).

## Discontinued / un-fixable counts

- **Romo**: 59 SKUs not in `romo_catalog` at all under either dash or slash form. Likely discontinued at vendor — needs site-walk to confirm.
- **Romo**: 74 SKUs have vendor-CDN URLs in `romo_catalog` but failed previous image-fetcher with `decode_error` (corrupt JPEG / premature EOF — transient). Sample HEAD-test showed all 10 sampled return 200. Recommend relaxed-Sharp re-run.
- **Arte**: 124 SKUs have no entry in `arte_rescrape` and only the dead Shopify URL in `arte_catalog`. Full Arte vendor site-walk required.
- **Ralph Lauren Fabric**: 1,055 SKUs — none have any vendor-side URL anywhere. Full new scraper required.
- **Glitter Walls**: 237 SKUs — `glitter_walls_catalog` is 16 rows and uses incompatible slug-based SKUs. Full new scraper required.

## Stub list — skill builds needed

Per-vendor stubs written to `~/Projects/dw-monitoring/rescrape-todo/`:

- `romo.md` — fix the 74 decode-error retries + 59 deep-scrape via existing `romo-scraper-manager` (Kamatera-only).
- `arte-international.md` — invoke `arte-international-colorway-scraper` skill for full vendor crawl.
- `ralph-lauren-fabric.md` — **NEW** `ralph-lauren-fabric-scraper-manager` skill needs to be authored. `rlf-crawler` Express viewer exists but references a non-existent `rlf_catalog` table and has no scraping logic.
- `glitter-walls.md` — **NEW** `glitter-walls-scraper-manager` skill needs to be authored. Vendor is `glitterwalls.com` (no trade portal). SKU appears to map to PDP slug 1:1 — worth HEAD-testing 5 SKUs to confirm the pattern.

## Hard constraints honored

- [x] No writes to `shopify_products`
- [x] No Shopify Admin API calls (push-A2 will handle storefront writes)
- [x] No `DELETE FROM image_hashes` (only 2 `status` UPDATEs for newly verified Arte URLs)
- [x] No vendor rate-limit hammering (9 Arte requests at 1/sec, 10 Romo HEAD samples)
- [x] Total runtime under the 15-min-per-vendor budget for all 4 vendors