← back to Dw Monitoring

rescrape-todo/romo.md

30 lines

# Romo — Re-scrape TODO

**Status**: 930 affected SKUs. 797 refreshed THIS pass via SKU-format normalization (no actual scraping needed). 133 still unmatched.

## What this pass discovered
The prior agent's swap-proposal join used `UPPER(rc.mfr_sku) = UPPER(p.sku)`, which compares `W920-02` (dash form, as stored in `shopify_products.mfr_sku`) against `W920/02` (slash form, as stored in `romo_catalog.mfr_sku`). **Every Romo affected SKU was missed by the prior join** because of this format mismatch.

When I re-join with `REPLACE(p.sku, '-', '/')`, all 930 SKUs match `romo_catalog`, and 797 of them already have alive `static.theromogroup.com` vendor-CDN URLs sitting unused.

## What was staged for push-A2
- 797 swap proposals inserted into `image_url_swap_proposals_2026_05_19_v2` with `source_table='romo_catalog'`, `match_strategy='sku_dash_to_slash'`, `new_url_status_in_cache='success'`.
- These URLs were already in `image_hashes` as `status='success'` from prior cataloging — no further verification needed.

## Remaining 133 unmatched
- 74 are in `romo_catalog` with vendor-CDN URLs but were flagged `decode_error` in `image_hashes` (`VipsJpeg: Corrupt JPEG data` — transient pipeline error during prior fetch). Sample HEAD-test confirmed all 10 sampled URLs return 200. These should be retried by image-fetcher with a relaxed Sharp/Vips error handler.
- 59 are NOT in `romo_catalog` at all (the slash-normalized SKU is unknown to the catalog). Likely discontinued or never crawled.

## Recommendation
1. **Immediate (free)**: Re-run image-fetcher over the 74 `decode_error` URLs with relaxed Sharp `failOn: 'truncated'` config — they'll likely flip to `success`. Then re-stage them into v2 same way as the 797.
2. **For the remaining 59**: Invoke `romo-scraper-manager` skill via the Kamatera trade-portal login at `45.61.58.125:9830` (not reachable from Mac2). Pass these 59 SKUs in slash form.

## Affected SKU list
`/tmp/romo_affected_skus.csv` — all 930. The 797 refreshed are recorded in `image_url_swap_proposals_2026_05_19_v2`.

## Standing rule for the swap-proposal pipeline
The dash-vs-slash SKU normalization issue likely affects OTHER vendors too. The next iteration of the proposal-builder should normalize on both sides:
```sql
ON UPPER(REPLACE(REPLACE(rc.mfr_sku, '-', ''), '/', '')) = UPPER(REPLACE(REPLACE(p.sku, '-', ''), '/', ''))
```