`README.md`

13 lines
# dw-scrape-scheduler

Staggered monthly feed-first vendor scrape. DTD verdict A (2026-07-13): one generic
feed-first runner + a single daily launchd dispatcher that runs only the Tier-A vendors
whose crawl_cron day == today, concurrency cap 2, local-models-only, no Gemini.

- `build-manifest.js` — vendor_registry -> data/manifest.json (tiers by platform+creds)
- `adapters/` — feed adapters (shopify LIVE; woo/magento TODO)
- `run-vendor.js` — scrape one vendor -> data/snapshots/ (PROOF stage: snapshot+count, no canonical write yet)
- `dispatcher.js` — daily selector + concurrency cap (--day/--vendor/--all/--dry-run/--cap)

Tier-C (portal-login/paid-Browserbase) is NEVER auto-scheduled (gated).