Crawl viability finding: Tier-B prices not in static HTML (JS/login/trade-only) — crawl uneconomical, route to letters; Browserbase crawl is a separate gated spend decision

589e3fa1c7ef665a0ee6f53221bc2eb005bf273e · 2026-06-17 10:29:27 -0700 · Steve Abrams

Files touched

A CRAWL-FINDINGS.md

Diff

commit 589e3fa1c7ef665a0ee6f53221bc2eb005bf273e
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Wed Jun 17 10:29:27 2026 -0700

    Crawl viability finding: Tier-B prices not in static HTML (JS/login/trade-only) — crawl uneconomical, route to letters; Browserbase crawl is a separate gated spend decision
---
 CRAWL-FINDINGS.md | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/CRAWL-FINDINGS.md b/CRAWL-FINDINGS.md
new file mode 100644
index 0000000..5e3194b
--- /dev/null
+++ b/CRAWL-FINDINGS.md
@@ -0,0 +1,18 @@
+# Cost-crawl viability (2026-06-17)
+
+Tested whether vendor retail/cost is scrapable cheaply (static HTML curl) for the Tier-B set.
+Result: **NOT viable via static HTML.** Sampled product pages:
+- Designers Guild — price JS-rendered (Knockout.js `ko.unwrap(price)`); not in HTML.
+- Arte International — no price in HTML (`price:0` / JS).
+- Scalamandre — empty `PRICE` data-attrs (JS / trade-login).
+- Elitis — `"price":0` (JS / not public).
+
+Most Tier-B vendors are **trade-only** and publish NO public price. So:
+1. **No cheap crawl path** — would require Browserbase (paid, per-vendor adapters) or trade logins.
+2. For most, there's **no public price to get at all** → the vendor LETTER is the only path.
+
+**Decision:** skip the crawl as uneconomical; route everything to the letter pipeline
+(approve/send web page already built). A Browserbase crawl for the handful of public-retail
+vendors is a separate, Steve-gated (spend) sub-project — not worth building blind.
+The price_sourcing schema already supports a future crawl (crawled_cost/crawl_status):
+if a crawl ever runs and finds a price, those SKUs auto-drop off the letter list.

← c8a693e Vendor price-request pipeline: mfr_sku extractor (21k SKUs, · back to Vendor Price Requests · VPR: exclude skip-own-brand (Hollywood) from letter list a3a5713 →