`tasks/prd-rolex-daily-scraper.md`

131 lines
# PRD: Daily Rolex Official Price Scraper

## Introduction

Scrape official retail prices from rolex.com daily to build a historical price database. Runs as a cron job at 6am daily. Captures all Rolex collections including Professional and Classic lines.

## Goals

- Capture official Rolex retail prices daily from rolex.com
- Build historical MSRP database over time
- Track price changes and updates across all collections
- Zero cost (direct website scraping)
- Cover ALL 12 Rolex collections comprehensively

## User Stories

### US-040: Create Rolex Website Scraper
**Description:** As a data analyst, I want a Python script to scrape current prices from rolex.com so that I have accurate MSRP data.

**Acceptance Criteria:**
- [ ] Scrape Submariner collection prices
- [ ] Scrape Daytona collection prices
- [ ] Scrape GMT-Master II collection prices
- [ ] Scrape Datejust collection prices
- [ ] Scrape Day-Date collection prices
- [ ] Scrape Explorer collection prices
- [ ] Scrape Sea-Dweller collection prices
- [ ] Scrape Yacht-Master collection prices
- [ ] Scrape Sky-Dweller collection prices
- [ ] Scrape Air-King collection prices
- [ ] Scrape Oyster Perpetual collection prices
- [ ] Scrape Cellini collection prices
- [ ] Output JSON with model, reference, price, date

### US-041: Create Rolex Price Storage System
**Description:** As a data analyst, I want daily price snapshots stored in JSON so that I can track price changes over time.

**Acceptance Criteria:**
- [ ] Create data/rolex-official-prices.json
- [ ] Append daily snapshots (don't overwrite history)
- [ ] Track price changes between days
- [ ] Include metadata (scrape timestamp, product count)

### US-042: Set Up Rolex Cron Job
**Description:** As a system administrator, I want the scraper to run automatically at 6am daily so that data collection is consistent.

**Acceptance Criteria:**
- [ ] Cron job runs at 6am daily
- [ ] Logs output to logs/rolex-scraper.log
- [ ] Handles errors gracefully without crashing
- [ ] Sends summary to stdout on completion

### US-043: Add Rolex API Endpoints
**Description:** As a frontend developer, I want API endpoints to access Rolex price data so that I can display it in the UI.

**Acceptance Criteria:**
- [ ] GET /api/rolex-prices returns all current prices
- [ ] GET /api/rolex-prices/:reference returns specific watch
- [ ] GET /api/rolex-prices/history returns historical data
- [ ] Response includes collection, model, reference, price

## Functional Requirements

- FR-1: Script must scrape all 12 Rolex collections from rolex.com/en-us/
- FR-2: Extract model name, reference number, and USD price for each watch
- FR-3: Rate limit requests (3 second delay) to avoid blocking
- FR-4: Store data in JSON format with daily snapshots
- FR-5: Deduplicate products by reference number
- FR-6: Handle connection timeouts gracefully with retries
- FR-7: Log all scraping activity to log file

## Non-Goals

- No scraping of authorized dealer prices
- No secondary market price tracking
- No image downloading
- No availability/stock tracking

## Technical Considerations

### Rolex Collections (12 total)
**Professional:**
- Submariner
- Cosmograph Daytona
- GMT-Master II
- Explorer / Explorer II
- Sea-Dweller / Deepsea
- Yacht-Master / Yacht-Master II
- Air-King

**Classic:**
- Datejust
- Day-Date
- Sky-Dweller
- Oyster Perpetual
- Cellini

### Data Storage Format
```json
{
  "metadata": {
    "source": "rolex.com",
    "lastUpdated": "2026-01-11T06:00:00Z"
  },
  "scrapes": [
    {
      "date": "2026-01-11",
      "products": [
        {"collection": "Submariner", "model": "Submariner Date", "reference": "m126610ln-0001", "price": 10250}
      ]
    }
  ]
}
```

### Cron Schedule
```
0 6 * * * /usr/bin/python3 /root/Projects/watches/scripts/rolex-daily-scraper.py >> /root/Projects/watches/logs/rolex-scraper.log 2>&1
```

## Success Metrics

- Successfully scrape 50+ Rolex models daily
- Zero missed daily scrapes over 30-day period
- Price data matches rolex.com within same day
- API response time under 200ms

## Open Questions

- None - proceeding with implementation