`scripts/README-d2-sweep.md`

82 lines
# TK-11331 Decision 2 — hands-off daily sweep

Finishes D2 (set `global.v_prods_quantity_order_min="5"` + `global.unit_of_measure="Sold Per Yard"`
on the non-compliant X-prefix @52/54 products) automatically, one throttled batch per day, with a
fail-loud verify gate and an auto-stop.

## Files
- `scripts/d2_daily_sweep.sh` — the wrapper (one batch per invocation).
- `deploy/com.steve.tk11331-d2-sweep.plist` — launchd job (04:45 daily). **NOT installed** — Steve installs.
- `data/d2-sweep-state.json` — persisted offset (source of truth for "where are we").
- `data/d2-sweep-runs.log` — human-readable per-run trail.
- `data/sweep-stdout.log` / `data/sweep-stderr.log` — launchd captured output.
- `data/d2-sweep-FAILED` — halt marker (present ⇒ wrapper refuses to run; **fail-loud**).
- `data/d2-sweep-DONE` — completion marker (present ⇒ nothing to do).

## Offset math
- `data/d2_set.json` holds **3600** non-compliant products (indices `0..3599`).
- **431** already applied+verified on Day-1 (indices `0..430`). Sweep resumes at offset **431**.
- Each run: apply `--offset=$OFFSET --limit=1000`, verify, then advance the offset by **1000**.

| Run | offset | rows applied | indices |
|-----|--------|--------------|---------|
| (Day-1, done) | 0   | 431  | 0..430    |
| Day-2 | 431  | 1000 | 431..1430  |
| Day-3 | 1431 | 1000 | 1431..2430 |
| Day-4 | 2431 | 1000 | 2431..3430 |
| Day-5 | 3431 | 169  | 3431..3599 |

3169 remaining ÷ ≤1000/day = **~4 daily runs**. The last day is a partial (169) because
`limit=1000` caps at the end of the set (`rows = min(1000, total - offset)`).

## Per-run behavior
1. Single-flight **lock** (`data/.d2-sweep.lock`) — a second fire while one is running just exits.
2. If `data/d2-sweep-FAILED` exists → **HALT** (exit 1), do nothing. Manual clear required.
3. If `data/d2-sweep-DONE` exists → nothing to do.
4. If `offset >= 3600` → write DONE, self-disable, ledger completion, exit.
5. `node d2_metafields.mjs --apply --offset=$OFFSET --limit=1000`
   — **PG mirror first, then Shopify** (authoritative), per rule 5. Records old values to `data/d2-restore-map.jsonl`.
6. `node verify_d2.mjs $OFFSET 1000` — re-reads via the **Shopify Admin API (authoritative, NOT the CDN)**.
7. **Fail-loud verify gate:** if `NON-compliant != 0` (or verify output is unreadable / `checked=0`)
   → write `data/d2-sweep-FAILED`, **do NOT advance the offset**, exit 1. The offset stays put so a
   fixed re-run reprocesses the same slice (writes are idempotent — same values).
8. Append the batch to `~/.claude/yolo-queue/executed-reversible/ledger.jsonl` via `log-exec.mjs`
   (`ticket=TK-11331`, `action=d2-daily-batch`, `blast_radius=<rows>`, `undo`, `verify`).
9. Advance the offset in `data/d2-sweep-state.json`.
10. When the advance reaches `>= 3600` → write DONE, ledger completion, **self-disable** the launchd
    job (`launchctl bootout gui/$UID/com.steve.tk11331-d2-sweep`).

## Idempotency & safety
- Re-running the same offset re-writes the identical values ("5" / "Sold Per Yard") — safe, no drift.
- On a bad verify the offset does **not** advance, so nothing is skipped.
- `RunAtLoad=false` — a reboot never fires an unattended batch on boot; it waits for the next 04:45.

## Install (Steve — GATED, one approve)
```sh
cp ~/Projects/tk-11331-exec/deploy/com.steve.tk11331-d2-sweep.plist ~/Library/LaunchAgents/
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.steve.tk11331-d2-sweep.plist
```

## Manual kill switch
```sh
launchctl bootout gui/$(id -u)/com.steve.tk11331-d2-sweep
# optional, after DONE: rm ~/Library/LaunchAgents/com.steve.tk11331-d2-sweep.plist
```

## Clear a FAILED halt (after investigating)
```sh
cat  ~/Projects/tk-11331-exec/data/d2-sweep-FAILED   # read the reason first
rm   ~/Projects/tk-11331-exec/data/d2-sweep-FAILED   # then the next 04:45 run resumes at the same offset
```

## Rollback
```sh
cd ~/Projects/tk-11331-exec && node d2_rollback.mjs --apply   # reverts the full data/d2-restore-map.jsonl
```

## Run one batch manually (test / catch-up, outside launchd)
```sh
bash ~/Projects/tk-11331-exec/scripts/d2_daily_sweep.sh
```
(Still throttled to one ≤1000-row batch per invocation.)