`_PM2_NOTES.md`

35 lines
# pm2 fleet notes (2026-04-30 nightly run)

## am-recrawl — STOPPED + investigated by codex
- Was: 90 restarts in ~24h (autorestart:true on a one-shot puppeteer scraper)
- Pattern: script runs ~65 Andrew Martin products → hits `TargetCloseError: Protocol error` → exits → pm2 restarts → repeat
- Action taken: `pm2 stop am-recrawl && pm2 save`
- **Codex root-cause investigation** (`codex-review-2026-04-30/am-recrawl/poke.md`):
  1. `scrape-andrewmartin.js:596-599` calls `closePool()` AFTER `main()` finally already called it at 590-592 — double-close
  2. `scraper-utils.js:184-185` `pool.end()` is not idempotent — fix: cache the close-promise (`if (!poolClosePromise) poolClosePromise = pool.end()`)
  3. `scrapeProductPage` allocates `page` outside `try`, so newPage/setUserAgent failures escape
  4. **Service is one-shot** (no listener), not long-running — pm2 `autorestart:true` is wrong
- NOT auto-applied: file lives in `~/kamatera-mirror/`, ambiguous whether sync would overwrite. Steve to-do: review codex's proposed patch (idempotent closePool, browser.isConnected guards, throw on disconnect, `--no-autorestart` on pm2) and apply on Kamatera, then re-sync.

## sf-admin — FIXED
- Was: would crash-loop on next restart because codex removed hardcoded SESSION_SECRET fallback (correct fix), but ecosystem.config.js sf-admin env had no SESSION_SECRET
- Action taken: generated 96-char random secret → `~/Projects/site-factory/admin/.session-secret` (chmod 600), added `.gitignore` line, wired SESSION_SECRET into ecosystem.config.js sf-admin env via fs.readFileSync, deleted+restarted sf-admin, verified HTTP 302 (OAuth redirect = healthy), `pm2 save`d.
- Currently: online, 0 restarts, accepts auth flow.

## visual-factory-orchestrator — STABLE (after 12 transient restarts)
- Crash-loop coincided with codex's edits on disk (server.js, src/llm.js around CLAUDE_CLI). Now stable with growing uptime, HTTP 200 on :9892. No further action.

## Stripe webhook routes (wholivedthere/bubbesblock/claimmyaddress)
- Codex modified `app/api/webhook/stripe/route.ts` in all 3 sites to fail-fast in production when STRIPE_WEBHOOK_SECRET is missing. Verified all 3 sites have STRIPE_WEBHOOK_SECRET set in `.env.local`. Next.js prod builds (`pm2 site-*`) are still serving old code — new code activates on next `npm run build && pm2 restart`.


## visual-factory-orchestrator — restart pattern RESOLVED (post-tonight)
- 23:01 PT: restarts=12 (right after codex round-1 edits — initial spike)
- 23:48 PT: 14 (slow drip — looked like a 30-min cycle)
- 03:24 PT: 16 (continued slow drip)
- 04:32 PT: 16 — stabilized for 70+ min, no new restarts

Hypothesis: the periodic restarts were tied to whatever was triggering them at-the-time, possibly the round-3 codex calls or Steve's parallel codex-review-r2 work earlier. By the time both stopped, vf-orch settled. No code change needed; the apparent "30-min cycle" was coincidence with those external triggers.

If it starts incrementing again later, look for: cron jobs that touch /Users/stevestudio2/Projects/visual-factory/, watchdog scripts that pm2-restart on schedule, or a `process.exit()` call buried in a stage runner.