`ROTATION-CHECKLIST.md`

113 lines
# Credential Rotation Checklist

> Origin: 2026-05-30 Kamatera compromise. These are **console-only** actions —
> Steve mints/changes the secret by hand, then routes it via `cli.js import-paste`.
> Claude cannot mint keys or run prod credential changes. **No secret VALUES go
> in this file** — only key names, mint URLs, and the route-back command.
> Queued 2026-06-01.

---

## 1. PG `dw_admin` password  ⬜ OUTSTANDING  (P1 — the actual compromise fix)

> **Re-verified 2026-06-21 (read-only): STILL GENUINELY OUTSTANDING — not stale.**
> Proof the rotation never landed: the old leaked plaintext `DW2024SecurePass`
> is still live in all 4 local consumer `.env.local` DATABASE_URL DSNs
> (Norma/sdcc, StudentLoanTracker/dw_unified, ClawCoder/clawcoder,
> ImportNewSkufromURL/dw_unified — all 127.0.0.1) and those apps connect
> cleanly → the local Mac2 `dw_admin` role STILL uses the compromised pw.
> The canonical fan key `PG_DW_ADMIN_PASSWORD` has **zero value** in the secrets
> registry (only the two dead aliases `DW_ADMIN_PG_PASSWORD` fWdg:1e3383c0 +
> `DW_ADMIN_DB_PASSWORD` 22WF:7397bbbe carry digests — master+desktop only,
> never fanned). `routes.json` now carries `PG_DW_ADMIN_PASSWORD` scaffolding
> (4 env_file dests) — **prep done, but no secret behind it; cli.js has never
> fanned dw_admin.** Prod role pw is SCRAM/unreadable; no evidence prod rotated.
> → The CNCP compromised flag (cncp server.js `out.prod.status:'compromised'`)
> stays SET. DTD 2026-06-21 verdict A, unanimous 3/3. Execute via
> `DW-ADMIN-ROTATION-RUNBOOK-2026-06-19.sh` (Steve-run, step-gated).

- **Exposed:** literal value was hardcoded in 14+ files on disk (scrubbed locally), and lives in Time Machine snapshots / Spotlight index / possibly remote git history.
- **Runbook:** `~/Projects/_shared/docs/pg-rotation-runbook.md` (~32 min, includes the Kamatera mirror — prod + local both).
- **Why it's the big one:** `dw_admin` is the shared admin used by wallco, the DW fleet, Norma's sdcc, etc. The Norma scoped rotation below does **not** fix this — it only moves Norma *off* `dw_admin`. `dw_admin` itself stays compromised until this runs.
- **Console steps:** pick a new ≥20-char pw → `ALTER ROLE dw_admin WITH PASSWORD '…'` on **prod Kamatera** (`sudo -u postgres psql`) AND **local Mac2** → restart every consumer (pm2 fleet) → fan out.
- **Route back after change:**
  ```
  printf 'DW_ADMIN_DB_PASSWORD=<NEW_PW>' | node ~/Projects/secrets-manager/cli.js import-paste
  ```
- **Verify:** `node ~/Projects/secrets-manager/cli.js check` (and confirm pm2 apps reconnect).

## 2. Gemini API key #1  ⬜ OUTSTANDING

- **Exposed:** hardcoded in 7 Patty routes + 3 DW docs.
- **Mint:** https://aistudio.google.com/apikey  → create new key, **delete the old one** in the same console.
- **Secret key name:** `GEMINI_API_KEY`  (confirm against `routes.json`).
- **Route back:**
  ```
  printf 'GEMINI_API_KEY=<NEW_KEY>' | node ~/Projects/secrets-manager/cli.js import-paste
  ```
- **Verify:** `node ~/Projects/secrets-manager/cli.js check` (hits Gemini /models to validate).

## 3. Gemini API key #2  ⬜ OUTSTANDING

- **Exposed:** hardcoded in `PoppyPetitions/lib/gemini.ts`.
- **Mint:** https://aistudio.google.com/apikey  → new key, delete old.
- **Secret key name:** confirm which route this maps to (`GEMINI_API_KEY_RALPH` or a poppy-specific key in `routes.json`).
- **Route back:**
  ```
  printf '<KEY_NAME>=<NEW_KEY>' | node ~/Projects/secrets-manager/cli.js import-paste
  ```
- **Verify:** `node ~/Projects/secrets-manager/cli.js check`.

---

## Related — Norma sdcc scoped rotation  (DTD-cleared 2026-06-01: Lane A)

Not a console mint — a scoped PG role swap. **DTD verdict: A (scoped-now), 2/2 valid panelists.**
Pre-flight (read-only, 2026-06-01) confirmed safe to run:
- `~/Projects/Norma/.env.local` exists ✓
- Norma currently connects as **`dw_admin`** on local `sdcc` → this moves it to a least-privilege scoped user ✓
- `norma_app_user` does **not** exist yet → script CREATEs it; no other consumer uses it, so the "scoped is illusory" caveat does NOT apply ✓
- Leaves shared `dw_admin` untouched (so the 5 social procs + sdcc-images viewer + replication keep working)

```
~/Projects/_shared/scripts/rotate-pg-scoped.sh ~/Projects/Norma sdcc norma_app_user NORMA_APP_DB_PASSWORD
```
**Caveats:** (a) the script is **LOCAL-only** (`127.0.0.1:5432`) — it does not touch prod Kamatera; (b) it's **interactive** (type the new ≥16-char pw twice at the prompt — the header's "pipe via stdin" is stale); (c) it does **not** rotate `dw_admin` — that's item #1 above.

---

## Newly identified (2026-06-03)

## 4. `SHOPIFY_ADMIN_TOKEN`  ✅ RESOLVED  (re-verified 2026-06-21 — was STALE)

> **Reconciled 2026-06-21 (read-only): this item was STALE — the token WORKS.**
> `SHOPIFY_ADMIN_TOKEN` (last4 `75b9`, registry digest `75b9:95b01e9b`) returns
> **HTTP 200** against `/admin/api/2024-04/shop.json`, and products were created
> with it this session. The "undefined / burned / no push since Apr 30" text
> below was the 2026-06-03 snapshot and no longer holds. NOTE: a working Shopify
> token is a **separate credential** and says nothing about the `dw_admin`
> compromise (item #1) — it does NOT justify clearing the compromised flag.
> (Any residual "no push" symptom is an env-sourcing/cron-wiring issue per the
> full-monte notes, not a dead token.)

- **(historical, 2026-06-03 — superseded by the note above)** likely the burned post-bubbe token. The full-monte Phase 8 catalog push reads `process.env.SHOPIFY_ADMIN_TOKEN` and currently gets `undefined` → every push auth-fails silently (catch-up "succeeded" counts are error-caught lies, 0 real writes). DW catalog has not pushed to Shopify since ~Apr 30.
- **Mint:** Shopify admin → Apps → (custom app) → API credentials → revoke old Admin API access token, install/reveal a new one.
- **Secret key name:** `SHOPIFY_ADMIN_TOKEN` (confirm against `routes.json`).
- **Route back:**
  ```
  printf 'SHOPIFY_ADMIN_TOKEN=<NEW_TOKEN>' | node ~/Projects/secrets-manager/cli.js import-paste
  ```
  Then ensure the full-monte env actually sources it (sourced `.env` or a crontab global `SHOPIFY_ADMIN_TOKEN=` line) — Phase 8 has no `.env` and cron doesn't export it.
- **Verify:** `node ~/Projects/secrets-manager/cli.js check` + a single Phase-8 dry push shows a real 200/201 write.

## 5. PG `repl_user` replication password  ⬜ OUTSTANDING  (unblocks the frozen dw_unified mirror)

- **Exposed / status:** the password lived only inside the Mac2 subscription's `subconninfo`, which was removed when the broken subscription was dropped 2026-06-03 — so it's now unavailable locally. Not a leak; a loss. The dw_unified mirror (Vendors/edges/settlement panels) stays frozen until this is reset and the subscription recreated.
- **Console steps (Kamatera publisher):** `ALTER ROLE repl_user WITH PASSWORD '<NEW_PW>';` (also clear the stray `spoon_all_designs_legacy_20260601` table from publication `dw_unified_pub` — see `~/cncp-starter/yolo-queue/pending-approval/mirror-resync-FIX.md`).
- **Secret key name:** propose `REPL_USER_DB_PASSWORD` (new route — does not exist in routes.json yet).
- **Route back:**
  ```
  printf 'REPL_USER_DB_PASSWORD=<NEW_PW>' | node ~/Projects/secrets-manager/cli.js import-paste
  ```
- **Verify:** rerun `REPL_PW=<NEW_PW> bash ~/cncp-starter/yolo-queue/run-mirror-resync.sh`; `max(synced_at)` jumps to ~now + the CNCP mirror-alert banner clears.