tick 6: README pass — covers ticks 2-5b features (classifier, queue, chat, idea-bar, env vars, endpoints)

4292cad79cf44ac098659bf606ff2d5e4e990a98 · 2026-05-11 12:31:28 -0700 · SteveStudio2
Files touched

M README.md
Diff

commit 4292cad79cf44ac098659bf606ff2d5e4e990a98
Author: SteveStudio2 <steve@designerwallcoverings.com>
Date:   Mon May 11 12:31:28 2026 -0700

    tick 6: README pass — covers ticks 2-5b features (classifier, queue, chat, idea-bar, env vars, endpoints)
---
 README.md | 73 ++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 53 insertions(+), 20 deletions(-)

diff --git a/README.md b/README.md
index 3d4a094..3aba639 100644
--- a/README.md
+++ b/README.md
@@ -1,16 +1,20 @@
 # cncp-failure-collector
 
-Tails pm2 + launchd for failures and posts them as CNCP tasks. Decided 2026-05-11 via best-practices reviewer + 4-LLM debate team consensus on architecture option B (dedicated `cncp-tasks.json` + tasks panel + collector daemon), tick-1 scope = data pipeline only.
+Tails pm2 + launchd for failures and posts them as CNCP failure-ledger rows. Local-only, pm2-managed, fail-soft.
+
+Decided 2026-05-11 via best-practices reviewer + 4-LLM debate-team consensus on architecture option B. Six iterative ticks shipped 2026-05-11.
 
 ## How it works
 
 Every `POLL_INTERVAL_MS` (default 30 s) the daemon:
 
-1. Runs `pm2 jlist`, diffs against `state.json` (kept in this dir). Any process where `restart_time` bumped or status flipped away from `online` is reported as a failure.
-2. Reads new bytes from `~/Library/Logs/com.steve.*.err.log` files (per-file offsets in `state.json` so a daemon restart doesn't re-post old chunks).
-3. POSTs each event to `http://127.0.0.1:3333/api/failures`. CNCP has a 60-s server-side dedup window per `(source, processName)` to absorb storm bursts.
+1. **pm2 scan** — runs `pm2 jlist`, diffs against `state.json`. Any process where `restart_time` bumped or status flipped away from `online` is posted. Bootstrap-silent (first run seeds baseline, no historical-restart spam). Skips itself.
+2. **launchd plist tailing** — scans `~/Library/LaunchAgents/com.steve.*.plist`, plutil-parses each for `StandardErrorPath` (or `StandardOutPath` fallback), builds a watchlist of ~97 jobs. Refreshed every 10 min. Reads only new bytes since last cycle. Caps at 200 watched files; skips files larger than 100 MB.
+3. **POST to CNCP** at `http://127.0.0.1:3333/api/failures`. CNCP applies sticky dedup (collapse repeat events into the same open row; only `resolved`/`dismissed` rows fork a new row on next event).
+4. **Auto-classify (fire-and-forget)** — on a NEW row response (`deduped:false`), the collector POSTs `/api/failures/<id>/classify` with no body. Per-process 5-min rate limit (in-memory `Map`). Server-side: single classifier in flight, FIFO queue (cap 10) drains on completion. Hermes 3 on **localhost** (Steve override 2026-05-11) returns severity / category / suggestedAction in ~8 s.
+5. **Heartbeat** every 10 quiet cycles (~5 min): logs `heartbeat — N quiet cycles, X pm2 tracked, Y logs tracked`.
 
-State file `state.json` is gitignored — it's per-machine runtime data.
+State (last restart counts, log byte offsets, plist scan timestamp) lives in `state.json` in this dir — gitignored, per-machine runtime data.
 
 ## Run
 
@@ -19,26 +23,55 @@ pm2 start ecosystem.config.js
 pm2 save
 ```
 
-One-shot dev test (no loop, just one scan):
+One-shot dev test:
 
 ```bash
 npm run once
 ```
 
-## Tick 2 (NOT in this tick — scoped out by debate team)
+## Environment
+
+| Var | Default | Purpose |
+|-----|---------|---------|
+| `CNCP_URL` | `http://127.0.0.1:3333` | Where to POST failures + trigger classify |
+| `POLL_INTERVAL_MS` | `30000` | Poll cadence (pm2 + launchd) |
+| `CLASSIFIER_ENABLED` | `1` | Set `0` to disable fire-and-forget classify (manual classify-by-UI still works) |
+
+CNCP-side env (in CNCP's own start, not this collector):
+
+| Var | Default | Purpose |
+|-----|---------|---------|
+| `HERMES_URL` | `http://127.0.0.1:11434` | Local Ollama for the classifier (Steve override of MS1 default) |
+| `HERMES_MODEL` | `hermes3:8b` | Pulled on MS2 + MS1 |
+| `GEORGE_ALERTS_ENABLED` | `0` (DRY-RUN) | Set `1` to actually email Steve on severity=high (1-hour throttle per processName via `cncp-alert-throttle.json`) |
+
+## CNCP endpoints
+
+- `GET /api/failures[?status=...&source=...]` — list, optionally filtered.
+- `POST /api/failures` — append (with sticky dedup); body `{source, processName, exitCode?, lastLines?, restartCount?}`.
+- `PATCH /api/failures/:id` — update status / severity / category / suggestedAction.
+- `DELETE /api/failures/:id` — remove.
+- `POST /api/failures/:id/classify` — synchronous (15 s timeout, single-flight queue). Hermes 3 returns `{category, severity, suggestedAction}`. Fail-soft: returns 200 with `classified:false` on any error.
+- `POST /api/failures/:id/chat` — refinement chat. Body `{message}`. Hermes can emit a trailing `{refine: {severity, category, suggestedAction}}` JSON line which auto-PATCHes the row.
+- `POST /api/idea-chat` — global "suggest an idea to build out" prompt; body `{prompt, savePin?}`. SavePin drops the outline into the Parking Lot.
+- `POST /api/failures/cleanup-transients` — manual trigger; the same logic runs 5 s after CNCP startup and every 30 min automatically. Dismisses `triage`/`low|medium`/`transient` rows quiet for 24 h.
+
+## CNCP UI
+
+`http://localhost:3333/` → **Failures** nav button (with open-count badge).
+
+- Card grid, sort + density slider (Steve standing-rule pattern).
+- Per-card: source + severity + category badges, processName, status dropdown (triage / investigating / resolved / dismissed), event count, last-output snippet, italic suggestedAction line.
+- Per-card **Refine** button — opens an inline Hermes chat thread. Persisted in `task.messages[]`.
+- Cross-top **idea-chat bar** — free-form "suggest an idea to build out" input. "Save to Lot" routes to Parking Lot.
+
+## Sidebar
 
-- Hermes 3 (`hermes3:8b`) classifier on **MS1 `http://192.168.1.133:11434`**, NOT localhost. Per-process-name rate limit.
-- POST `/api/failures/:id/classify` returns `{category, severity, suggestedAction}` as async enrichment, not inline.
-- CNCP Failures panel UI (sidebar entry + render card).
-- Optional: outbound webhook to George for high-severity tasks.
-- **launchd error-log tailing** — Steve's convention is per-project paths like `~/Projects/animals/logs/hawk.stderr.log`, NOT `~/Library/Logs/com.steve.*.err.log`. Tick 2 needs to: parse every `~/Library/LaunchAgents/com.steve.*.plist` for `StandardErrorPath`, build a watchlist, tail each. Current code scans `~/Library/Logs/` which is the wrong location for Steve's stack — it no-ops harmlessly today.
+Failures shell shows `N open / M total` with top-6 source-badged process names; click to open the main view.
 
-## Reviewer flags from tick 1 design (all addressed)
+## What's NOT in here
 
-- Local-first: writes only to `127.0.0.1:3333`, no cloud egress.
-- No `ANTHROPIC_API_KEY` read.
-- pm2 poll + per-file tail (not `pm2 logs --json` stream).
-- 60-s server-side dedup window per `(source, processName)`.
-- Daemon survives CNCP being down (retries each cycle).
-- Hardened pm2 entry: max-restarts 10, min-uptime 30 s, exp backoff, 200 M memory cap.
-- Gitified before writing code.
+- **Cloud agents** — local-only by Steve's standing rule.
+- **Anthropic API** — never. All inference via local Ollama.
+- **Auto-restart** — that stays with `process-hawk` + per-project hawks. This is the observability ledger.
+- **Email sending in production** — `GEORGE_ALERTS_ENABLED` defaults `0`. Flip when comfortable.
← 8a79514 tick 4: fire-and-forget classify on new rows (5min per-proce · back to Cncp Failure Collector · tick 7: prune classifier rate-limit Map of entries older tha 78ad17a →