← back to The Ai Factory
PLAN.md
118 lines
# The AI Factory — PLAN.md
**Status:** scaffold (2026-04-30)
**Owner:** Steve
**Local path:** `~/Projects/the-ai-factory/`
**Ports:** orchestrator `9890`, viewer `9891`
**Pattern reference:** Site Factory (`~/Projects/site-factory/`, ports 9880/9881)
---
## What it is (working hypothesis — confirm before building features)
Orchestrator that turns a single prompt ("build me an X") into a **fully-shipped AI artifact**: a Claude-Code subagent, a `/skill`, a small standalone agent (pm2 service), or a one-shot script. The Factory owns the pipeline, the critic loop, and the registry — humans review the output, not every step.
This is a meta-factory. Site Factory builds sites. Hormuzy builds design. The AI Factory builds the AI things themselves.
> If this hypothesis is wrong, edit this section first — every other choice below flows from it.
## Pipeline (mirrors Site Factory's 11-stage pattern)
1. **Intake** — prompt → structured spec (artifact_type, name, surface, scope)
2. **Research** — pull adjacent skills/agents from `~/.claude/skills/` + `~/.claude/agents/` so we don't duplicate
3. **Scaffold** — write files (subagent .md, skill SKILL.md, agent index.js, etc.)
4. **Wire** — register with pm2 / register in MEMORY.md / add to skill index
5. **Smoke test** — invoke the artifact end-to-end against a canned input
6. **Critic (Manager pass)** — second-model review using `claude-api` skill rules: caching, model choice, prompt clarity
7. **Iterate** — apply critic notes, re-test
8. **Document** — README.md inside the artifact + one-line entry in this Factory's registry
9. **Memory write** — emit a memory entry for `MEMORY.md` so future sessions know it exists
10. **Activate** — flip status to `live`
11. **Watchdog hook** — if it's a long-running service, register with `process-hawk`
## Stack
- **Runtime:** Node 20 + Express
- **DB:** PostgreSQL local (`ai_factory` schema in `dw_unified` DB, or its own DB — TBD on first run)
- **Auth:** none initially (loopback only); add SSO via `dw-sso-integration` skill before exposing externally
- **LLM:** **NO Anthropic API** (Steve 2026-04-30: "if api costs money.. no"). Hybrid:
- **Ollama (local, free)** for mechanical stages — `qwen3:14b` for spec extraction / scaffold-template fill / doc generation. `gemma3:4b` for cheap classification.
- **Claude CLI subprocess** (`/Users/stevestudio2/.local/bin/claude -p ...`) for the Manager critic pass only. Goes through Steve's Max sub, not the API → no per-token charges. One call per run keeps quota burn near zero.
- Adapters live in `src/llm.js`.
- **Process manager:** pm2; service name `ai-factory-orchestrator` (`:9890`) and `ai-factory-viewer` (`:9891`)
- **Watchdog:** add to `com.steve.pd-hawk` style launchd OR `process-hawk` skill once stable
## File layout
```
the-ai-factory/
├── PLAN.md ← this file
├── README.md
├── package.json
├── .env.example ← ANTHROPIC_API_KEY, PG_*, ADMIN_TOKEN
├── server.js ← Express orchestrator on :9890
├── viewer.js ← static viewer on :9891 (matches sf-viewer)
├── src/
│ ├── pipeline.js ← 11-stage runner
│ ├── stages/ ← one file per stage
│ ├── critic.js ← Manager pass
│ └── registry.js ← what we've built
├── sql/
│ └── 001_init.sql ← ai_factory.runs / ai_factory.artifacts / ai_factory.events
├── public/
│ └── index.html ← viewer
└── scripts/
└── run.js ← CLI entry: `node scripts/run.js "build me a foo"`
```
## Decisions (2026-04-30)
1. **Artifact scope (Q1)** — START NARROW: Claude Code subagents (`~/.claude/agents/*.md`) + `/skills` (`~/.claude/skills/<name>/SKILL.md`) only. pm2 agents come in v2 once the critic loop is proven on the simpler artifacts. *(default — flag if you want broader from day 1)*
2. **DB home (Q2)** — **STANDALONE DB `ai_factory`** (not inside dw_unified — Steve flagged 2026-04-30: "be careful, create a new stand alone db"). Tables in `public`. Orchestrator boots with a hard safety guard that refuses `PG_DATABASE=dw_unified`.
3. **Public-facing? (Q3)** — **NO. All backroom internal.** Steve confirmed 2026-04-30. Loopback-only on Mac Studio 2 forever; no Cloudflare, no domain, no SSO, no Kamatera mirror. `127.0.0.1` binding stays.
4. **Trigger surface (Q4)** — viewer form (`:9891`) + CLI (`scripts/run.js`). A `/ai-factory` slash command is a nice-to-have once the v1 pipeline ships; not blocking. *(default)*
## Standing rules that apply
- Stays local on Mac Studio 2 (no cloud agents).
- **Loopback only** — `127.0.0.1` binding for both `:9890` and `:9891`. No external exposure ever.
- **NO Anthropic API** — Steve 2026-04-30. Use local Ollama + Claude CLI subprocess only.
- Any new long-running service → `pm2 start && pm2 save`.
- If the Factory generates a *site*, hand off to Site Factory / `dw-site-build`. Don't reimplement.
- If the Factory generates a *domain* + site, hand off to Onboard Domain Agent.
## Next actions
- [x] Decisions Q1–Q4 locked (2026-04-30)
- [x] MEMORY.md entry written
- [x] Standalone DB `ai_factory` created + schema applied
- [x] Orchestrator + viewer boot clean on `:9890` / `:9891`
- [x] No-API decision locked (hybrid Ollama + Claude CLI)
- [x] Stage 1 (intake) WIRED — qwen3:14b extracts spec as JSON
- [x] Stage 3 (scaffold) WIRED — qwen3:14b template-fills subagent .md OR skill SKILL.md, writes to `output/run-<id>/...`, frontmatter validated
- [x] Stage 6 (critic) WIRED — Claude CLI subprocess returns `{approved, score, issues, fix_instructions}` JSON
- [x] Stage 7 (iterate) WIRED — qwen3:14b applies fix_instructions, overwrites file (single pass)
- [x] Verified end-to-end on `shopify-cost-breakdown` skill prompt: 145s total, critic scored v1 at 62 → iterate applied 10 fixes → final SKILL.md is shippable. See `output/run-4/`.
- [x] **Re-critic loop WIRED** — after iterate we re-score v2; the run's final status reflects the revised review, not the v1 review. Verified on run-5 (`sql-tldr-summarizer`): v1=62/false → iterate(8 fixes) → v2=82/**true** → status=`awaiting_activation_v2`.
- [x] **Activate-with-approval WIRED** — `POST /runs/:id/activate` is the only path that touches `~/.claude/agents/` or `~/.claude/skills/`. Gates: (a) run status must start with `awaiting_activation` unless `?force=true`, (b) won't overwrite an existing target unless `?overwrite=true`, (c) refuses to copy from outside the sandbox dir. Viewer surfaces an Activate button on eligible runs.
- [x] **Stage 9 (memory_write) WIRED** — runs as part of the activate handler (best-effort; activation succeeds even if memory append fails). Appends one line to `MEMORY.md` of the form `- AI Factory shipped: ``<name>`` (<type>) — <scope> [run #N, YYYY-MM-DD, lives in ~/.claude/...]`. Skips if the same artifact name is already in the index. Verified on run-5: line appears, file lives at `~/.claude/skills/sql-tldr-summarizer/SKILL.md`, run status=activated.
- [x] **Stage 2 (research) WIRED** — pure-local bigram-Jaccard scan of `~/.claude/agents/` and `~/.claude/skills/` for similar names. Logs an `exact-name` warning if a conflict exists. Non-blocking; runs in parallel with scaffold (Promise.all).
- [x] **Stage 5 (smoke_test) WIRED** — pure-local structural validation (frontmatter, name match, body length, no unrendered placeholders, tools/model for subagents). Runs after intake's first-pass scaffold; re-runs on v2 if iterate fired.
- [x] **Stage 8 (document) WIRED** — for skills only, drops a `README.md` next to `SKILL.md` in the artifact dir. Subagents skipped (single-file, self-documenting). Runs in parallel with critic.
- [x] **Stage 4 (wire) explicit no-op** — for v1 subagent/skill artifacts there's no external registration step (the harness auto-discovers). Logged as no-op for clarity.
- [x] **Stage 11 (watchdog_hook) explicit no-op** — only relevant for v2 service-type artifacts.
- [x] **Pipeline parallelization** — research‖scaffold and document‖critic run as `Promise.all`. Should shave ~10–30s off the ~145s baseline (qwen3:14b scaffold ≈30s parallel with research's instant local read; document's instant write parallel with critic's ~25s Claude CLI call).
- [x] **Viewer drill-down** — clicking a run row expands it inline to show the full event timeline (stage, name, level, message, payload as collapsible JSON). No separate page.
- [ ] Confirm working hypothesis (one-line edit at top) — Steve
- [ ] Stage 8 (document) — generate a README inside the artifact dir (skills only)
- [ ] Stage 11 (watchdog_hook) — for service-type artifacts only (v2)
- [ ] Viewer drill-down — per-run event timeline (currently only lists runs)
## Out of scope (per Q3 = backroom internal)
- ~~Domain registration~~
- ~~SSO via dw-sso-integration~~
- ~~Cloudflare zone~~
- ~~Kamatera mirror~~
- ~~theaifactory.com~~