← back to Cli Printing Press
feat(cli): generator pipeline improvements — auth inference, verify env, sync paths (#103)
6da6fd098fa6dad43074794157cbb2b265979a30 · 2026-04-01 08:27:31 -0700 · Trevin Chow
* feat(cli): infer query-param auth from parameter names when spec lacks securitySchemes
If >30% of operations have a query parameter named key/api_key/apikey/
access_token/token and no securitySchemes is declared, infer query-param
auth. Sets Auth.In="query" and Auth.Header to the most common param name.
Guard: only infers when primary detection found no auth scheme.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): verify passes all auth env vars from spec during testing
RunVerify now iterates spec.Auth.EnvVars instead of using a single
hardcoded env var. In mock mode, all auth env vars get mock-token.
In live mode, all set env vars are passed through to subprocesses.
Extracted env-building into a closure to eliminate duplication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): sync path resolution — profiler stores actual endpoint paths
SyncableResources changed from []string to []SyncableResource{Name, Path}.
The profiler now captures the actual endpoint path (e.g., /ISteamApps/
GetAppList/v2/) instead of just the resource name. The sync template
generates a syncResourcePath() lookup map so sync calls the correct URL.
Fixes sync for non-REST APIs like Steam where resource names don't match
API paths. REST APIs are unaffected — their stored path is /<resource>.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(cli): Steam run 3 retro and generator pipeline improvements plan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): Codex review fixes — workflow template, verify api-key injection, deterministic sync
1. Workflow template: use {{.Name}} not {{.}} for SyncableResource entries
so channel_workflow.go.tmpl renders resource names, not struct literals.
2. Verify live mode: inject --api-key under ALL spec auth env var names,
not just the derived envVarName. Fixes live verify for CLIs that read
STEAM_API_KEY when the derived name is STEAM_WEB_TOKEN.
3. Sync path: pick shortest path when multiple paginated list endpoints
exist for the same resource, ensuring deterministic generation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(ci): hide lefthook meta banner, keep summary and results
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Files touched
A docs/plans/2026-04-01-002-fix-generator-pipeline-improvements-plan.mdA docs/retros/2026-04-01-steam-run3-retro.mdM internal/generator/generator.goM internal/generator/templates/channel_workflow.go.tmplM internal/generator/templates/sync.go.tmplM internal/llmpolish/vision.goM internal/llmpolish/vision_test.goM internal/openapi/parser.goM internal/pipeline/runtime.goM internal/profiler/profiler.goM internal/profiler/profiler_test.goM lefthook.yml
Diff
commit 6da6fd098fa6dad43074794157cbb2b265979a30
Author: Trevin Chow <trevin@trevinchow.com>
Date: Wed Apr 1 08:27:31 2026 -0700
feat(cli): generator pipeline improvements — auth inference, verify env, sync paths (#103)
* feat(cli): infer query-param auth from parameter names when spec lacks securitySchemes
If >30% of operations have a query parameter named key/api_key/apikey/
access_token/token and no securitySchemes is declared, infer query-param
auth. Sets Auth.In="query" and Auth.Header to the most common param name.
Guard: only infers when primary detection found no auth scheme.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): verify passes all auth env vars from spec during testing
RunVerify now iterates spec.Auth.EnvVars instead of using a single
hardcoded env var. In mock mode, all auth env vars get mock-token.
In live mode, all set env vars are passed through to subprocesses.
Extracted env-building into a closure to eliminate duplication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): sync path resolution — profiler stores actual endpoint paths
SyncableResources changed from []string to []SyncableResource{Name, Path}.
The profiler now captures the actual endpoint path (e.g., /ISteamApps/
GetAppList/v2/) instead of just the resource name. The sync template
generates a syncResourcePath() lookup map so sync calls the correct URL.
Fixes sync for non-REST APIs like Steam where resource names don't match
API paths. REST APIs are unaffected — their stored path is /<resource>.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(cli): Steam run 3 retro and generator pipeline improvements plan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): Codex review fixes — workflow template, verify api-key injection, deterministic sync
1. Workflow template: use {{.Name}} not {{.}} for SyncableResource entries
so channel_workflow.go.tmpl renders resource names, not struct literals.
2. Verify live mode: inject --api-key under ALL spec auth env var names,
not just the derived envVarName. Fixes live verify for CLIs that read
STEAM_API_KEY when the derived name is STEAM_WEB_TOKEN.
3. Sync path: pick shortest path when multiple paginated list endpoints
exist for the same resource, ensuring deterministic generation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(ci): hide lefthook meta banner, keep summary and results
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---
...002-fix-generator-pipeline-improvements-plan.md | 240 +++++++++++++++++++++
docs/retros/2026-04-01-steam-run3-retro.md | 105 +++++++++
internal/generator/generator.go | 8 +-
.../generator/templates/channel_workflow.go.tmpl | 2 +-
internal/generator/templates/sync.go.tmpl | 19 +-
internal/llmpolish/vision.go | 6 +-
internal/llmpolish/vision_test.go | 9 +-
internal/openapi/parser.go | 76 ++++++-
internal/pipeline/runtime.go | 44 ++--
internal/profiler/profiler.go | 49 ++++-
internal/profiler/profiler_test.go | 6 +-
lefthook.yml | 6 +
12 files changed, 538 insertions(+), 32 deletions(-)
diff --git a/docs/plans/2026-04-01-002-fix-generator-pipeline-improvements-plan.md b/docs/plans/2026-04-01-002-fix-generator-pipeline-improvements-plan.md
new file mode 100644
index 00000000..03c73af2
--- /dev/null
+++ b/docs/plans/2026-04-01-002-fix-generator-pipeline-improvements-plan.md
@@ -0,0 +1,240 @@
+---
+title: "fix: Generator pipeline improvements — domain Search, auth inference, verify env, sync paths"
+type: fix
+status: active
+date: 2026-04-01
+origin: docs/retros/2026-04-01-steam-run3-retro.md
+---
+
+# Fix: Generator Pipeline Improvements
+
+## Overview
+
+Four generalized machine improvements carried forward from three Steam retros. Each affects every future generated CLI, not just Steam. Domain-specific Search in the store template (+3 scorecard), auth inference from query params (+2 scorecard), verify auth env passthrough (+5% verify), and sync path resolution for non-REST APIs (fixes broken sync).
+
+## Problem Frame
+
+At 85/100, the Steam CLI's remaining gaps are in the generator and verify pipeline — not the scorer or the CLI itself. The store generates Upsert per entity but only generic Search. The OpenAPI parser misses auth when specs lack `securitySchemes`. Verify can't test auth-requiring commands because it doesn't pass env vars. Sync builds paths from resource names instead of actual endpoints.
+
+(see origin: docs/retros/2026-04-01-steam-run3-retro.md, findings #1, #2, #3, #6)
+
+## Requirements Trace
+
+- R1. Generated store has domain-specific Search methods (e.g., `SearchPlayers()`) alongside Upsert methods
+- R2. OpenAPI parser infers query-param auth when >30% of operations have a `key`/`api_key` parameter
+- R3. Verify passes all auth env vars from the spec during command testing
+- R4. Sync uses actual endpoint paths from the profiler, not `"/" + resourceName`
+
+## Scope Boundaries
+
+- Not changing the scorecard (PRs #100-102 already fixed scorer issues)
+- Not changing the retro skill
+- Not fixing flag description quality (depends on spec content)
+- Not adding sync path params bonus (inherent to API structure)
+
+## Context & Research
+
+### Relevant Code and Patterns
+
+**Store Search:**
+- `store.go.tmpl:341-372` — already generates `Search{{pascal .Name}}()` for tables with FTS5. The template iterates `{{range .Tables}}{{if .FTS5}}`. The issue: the scorecard checks for `\.Search[A-Z]` patterns, but the generated CLI may not have searchable tables or the profiler may not flag enough fields.
+- `profiler.go:533-548` — `collectStringFields()` identifies searchable fields. `SearchableFields` map is passed to the generator.
+
+**Auth Inference:**
+- `parser.go:240-307` — `mapAuth()` extracts security schemes. Returns early if no scheme found. The fallback scan should go here.
+- `spec.go:27-38` — `AuthConfig` struct already has `In` field for query-param auth.
+
+**Verify Env:**
+- `runtime.go:123-147` — `RunVerify()` constructs env with one token var. Needs to iterate `spec.Auth.EnvVars`.
+- `runtime.go:462` — `runCommandTests()` receives `env []string` and passes to subprocess.
+
+**Sync Path:**
+- `profiler.go:63` — `SyncableResources []string` stores only names.
+- `profiler.go:141` — adds resource name to syncable map but discards the endpoint path.
+- `sync.go.tmpl:178` — `path := "/" + resource` is the broken path construction.
+- `spec.go:53` — `Endpoint.Path` has the actual API path.
+
+### Prior Art
+
+- PRs #100-102 established the retro→plan→implement pattern. Each fix follows: read the code, understand the gap, make a targeted change, test with existing fixtures + new test cases.
+
+## Key Technical Decisions
+
+- **SyncableResources struct change:** Change from `[]string` to `[]SyncableResource{Name, Path}`. This ripples through the profiler, generator data struct, and sync template — but each change is mechanical. The profiler already has the endpoint path available at the point where it adds to syncable.
+
+- **Auth inference threshold: 30%** — If >30% of operations have a `key`/`api_key` query param, infer auth. This avoids false positives on APIs where only 1-2 endpoints accept optional keys. Steam has 47/158 (30%) — right at the threshold.
+
+- **Verify env: iterate spec.Auth.EnvVars** — The verify runtime already passes one env var. Extending to iterate the full list is a small change. The verify config already carries the spec.
+
+## Open Questions
+
+### Deferred to Implementation
+
+- **Exact FTS5 table generation logic** — The store template already generates FTS5 tables for some resources. Need to trace when the profiler marks a table as having FTS5 vs not, to understand whether domain-specific Search methods are already emitted and just not used, or not emitted at all.
+- **Auth inference: what happens when both securitySchemes AND query-param keys exist** — The guard should be: only infer if securitySchemes detection produced `Type: "none"`. If bearer/oauth was already detected, don't override.
+
+## Implementation Units
+
+- [ ] **Unit 1: Domain-specific Search methods in store template**
+
+**Goal:** Every generated store emits `Search<Entity>()` methods for entities with searchable fields, enabling the scorecard's data_pipeline +3.
+
+**Requirements:** R1
+
+**Dependencies:** None
+
+**Files:**
+- Modify: `internal/generator/templates/store.go.tmpl`
+- Modify: `internal/profiler/profiler.go` (if SearchableFields needs enhancement)
+- Test: `internal/profiler/profiler_test.go`
+
+**Approach:**
+- The store template already has `Search{{pascal .Name}}` generation gated on `{{if .FTS5}}`. Verify this is working and producing the `\.Search[A-Z]` pattern the scorecard looks for.
+- If the profiler's `SearchableFields` is too narrow (missing resources), expand `collectStringFields` to include more field types.
+- If the template generates the methods but they're not detected by the scorecard, check the scorecard's regex against the actual generated code.
+
+**Patterns to follow:**
+- Existing `Upsert{{pascal .Name}}` pattern in `store.go.tmpl`
+
+**Test scenarios:**
+- Happy path: Generate from Petstore spec → store has `SearchPets()` method (pets have string name/status fields)
+- Happy path: Generate from Discord spec → store has `SearchMessages()` (messages have content field)
+- Edge case: Spec with no string fields → no Search methods emitted (no crash)
+- Integration: Run scorecard on generated CLI → data_pipeline_integrity ≥9/10
+
+**Verification:**
+- Generated store.go contains `func (s *Store) Search<Entity>` for entities with searchable fields
+- Scorecard data_pipeline score improves
+
+---
+
+- [ ] **Unit 2: Auth inference from query parameter names**
+
+**Goal:** OpenAPI parser infers query-param auth when specs lack `securitySchemes` but have common auth param names
+
+**Requirements:** R2
+
+**Dependencies:** None
+
+**Files:**
+- Modify: `internal/openapi/parser.go` (`mapAuth` function)
+- Test: `internal/openapi/parser_test.go`
+
+**Approach:**
+- In `mapAuth()`, after `selectSecurityScheme()` returns nil (no scheme found):
+- Walk all operations in `doc.Paths`, collect query params matching common auth names
+- Auth param names to check: `key`, `api_key`, `apikey`, `access_token`, `token`
+- Count how many operations have at least one auth-like query param
+- If count / total operations > 0.3 (30%), set `auth.Type = "api_key"`, `auth.In = "query"`, `auth.Header = <most_common_param_name>`
+- Guard: only infer if the primary detection found no auth (`auth.Type == "" || auth.Type == "none"`)
+
+**Patterns to follow:**
+- Existing `mapAuth` flow: extract scheme → map to AuthConfig. The fallback scan is a new step after the primary detection.
+
+**Test scenarios:**
+- Happy path: Steam-like spec with no securitySchemes but `key` param on 47/158 operations → auth detected as `in: query, header: key`
+- Happy path: Stripe spec with explicit bearer auth → fallback does NOT run (guard works)
+- Edge case: Spec with `key` param on only 2/100 operations → no auth inferred (below 30%)
+- Edge case: Spec with both `key` and `api_key` params → uses the more common one
+- Negative: Spec with no query params at all → no auth inferred, no crash
+
+**Verification:**
+- Parse Steam public spec → `Auth.In == "query"` and `Auth.Header == "key"`
+- Parse Stripe spec → bearer auth unchanged
+- Scorecard auth score improves for specs with undeclared query-param auth
+
+---
+
+- [ ] **Unit 3: Verify auth env passthrough**
+
+**Goal:** Verify passes all auth-related env vars from the spec to command test subprocesses
+
+**Requirements:** R3
+
+**Dependencies:** None (independent, but benefits from Unit 2 which provides more auth env vars)
+
+**Files:**
+- Modify: `internal/pipeline/runtime.go` (`RunVerify` function, env construction)
+- Test: `internal/pipeline/runtime_test.go`
+
+**Approach:**
+- In `RunVerify`, where the env is constructed (lines 123-147), iterate `spec.Auth.EnvVars` instead of using a single hardcoded env var name
+- For each env var in the list, check if it's set in the current environment and pass it through
+- In mock mode, pass all auth env vars with `mock-token-for-testing` value
+- The verify config already carries the spec — just need to read `cfg.Spec.Auth.EnvVars`
+
+**Patterns to follow:**
+- Existing env construction pattern at lines 123-147
+
+**Test scenarios:**
+- Happy path: Spec with `Auth.EnvVars: ["STEAM_API_KEY"]` → env includes `STEAM_API_KEY` during test
+- Happy path: Spec with multiple env vars `["DISCORD_TOKEN", "DISCORD_BOT_TOKEN"]` → both passed
+- Edge case: No env vars in spec → env construction unchanged (no crash)
+- Integration: Run verify on Steam CLI with STEAM_API_KEY set → auth-requiring commands pass dry-run
+
+**Verification:**
+- Verify pass rate improves for CLIs with auth (75% → ~95% for Steam)
+
+---
+
+- [ ] **Unit 4: Sync path resolution for non-REST APIs**
+
+**Goal:** Sync uses actual endpoint paths from the profiler instead of naive `"/" + resource` construction
+
+**Requirements:** R4
+
+**Dependencies:** None
+
+**Files:**
+- Modify: `internal/profiler/profiler.go` (`SyncableResources` type, population logic)
+- Modify: `internal/generator/generator.go` (data struct passed to sync template)
+- Modify: `internal/generator/templates/sync.go.tmpl` (use stored path)
+- Test: `internal/profiler/profiler_test.go`
+
+**Approach:**
+- Define a `SyncableResource` struct with `Name` and `Path` fields in `profiler.go`
+- Change `SyncableResources` from `[]string` to `[]SyncableResource`
+- In the profiler's syncable detection (line 141), when adding a resource, also store the endpoint's `Path` field
+- Update the generator's data struct to pass the new type to the sync template
+- In `sync.go.tmpl`, replace `path := "/" + resource` with the stored path
+- Keep `resource` as the display name / sync state key — only the API path changes
+
+**Patterns to follow:**
+- Existing `TableDef` struct pattern in `generator.go` — struct with Name + metadata passed to templates
+
+**Test scenarios:**
+- Happy path: Steam spec with `/ISteamApps/GetAppList/v2/` → sync uses actual path, not `/isteam-apps`
+- Happy path: Stripe spec with `/customers` → sync still uses `/customers` (REST path = resource name)
+- Edge case: Resource with no list endpoint → not in SyncableResources (no crash)
+- Negative: Profiler output for Petstore → SyncableResources have correct paths
+
+**Verification:**
+- Generate from Steam spec → `sync --resources isteam-apps` calls `/ISteamApps/GetAppList/v2/`
+- Generate from Stripe/Petstore spec → sync paths unchanged
+
+## System-Wide Impact
+
+- **Store template (Unit 1):** Adds methods to every generated store. Existing CLIs unaffected. New CLIs get more methods — compilation is additive (new functions, no changed signatures).
+- **Auth inference (Unit 2):** Changes how the parser populates `AuthConfig`. Affects the generated `config.go` and `client.go` for APIs with undeclared auth. APIs with declared auth are unaffected (guard).
+- **Verify env (Unit 3):** Changes verify's subprocess environment. More env vars passed = more commands can complete during testing. No regression risk — additional env vars don't break commands that don't use them.
+- **Sync path (Unit 4):** Changes the `SyncableResources` type from `[]string` to `[]SyncableResource`. This ripples through profiler → generator → template. The struct carries the same data plus the path — no data is lost.
+- **Unchanged:** Scorecard dimensions, dogfood detection, verify command discovery, template output for non-sync commands.
+
+## Risks & Dependencies
+
+| Risk | Mitigation |
+|------|------------|
+| Auth inference false positive (infers auth on public API) | 30% threshold + guard against overriding detected auth |
+| SyncableResource struct change breaks profiler tests | Update profiler_test.go assertions to use the new struct |
+| Store Search methods already emitted but not scoring | Investigate before implementing — may be a scorecard detection gap |
+| Sync template change breaks existing REST CLIs | Store the REST path (`/customers`) as-is — only non-REST CLIs get different paths |
+
+## Sources & References
+
+- **Origin:** [docs/retros/2026-04-01-steam-run3-retro.md](docs/retros/2026-04-01-steam-run3-retro.md) — findings #1, #2, #3, #6
+- **Prior retros:** [docs/retros/2026-03-31-steam-retro.md](docs/retros/2026-03-31-steam-retro.md) (findings #5, #6, #8), [docs/retros/2026-03-31-steam-run2-retro.md](docs/retros/2026-03-31-steam-run2-retro.md) (findings #8, #11)
+- Store template: `internal/generator/templates/store.go.tmpl`
+- Auth parser: `internal/openapi/parser.go` (`mapAuth`)
+- Verify runtime: `internal/pipeline/runtime.go` (`RunVerify`)
+- Profiler: `internal/profiler/profiler.go` (`SyncableResources`)
+- PRs #100-102 (mvanhorn/cli-printing-press)
diff --git a/docs/retros/2026-04-01-steam-run3-retro.md b/docs/retros/2026-04-01-steam-run3-retro.md
new file mode 100644
index 00000000..156099e0
--- /dev/null
+++ b/docs/retros/2026-04-01-steam-run3-retro.md
@@ -0,0 +1,105 @@
+# Printing Press Retro: Steam Web API (Run 3)
+
+## Session Stats
+- API: Steam Web API
+- Spec source: Zuplo/Steam-OpenAPI (158 operations, OpenAPI 3.0, reused from Run 1)
+- Scorecard: **85/100 Grade A** (up from 68 → 70 → 84)
+- Verify pass rate: 75% (61/81 passed, 0 critical)
+- Machine improvements applied: PRs #100, #101, #102
+- Manual code edits: 1 (root description rewrite, STEAM_API_KEY env var)
+- Features built from scratch: 22 (10 wrapper + 6 transcendence + 6 insight)
+
+## Journey: 68 → 85 in 3 runs
+
+| Run | Score | Grade | What changed |
+|-----|-------|-------|-------------|
+| Run 1 | 68 | B | Baseline generation + polish |
+| Run 2 | 84 | A | PR #100 scorer fixes (verify naming, dogfood dead-flag, dry-run cache) |
+| Run 3 | 85 | A | PR #101 behavioral detection + PR #102 template improvements |
+
+**17 points recovered through machine improvements.** The CLI pattern is the same; the machine got smarter.
+
+## Findings
+
+### 1. Data pipeline 7/10 — generic Search methods (Generator gap — scorer is correct)
+
+- **Scorer correct?** Yes. The scorer gives +3 for domain-specific Search methods like `SearchPlayers()`, `SearchGames()`. The generator emits domain-specific Upsert methods (e.g., `UpsertIsteamUserStats`) but only generic `Search()`. This is a real generator gap — the store template doesn't emit per-entity search methods.
+- **Root cause:** `store.go.tmpl` generates Upsert per domain table but Search is only generic.
+- **Frequency:** Every API.
+- **Recommendation: Fix the generator.** Emit domain-specific Search methods alongside Upsert in the store template.
+- **Impact:** 3 points.
+
+### 2. Sync correctness 7/10 — missing path parameters (Partially inherent, partially fixable)
+
+- **Scorer correct?** Yes. The +3 bonus for `/{` path params in sync.go rewards APIs with parameterized list endpoints (like `/users/{team_id}/members`). Steam's list endpoints (`/ISteamApps/GetAppList/v2/`) don't have path params — this is inherent to Steam's API structure.
+- **However:** The sync path resolution issue from Run 1 retro (profiler stores resource names, not endpoint paths) is still unfixed. Fixing that wouldn't change the path_params score for Steam, but would fix sync functionality for non-REST APIs generally.
+- **Frequency:** API subclass — APIs without path params in list endpoints. The path resolution fix affects non-REST APIs broadly.
+- **Recommendation:** Two separate items:
+ 1. The +3 sync bonus for path params is a scorer design choice — not a bug. Skip for Steam.
+ 2. The sync path resolution (profiler stores actual endpoint paths) should still be fixed for non-REST APIs. This is WU-5 from the Run 1 retro.
+- **Impact:** 3 points (but 0 recoverable for Steam specifically).
+
+### 3. Auth 8/10 — spec lacks securitySchemes (Partially scorer, partially generator)
+
+- **Scorer correct?** Partially. The Zuplo Steam spec has no `securitySchemes` section — auth is only expressed as a `key` query parameter on 47/158 operations. The scorer can't award auth points because there's nothing to compare against. The generator correctly saw no auth scheme.
+- **Root cause:** Two issues: (a) spec quality — no securitySchemes declaration, (b) generator doesn't infer query-param auth from parameter names.
+- **Recommendation:** Generator enhancement: if >30% of operations have a `key` or `api_key` query param, infer query-param auth. This was finding #5 from Run 1 retro — still unfixed.
+- **Impact:** 2 points.
+
+### 4. Type fidelity 3/5 — improved but not perfect (Both generator and scorer)
+
+- **Scorer correct?** Yes for the remaining 2 points. PR #102 fixed ID params (IntVar → StringVar), gaining 1 point. The remaining deductions: (a) flag description quality — some generated commands have short descriptions (<5 words avg), (b) required flag count may be low for some commands.
+- **Frequency:** Every API.
+- **Recommendation:** Generator template: ensure flag descriptions in the template include the parameter's spec description, not just its name. The `oneline .Description` is already used — the issue is specs with terse descriptions.
+- **Impact:** 2 points (hard to improve without better spec content).
+
+### 5. Dead code 4/5 — 1 remaining dead function (Investigate)
+
+- **Scorer correct?** Need to verify. Dogfood reports 1 dead function. This could be a real dead function added by Claude during wrapper command building, or another false positive.
+- **Recommendation:** Check which function is reported dead. If real, remove it. If false positive, it's another dogfood detection gap.
+- **Impact:** 1 point.
+
+### 6. Verify 75% — 20 commands score 1/3 (Known limitation)
+
+- **Scorer correct?** Partially. The 20 commands pass --help but fail dry-run and exec because they need `STEAM_API_KEY` in the environment during verify. Verify doesn't pass auth env vars during testing.
+- **Recommendation:** Enhance verify to read auth env var names from the CLI's config template and pass them during testing. This was finding #11 from Run 2 retro — still unfixed.
+- **Impact:** 5% verify improvement (75% → ~95%).
+
+## Prioritized Improvements
+
+### Do Now
+| # | Fix | Component | Impact | Complexity |
+|---|-----|-----------|--------|------------|
+| 5 | Remove 1 dead function | Generated CLI | 1 point | Trivial |
+
+### Do Next (from prior retros, still unfixed)
+| # | Fix | Component | Impact | Complexity |
+|---|-----|-----------|--------|------------|
+| 1 | Emit domain-specific Search methods alongside Upsert | `store.go.tmpl` | 3 points | Medium |
+| 3 | Infer query-param auth from parameter names | OpenAPI parser | 2 points | Medium |
+| 6 | Pass auth env vars to verify during testing | `verify` runtime | 5% verify | Medium |
+| 2 | Sync path resolution for non-REST APIs | Profiler + sync template | 0 for Steam, helps other APIs | Medium |
+
+### Skip
+| # | Fix | Why |
+|---|-----|-----|
+| 2 (partial) | Sync path params bonus | Inherent to Steam's API — no path params in list endpoints |
+| 4 | Flag description quality | Depends on spec content quality, not generator |
+
+## What the Machine Got Right
+
+**The retro→plan→implement loop works.** Three iterations of retro → identify scorer bugs vs real gaps → fix the right thing → regenerate produced a 17-point improvement (68→85) with zero changes to the retro/build process itself. The improvement compounds: each fix helps every future CLI, not just Steam.
+
+**Specific wins from machine improvements:**
+- **Path validity 0→10** (PR #101): scanning all files instead of sampling 10 eliminated the wrapper-command bias
+- **Insight 4→9** (PR #101): behavioral detection found `completionist.go`, `playtime.go`, `rare.go` that filename-prefix detection missed
+- **Workflows 8→10** (PR #101): counting total API calls instead of unique HTTP methods correctly identified `profile.go` (5 c.Get calls)
+- **Dead code 4→5→4** (PR #100, #101): dead-flag false positives eliminated; 1 real dead function remains from Claude's build phase
+- **Type fidelity 2→3** (PR #102): ID params use StringVar; dead import markers removed
+- **README 7→9** (PR #102): cookbook section with 5 workflow examples
+
+**The scorer validity lens from the retro skill paid off.** Without it, we would have spent effort adding more insight commands (gaming the filename prefix list) instead of fixing the scorer to detect behavior. The first retro correctly identified path_validity 0/10 as a scorer bug worth 10 points — more than all the generator improvements combined.
+
+## Anti-patterns to Avoid
+
+- **Diminishing returns on scorecard optimization.** We're at 85/100. The remaining 15 points include 3 from sync (inherent to Steam), 2 from auth (spec quality), 2 from type fidelity (spec quality). The practical ceiling for this API+spec combination is ~90. Further machine improvements should be validated against a DIFFERENT API to confirm they generalize, not tuned further on Steam.
diff --git a/internal/generator/generator.go b/internal/generator/generator.go
index 11182ce9..d8988387 100644
--- a/internal/generator/generator.go
+++ b/internal/generator/generator.go
@@ -369,7 +369,7 @@ func (g *Generator) Generate() error {
}
storeData := struct {
*spec.APISpec
- SyncableResources []string
+ SyncableResources []profiler.SyncableResource
SearchableFields map[string][]string
Tables []TableDef
}{
@@ -395,7 +395,7 @@ func (g *Generator) Generate() error {
visionData := struct {
*spec.APISpec
- SyncableResources []string
+ SyncableResources []profiler.SyncableResource
SearchableFields map[string][]string
Tables []TableDef
}{
@@ -426,7 +426,7 @@ func (g *Generator) Generate() error {
if g.VisionSet.Store {
workflowData := struct {
*spec.APISpec
- SyncableResources []string
+ SyncableResources []profiler.SyncableResource
SearchableFields map[string][]string
}{
APISpec: g.Spec,
@@ -470,7 +470,7 @@ func (g *Generator) Generate() error {
if g.VisionSet.MCP {
mcpData := struct {
*spec.APISpec
- SyncableResources []string
+ SyncableResources []profiler.SyncableResource
SearchableFields map[string][]string
Tables []TableDef
VisionSet VisionTemplateSet
diff --git a/internal/generator/templates/channel_workflow.go.tmpl b/internal/generator/templates/channel_workflow.go.tmpl
index d8619527..5a8f2ee5 100644
--- a/internal/generator/templates/channel_workflow.go.tmpl
+++ b/internal/generator/templates/channel_workflow.go.tmpl
@@ -57,7 +57,7 @@ and full resync. After archiving, use 'search' for instant full-text search.`,
}
defer s.Close()
- resources := []string{ {{- range .SyncableResources}}"{{.}}", {{end}} }
+ resources := []string{ {{- range .SyncableResources}}"{{.Name}}", {{end}} }
totalSynced := 0
for _, resource := range resources {
diff --git a/internal/generator/templates/sync.go.tmpl b/internal/generator/templates/sync.go.tmpl
index b5802f1d..1f4b5e42 100644
--- a/internal/generator/templates/sync.go.tmpl
+++ b/internal/generator/templates/sync.go.tmpl
@@ -175,7 +175,7 @@ func syncResource(c interface {
fmt.Fprintf(os.Stderr, `{"event":"sync_start","resource":"%s"}`+"\n", resource)
}
- path := "/" + resource
+ path := syncResourcePath(resource)
var totalCount int
// Resume cursor from sync_state (unless --full cleared it)
@@ -435,11 +435,26 @@ func parseSinceDuration(s string) (time.Time, error) {
func defaultSyncResources() []string {
return []string{
{{- range .SyncableResources}}
- "{{.}}",
+ "{{.Name}}",
{{- end}}
}
}
+// syncResourcePath maps resource names to their actual API endpoint paths.
+// For REST APIs this is typically "/<resource>". For non-REST APIs (e.g., Steam)
+// this preserves the actual endpoint path like "/ISteamApps/GetAppList/v2".
+func syncResourcePath(resource string) string {
+ paths := map[string]string{
+{{- range .SyncableResources}}
+ "{{.Name}}": "{{.Path}}",
+{{- end}}
+ }
+ if p, ok := paths[resource]; ok {
+ return p
+ }
+ return "/" + resource
+}
+
func extractID(obj map[string]any) string {
for _, key := range []string{"id", "ID", "uuid", "slug", "name"} {
if v, ok := obj[key]; ok {
diff --git a/internal/llmpolish/vision.go b/internal/llmpolish/vision.go
index cd428d92..11637ccf 100644
--- a/internal/llmpolish/vision.go
+++ b/internal/llmpolish/vision.go
@@ -68,7 +68,11 @@ func buildVisionPrompt(profile *profiler.APIProfile, apiSpec *spec.APISpec) stri
fmt.Fprintf(&builder, "- Needs local search: %t\n", profile.NeedsSearch)
fmt.Fprintf(&builder, "- Has real-time events: %t\n", profile.HasRealtime)
fmt.Fprintf(&builder, "- Has chronological data: %t\n", profile.HasChronological)
- fmt.Fprintf(&builder, "- Syncable resources: %s\n", formatStringSlice(profile.SyncableResources))
+ syncNames := make([]string, len(profile.SyncableResources))
+ for i, sr := range profile.SyncableResources {
+ syncNames[i] = sr.Name
+ }
+ fmt.Fprintf(&builder, "- Syncable resources: %s\n", formatStringSlice(syncNames))
fmt.Fprintf(&builder, "- Searchable fields per resource: %s\n", formatSearchableFields(profile.SearchableFields))
if apiSpec.Description != "" {
fmt.Fprintf(&builder, "\nAPI description: %s\n", apiSpec.Description)
diff --git a/internal/llmpolish/vision_test.go b/internal/llmpolish/vision_test.go
index 54804757..820be95d 100644
--- a/internal/llmpolish/vision_test.go
+++ b/internal/llmpolish/vision_test.go
@@ -36,9 +36,12 @@ func TestParseVisionCustomization(t *testing.T) {
func TestBuildVisionPrompt(t *testing.T) {
profile := &profiler.APIProfile{
- HighVolume: true,
- NeedsSearch: true,
- SyncableResources: []string{"messages", "channels"},
+ HighVolume: true,
+ NeedsSearch: true,
+ SyncableResources: []profiler.SyncableResource{
+ {Name: "messages", Path: "/channels/{channel_id}/messages"},
+ {Name: "channels", Path: "/guilds/{guild_id}/channels"},
+ },
SearchableFields: map[string][]string{
"messages": {"content", "author"},
},
diff --git a/internal/openapi/parser.go b/internal/openapi/parser.go
index 74731c38..5d588827 100644
--- a/internal/openapi/parser.go
+++ b/internal/openapi/parser.go
@@ -241,7 +241,7 @@ func mapAuth(doc *openapi3.T, name string) spec.AuthConfig {
auth := spec.AuthConfig{Type: "none"}
schemeName, scheme := selectSecurityScheme(doc)
if scheme == nil {
- return auth
+ return inferQueryParamAuth(doc, name, auth)
}
auth.Scheme = schemeName
@@ -306,6 +306,80 @@ func mapAuth(doc *openapi3.T, name string) spec.AuthConfig {
return auth
}
+// commonAuthQueryParams are query parameter names that commonly carry API keys.
+var commonAuthQueryParams = map[string]bool{
+ "key": true,
+ "api_key": true,
+ "apikey": true,
+ "access_token": true,
+ "token": true,
+}
+
+// inferQueryParamAuth scans all operations for query parameters that look like
+// API keys. If more than 30% of operations carry one, we infer query-param auth.
+// This handles specs that omit securitySchemes but pass keys via query string.
+func inferQueryParamAuth(doc *openapi3.T, name string, fallback spec.AuthConfig) spec.AuthConfig {
+ if doc == nil || doc.Paths == nil {
+ return fallback
+ }
+
+ paramCounts := map[string]int{}
+ totalOps := 0
+
+ for _, pathKey := range doc.Paths.InMatchingOrder() {
+ pathItem := doc.Paths.Value(pathKey)
+ if pathItem == nil {
+ continue
+ }
+ for _, op := range pathItem.Operations() {
+ if op == nil {
+ continue
+ }
+ totalOps++
+ seen := false
+ // Check path-level parameters first, then operation-level.
+ for _, params := range []openapi3.Parameters{pathItem.Parameters, op.Parameters} {
+ for _, pRef := range params {
+ if pRef == nil || pRef.Value == nil {
+ continue
+ }
+ p := pRef.Value
+ if p.In == openapi3.ParameterInQuery && commonAuthQueryParams[strings.ToLower(p.Name)] && !seen {
+ paramCounts[p.Name]++
+ seen = true
+ }
+ }
+ }
+ }
+ }
+
+ if totalOps == 0 {
+ return fallback
+ }
+
+ // Find the most common auth-like param name.
+ var best string
+ var bestCount int
+ for pName, cnt := range paramCounts {
+ if cnt > bestCount {
+ best = pName
+ bestCount = cnt
+ }
+ }
+
+ if bestCount == 0 || float64(bestCount)/float64(totalOps) <= 0.3 {
+ return fallback
+ }
+
+ envPrefix := strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
+ return spec.AuthConfig{
+ Type: "api_key",
+ In: "query",
+ Header: best,
+ EnvVars: []string{envPrefix + "_API_KEY"},
+ }
+}
+
func selectSecurityScheme(doc *openapi3.T) (string, *openapi3.SecurityScheme) {
if doc == nil || doc.Components == nil || len(doc.Components.SecuritySchemes) == 0 {
return "", nil
diff --git a/internal/pipeline/runtime.go b/internal/pipeline/runtime.go
index c8d921e6..e95a5480 100644
--- a/internal/pipeline/runtime.go
+++ b/internal/pipeline/runtime.go
@@ -119,32 +119,48 @@ func RunVerify(cfg VerifyConfig) (*VerifyReport, error) {
classifyCommandKind(&commands[i], spec)
}
- // 7. Run tests
- for i, cmd := range commands {
+ // Collect auth env var names from the spec, falling back to the derived name.
+ authEnvVars := []string{envVarName}
+ if spec != nil && len(spec.Auth.EnvVars) > 0 {
+ authEnvVars = spec.Auth.EnvVars
+ }
+
+ // buildEnv constructs the environment for test subprocesses, passing
+ // all auth-related env vars so auth-requiring commands can complete.
+ buildEnv := func() []string {
env := os.Environ()
if report.Mode == "live" {
- env = append(env, envVarName+"="+cfg.APIKey)
+ for _, ev := range authEnvVars {
+ if val := os.Getenv(ev); val != "" {
+ env = append(env, ev+"="+val)
+ }
+ }
+ // Also pass the explicit --api-key under ALL auth env var names so the
+ // generated CLI finds it regardless of which env var it reads.
+ if cfg.APIKey != "" {
+ for _, ev := range authEnvVars {
+ env = append(env, ev+"="+cfg.APIKey)
+ }
+ }
} else {
env = append(env, baseURLEnvVar+"="+baseURLOverride)
- env = append(env, envVarName+"=mock-token-for-testing")
+ for _, ev := range authEnvVars {
+ env = append(env, ev+"=mock-token-for-testing")
+ }
}
+ return env
+ }
+ // 7. Run tests
+ for i, cmd := range commands {
+ env := buildEnv()
result := runCommandTests(binaryPath, cmd, report.Mode, env)
commands[i] = cmd // preserve classification
report.Results = append(report.Results, result)
}
// 8. Data pipeline test
- report.DataPipeline = runDataPipelineTest(binaryPath, report.Mode, func() []string {
- env := os.Environ()
- if report.Mode == "live" {
- env = append(env, envVarName+"="+cfg.APIKey)
- } else {
- env = append(env, baseURLEnvVar+"="+baseURLOverride)
- env = append(env, envVarName+"=mock-token-for-testing")
- }
- return env
- })
+ report.DataPipeline = runDataPipelineTest(binaryPath, report.Mode, buildEnv)
// 9. Compute aggregate
for _, r := range report.Results {
diff --git a/internal/profiler/profiler.go b/internal/profiler/profiler.go
index 2fe39854..346fbcef 100644
--- a/internal/profiler/profiler.go
+++ b/internal/profiler/profiler.go
@@ -44,6 +44,12 @@ type PaginationProfile struct {
DefaultPageSize int `json:"default_page_size"` // detected or default 100
}
+// SyncableResource describes a resource that supports paginated list sync.
+type SyncableResource struct {
+ Name string
+ Path string
+}
+
// APIProfile describes the shape of an API and what power-user features it warrants.
type APIProfile struct {
HighVolume bool
@@ -60,7 +66,7 @@ type APIProfile struct {
TotalEndpoints int
ReadRatio float64
- SyncableResources []string
+ SyncableResources []SyncableResource
SearchableFields map[string][]string
Domain DomainSignals
@@ -79,7 +85,7 @@ func Profile(s *spec.APISpec) *APIProfile {
}
resourceNames := collectResourceNames(s.Resources)
- syncable := make(map[string]struct{})
+ syncable := make(map[string]string) // resource name -> list endpoint path
searchable := make(map[string]map[string]struct{})
listResources := make(map[string]struct{})
@@ -138,7 +144,12 @@ func Profile(s *spec.APISpec) *APIProfile {
listResources[resourceName] = struct{}{}
if endpoint.Pagination != nil {
p.ListEndpoints++
- syncable[resourceName] = struct{}{}
+ // Pick the shortest path for determinism when multiple
+ // paginated list endpoints exist for the same resource.
+ // Shorter paths are typically the primary list endpoint.
+ if existing, ok := syncable[resourceName]; !ok || len(endpoint.Path) < len(existing) {
+ syncable[resourceName] = endpoint.Path
+ }
}
}
@@ -210,7 +221,7 @@ func Profile(s *spec.APISpec) *APIProfile {
}
p.NeedsSearch = len(listResources) >= 3 && float64(searchEndpointCount)/float64(len(listResources)) < 0.5
- p.SyncableResources = sortedKeys(syncable)
+ p.SyncableResources = sortedSyncableResources(syncable)
for resource, fields := range searchable {
p.SearchableFields[resource] = sortedKeys(fields)
}
@@ -236,7 +247,7 @@ func (p *APIProfile) ToVisionaryPlan(apiName string) *vision.VisionaryPlan {
plan := &vision.VisionaryPlan{
APIName: apiName,
Identity: vision.APIIdentity{
- CoreEntities: p.SyncableResources,
+ CoreEntities: syncableResourceNames(p.SyncableResources),
DataProfile: vision.DataProfile{
Volume: lowHigh(p.HighVolume),
SearchNeed: lowHigh(p.NeedsSearch),
@@ -316,6 +327,11 @@ func (p *APIProfile) RecommendedFeatures() []string {
return features
}
+// SyncableResourceNames returns the names of the syncable resources.
+func (p *APIProfile) SyncableResourceNames() []string {
+ return syncableResourceNames(p.SyncableResources)
+}
+
func featureIdeaFor(name string, p *APIProfile) vision.FeatureIdea {
switch name {
case "sync":
@@ -587,6 +603,29 @@ func sortedKeys[V any](m map[string]V) []string {
return keys
}
+// sortedSyncableResources converts a name->path map into a sorted slice of SyncableResource.
+func sortedSyncableResources(m map[string]string) []SyncableResource {
+ names := make([]string, 0, len(m))
+ for k := range m {
+ names = append(names, k)
+ }
+ sort.Strings(names)
+ resources := make([]SyncableResource, len(names))
+ for i, name := range names {
+ resources[i] = SyncableResource{Name: name, Path: m[name]}
+ }
+ return resources
+}
+
+// syncableResourceNames extracts just the names from a slice of SyncableResource.
+func syncableResourceNames(resources []SyncableResource) []string {
+ names := make([]string, len(resources))
+ for i, r := range resources {
+ names[i] = r.Name
+ }
+ return names
+}
+
func detectDomainSignals(s *spec.APISpec) DomainSignals {
if s == nil {
return DomainSignals{Archetype: ArchetypeGeneric}
diff --git a/internal/profiler/profiler_test.go b/internal/profiler/profiler_test.go
index 680b9756..b2746470 100644
--- a/internal/profiler/profiler_test.go
+++ b/internal/profiler/profiler_test.go
@@ -26,7 +26,11 @@ func TestProfileDiscord(t *testing.T) {
assert.True(t, profile.HasChronological)
assert.True(t, profile.HasDependencies)
assert.ElementsMatch(t, []string{"sync", "search", "store", "export", "import", "tail", "analytics"}, profile.RecommendedFeatures())
- assert.Contains(t, profile.SyncableResources, "messages")
+ syncNames := make([]string, len(profile.SyncableResources))
+ for i, sr := range profile.SyncableResources {
+ syncNames[i] = sr.Name
+ }
+ assert.Contains(t, syncNames, "messages")
assert.Contains(t, profile.SearchableFields["messages"], "content")
}
diff --git a/lefthook.yml b/lefthook.yml
index 23370088..df3925ae 100644
--- a/lefthook.yml
+++ b/lefthook.yml
@@ -1,3 +1,9 @@
+output:
+ - summary
+ - success
+ - failure
+ - execution
+
post-checkout:
commands:
build:
← 3af5d2d7 fix(ci): add post-merge hook to rebuild binary after git pul
·
back to Cli Printing Press
·
fix(cli): machine context compensation — scorer, generator, 55c5525b →