feat(catalog): add telegram, launchdarkly, sentry from dogfood gauntlet

0f1beba24f7818d80ddf5c19cb92a4a8e127776c · 2026-03-24 09:29:24 -0700 · Matt Van Horn

Ran 10 new APIs through the generator. Results: 3/10 pass all 7 gates.
- PASS: Telegram (74 paths), Sentry (126 paths), LaunchDarkly (221 paths)
- FAIL: Fly.io, Spotify, Vercel, Supabase, Trello, Jira, Cloudflare

6 generator bugs documented in dogfood-gauntlet-findings.md.
Top fix: $ref/$schema sanitization would unlock 3 more APIs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Files touched

A catalog/launchdarkly.yaml
A catalog/sentry.yaml
A catalog/telegram.yaml
A docs/plans/2026-03-24-feat-dogfood-gauntlet-10-new-apis-plan.md
A docs/plans/dogfood-gauntlet-findings.md

Diff

commit 0f1beba24f7818d80ddf5c19cb92a4a8e127776c
Author: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Date:   Tue Mar 24 09:29:24 2026 -0700

    feat(catalog): add telegram, launchdarkly, sentry from dogfood gauntlet
    
    Ran 10 new APIs through the generator. Results: 3/10 pass all 7 gates.
    - PASS: Telegram (74 paths), Sentry (126 paths), LaunchDarkly (221 paths)
    - FAIL: Fly.io, Spotify, Vercel, Supabase, Trello, Jira, Cloudflare
    
    6 generator bugs documented in dogfood-gauntlet-findings.md.
    Top fix: $ref/$schema sanitization would unlock 3 more APIs.
    
    Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---
 catalog/launchdarkly.yaml                          |  11 ++
 catalog/sentry.yaml                                |  11 ++
 catalog/telegram.yaml                              |  11 ++
 ...03-24-feat-dogfood-gauntlet-10-new-apis-plan.md | 148 +++++++++++++++++++++
 docs/plans/dogfood-gauntlet-findings.md            |  95 +++++++++++++
 5 files changed, 276 insertions(+)

diff --git a/catalog/launchdarkly.yaml b/catalog/launchdarkly.yaml
new file mode 100644
index 00000000..fdb99b18
--- /dev/null
+++ b/catalog/launchdarkly.yaml
@@ -0,0 +1,11 @@
+name: launchdarkly
+display_name: LaunchDarkly
+description: Feature flag management - flags, environments, segments, experiments, audit logs
+category: developer-tools
+spec_url: https://app.launchdarkly.com/api/v2/openapi.json
+spec_format: json
+openapi_version: "3.0"
+tier: community
+verified_date: "2026-03-24"
+homepage: https://launchdarkly.com
+notes: "221 paths, API key auth. Flat 'api' resource due to /api/v2/ prefix."
diff --git a/catalog/sentry.yaml b/catalog/sentry.yaml
new file mode 100644
index 00000000..684a1e31
--- /dev/null
+++ b/catalog/sentry.yaml
@@ -0,0 +1,11 @@
+name: sentry
+display_name: Sentry
+description: Error tracking and performance monitoring - projects, issues, events, releases
+category: developer-tools
+spec_url: https://raw.githubusercontent.com/getsentry/sentry-api-schema/main/openapi-derefed.json
+spec_format: json
+openapi_version: "3.0"
+tier: community
+verified_date: "2026-03-24"
+homepage: https://sentry.io
+notes: "126 paths, bearer auth. Templated base URL {region}.sentry.io needs manual config."
diff --git a/catalog/telegram.yaml b/catalog/telegram.yaml
new file mode 100644
index 00000000..ae99c611
--- /dev/null
+++ b/catalog/telegram.yaml
@@ -0,0 +1,11 @@
+name: telegram
+display_name: Telegram Bot
+description: Telegram Bot API for building bots - send messages, manage chats, webhooks, stickers
+category: communication
+spec_url: https://api.apis.guru/v2/specs/telegram.org/5.0.0/openapi.json
+spec_format: json
+openapi_version: "3.0"
+tier: community
+verified_date: "2026-03-24"
+homepage: https://core.telegram.org/bots/api
+notes: "Flat command structure (no sub-resources). 50+ commands. Token-in-URL auth."
diff --git a/docs/plans/2026-03-24-feat-dogfood-gauntlet-10-new-apis-plan.md b/docs/plans/2026-03-24-feat-dogfood-gauntlet-10-new-apis-plan.md
new file mode 100644
index 00000000..398d0ffc
--- /dev/null
+++ b/docs/plans/2026-03-24-feat-dogfood-gauntlet-10-new-apis-plan.md
@@ -0,0 +1,148 @@
+---
+title: "Dogfood Gauntlet - Generate and Test 10 New API CLIs"
+type: feat
+status: completed
+date: 2026-03-24
+---
+
+# Dogfood Gauntlet - Generate and Test 10 New API CLIs
+
+## Overview
+
+The printing press has been tested against 4 APIs (Petstore, Stytch, Discord, Gmail). That's not enough. We need to throw diverse, real-world specs at it and see what breaks. Not Steinberger's APIs (Google suite, Discord, WhatsApp) - those are covered. Not the ones already in our registry. Fresh specs that stress different parts of the generator.
+
+The goal: run `printing-press generate` against 10 new APIs, capture what passes and what breaks, fix the generator (not the output), and add the working ones to the catalog.
+
+## The 10 APIs
+
+Selected for diversity of auth, spec size, resource nesting, and pagination patterns:
+
+| # | API | Spec URL | Size | Auth | Free? | Why It's Interesting |
+|---|-----|----------|------|------|-------|---------------------|
+| 1 | Cloudflare | `https://raw.githubusercontent.com/cloudflare/api-schemas/main/openapi.json` | 1716 paths | API key, Bearer | Yes | Largest spec - stress test truncation, zone-scoped resources |
+| 2 | Jira | `https://api.apis.guru/v2/specs/atlassian.com/jira/1001.0.0-SNAPSHOT/openapi.json` | 317 paths | OAuth2, Basic | Yes | Deep nesting (project/board/sprint/issue/comment), JQL |
+| 3 | Spotify | `https://api.apis.guru/v2/specs/spotify.com/sonallux/2023.2.27/openapi.json` | 68 paths | OAuth2 | Yes | Cursor pagination, nested resources, search with filters |
+| 4 | Telegram Bot | `https://api.apis.guru/v2/specs/telegram.org/5.0.0/openapi.json` | 74 paths | Token-in-URL | Yes | File uploads, inline keyboards, webhook setup |
+| 5 | Trello | `https://api.apis.guru/v2/specs/trello.com/1.0/openapi.json` | 264 paths | API key+token | Yes | Board/list/card/checklist nesting, batch endpoint |
+| 6 | Vercel | `https://api.apis.guru/v2/specs/vercel.com/0.0.1/openapi.json` | 85 paths | Bearer, OAuth2 | Yes | Deploy lifecycle, env vars, domain mgmt |
+| 7 | Sentry | `https://raw.githubusercontent.com/getsentry/sentry-api-schema/main/openapi-derefed.json` | 126 paths | Bearer | Yes | Org/project/issue hierarchy, cursor pagination |
+| 8 | Fly.io Machines | `https://docs.machines.dev/spec/openapi3.json` | 51 paths | Bearer | Yes | Machine lifecycle, volumes, leases, long-poll waits |
+| 9 | LaunchDarkly | `https://app.launchdarkly.com/api/v2/openapi.json` | 221 paths | API key | Yes (trial) | Feature flags, semantic patching, experiments |
+| 10 | Supabase | `https://api.supabase.com/api/v1-json` | 105 paths | Bearer | Yes | Project lifecycle, database mgmt, edge functions |
+
+## Acceptance Criteria
+
+- [ ] All 10 specs downloaded and parsed without crash
+- [ ] At least 7/10 pass all 7 quality gates (compile, vet, build, binary, help, version, doctor)
+- [ ] Bugs found in the generator are documented with spec name and reproduction steps
+- [ ] Each passing CLI has been dogfooded: `--help`, `doctor`, at least 1 resource `--help`
+- [ ] Working CLIs added to catalog/ as community tier entries
+- [ ] Known specs registry updated with all 10 URLs
+- [ ] Bugs-found list written to docs/plans/dogfood-gauntlet-findings.md
+
+## Implementation Units
+
+### Unit 1: Add All 10 to Known Specs Registry
+
+**Files:** `internal/pipeline/discover.go`
+
+Add the 10 new APIs to `KnownSpecs` map. None are sandbox-safe (no Tier 3 dogfooding).
+
+### Unit 2: Run the Gauntlet
+
+**Approach:** For each API, in order from smallest to largest:
+
+```bash
+# Build the press
+go build -o ./printing-press ./cmd/printing-press
+
+# For each API:
+./printing-press generate \
+  --spec <spec-url> \
+  --output /tmp/dogfood-<name>-cli \
+  2>&1 | tee /tmp/dogfood-<name>-output.txt
+
+# If 7 gates pass:
+cd /tmp/dogfood-<name>-cli
+go build -o <name>-cli ./cmd/<name>-cli
+./<name>-cli --help
+./<name>-cli doctor
+./<name>-cli <first-resource> --help
+
+# Record: pass/fail, gate output, help output, any crashes
+```
+
+Run order (smallest first, fail fast):
+1. Fly.io (51 paths)
+2. Spotify (68 paths)
+3. Telegram (74 paths)
+4. Vercel (85 paths)
+5. Supabase (105 paths)
+6. Sentry (126 paths)
+7. LaunchDarkly (221 paths)
+8. Trello (264 paths)
+9. Jira (317 paths)
+10. Cloudflare (1716 paths)
+
+### Unit 3: Fix Generator Bugs Found
+
+**Files:** `internal/openapi/parser.go`, `internal/generator/templates/*.tmpl`
+
+For each API that fails:
+1. Read the error output
+2. Identify root cause in the parser or templates (NOT in the generated output)
+3. Fix the generator
+4. Run `go test ./...` (don't break existing specs)
+5. Re-run the failing API
+6. Repeat until it passes or document as known limitation
+
+Common expected failures:
+- Huge specs hitting resource/endpoint caps (Cloudflare at 1716 paths)
+- Complex auth schemes not mapping cleanly (Telegram token-in-URL)
+- Deep nesting exceeding sub-resource detection (Jira 5-level nesting)
+- Unusual parameter types or content types
+
+### Unit 4: Add Passing CLIs to Catalog
+
+**Files:** `catalog/<name>.yaml` for each passing API
+
+Create catalog entry with:
+- name, display_name, description, category, spec_url, spec_format
+- tier: community
+- verified_date: 2026-03-24
+
+### Unit 5: Write Findings Report
+
+**Files:** `docs/plans/dogfood-gauntlet-findings.md`
+
+Document:
+- Pass/fail per API with gate output
+- Bugs found and fixed (with commit SHAs)
+- Bugs found but NOT fixed (known limitations)
+- Generator improvements needed
+- Quality score per CLI (if dogfood phase was run)
+- Recommendations for the Steinberger Parity plan
+
+## Scope Boundaries
+
+- Fix the generator, never the generated output
+- Don't call any real APIs (no auth needed, just generate and compile)
+- Don't run the full pipeline (`printing-press print`) - just `generate`
+- Don't test all endpoints of each CLI - just help, doctor, and one resource
+- If a spec can't be downloaded (404, rate limit), skip it and note why
+- Max 2 retries per spec after generator fixes
+
+## Success Metrics
+
+- **7+ of 10 pass all gates** = the press handles diverse specs well
+- **5-6 pass** = generator needs targeted fixes for common patterns
+- **< 5 pass** = fundamental parser/template gaps to address
+
+## Sources
+
+- Known specs registry: `internal/pipeline/discover.go`
+- Catalog entries: `catalog/*.yaml`
+- Generator: `internal/generator/generator.go`, `internal/generator/templates/`
+- Parser: `internal/openapi/parser.go`
+- Quality gates: `internal/generator/validate.go`
+- apis.guru directory: https://apis.guru/
diff --git a/docs/plans/dogfood-gauntlet-findings.md b/docs/plans/dogfood-gauntlet-findings.md
new file mode 100644
index 00000000..47f44730
--- /dev/null
+++ b/docs/plans/dogfood-gauntlet-findings.md
@@ -0,0 +1,95 @@
+# Dogfood Gauntlet Findings - 2026-03-24
+
+## Scorecard: 3/10 Pass All 7 Gates
+
+| # | API | Paths | Result | Failure Point | Root Cause |
+|---|-----|-------|--------|---------------|------------|
+| 1 | Fly.io | 51 | FAIL | go vet | types.go: generated types have syntax errors (`.` in type names from complex $ref schemas) |
+| 2 | Spotify | 68 | FAIL | binary crash | CLI name derived from spec title is absurdly long ("spotify-web-with-fixes-and-improvements-from-sonallux-cli"), causes panic in command registration |
+| 3 | Telegram | 74 | PASS | - | 7/7 gates pass. 2 endpoints skipped (resource limit 50). Flat structure (no sub-resources). |
+| 4 | Vercel | 85 | FAIL | go vet | `$schema` JSON reference keys leak into generated Go code as `$schema` variable names (invalid Go identifier) |
+| 5 | Supabase | 105 | FAIL | parse | `base_url is required` - spec has no `servers` field, parser requires it |
+| 6 | Sentry | 126 | PASS | - | 7/7 gates pass. Flat "api" resource. Doctor shows templated URL bug: `https://{region}.sentry.io` has `{region}` literal in host. |
+| 7 | LaunchDarkly | 221 | PASS | - | 7/7 gates pass. Flat "api" resource. Doctor shows API reachable (200). Clean output. |
+| 8 | Trello | 264 | FAIL | go build | Syntax errors from `/` characters in generated enum type names (Trello uses `modelTypes/card` style enums) |
+| 9 | Jira | 317 | FAIL | go vet | types.go: malformed type definitions from deeply nested $ref schemas with numeric keys |
+| 10 | Cloudflare | 1716 | FAIL | parse | `name is required` - spec missing required `info.title` field (Cloudflare uses `x-api-name` extension) |
+
+## Grade: D (3/10 pass)
+
+Below the 7/10 target. Generator needs targeted fixes for common patterns.
+
+## Bugs Found
+
+### Bug 1: `$ref` / `$schema` keys leak into Go identifiers
+- **Affected:** Fly.io, Vercel, Jira
+- **Root cause:** JSON Schema `$ref` and `$schema` keys are passed through to Go type/variable names without sanitizing the `$` prefix
+- **Fix needed in:** `internal/openapi/parser.go` (type name sanitization) and `internal/generator/templates/types.go.tmpl`
+- **Severity:** High - affects any spec with complex schemas
+
+### Bug 2: Spec title used as CLI name without truncation
+- **Affected:** Spotify ("Spotify Web API with fixes and improvements from sonallux")
+- **Root cause:** `--name` defaults to kebab-cased spec title, no length limit
+- **Fix needed in:** `internal/cli/root.go` (name derivation) or `internal/openapi/parser.go`
+- **Severity:** Medium - workaround is `--name spotify`
+
+### Bug 3: Missing `servers` field causes parse failure
+- **Affected:** Supabase
+- **Root cause:** Parser requires `base_url` but some specs have no `servers` array
+- **Fix needed in:** `internal/openapi/parser.go` (fallback to empty string or host from spec URL)
+- **Severity:** Medium - affects specs that rely on relative URLs
+
+### Bug 4: Cloudflare spec uses `x-api-name` instead of `info.title`
+- **Affected:** Cloudflare
+- **Root cause:** Parser requires `info.title` but Cloudflare uses a custom extension
+- **Fix needed in:** `internal/openapi/parser.go` or `internal/spec/spec.go` (fallback to x-api-name)
+- **Severity:** Low - Cloudflare-specific
+
+### Bug 5: Enum values with `/` generate invalid Go type names
+- **Affected:** Trello
+- **Root cause:** Trello uses `modelTypes/card` style enum values that become Go identifiers
+- **Fix needed in:** `internal/openapi/parser.go` (enum value sanitization)
+- **Severity:** Medium - affects specs with path-like enum values
+
+### Bug 6: Templated base URL not resolved
+- **Affected:** Sentry (`https://{region}.sentry.io`)
+- **Root cause:** Server URL templates with `{variable}` placeholders are used verbatim instead of resolved
+- **Fix needed in:** `internal/openapi/parser.go` (resolve server URL templates using default values)
+- **Severity:** Low - CLI still compiles, doctor just shows unreachable
+
+## Dogfood Observations
+
+### Telegram (PASS)
+- 50+ commands at top level (flat, no sub-resources)
+- All commands are verb-noun style: `send-message`, `answer-callback-query`
+- Doctor: base_url has `{token}` placeholder (same templated URL issue as Sentry)
+- Descriptions say "Manage X" - generic, not helpful
+
+### Sentry (PASS)
+- Single "api" top-level resource (flat hierarchy)
+- All 50 endpoints under `api` - the sub-resource detection doesn't help because all paths start with `/api/0/`
+- Doctor: templated URL `{region}` literal in host
+- CLI name: "reference-cli" (from spec title "Sentry Public API" being overridden by... something)
+
+### LaunchDarkly (PASS)
+- Single "api" top-level resource (same flat problem)
+- Doctor: API reachable, 200 response
+- Clean help output, good descriptions
+- Truncated spec description in `--help` banner (too long)
+
+## Priority Fix Order
+
+1. **$ref/$schema sanitization** (fixes 3 APIs: Fly.io, Vercel, Jira)
+2. **Spec title truncation for CLI name** (fixes Spotify)
+3. **Missing servers fallback** (fixes Supabase)
+4. **Enum value sanitization** (fixes Trello)
+5. **Server URL template resolution** (fixes Sentry/Telegram doctor)
+6. **x-api-name fallback** (fixes Cloudflare)
+
+Fixes 1-4 would bring the pass rate from 3/10 to 8/10.
+
+## Recommendations
+
+- The "flat api resource" problem (Sentry, LaunchDarkly) is a parser issue: all paths start with `/api/v2/` or `/api/0/` and the first segment becomes the only resource. The parser should strip common API path prefixes before grouping.
+- Consider bumping the endpoint-per-resource limit from 50 to 100 for APIs with flat structures.
+- The spec title to CLI name derivation needs a hard cap (e.g., 30 chars) and a smart truncation that picks the most meaningful words.

← d81e961a feat(pipeline): add 10 new APIs to known specs registry for · back to Cli Printing Press · fix(parser): sanitize schema names, cap title length, handle 3f096bce →