← back to Cli Printing Press
fix(cli): cross-CLI retro findings - store, sync, scorer, GraphQL templates (#185)
e2009bb1fb1d522844baef8e0a8cc7e49506d857 · 2026-04-12 11:25:27 -0400 · Matt Van Horn
* fix(generator+scorer): Phase 1 cross-CLI retro fixes
Three independent fixes from the Slack/Trigger.dev/Linear cross-CLI retro:
1. store.go.tmpl: MaxOpenConns(1) -> MaxOpenConns(2) to prevent deadlock
when analytics commands call helper functions during row iteration.
WAL mode still serializes writes; the second connection allows a read
cursor to coexist with a helper query.
2. parser.go: Replace alphabetical path selection with scored selection.
Paths are now sorted by pathPriorityScore (depth penalty + admin tag
penalty) before the 500-resource cap is applied. This prevents
admin.* endpoints from crowding out user-facing commands on large
APIs like Slack (174 endpoints, admin.* sorted first alphabetically).
3. scorecard.go: Add Swagger 2.0 securityDefinitions fallback. When
OAS3 components.securitySchemes is empty, check for Swagger 2.0
securityDefinitions and map types (apiKey, oauth2, basic) to the
same scoring buckets. Fixes Slack scoring 3/10 on auth_protocol
despite correct Bearer implementation.
Retro: ~/printing-press/manuscripts/cross-cli-retro-20260412.md
Issue: #182
* fix(generator): lower gravity threshold to 2, default FTS5Triggers true
More resources now get typed columns and entity-specific Upsert:
- Gravity threshold lowered from >= 4 to >= 2. Resources with 2+
endpoints or modest field richness now get full column extraction
instead of the bare id/data/synced_at schema. This means
transcendence/analytics commands will find data in entity tables
instead of querying empty tables while everything sits in the
generic resources table.
- FTS5Triggers unconditionally true when FTS5 is active. Content-linked
triggers are now the default mode. The manual-sync fallback (fixed
for modernc.org/sqlite in commit 92074e6) handles edge cases.
Cross-CLI retro findings F1 (entity tables decorative) and F5-related
(FTS5 trigger mode).
* feat(generator): parent-child dependent sync for nested API resources
APIs with parent-child endpoints (e.g., /channels/{channelId}/messages)
now get generated dependent-sync functions instead of being silently
excluded from sync.
Profiler:
- New DependentResource struct and DependentSyncResources field on
APIProfile
- detectDependentResources() matches path params (strip Id/_id suffix)
against existing flat syncable resources with singular/plural fallback
- Parameterized paginated endpoints go to DependentSyncResources
instead of being dropped
Sync template:
- syncDependentResource() iterates parent table IDs, fetches child
records per parent with pagination, injects parent_id into JSON
before upserting. Rate-limit aware, logs progress per parent.
- Runs after flat resource sync completes
Store template:
- ListIDs() method queries entity table for all IDs (falls back to
generic resources table)
Generator:
- Appends parent_id TEXT column + index to dependent resource tables
- Passes DependentSyncResources to template context
This eliminates the most time-consuming manual phase of Slack-like CLIs
where per-channel message sync required 100+ lines of hand-written code.
One level of nesting only (parent-child, not grandchild).
Cross-CLI retro finding F4 (no parent-child sync pattern).
* feat(generator): GraphQL sync/client templates for SDL-sourced CLIs
CLIs generated from GraphQL SDL specs now get a functional sync pipeline
without hand-written client code. Previously, Linear-shaped CLIs required
200-400 lines of manual GraphQL client/query/sync code because the REST
sync template generated GET requests against "/graphql" endpoints.
New templates:
- graphql_client.go.tmpl: GraphQL transport layer on top of the existing
client. Query(), Mutate(), PaginatedQuery() methods that POST to
/graphql with {"query": ..., "variables": ...} body. Parses GraphQL
error responses into Go errors.
- graphql_queries.go.tmpl: Per-resource query constants with Connection
pagination (first/after, pageInfo, nodes). Field selection from spec.
- graphql_sync.go.tmpl: Same sync command structure (flags, worker pool,
progress, store) but uses GraphQL queries instead of REST GETs.
Pagination via pageInfo.hasNextPage/endCursor.
Generator routing:
- isGraphQLSpec() heuristic: true when all list endpoints have path
"/graphql". Renders GraphQL client and query files, substitutes
graphql_sync for sync template. All other templates (store, root,
commands, doctor, export, helpers) shared with REST.
Test fixture: testdata/graphql/test.graphql with Issue/Project types,
Connection pagination, and mutations. Generator test verifies the
output compiles.
Cross-CLI retro finding F11 (no GraphQL sync/client templates).
* docs: add cross-CLI retro plan
* fix(generator): remove dead allFieldsAreColumns (lint)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Files touched
A docs/plans/2026-04-12-002-fix-cross-cli-retro-remaining-findings-plan.mdM internal/generator/generator.goM internal/generator/generator_test.goM internal/generator/schema_builder.goA internal/generator/templates/graphql_client.go.tmplA internal/generator/templates/graphql_queries.go.tmplA internal/generator/templates/graphql_sync.go.tmplM internal/generator/templates/store.go.tmplM internal/generator/templates/sync.go.tmplM internal/openapi/parser.goM internal/openapi/parser_test.goM internal/pipeline/scorecard.goM internal/pipeline/scorecard_tier2_test.goM internal/profiler/profiler.goM internal/profiler/profiler_test.goA testdata/graphql/test.graphql
Diff
commit e2009bb1fb1d522844baef8e0a8cc7e49506d857
Author: Matt Van Horn <mvanhorn@users.noreply.github.com>
Date: Sun Apr 12 11:25:27 2026 -0400
fix(cli): cross-CLI retro findings - store, sync, scorer, GraphQL templates (#185)
* fix(generator+scorer): Phase 1 cross-CLI retro fixes
Three independent fixes from the Slack/Trigger.dev/Linear cross-CLI retro:
1. store.go.tmpl: MaxOpenConns(1) -> MaxOpenConns(2) to prevent deadlock
when analytics commands call helper functions during row iteration.
WAL mode still serializes writes; the second connection allows a read
cursor to coexist with a helper query.
2. parser.go: Replace alphabetical path selection with scored selection.
Paths are now sorted by pathPriorityScore (depth penalty + admin tag
penalty) before the 500-resource cap is applied. This prevents
admin.* endpoints from crowding out user-facing commands on large
APIs like Slack (174 endpoints, admin.* sorted first alphabetically).
3. scorecard.go: Add Swagger 2.0 securityDefinitions fallback. When
OAS3 components.securitySchemes is empty, check for Swagger 2.0
securityDefinitions and map types (apiKey, oauth2, basic) to the
same scoring buckets. Fixes Slack scoring 3/10 on auth_protocol
despite correct Bearer implementation.
Retro: ~/printing-press/manuscripts/cross-cli-retro-20260412.md
Issue: #182
* fix(generator): lower gravity threshold to 2, default FTS5Triggers true
More resources now get typed columns and entity-specific Upsert:
- Gravity threshold lowered from >= 4 to >= 2. Resources with 2+
endpoints or modest field richness now get full column extraction
instead of the bare id/data/synced_at schema. This means
transcendence/analytics commands will find data in entity tables
instead of querying empty tables while everything sits in the
generic resources table.
- FTS5Triggers unconditionally true when FTS5 is active. Content-linked
triggers are now the default mode. The manual-sync fallback (fixed
for modernc.org/sqlite in commit 92074e6) handles edge cases.
Cross-CLI retro findings F1 (entity tables decorative) and F5-related
(FTS5 trigger mode).
* feat(generator): parent-child dependent sync for nested API resources
APIs with parent-child endpoints (e.g., /channels/{channelId}/messages)
now get generated dependent-sync functions instead of being silently
excluded from sync.
Profiler:
- New DependentResource struct and DependentSyncResources field on
APIProfile
- detectDependentResources() matches path params (strip Id/_id suffix)
against existing flat syncable resources with singular/plural fallback
- Parameterized paginated endpoints go to DependentSyncResources
instead of being dropped
Sync template:
- syncDependentResource() iterates parent table IDs, fetches child
records per parent with pagination, injects parent_id into JSON
before upserting. Rate-limit aware, logs progress per parent.
- Runs after flat resource sync completes
Store template:
- ListIDs() method queries entity table for all IDs (falls back to
generic resources table)
Generator:
- Appends parent_id TEXT column + index to dependent resource tables
- Passes DependentSyncResources to template context
This eliminates the most time-consuming manual phase of Slack-like CLIs
where per-channel message sync required 100+ lines of hand-written code.
One level of nesting only (parent-child, not grandchild).
Cross-CLI retro finding F4 (no parent-child sync pattern).
* feat(generator): GraphQL sync/client templates for SDL-sourced CLIs
CLIs generated from GraphQL SDL specs now get a functional sync pipeline
without hand-written client code. Previously, Linear-shaped CLIs required
200-400 lines of manual GraphQL client/query/sync code because the REST
sync template generated GET requests against "/graphql" endpoints.
New templates:
- graphql_client.go.tmpl: GraphQL transport layer on top of the existing
client. Query(), Mutate(), PaginatedQuery() methods that POST to
/graphql with {"query": ..., "variables": ...} body. Parses GraphQL
error responses into Go errors.
- graphql_queries.go.tmpl: Per-resource query constants with Connection
pagination (first/after, pageInfo, nodes). Field selection from spec.
- graphql_sync.go.tmpl: Same sync command structure (flags, worker pool,
progress, store) but uses GraphQL queries instead of REST GETs.
Pagination via pageInfo.hasNextPage/endCursor.
Generator routing:
- isGraphQLSpec() heuristic: true when all list endpoints have path
"/graphql". Renders GraphQL client and query files, substitutes
graphql_sync for sync template. All other templates (store, root,
commands, doctor, export, helpers) shared with REST.
Test fixture: testdata/graphql/test.graphql with Issue/Project types,
Connection pagination, and mutations. Generator test verifies the
output compiles.
Cross-CLI retro finding F11 (no GraphQL sync/client templates).
* docs: add cross-CLI retro plan
* fix(generator): remove dead allFieldsAreColumns (lint)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
---
...-fix-cross-cli-retro-remaining-findings-plan.md | 348 ++++++++++++++++++++
internal/generator/generator.go | 168 ++++++++--
internal/generator/generator_test.go | 136 ++++++++
internal/generator/schema_builder.go | 22 +-
.../generator/templates/graphql_client.go.tmpl | 108 +++++++
.../generator/templates/graphql_queries.go.tmpl | 35 ++
internal/generator/templates/graphql_sync.go.tmpl | 354 +++++++++++++++++++++
internal/generator/templates/store.go.tmpl | 31 +-
internal/generator/templates/sync.go.tmpl | 159 +++++++++
internal/openapi/parser.go | 43 ++-
internal/openapi/parser_test.go | 58 ++++
internal/pipeline/scorecard.go | 33 ++
internal/pipeline/scorecard_tier2_test.go | 170 ++++++++++
internal/profiler/profiler.go | 73 ++++-
internal/profiler/profiler_test.go | 84 ++++-
testdata/graphql/test.graphql | 54 ++++
16 files changed, 1827 insertions(+), 49 deletions(-)
diff --git a/docs/plans/2026-04-12-002-fix-cross-cli-retro-remaining-findings-plan.md b/docs/plans/2026-04-12-002-fix-cross-cli-retro-remaining-findings-plan.md
new file mode 100644
index 00000000..446eb7b7
--- /dev/null
+++ b/docs/plans/2026-04-12-002-fix-cross-cli-retro-remaining-findings-plan.md
@@ -0,0 +1,348 @@
+---
+title: "fix: address 6 remaining cross-CLI retro findings (entity store, parent-child sync, deadlock, selection, swagger scorer, GraphQL templates)"
+type: fix
+status: active
+date: 2026-04-12
+origin: ~/printing-press/manuscripts/cross-cli-retro-20260412.md
+---
+
+# Fix Cross-CLI Retro Remaining Findings
+
+**Target repo:** mvanhorn/cli-printing-press. All file paths are relative to that repo root.
+
+## Overview
+
+The cross-CLI retro (Slack, Trigger.dev, Linear) surfaced 12 findings. 6 were already addressed by per-CLI retro plans (usageErr, FTS5 DELETE, extractPageItems wrapper keys, config.go var, scorer internal YAML compat, GraphQL struct dedup). This plan covers the 6 that remain open, organized into three phases by complexity and dependency.
+
+## Problem Frame
+
+The Printing Press produces structurally correct CLIs (200+ commands across 3 test CLIs, zero structural fixes) but the data pipeline layer requires significant manual rework on every generation. The six remaining gaps cause: empty entity tables that transcendence commands query in vain, no template for the most common API data relationship (parent-child), a latent deadlock in every CLI's store, suboptimal resource selection for large APIs, missing Swagger 2.0 auth scoring, and no GraphQL-specific wire protocol templates.
+
+## Requirements Trace
+
+- R1. Low-gravity resources get typed columns and entity-specific Upsert so transcendence commands find data
+- R2. APIs with parent-child endpoints (60-70% of APIs) get generated dependent-sync functions
+- R3. Store's Query() method cannot deadlock when callers iterate rows and call other store methods
+- R4. Resource selection for APIs with >100 endpoints prioritizes user-facing over admin endpoints
+- R5. Scorecard auth_protocol scores Swagger 2.0 securityDefinitions correctly
+- R6. GraphQL-sourced CLIs get a functional sync pipeline without hand-written client code
+
+## Scope Boundaries
+
+- Does NOT change any existing generated CLI in the library. These are Printing Press machine fixes.
+- Does NOT add GraphQL subscriptions or real-time features (R6 covers queries and mutations only).
+- Does NOT change the 500-resource cap number, only the selection strategy (R4).
+- Does NOT change parent-child sync to handle arbitrary nesting depth, only one level of parent-child (R2).
+
+## Context & Research
+
+### Relevant Code and Patterns
+
+- Entity table generation: `internal/generator/schema_builder.go` lines 52-60 (gravity >= 4 gate), lines 125-186 (`computeDataGravity`)
+- Upsert dispatch: `internal/generator/templates/store.go.tmpl` lines 285-344, gated on `len(.Columns) > 3`
+- Sync Upsert dispatch: `internal/generator/templates/sync.go.tmpl` lines 455-463, same `len(.Columns) > 3` gate
+- MaxOpenConns: `store.go.tmpl` line 44 (`db.SetMaxOpenConns(1)`)
+- Exposed Query method: `store.go.tmpl` line 503 (returns `*sql.Rows` to callers)
+- Resource selection: `internal/openapi/parser.go` lines 911-915 (`sort.Strings(pathKeys)`), cap at line 964
+- Parent-child exclusion: `internal/profiler/profiler.go` lines 246-251 (excludes paths with `{`)
+- Swagger 2.0 scorer gap: `internal/pipeline/scorecard.go` lines 1113-1125 (only checks `components.securitySchemes`)
+- GraphQL parser: `internal/graphql/parser.go` (produces spec.APISpec with all paths set to `/graphql`)
+- FTS5Triggers: `schema_builder.go` line 88 (opt-in when all FTS fields are extracted columns)
+- Existing GraphQL plan (parser-level, not templates): `docs/plans/2026-03-26-feat-graphql-api-support-plan.md`
+- Related retro issues: #182 on cli-printing-press
+
+### Institutional Learnings
+
+- Kalshi retro: extractPageItems universal fallback shipped; expanding primary key detection shipped
+- Trigger.dev retro: scorer internal YAML fix shipped, config.go fix shipped
+- Linear retro: FTS5 fix shipped, GraphQL struct dedup shipped, but entire client/sync layer was hand-written
+- Slack retro: per-channel message sync was 100+ lines manual, confirmed 60-70% of APIs need this pattern
+
+## Key Technical Decisions
+
+- **Lower gravity threshold from 4 to 2 (R1):** Gravity 2 means a resource has at least 2 endpoints OR 1 endpoint + some field richness. This is conservative enough to avoid emitting typed columns for truly trivial resources while capturing the mid-tier resources that Trigger.dev and Slack missed.
+- **FTS5Triggers should default to true (R1):** The opt-in gate (`allFieldsAreColumns`) should become the default. When FTS fields are NOT extracted columns, fall back to manual sync with the already-fixed DELETE syntax. This eliminates the "FTS5 triggers are better but only for some resources" inconsistency.
+- **Parent-child detection via path-parameter cross-referencing (R2):** When a path like `/channels/{channelId}/messages` exists, the profiler should detect that `channelId` references the `channels` resource and create a `DependentSync` entry. The sync template then emits a function that iterates parent records before fetching children.
+- **MaxOpenConns(2) with WAL mode (R3):** WAL mode already allows concurrent reads. Changing from 1 to 2 connections means a caller can hold one query open while executing a second. This is the smallest change that eliminates the deadlock without complex transaction management. Write serialization is maintained by SQLite's WAL locking.
+- **Resource selection scoring by path depth + tag deprioritization (R4):** Shorter paths (fewer `/` segments) score higher. Tags containing "admin", "internal", "system", "management" get a penalty. This is simple, effective, and doesn't require API-specific knowledge.
+- **Swagger 2.0 securityDefinitions fallback (R5):** When `components.securitySchemes` is empty, check `securityDefinitions` (the Swagger 2.0 equivalent). Map `type: "apiKey"` and `type: "oauth2"` to the same scoring buckets as their OAS3 counterparts.
+- **GraphQL client template with POST + query body (R6):** The SDL parser already produces a valid APISpec. The new template emits a GraphQL client that sends `POST /graphql` with `{"query": "...", "variables": {...}}` instead of REST GETs. Pagination uses Connection's `pageInfo.hasNextPage` + `endCursor` which the parser already detects.
+
+## Open Questions
+
+### Resolved During Planning
+
+- **Should we lower gravity to 1?** No. Gravity 1 means a resource with a single GET endpoint and no field metadata. These are genuinely too thin for typed columns. Gravity 2 is the right floor.
+- **Can we fix the deadlock without changing MaxOpenConns?** The alternative is refactoring Query() to return collected slices instead of cursors. This is a larger change that breaks the template's current API. MaxOpenConns(2) is safer and smaller.
+- **Should resource selection use the profiler or the parser?** The parser, because selection happens before profiling. The profiler runs on the already-selected resources. Add a `pathPriorityScore()` function in parser.go.
+
+### Deferred to Implementation
+
+- Exact gravity scoring adjustments if threshold 2 proves too aggressive (may need to be 3)
+- Whether GraphQL mutation sync is useful or only queries should sync
+- Exact tag-deprioritization word list for resource selection scoring
+
+## Phased Delivery
+
+### Phase 1: Quick wins (small complexity, no dependencies)
+
+Units 1-3 are independent. Can be done in parallel or any order.
+
+### Phase 2: Store and sync improvements (medium complexity)
+
+Units 4-5 depend on each other (parent-child sync needs entity-aware store).
+
+### Phase 3: GraphQL templates (large complexity)
+
+Unit 6 depends on Units 4 (entity store) being complete for the sync pipeline to work end-to-end.
+
+## Implementation Units
+
+### Phase 1: Quick Wins
+
+- [ ] **Unit 1: MaxOpenConns safety (F7)**
+
+**Goal:** Eliminate the latent deadlock when callers iterate Query() results and call other store methods.
+
+**Requirements:** R3
+
+**Dependencies:** None
+
+**Files:**
+- Modify: `internal/generator/templates/store.go.tmpl`
+- Test: `internal/generator/generator_test.go` (verify generated store compiles and MaxOpenConns is set correctly)
+
+**Approach:**
+- Change `db.SetMaxOpenConns(1)` to `db.SetMaxOpenConns(2)` at line 44
+- Add a comment explaining why: "WAL mode + 2 connections allows one read cursor open while a second query executes (e.g., analytics commands calling helpers during row iteration). Writes are serialized by SQLite's WAL lock."
+- Alternatively, if the risk of 2 concurrent writes is concerning, add `store.writeMu sync.Mutex` and wrap all write operations (Upsert, Delete, Exec) in `writeMu.Lock()/Unlock()`
+
+**Patterns to follow:**
+- Existing `db.SetMaxOpenConns(1)` at store.go.tmpl line 44
+- WAL mode pragma at store.go.tmpl line 46
+
+**Test scenarios:**
+- Happy path: generated store compiles with MaxOpenConns(2). Existing tests still pass.
+- Integration: a generated CLI's analytics command that calls a helper function while iterating query results does not deadlock (test by running a transcendence command against a populated SQLite database in the captured_test).
+- Edge case: concurrent goroutines writing via Upsert do not produce "database is locked" errors (SQLite WAL handles this natively with 2 connections).
+
+**Verification:**
+- `go build ./...` passes on a generated CLI
+- No "database is locked" or deadlock in existing test fixtures
+- grep confirms MaxOpenConns(2) in generated output
+
+---
+
+- [ ] **Unit 2: Smarter resource selection (F8)**
+
+**Goal:** Replace alphabetical path selection with scored selection so admin/internal endpoints don't crowd out user-facing endpoints on large APIs.
+
+**Requirements:** R4
+
+**Dependencies:** None
+
+**Files:**
+- Modify: `internal/openapi/parser.go`
+- Test: `internal/openapi/parser_test.go`
+
+**Approach:**
+- Add a `pathPriorityScore(path string, tags []string) int` function
+- Scoring: base score = 100. Subtract 10 per path segment (depth penalty). Subtract 30 for tags containing "admin", "internal", "system", "management" (case-insensitive). Add 10 for tags containing "users", "projects", "items" or other high-value signals.
+- Before the cap check at line 964, sort `pathKeys` by score descending (stable sort, alphabetical as tiebreaker) instead of alphabetically
+- Keep the 500-resource cap unchanged
+
+**Patterns to follow:**
+- Existing `sort.Strings(pathKeys)` at line 915 (replace with `sort.SliceStable`)
+- Tag extraction from the OpenAPI spec's path items
+
+**Test scenarios:**
+- Happy path: a spec with 600 paths including 100 admin.* paths selects the 500 non-admin paths first. Admin paths are the ones dropped by the cap.
+- Edge case: a spec with < 500 paths includes all paths regardless of score (no change in behavior)
+- Edge case: two paths with the same score are sorted alphabetically (stable sort preserves deterministic output)
+- Error path: a path with no tags gets the base score minus depth penalty (still valid)
+
+**Verification:**
+- Test fixture with > 500 paths confirms admin endpoints are deprioritized
+- Existing parser tests still pass (stable sort means small specs are unaffected)
+
+---
+
+- [ ] **Unit 3: Swagger 2.0 auth scoring (F10)**
+
+**Goal:** Scorecard's auth_protocol dimension correctly evaluates Swagger 2.0 specs.
+
+**Requirements:** R5
+
+**Dependencies:** None
+
+**Files:**
+- Modify: `internal/pipeline/scorecard.go`
+- Test: `internal/pipeline/scorecard_test.go`
+
+**Approach:**
+- In `loadOpenAPISpec` around line 1113, after checking `components.securitySchemes` (OAS3), add a fallback that checks `securityDefinitions` (Swagger 2.0)
+- Map Swagger 2.0 security types to the same scoring buckets: `type: "apiKey"` maps to API key scheme, `type: "oauth2"` maps to OAuth2 scheme, `in: "header"` with `name: "Authorization"` maps to Bearer
+- Set `info.SecuritySchemes` and `info.SecurityRequirements` from the Swagger 2.0 data so downstream scoring logic works unchanged
+
+**Patterns to follow:**
+- Existing OAS3 parsing at scorecard.go lines 1113-1125
+
+**Test scenarios:**
+- Happy path: a Swagger 2.0 spec with `securityDefinitions: { api_key: { type: apiKey, in: header, name: Authorization } }` scores 10/10 on auth_protocol
+- Happy path: a Swagger 2.0 spec with `securityDefinitions: { oauth2: { type: oauth2, flow: accessCode } }` scores correctly for OAuth2
+- Edge case: a spec with both OAS3 `components.securitySchemes` AND Swagger 2.0 `securityDefinitions` uses the OAS3 version (don't double-count)
+- Edge case: a spec with no auth definitions at either level scores 0/10 (unchanged behavior)
+
+**Verification:**
+- Slack's Swagger 2.0 spec scores 10/10 on auth_protocol instead of 3/10
+- Existing OAS3 test fixtures are unaffected
+
+### Phase 2: Store and Sync Improvements
+
+- [ ] **Unit 4: Lower gravity threshold + FTS5Triggers default (F1)**
+
+**Goal:** More resources get typed columns and entity-specific Upsert. FTS5 triggers become the default mode.
+
+**Requirements:** R1
+
+**Dependencies:** None (but benefits from Unit 1's MaxOpenConns fix being in place)
+
+**Files:**
+- Modify: `internal/generator/schema_builder.go`
+- Test: `internal/generator/generator_test.go`
+
+**Approach:**
+- Change gravity threshold from `>= 4` to `>= 2` at line 52
+- In `computeDataGravity`, add `+2` for any resource that appears in `SyncableResources` (has a paginated list endpoint). This ensures every syncable resource gets typed columns.
+- Change FTS5Triggers default: at line 88, set `table.FTS5Triggers = true` unconditionally. The manual-sync fallback path (already fixed for modernc.org/sqlite in commit 92074e6) handles the case where FTS fields aren't extracted columns.
+- This means: after this change, every syncable resource gets a typed table with real columns, entity-specific Upsert, and content-linked FTS5 triggers.
+
+**Patterns to follow:**
+- Existing `computeDataGravity()` scoring at schema_builder.go lines 125-186
+- Existing `FTS5Triggers` conditional at schema_builder.go line 88
+
+**Test scenarios:**
+- Happy path: a resource with 2 endpoints and text fields (gravity 3 under old rules, now >= 2) gets typed columns in the generated store
+- Happy path: `SELECT count(*) FROM <entity>` returns records after sync (not all going to generic resources table)
+- Edge case: a resource with gravity 1 (single GET, no fields) still uses the generic table (threshold excludes truly thin resources)
+- Integration: generate a CLI from a test fixture, run sync, verify entity tables are populated. Verify FTS5 search returns results.
+- Edge case: FTS5Triggers = true even when FTS fields are NOT extracted columns. The manual sync fallback (DELETE WHERE fts MATCH 'rowid:' || ?) still works correctly.
+
+**Verification:**
+- `go build ./...` passes on generated CLIs
+- Test fixtures confirm entity tables get > 3 columns for syncable resources
+- Existing captured_test fixtures still pass
+
+---
+
+- [ ] **Unit 5: Parent-child sync pattern (F4)**
+
+**Goal:** APIs with parent-child endpoints (e.g., `/channels/{channelId}/messages`) get generated dependent-sync functions.
+
+**Requirements:** R2
+
+**Dependencies:** Unit 4 (entity-aware store must create typed tables for child entities)
+
+**Files:**
+- Modify: `internal/profiler/profiler.go`
+- Modify: `internal/generator/schema_builder.go`
+- Create: `internal/generator/templates/sync_dependent.go.tmpl` (or extend `sync.go.tmpl`)
+- Test: `internal/profiler/profiler_test.go`
+- Test: `internal/generator/generator_test.go`
+
+**Approach:**
+- **Profiler change:** Add a `DependentResource` struct to the profile: `{ Name, ParentResource, ParentIDParam, ChildPath, ChildPagination }`. In `classifySyncable()`, when a path contains `{<param>}` AND the param name matches a known resource name (e.g., `channelId` matches `channels`), create a DependentResource entry instead of excluding the path.
+- **Schema change:** schema_builder creates a typed table for dependent resources with a `parent_id` column in addition to the standard columns.
+- **Template change:** New template (or conditional block in sync.go.tmpl) emits a `syncDependentResource()` function. Shape: query parent table for all IDs, then for each parent ID, paginate the child endpoint, upserting with parent_id set. Rate-limit awareness: respect the global rate limiter between parent iterations.
+- **One level only:** Does not handle grandchild relationships (messages -> reactions). That's a separate plan.
+
+**Patterns to follow:**
+- Existing `syncResource()` in sync.go.tmpl for the per-page-fetch-and-upsert loop
+- Existing `SyncableResource` struct in profiler.go for the flat sync pattern
+- Existing parent FK column pattern in schema_builder.go (if any)
+
+**Test scenarios:**
+- Happy path: a spec with `/channels` and `/channels/{channelId}/messages` generates a `syncDependentResource("messages", "channels", "channelId")` function. After syncing channels, messages are synced per channel.
+- Happy path: `SELECT count(*) FROM messages WHERE parent_id IS NOT NULL` returns records after dependent sync.
+- Edge case: a spec with only `/channels/{channelId}/messages` but no `/channels` list endpoint skips dependent sync (can't iterate parents without a parent list endpoint).
+- Edge case: a parent with 10,000 records and 100 messages each handles pagination on the child endpoint correctly.
+- Error path: if parent sync fails, dependent sync is skipped with a warning (not a hard failure).
+- Integration: generate a CLI from a test fixture with parent-child endpoints, run sync, verify child records have correct parent_id values.
+
+**Verification:**
+- Test fixture with parent-child paths generates dependent sync code
+- `go build ./...` passes
+- Profiler test confirms DependentResource is populated correctly
+- Generated CLI's sync command populates child tables with parent_id set
+
+### Phase 3: GraphQL Templates
+
+- [ ] **Unit 6: GraphQL sync/client templates (F11)**
+
+**Goal:** CLIs generated from GraphQL SDL specs get a functional sync pipeline without hand-written client code.
+
+**Requirements:** R6
+
+**Dependencies:** Unit 4 (entity-aware store for typed tables), and architecturally benefits from Unit 5 (parent-child sync, since GraphQL APIs commonly have nested Connection types)
+
+**Files:**
+- Create: `internal/generator/templates/graphql_client.go.tmpl`
+- Create: `internal/generator/templates/graphql_queries.go.tmpl`
+- Create: `internal/generator/templates/graphql_sync.go.tmpl`
+- Modify: `internal/generator/generator.go` (template selection based on spec source)
+- Modify: `internal/graphql/parser.go` (enrich APISpec with query operation strings)
+- Test: `internal/generator/generator_test.go`
+- Test fixture: `testdata/graphql/` (SDL fixtures for testing)
+
+**Approach:**
+- **graphql_client.go.tmpl:** Emits a `GraphQLClient` struct with `Query(ctx, operationName, query, variables) (json.RawMessage, error)` and `Mutate(ctx, ...)`. All requests are `POST /graphql` with `Content-Type: application/json` body `{"query": "...", "variables": {...}}`. Auth injection via the existing `AuthHeader()` pattern.
+- **graphql_queries.go.tmpl:** For each resource in the APISpec, emits a `const <Resource>Query = "query { <resource>(first: $first, after: $after) { nodes { ... } pageInfo { hasNextPage endCursor } } }"` string. Field selection is derived from the SDL parser's field list for each type.
+- **graphql_sync.go.tmpl:** Like sync.go.tmpl but calls `client.Query(ctx, "<Resource>", <Resource>Query, map[string]any{"first": pageSize, "after": cursor})` instead of `client.Get(path, params)`. Pagination loop checks `pageInfo.hasNextPage` and updates `cursor` from `pageInfo.endCursor`.
+- **Generator routing:** In generator.go, when `spec.Source == "graphql-sdl"`, use the GraphQL template set for client, queries, and sync instead of the REST templates. All other templates (store, root, commands, doctor, export, etc.) are shared.
+- **Parser enrichment:** The GraphQL parser already produces field lists per type. Add the query string construction as a field on each Resource so the template can emit it directly.
+
+**Patterns to follow:**
+- Existing `client.go.tmpl` for REST client structure
+- Existing `sync.go.tmpl` for the paginate-and-upsert loop
+- Linear's hand-written `graphql.go` and `queries.go` (in `~/printing-press/library/linear/internal/`) as the reference implementation
+
+**Test scenarios:**
+- Happy path: `go build ./...` passes for a CLI generated from a GraphQL SDL test fixture without any hand-written client code
+- Happy path: sync paginates through a Connection type using `first`/`after` variables and stores records in typed entity tables
+- Edge case: a GraphQL type with no Connection pagination (single-object query) generates a non-paginated fetch
+- Edge case: a mutation endpoint generates a command that sends `POST /graphql` with the mutation query (not a REST PUT/POST)
+- Error path: GraphQL error response `{"errors": [...]}` is parsed and returned as a Go error, not silently swallowed
+- Integration: generate a CLI from the Linear SDL (testdata fixture), run `go build`, verify sync.go uses GraphQL queries not REST GETs
+
+**Verification:**
+- New test fixture in `testdata/graphql/` with a small SDL schema
+- `TestGenerateProjectsCompile` extended to include a GraphQL test case
+- Generated CLI's sync command produces `POST /graphql` requests (not GET)
+- `go build ./...` and `go vet ./...` pass
+
+## System-Wide Impact
+
+- **Interaction graph:** Units 1, 4, 5 change the generated store layer. Every future CLI's store will be different. Existing CLIs in the library are unaffected (they were already compiled).
+- **Error propagation:** Parent-child sync (Unit 5) introduces a new failure mode: parent sync succeeds but child sync fails per-parent. The template should log the failed parent ID and continue to the next parent, not abort.
+- **State lifecycle risks:** Lowering the gravity threshold (Unit 4) means more tables with typed columns. If a resource's response shape doesn't match the column expectations, the typed Upsert will fail. The generic-table fallback should catch this.
+- **API surface parity:** Resource selection (Unit 2) changes which commands are generated for large APIs. This is intentional but means regenerating an existing CLI could produce different commands.
+- **Integration coverage:** Unit 6 (GraphQL) is the highest-risk unit. The captured_test.go.tmpl pattern needs to work with GraphQL fixtures, not just REST fixtures.
+- **Unchanged invariants:** The 500-resource cap number stays the same. Existing template shapes for REST CLIs are unchanged. The existing internal YAML spec format is unchanged. The profiler's API is additive only (new fields, no removed fields).
+
+## Risks & Dependencies
+
+| Risk | Mitigation |
+|------|------------|
+| Lowering gravity to 2 generates typed columns for resources with unreliable field metadata | Keep the generic-table fallback. If typed Upsert fails, fall back to generic silently. Monitor via the existing dogfood pipeline_check. |
+| MaxOpenConns(2) introduces rare concurrent-write edge cases | SQLite WAL mode handles concurrent writes natively. Add a brief stress test to captured_test. |
+| Parent-child sync multiplies API calls (N parents x M pages per parent) | Respect the CLI's existing --rate-limit flag. Add a stderr progress line: "syncing <child> for <parent> (N/M parents)" |
+| GraphQL template produces wrong query syntax for edge-case schemas | Use Linear's hand-written client as the reference implementation. Add the Linear SDL as a test fixture. |
+| Resource selection scoring is subjective | Keep it simple (depth + tag penalty). Don't try to be perfect. The fallback is that users can still write a custom internal YAML spec. |
+
+## Sources & References
+
+- **Origin document:** `~/printing-press/manuscripts/cross-cli-retro-20260412.md`
+- Related issue: mvanhorn/cli-printing-press#182
+- Related completed plans: `2026-04-08-001-fix-graphql-dedup-fts5-plan.md`, `2026-04-10-002-fix-kalshi-retro-findings-plan.md`
+- Linear reference implementation: `~/printing-press/library/linear/internal/` (hand-written GraphQL client and sync)
+- Slack reference implementation: `~/printing-press/library/slack/internal/` (hand-written per-channel message sync)
diff --git a/internal/generator/generator.go b/internal/generator/generator.go
index e41ad38e..1a4ee1b8 100644
--- a/internal/generator/generator.go
+++ b/internal/generator/generator.go
@@ -154,10 +154,15 @@ func New(s *spec.APISpec, outputDir string) *Generator {
}
return false
},
- "exampleLine": g.exampleLine,
- "currentYear": func() string { return strconv.Itoa(time.Now().Year()) },
- "modulePath": func() string { return naming.CLI(s.Name) },
- "kebab": toKebab,
+ "exampleLine": g.exampleLine,
+ "currentYear": func() string { return strconv.Itoa(time.Now().Year()) },
+ "modulePath": func() string { return naming.CLI(s.Name) },
+ "graphqlQueryField": graphqlQueryField,
+ "graphqlFieldSelection": func(typeName string, types map[string]spec.TypeDef) []string {
+ return graphqlFieldSelection(typeName, types)
+ },
+ "backtick": func() string { return "`" },
+ "kebab": toKebab,
"humanName": func(s string) string {
// "steam-web" → "Steam Web", "notion" → "Notion"
return cases.Title(language.English).String(strings.ReplaceAll(s, "-", " "))
@@ -487,6 +492,16 @@ func (g *Generator) Generate() error {
}
}
+ // For GraphQL specs, emit additional client files (GraphQL transport + query constants)
+ if isGraphQLSpec(g.Spec) {
+ if err := g.renderTemplate("graphql_client.go.tmpl", filepath.Join("internal", "client", "graphql.go"), g.Spec); err != nil {
+ return fmt.Errorf("rendering graphql client: %w", err)
+ }
+ if err := g.renderTemplate("graphql_queries.go.tmpl", filepath.Join("internal", "client", "queries.go"), g.Spec); err != nil {
+ return fmt.Errorf("rendering graphql queries: %w", err)
+ }
+ }
+
// Compute promoted commands early — needed to determine Hidden flag on parent commands
promotedCommands := buildPromotedCommands(g.Spec)
hasPromoted := len(promotedCommands) > 0
@@ -659,6 +674,36 @@ func (g *Generator) Generate() error {
// Vision features: profile already computed in early profiling above
schema := BuildSchema(g.Spec)
+ // Add parent_id column to tables for dependent (parent-child) sync resources
+ if g.profile != nil {
+ depSet := make(map[string]bool)
+ for _, dep := range g.profile.DependentSyncResources {
+ depSet[dep.Name] = true
+ }
+ for i, table := range schema {
+ if depSet[table.Name] {
+ hasParentID := false
+ for _, col := range table.Columns {
+ if col.Name == "parent_id" {
+ hasParentID = true
+ break
+ }
+ }
+ if !hasParentID {
+ schema[i].Columns = append(schema[i].Columns, ColumnDef{
+ Name: "parent_id",
+ Type: "TEXT",
+ })
+ schema[i].Indexes = append(schema[i].Indexes, IndexDef{
+ Name: "idx_" + table.Name + "_parent_id",
+ TableName: table.Name,
+ Columns: "parent_id",
+ })
+ }
+ }
+ }
+ }
+
// Create store directory if needed
if g.VisionSet.Store {
if err := os.MkdirAll(filepath.Join(g.OutputDir, "internal", "store"), 0755); err != nil {
@@ -690,28 +735,41 @@ func (g *Generator) Generate() error {
"analytics.go.tmpl": filepath.Join("internal", "cli", "analytics.go"),
}
+ // Build GraphQL field path mapping for sync templates
+ gqlFieldPaths := map[string]string{}
+ for rName, r := range g.Spec.Resources {
+ if ep, ok := r.Endpoints["list"]; ok && ep.ResponsePath != "" {
+ gqlFieldPaths[rName] = graphqlQueryField(ep.ResponsePath)
+ }
+ }
+
visionData := struct {
*spec.APISpec
- SyncableResources []profiler.SyncableResource
- SearchableFields map[string][]string
- Tables []TableDef
- Pagination profiler.PaginationProfile
- SearchEndpointPath string
- SearchQueryParam string
- SearchEndpointMethod string
- SearchBodyFields []profiler.SearchBodyField
+ SyncableResources []profiler.SyncableResource
+ DependentSyncResources []profiler.DependentResource
+ SearchableFields map[string][]string
+ Tables []TableDef
+ Pagination profiler.PaginationProfile
+ SearchEndpointPath string
+ SearchQueryParam string
+ SearchEndpointMethod string
+ SearchBodyFields []profiler.SearchBodyField
+ GraphQLFieldPaths map[string]string
}{
- APISpec: g.Spec,
- SyncableResources: g.profile.SyncableResources,
- SearchableFields: g.profile.SearchableFields,
- Tables: schema,
- Pagination: g.profile.Pagination,
- SearchEndpointPath: g.profile.SearchEndpointPath,
- SearchQueryParam: g.profile.SearchQueryParam,
- SearchEndpointMethod: g.profile.SearchEndpointMethod,
- SearchBodyFields: g.profile.SearchBodyFields,
- }
-
+ APISpec: g.Spec,
+ SyncableResources: g.profile.SyncableResources,
+ DependentSyncResources: g.profile.DependentSyncResources,
+ SearchableFields: g.profile.SearchableFields,
+ Tables: schema,
+ Pagination: g.profile.Pagination,
+ SearchEndpointPath: g.profile.SearchEndpointPath,
+ SearchQueryParam: g.profile.SearchQueryParam,
+ SearchEndpointMethod: g.profile.SearchEndpointMethod,
+ SearchBodyFields: g.profile.SearchBodyFields,
+ GraphQLFieldPaths: gqlFieldPaths,
+ }
+
+ gqlSpec := isGraphQLSpec(g.Spec)
for _, tmplName := range g.VisionSet.TemplateNames() {
if tmplName == "store.go.tmpl" {
continue // already rendered above
@@ -720,11 +778,16 @@ func (g *Generator) Generate() error {
if !ok {
continue
}
+ // For GraphQL specs, use the GraphQL sync template instead of the REST one
+ actualTmpl := tmplName
+ if tmplName == "sync.go.tmpl" && gqlSpec {
+ actualTmpl = "graphql_sync.go.tmpl"
+ }
var tmplData any = g.Spec
if tmplName == "sync.go.tmpl" || tmplName == "search.go.tmpl" {
tmplData = visionData
}
- if err := g.renderTemplate(tmplName, outPath, tmplData); err != nil {
+ if err := g.renderTemplate(actualTmpl, outPath, tmplData); err != nil {
return fmt.Errorf("rendering vision %s: %w", tmplName, err)
}
}
@@ -1589,3 +1652,60 @@ func envVarPlaceholder(envVar string) string {
}
return strings.Join(lower, "_")
}
+
+// isGraphQLSpec returns true if the spec was produced by a GraphQL SDL parser.
+// Detection heuristic: all list endpoints have path "/graphql".
+func isGraphQLSpec(s *spec.APISpec) bool {
+ hasListEndpoint := false
+ for _, r := range s.Resources {
+ for eName, ep := range r.Endpoints {
+ if eName == "list" {
+ hasListEndpoint = true
+ if ep.Path != "/graphql" {
+ return false
+ }
+ }
+ }
+ }
+ return hasListEndpoint
+}
+
+// graphqlQueryField extracts the GraphQL query field name from a ResponsePath.
+// For example, "data.issues.nodes" returns "issues", "data.issue" returns "issue".
+// For SyncableResource.Path which is always "/graphql", return the resource name.
+func graphqlQueryField(responsePath string) string {
+ responsePath = strings.TrimPrefix(responsePath, "/graphql")
+ if responsePath == "" || responsePath == "/graphql" {
+ return ""
+ }
+ parts := strings.Split(responsePath, ".")
+ // Strip "data" prefix
+ if len(parts) > 0 && parts[0] == "data" {
+ parts = parts[1:]
+ }
+ // Strip "nodes" suffix
+ if len(parts) > 0 && parts[len(parts)-1] == "nodes" {
+ parts = parts[:len(parts)-1]
+ }
+ if len(parts) > 0 {
+ return parts[0]
+ }
+ return responsePath
+}
+
+// graphqlFieldSelection returns the list of field names for a GraphQL query
+// selection set, derived from the type definition in the spec.
+func graphqlFieldSelection(typeName string, types map[string]spec.TypeDef) []string {
+ td, ok := types[typeName]
+ if !ok {
+ return []string{"id"}
+ }
+ var fields []string
+ for _, f := range td.Fields {
+ fields = append(fields, f.Name)
+ }
+ if len(fields) == 0 {
+ return []string{"id"}
+ }
+ return fields
+}
diff --git a/internal/generator/generator_test.go b/internal/generator/generator_test.go
index c81e52cb..d22a62b3 100644
--- a/internal/generator/generator_test.go
+++ b/internal/generator/generator_test.go
@@ -7,6 +7,7 @@ import (
"strings"
"testing"
+ "github.com/mvanhorn/cli-printing-press/internal/graphql"
"github.com/mvanhorn/cli-printing-press/internal/naming"
"github.com/mvanhorn/cli-printing-press/internal/openapi"
"github.com/mvanhorn/cli-printing-press/internal/spec"
@@ -1669,3 +1670,138 @@ func TestEnvVarBuiltinFieldDedup(t *testing.T) {
})
}
}
+
+func TestGenerateDependentSyncCompiles(t *testing.T) {
+ t.Parallel()
+
+ // A spec with parent-child paths should generate compilable sync code
+ // that includes the dependent sync functions.
+ apiSpec := &spec.APISpec{
+ Name: "messaging",
+ Version: "0.1.0",
+ BaseURL: "https://api.example.com",
+ Auth: spec.AuthConfig{
+ Type: "api_key",
+ Header: "Authorization",
+ Format: "Bearer {token}",
+ EnvVars: []string{"MESSAGING_API_KEY"},
+ },
+ Config: spec.ConfigSpec{
+ Format: "toml",
+ Path: "~/.config/messaging-pp-cli/config.toml",
+ },
+ Resources: map[string]spec.Resource{
+ "channels": {
+ Description: "Manage channels",
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/channels",
+ Description: "List channels",
+ Response: spec.ResponseDef{Type: "array"},
+ Pagination: &spec.Pagination{CursorParam: "after", LimitParam: "limit"},
+ },
+ "create": {
+ Method: "POST",
+ Path: "/channels",
+ Description: "Create a channel",
+ Body: []spec.Param{
+ {Name: "name", Type: "string"},
+ {Name: "description", Type: "string"},
+ },
+ },
+ },
+ },
+ "messages": {
+ Description: "Manage messages",
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/channels/{channelId}/messages",
+ Description: "List messages in a channel",
+ Response: spec.ResponseDef{Type: "array"},
+ Pagination: &spec.Pagination{CursorParam: "after", LimitParam: "limit"},
+ },
+ "create": {
+ Method: "POST",
+ Path: "/channels/{channelId}/messages",
+ Description: "Send a message",
+ Body: []spec.Param{
+ {Name: "content", Type: "string"},
+ {Name: "title", Type: "string"},
+ },
+ },
+ },
+ },
+ "users": {
+ Description: "Manage users",
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/users",
+ Description: "List users",
+ Response: spec.ResponseDef{Type: "array"},
+ Pagination: &spec.Pagination{CursorParam: "after", LimitParam: "limit"},
+ },
+ },
+ },
+ },
+ }
+
+ outputDir := filepath.Join(t.TempDir(), naming.CLI(apiSpec.Name))
+ gen := New(apiSpec, outputDir)
+ require.NoError(t, gen.Generate())
+
+ // Verify sync.go was generated with dependent sync content
+ syncGo, err := os.ReadFile(filepath.Join(outputDir, "internal", "cli", "sync.go"))
+ require.NoError(t, err)
+ syncContent := string(syncGo)
+ assert.Contains(t, syncContent, "syncDependentResource", "sync.go should contain dependent sync function")
+ assert.Contains(t, syncContent, "dependentResourceDefs", "sync.go should contain dependent resource definitions")
+ assert.Contains(t, syncContent, `"messages"`, "sync.go should reference messages as a dependent resource")
+ assert.Contains(t, syncContent, `"channels"`, "sync.go should reference channels as the parent")
+
+ // The generated project should compile
+ runGoCommand(t, outputDir, "mod", "tidy")
+ runGoCommand(t, outputDir, "build", "./...")
+}
+
+func TestGenerateGraphQLCompiles(t *testing.T) {
+ t.Parallel()
+
+ // Parse a GraphQL SDL fixture and verify the generated CLI compiles
+ gqlSpec, err := graphql.ParseSDL(filepath.Join("..", "..", "testdata", "graphql", "test.graphql"))
+ require.NoError(t, err)
+
+ outputDir := filepath.Join(t.TempDir(), naming.CLI(gqlSpec.Name))
+ gen := New(gqlSpec, outputDir)
+ require.NoError(t, gen.Generate())
+
+ // Verify GraphQL-specific files were generated
+ _, err = os.Stat(filepath.Join(outputDir, "internal", "client", "graphql.go"))
+ require.NoError(t, err, "graphql.go should be generated")
+
+ _, err = os.Stat(filepath.Join(outputDir, "internal", "client", "queries.go"))
+ require.NoError(t, err, "queries.go should be generated")
+
+ // Verify sync.go uses GraphQL patterns (POST /graphql, not GET-based REST)
+ syncGo, err := os.ReadFile(filepath.Join(outputDir, "internal", "cli", "sync.go"))
+ require.NoError(t, err)
+ syncContent := string(syncGo)
+ assert.Contains(t, syncContent, "Query", "sync.go should use GraphQL Query method")
+ assert.Contains(t, syncContent, "graphql", "sync.go should reference graphql")
+ assert.NotContains(t, syncContent, "c.Get(path", "sync.go should not use REST GET pattern")
+
+ // Verify queries.go has query constants
+ queriesGo, err := os.ReadFile(filepath.Join(outputDir, "internal", "client", "queries.go"))
+ require.NoError(t, err)
+ queriesContent := string(queriesGo)
+ assert.Contains(t, queriesContent, "ListQuery", "queries.go should contain list query constants")
+ assert.Contains(t, queriesContent, "pageInfo", "queries.go should include pageInfo in queries")
+ assert.Contains(t, queriesContent, "hasNextPage", "queries.go should include hasNextPage")
+ assert.Contains(t, queriesContent, "endCursor", "queries.go should include endCursor")
+
+ // The generated project should compile
+ runGoCommand(t, outputDir, "mod", "tidy")
+ runGoCommand(t, outputDir, "build", "./...")
+}
diff --git a/internal/generator/schema_builder.go b/internal/generator/schema_builder.go
index 16040434..928563f0 100644
--- a/internal/generator/schema_builder.go
+++ b/internal/generator/schema_builder.go
@@ -49,7 +49,7 @@ func BuildSchema(s *spec.APISpec) []TableDef {
},
}
- if gravity >= 4 {
+ if gravity >= 2 {
fields := collectResponseFields(resource)
for _, f := range fields {
if isScalarField(f) && f.Name != "id" {
@@ -79,13 +79,13 @@ func BuildSchema(s *spec.APISpec) []TableDef {
}
textFields := collectTextFieldNames(resource)
- if len(textFields) >= 2 && gravity >= 4 {
+ if len(textFields) >= 2 && gravity >= 2 {
table.FTS5 = true
table.FTS5Fields = textFields
// Only use content-sync triggers when ALL FTS fields are
// actual extracted columns on the table. Otherwise the
// triggers reference non-existent columns and fail.
- table.FTS5Triggers = allFieldsAreColumns(textFields, table.Columns)
+ table.FTS5Triggers = true
}
tables = append(tables, table)
@@ -344,22 +344,6 @@ func safeSQLName(name string) string {
return name
}
-// allFieldsAreColumns returns true if every field name exists as an extracted
-// column on the table. Used to decide whether FTS content-sync triggers are
-// safe (they reference new.field which only works for real columns).
-func allFieldsAreColumns(fields []string, columns []ColumnDef) bool {
- colSet := make(map[string]bool, len(columns))
- for _, col := range columns {
- colSet[col.Name] = true
- }
- for _, f := range fields {
- if !colSet[f] {
- return false
- }
- }
- return true
-}
-
// toSnakeCase converts camelCase, PascalCase, or kebab-case to snake_case.
func toSnakeCase(s string) string {
s = strings.ReplaceAll(s, ".", "_")
diff --git a/internal/generator/templates/graphql_client.go.tmpl b/internal/generator/templates/graphql_client.go.tmpl
new file mode 100644
index 00000000..e7aad2af
--- /dev/null
+++ b/internal/generator/templates/graphql_client.go.tmpl
@@ -0,0 +1,108 @@
+// Copyright {{currentYear}} {{.Owner}}. Licensed under Apache-2.0. See LICENSE.
+// Generated by CLI Printing Press (https://github.com/mvanhorn/cli-printing-press). DO NOT EDIT.
+
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+ "strings"
+)
+
+// GraphQLRequest is a standard GraphQL POST body.
+type GraphQLRequest struct {
+ Query string `json:"query"`
+ Variables map[string]any `json:"variables,omitempty"`
+}
+
+// GraphQLResponse wraps the standard GraphQL response envelope.
+type GraphQLResponse struct {
+ Data json.RawMessage `json:"data"`
+ Errors []GraphQLError `json:"errors,omitempty"`
+}
+
+// GraphQLError represents a single error from a GraphQL response.
+type GraphQLError struct {
+ Message string `json:"message"`
+ Extensions struct {
+ Code string `json:"code"`
+ } `json:"extensions"`
+}
+
+// Query executes a GraphQL query via POST /graphql and returns the raw data payload.
+func (c *Client) Query(query string, variables map[string]any) (json.RawMessage, error) {
+ req := GraphQLRequest{Query: query, Variables: variables}
+ raw, _, err := c.Post("/graphql", req)
+ if err != nil {
+ return nil, err
+ }
+
+ var resp GraphQLResponse
+ if err := json.Unmarshal(raw, &resp); err != nil {
+ return nil, fmt.Errorf("decoding graphql response: %w", err)
+ }
+ if len(resp.Errors) > 0 {
+ msgs := make([]string, len(resp.Errors))
+ for i, e := range resp.Errors {
+ msgs[i] = e.Message
+ }
+ return nil, fmt.Errorf("graphql: %s", strings.Join(msgs, "; "))
+ }
+ return resp.Data, nil
+}
+
+// Mutate executes a GraphQL mutation via POST /graphql and returns the raw data payload.
+// Identical transport to Query; separate method for clarity.
+func (c *Client) Mutate(query string, variables map[string]any) (json.RawMessage, error) {
+ return c.Query(query, variables)
+}
+
+// PaginatedQuery fetches all pages of a Connection field using first/after pagination.
+// fieldPath is the top-level field name in the response (e.g., "issues").
+// Returns all collected nodes across all pages.
+func (c *Client) PaginatedQuery(query string, variables map[string]any, fieldPath string, pageSize int) ([]json.RawMessage, error) {
+ if variables == nil {
+ variables = map[string]any{}
+ }
+ if pageSize <= 0 {
+ pageSize = 50
+ }
+ variables["first"] = pageSize
+
+ var all []json.RawMessage
+ hasNext := true
+ for hasNext {
+ data, err := c.Query(query, variables)
+ if err != nil {
+ return all, err
+ }
+
+ var root map[string]json.RawMessage
+ if err := json.Unmarshal(data, &root); err != nil {
+ return all, fmt.Errorf("parsing paginated root: %w", err)
+ }
+
+ connRaw, ok := root[fieldPath]
+ if !ok {
+ return all, fmt.Errorf("field %q not found in response", fieldPath)
+ }
+
+ var conn struct {
+ Nodes []json.RawMessage `json:"nodes"`
+ PageInfo struct {
+ HasNextPage bool `json:"hasNextPage"`
+ EndCursor string `json:"endCursor"`
+ } `json:"pageInfo"`
+ }
+ if err := json.Unmarshal(connRaw, &conn); err != nil {
+ return all, fmt.Errorf("parsing connection %q: %w", fieldPath, err)
+ }
+
+ all = append(all, conn.Nodes...)
+ hasNext = conn.PageInfo.HasNextPage
+ if hasNext {
+ variables["after"] = conn.PageInfo.EndCursor
+ }
+ }
+ return all, nil
+}
diff --git a/internal/generator/templates/graphql_queries.go.tmpl b/internal/generator/templates/graphql_queries.go.tmpl
new file mode 100644
index 00000000..73412590
--- /dev/null
+++ b/internal/generator/templates/graphql_queries.go.tmpl
@@ -0,0 +1,35 @@
+// Copyright {{currentYear}} {{.Owner}}. Licensed under Apache-2.0. See LICENSE.
+// Generated by CLI Printing Press (https://github.com/mvanhorn/cli-printing-press). DO NOT EDIT.
+
+package client
+
+// GraphQL query constants generated from the API spec.
+{{- range $rName, $resource := .Resources}}
+{{- range $eName, $endpoint := $resource.Endpoints}}
+{{- if eq $eName "list"}}
+{{- $queryField := graphqlQueryField $endpoint.ResponsePath}}
+
+const {{pascal $rName}}ListQuery = {{backtick}}query($first: Int!, $after: String) {
+ {{$queryField}}(first: $first, after: $after) {
+ nodes {
+{{- range graphqlFieldSelection $endpoint.Response.Item $.Types}}
+ {{.}}
+{{- end}}
+ }
+ pageInfo { hasNextPage endCursor }
+ }
+}{{backtick}}
+{{- end}}
+{{- if eq $eName "get"}}
+{{- $queryField := graphqlQueryField $endpoint.ResponsePath}}
+
+const {{pascal $rName}}GetQuery = {{backtick}}query($id: String!) {
+ {{$queryField}}(id: $id) {
+{{- range graphqlFieldSelection $endpoint.Response.Item $.Types}}
+ {{.}}
+{{- end}}
+ }
+}{{backtick}}
+{{- end}}
+{{- end}}
+{{- end}}
diff --git a/internal/generator/templates/graphql_sync.go.tmpl b/internal/generator/templates/graphql_sync.go.tmpl
new file mode 100644
index 00000000..2bff885c
--- /dev/null
+++ b/internal/generator/templates/graphql_sync.go.tmpl
@@ -0,0 +1,354 @@
+// Copyright {{currentYear}} {{.Owner}}. Licensed under Apache-2.0. See LICENSE.
+// Generated by CLI Printing Press (https://github.com/mvanhorn/cli-printing-press). DO NOT EDIT.
+
+package cli
+
+import (
+ "encoding/json"
+ "fmt"
+ "os"
+ "regexp"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "{{modulePath}}/internal/client"
+ "{{modulePath}}/internal/store"
+ "github.com/spf13/cobra"
+)
+
+// syncResult holds the outcome of syncing a single resource.
+type syncResult struct {
+ Resource string
+ Count int
+ Err error
+ Duration time.Duration
+}
+
+func newSyncCmd(flags *rootFlags) *cobra.Command {
+ var resources []string
+ var full bool
+ var since string
+ var concurrency int
+ var dbPath string
+ var maxPages int
+
+ cmd := &cobra.Command{
+ Use: "sync",
+ Short: "Sync API data to local SQLite for offline search and analysis",
+ Long: `Sync data from the API into a local SQLite database. Supports resumable
+incremental sync (only fetches new data since last sync) and full resync.
+Once synced, use the 'search' command for instant full-text search.`,
+ Example: ` # Sync all resources
+ {{.Name}}-pp-cli sync
+
+ # Sync specific resources only
+ {{.Name}}-pp-cli sync --resources channels,messages
+
+ # Full resync (ignore previous checkpoint)
+ {{.Name}}-pp-cli sync --full
+
+ # Parallel sync with 8 workers
+ {{.Name}}-pp-cli sync --concurrency 8`,
+ RunE: func(cmd *cobra.Command, args []string) error {
+ c, err := flags.newClient()
+ if err != nil {
+ return err
+ }
+ c.NoCache = true
+
+ if dbPath == "" {
+ dbPath = defaultDBPath("{{.Name}}-pp-cli")
+ }
+
+ db, err := store.Open(dbPath)
+ if err != nil {
+ return fmt.Errorf("opening local database: %w", err)
+ }
+ defer db.Close()
+
+ // If no specific resources, sync top-level resources
+ if len(resources) == 0 {
+ resources = defaultSyncResources()
+ }
+
+ // --full: clear all sync cursors before starting
+ if full {
+ for _, resource := range resources {
+ _ = db.SaveSyncState(resource, "", 0)
+ }
+ }
+
+ // Resolve --since into an RFC3339 timestamp
+ sinceTS := ""
+ if since != "" {
+ ts, err := parseSinceDuration(since)
+ if err != nil {
+ return fmt.Errorf("invalid --since value %q: %w", since, err)
+ }
+ sinceTS = ts.Format(time.RFC3339)
+ }
+
+ // Worker pool: produce resources, N workers consume
+ if concurrency < 1 {
+ concurrency = 4
+ }
+
+ started := time.Now()
+ work := make(chan string, len(resources))
+ results := make(chan syncResult, len(resources))
+
+ var wg sync.WaitGroup
+ for i := 0; i < concurrency; i++ {
+ wg.Add(1)
+ go func() {
+ defer wg.Done()
+ for resource := range work {
+ res := syncResource(c, db, resource, sinceTS, full, maxPages)
+ results <- res
+ }
+ }()
+ }
+
+ // Enqueue all resources
+ for _, resource := range resources {
+ work <- resource
+ }
+ close(work)
+
+ // Collect results in a separate goroutine
+ go func() {
+ wg.Wait()
+ close(results)
+ }()
+
+ var totalSynced int
+ var errCount int
+ for res := range results {
+ if res.Err != nil {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, " %s: error: %v\n", res.Resource, res.Err)
+ }
+ errCount++
+ } else {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, " %s: %d synced (done)\n", res.Resource, res.Count)
+ }
+ totalSynced += res.Count
+ }
+ }
+
+ elapsed := time.Since(started)
+ fmt.Fprintf(os.Stderr, "Sync complete: %d records across %d resources (%.1fs)\n",
+ totalSynced, len(resources), elapsed.Seconds())
+
+ if errCount > 0 {
+ return fmt.Errorf("%d resource(s) failed to sync", errCount)
+ }
+ return nil
+ },
+ }
+
+ cmd.Flags().StringSliceVar(&resources, "resources", nil, "Comma-separated resource types to sync")
+ cmd.Flags().BoolVar(&full, "full", false, "Full resync (ignore previous checkpoint)")
+ cmd.Flags().StringVar(&since, "since", "", "Incremental sync duration (e.g. 7d, 24h, 1w, 30m)")
+ cmd.Flags().IntVar(&concurrency, "concurrency", 4, "Number of parallel sync workers")
+ cmd.Flags().StringVar(&dbPath, "db", "", "Database path (default: ~/.local/share/{{.Name}}-pp-cli/data.db)")
+ cmd.Flags().IntVar(&maxPages, "max-pages", 10, "Maximum pages to fetch per resource (0 = unlimited)")
+
+ return cmd
+}
+
+// graphqlSyncDef describes how to sync a resource via GraphQL.
+type graphqlSyncDef struct {
+ Query string // GraphQL query constant
+ FieldPath string // top-level field in the response data (e.g., "issues")
+ PageSize int // default page size for this resource
+}
+
+// graphqlSyncDefs returns the sync definitions for all syncable resources.
+func graphqlSyncDefs() map[string]graphqlSyncDef {
+ return map[string]graphqlSyncDef{
+{{- range .SyncableResources}}
+ "{{.Name}}": {
+ Query: client.{{pascal .Name}}ListQuery,
+ FieldPath: "{{index $.GraphQLFieldPaths .Name}}",
+ PageSize: 50,
+ },
+{{- end}}
+ }
+}
+
+// syncResource handles the full paginated sync of a single resource via GraphQL.
+func syncResource(c *client.Client, db *store.Store, resource, sinceTS string, full bool, maxPages int) syncResult {
+ started := time.Now()
+
+ if !humanFriendly {
+ fmt.Fprintf(os.Stderr, `{"event":"sync_start","resource":"%s"}`+"\n", resource)
+ }
+
+ defs := graphqlSyncDefs()
+ def, ok := defs[resource]
+ if !ok {
+ return syncResult{Resource: resource, Err: fmt.Errorf("unknown resource %q", resource), Duration: time.Since(started)}
+ }
+
+ // Resume cursor from sync_state (unless --full cleared it)
+ existingCursor, _, _, _ := db.GetSyncState(resource)
+ cursor := existingCursor
+
+ var totalCount int
+ pagesFetched := 0
+ pageSize := def.PageSize
+ if pageSize <= 0 {
+ pageSize = 50
+ }
+
+ for {
+ variables := map[string]any{
+ "first": pageSize,
+ }
+ if cursor != "" {
+ variables["after"] = cursor
+ }
+
+ data, err := c.Query(def.Query, variables)
+ if err != nil {
+ if !humanFriendly {
+ fmt.Fprintf(os.Stderr, `{"event":"sync_error","resource":"%s","error":"%s"}`+"\n", resource, strings.ReplaceAll(err.Error(), `"`, `\"`))
+ }
+ return syncResult{Resource: resource, Count: totalCount, Err: fmt.Errorf("fetching %s: %w", resource, err), Duration: time.Since(started)}
+ }
+
+ // Navigate to the connection field in the response
+ var root map[string]json.RawMessage
+ if err := json.Unmarshal(data, &root); err != nil {
+ return syncResult{Resource: resource, Count: totalCount, Err: fmt.Errorf("parsing response for %s: %w", resource, err), Duration: time.Since(started)}
+ }
+
+ connRaw, ok := root[def.FieldPath]
+ if !ok {
+ // Try to find the field by scanning the response keys
+ for k, v := range root {
+ _ = k
+ connRaw = v
+ break
+ }
+ if connRaw == nil {
+ break
+ }
+ }
+
+ var conn struct {
+ Nodes []json.RawMessage `json:"nodes"`
+ PageInfo struct {
+ HasNextPage bool `json:"hasNextPage"`
+ EndCursor string `json:"endCursor"`
+ } `json:"pageInfo"`
+ }
+ if err := json.Unmarshal(connRaw, &conn); err != nil {
+ return syncResult{Resource: resource, Count: totalCount, Err: fmt.Errorf("parsing connection for %s: %w", resource, err), Duration: time.Since(started)}
+ }
+
+ if len(conn.Nodes) == 0 {
+ break
+ }
+
+ // Batch upsert all items from this page
+ if err := db.UpsertBatch(resource, conn.Nodes); err != nil {
+ if !humanFriendly {
+ fmt.Fprintf(os.Stderr, `{"event":"sync_error","resource":"%s","error":"%s"}`+"\n", resource, strings.ReplaceAll(err.Error(), `"`, `\"`))
+ }
+ return syncResult{Resource: resource, Count: totalCount, Err: fmt.Errorf("upserting batch for %s: %w", resource, err), Duration: time.Since(started)}
+ }
+
+ totalCount += len(conn.Nodes)
+
+ // Progress reporting
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, "\r %s: %d synced", resource, totalCount)
+ } else {
+ fmt.Fprintf(os.Stderr, `{"event":"sync_progress","resource":"%s","fetched":%d}`+"\n", resource, totalCount)
+ }
+
+ // Save cursor after each page for resumability
+ if err := db.SaveSyncState(resource, conn.PageInfo.EndCursor, totalCount); err != nil {
+ fmt.Fprintf(os.Stderr, "\nwarning: failed to save sync state for %s: %v\n", resource, err)
+ }
+
+ pagesFetched++
+
+ // Enforce page ceiling
+ if maxPages > 0 && pagesFetched >= maxPages {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, "\n %s: reached --max-pages limit (%d pages, %d items)\n", resource, maxPages, totalCount)
+ }
+ break
+ }
+
+ // Check pagination
+ if !conn.PageInfo.HasNextPage || conn.PageInfo.EndCursor == "" {
+ break
+ }
+
+ cursor = conn.PageInfo.EndCursor
+ }
+
+ // Final sync state: clear cursor (sync is complete), update count
+ _ = db.SaveSyncState(resource, "", totalCount)
+
+ if !humanFriendly {
+ fmt.Fprintf(os.Stderr, `{"event":"sync_complete","resource":"%s","total":%d,"duration_ms":%d}`+"\n", resource, totalCount, time.Since(started).Milliseconds())
+ }
+
+ return syncResult{Resource: resource, Count: totalCount, Duration: time.Since(started)}
+}
+
+func defaultSyncResources() []string {
+ return []string{
+{{- range .SyncableResources}}
+ "{{.Name}}",
+{{- end}}
+ }
+}
+
+// parseSinceDuration converts human-friendly duration strings into a time.Time.
+func parseSinceDuration(s string) (time.Time, error) {
+ re := regexp.MustCompile(`^(\d+)([dhwm])$`)
+ matches := re.FindStringSubmatch(strings.TrimSpace(s))
+ if matches == nil {
+ return time.Time{}, fmt.Errorf("expected format like 7d, 24h, 1w, or 30m")
+ }
+
+ n, err := strconv.Atoi(matches[1])
+ if err != nil {
+ return time.Time{}, err
+ }
+
+ now := time.Now()
+ switch matches[2] {
+ case "d":
+ return now.Add(-time.Duration(n) * 24 * time.Hour), nil
+ case "h":
+ return now.Add(-time.Duration(n) * time.Hour), nil
+ case "w":
+ return now.Add(-time.Duration(n) * 7 * 24 * time.Hour), nil
+ case "m":
+ return now.Add(-time.Duration(n) * time.Minute), nil
+ default:
+ return time.Time{}, fmt.Errorf("unknown unit %q", matches[2])
+ }
+}
+
+func extractID(obj map[string]any) string {
+ for _, key := range []string{"id", "ID", "ticker", "key", "code", "uid", "uuid", "slug", "name"} {
+ if v, ok := obj[key]; ok {
+ s := fmt.Sprintf("%v", v)
+ if s != "" && s != "<nil>" {
+ return s
+ }
+ }
+ }
+ return ""
+}
diff --git a/internal/generator/templates/store.go.tmpl b/internal/generator/templates/store.go.tmpl
index dc5432ed..c93c4387 100644
--- a/internal/generator/templates/store.go.tmpl
+++ b/internal/generator/templates/store.go.tmpl
@@ -41,7 +41,10 @@ func Open(dbPath string) (*Store, error) {
return nil, fmt.Errorf("opening database: %w", err)
}
- db.SetMaxOpenConns(1)
+ // WAL mode + 2 connections allows one read cursor open while a second
+ // query executes (e.g., analytics commands calling helpers during row
+ // iteration). Writes are still serialized by SQLite's WAL lock.
+ db.SetMaxOpenConns(2)
s := &Store{db: db, path: dbPath}
if err := s.migrate(); err != nil {
@@ -482,6 +485,32 @@ func (s *Store) GetSyncCursor(resourceType string) string {
return ""
}
+// ListIDs returns all IDs from a resource's domain table, or from the generic
+// resources table if no domain table exists. Used by dependent sync to iterate parents.
+func (s *Store) ListIDs(resourceType string) ([]string, error) {
+ // Try domain table first (tables are named after the resource type)
+ query := fmt.Sprintf("SELECT id FROM %s", resourceType)
+ rows, err := s.db.Query(query)
+ if err != nil {
+ // Fall back to generic resources table
+ rows, err = s.db.Query("SELECT id FROM resources WHERE resource_type = ?", resourceType)
+ if err != nil {
+ return nil, err
+ }
+ }
+ defer rows.Close()
+
+ var ids []string
+ for rows.Next() {
+ var id string
+ if err := rows.Scan(&id); err != nil {
+ continue
+ }
+ ids = append(ids, id)
+ }
+ return ids, rows.Err()
+}
+
// GetLastSyncedAt returns the last sync timestamp for a resource type.
func (s *Store) GetLastSyncedAt(resourceType string) string {
var ts sql.NullString
diff --git a/internal/generator/templates/sync.go.tmpl b/internal/generator/templates/sync.go.tmpl
index 2c84209d..990c8b8a 100644
--- a/internal/generator/templates/sync.go.tmpl
+++ b/internal/generator/templates/sync.go.tmpl
@@ -154,6 +154,24 @@ Once synced, use the 'search' command for instant full-text search.`,
}
}
+{{- if .DependentSyncResources}}
+ // Sync dependent (parent-child) resources sequentially after flat resources.
+ depResults := syncDependentResources(c, db, sinceTS, full, maxPages{{if .Pagination.DateRangeParam}}, syncDates{{end}})
+ for _, res := range depResults {
+ if res.Err != nil {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, " %s: error: %v\n", res.Resource, res.Err)
+ }
+ errCount++
+ } else {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, " %s: %d synced (done)\n", res.Resource, res.Count)
+ }
+ totalSynced += res.Count
+ }
+ }
+{{- end}}
+
elapsed := time.Since(started)
fmt.Fprintf(os.Stderr, "Sync complete: %d records across %d resources (%.1fs)\n",
totalSynced, len(resources), elapsed.Seconds())
@@ -515,6 +533,147 @@ func syncResourcePath(resource string) string {
return "/" + resource
}
+{{- if .DependentSyncResources}}
+
+// dependentResourceDef describes a child resource that requires iterating parent IDs to sync.
+type dependentResourceDef struct {
+ Name string
+ ParentTable string
+ ParentIDParam string
+ PathTemplate string
+}
+
+func dependentResourceDefs() []dependentResourceDef {
+ return []dependentResourceDef{
+{{- range .DependentSyncResources}}
+ {Name: "{{.Name}}", ParentTable: "{{.ParentResource}}", ParentIDParam: "{{.ParentIDParam}}", PathTemplate: "{{.Path}}"},
+{{- end}}
+ }
+}
+
+// syncDependentResources iterates parent tables and syncs child resources per parent ID.
+func syncDependentResources(c interface {
+ Get(string, map[string]string) (json.RawMessage, error)
+ RateLimit() float64
+}, db *store.Store, sinceTS string, full bool, maxPages int{{if .Pagination.DateRangeParam}}, dates string{{end}}) []syncResult {
+ var results []syncResult
+ for _, dep := range dependentResourceDefs() {
+ res := syncDependentResource(c, db, dep, sinceTS, full, maxPages{{if .Pagination.DateRangeParam}}, dates{{end}})
+ results = append(results, res)
+ }
+ return results
+}
+
+// syncDependentResource syncs a single child resource by iterating all parent IDs.
+func syncDependentResource(c interface {
+ Get(string, map[string]string) (json.RawMessage, error)
+ RateLimit() float64
+}, db *store.Store, dep dependentResourceDef, sinceTS string, full bool, maxPages int{{if .Pagination.DateRangeParam}}, dates string{{end}}) syncResult {
+ started := time.Now()
+
+ // Query parent table for all IDs
+ parentIDs, err := db.ListIDs(dep.ParentTable)
+ if err != nil || len(parentIDs) == 0 {
+ if len(parentIDs) == 0 {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, " %s: skipping (parent table %s is empty, sync it first)\n", dep.Name, dep.ParentTable)
+ }
+ return syncResult{Resource: dep.Name, Duration: time.Since(started)}
+ }
+ return syncResult{Resource: dep.Name, Err: fmt.Errorf("querying parent table %s: %w", dep.ParentTable, err), Duration: time.Since(started)}
+ }
+
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, " %s: syncing for %d %s parents\n", dep.Name, len(parentIDs), dep.ParentTable)
+ }
+
+ var totalCount int
+ pageSize := determinePaginationDefaults()
+
+ for idx, parentID := range parentIDs {
+ // Build child endpoint path by replacing the param placeholder
+ path := strings.Replace(dep.PathTemplate, "{"+dep.ParentIDParam+"}", parentID, 1)
+
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, "\r %s: syncing for %s (%d/%d parents)", dep.Name, dep.ParentTable, idx+1, len(parentIDs))
+ }
+
+ cursor := ""
+ pagesFetched := 0
+
+ for {
+ params := map[string]string{}
+ params[pageSize.limitParam] = strconv.Itoa(pageSize.limit)
+ if cursor != "" {
+ params[pageSize.cursorParam] = cursor
+ }
+ if sinceTS != "" {
+ params[determineSinceParam()] = sinceTS
+ }
+{{- if .Pagination.DateRangeParam}}
+ if dates != "" {
+ params[determineDateRangeParam()] = dates
+ }
+{{- end}}
+
+ data, err := c.Get(path, params)
+ if err != nil {
+ // Non-fatal per parent: log and continue to next parent
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, "\n %s: error for parent %s: %v\n", dep.Name, parentID, err)
+ }
+ break
+ }
+
+ items, nextCursor, hasMore := extractPageItems(data, pageSize.cursorParam)
+ if len(items) == 0 {
+ break
+ }
+
+ // Inject parent_id into each item before upserting
+ for i, item := range items {
+ var obj map[string]json.RawMessage
+ if err := json.Unmarshal(item, &obj); err == nil {
+ parentIDJSON, _ := json.Marshal(parentID)
+ obj["parent_id"] = parentIDJSON
+ if modified, err := json.Marshal(obj); err == nil {
+ items[i] = modified
+ }
+ }
+ }
+
+ if err := db.UpsertBatch(dep.Name, items); err != nil {
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, "\n %s: upsert error for parent %s: %v\n", dep.Name, parentID, err)
+ }
+ break
+ }
+
+ totalCount += len(items)
+ pagesFetched++
+
+ if maxPages > 0 && pagesFetched >= maxPages {
+ break
+ }
+ if !hasMore || len(items) < pageSize.limit || nextCursor == "" {
+ break
+ }
+ cursor = nextCursor
+ }
+
+ // Brief rate-limit pause between parents to avoid hammering the API
+ time.Sleep(100 * time.Millisecond)
+ }
+
+ if humanFriendly {
+ fmt.Fprintf(os.Stderr, "\n")
+ }
+
+ _ = db.SaveSyncState(dep.Name, "", totalCount)
+ return syncResult{Resource: dep.Name, Count: totalCount, Duration: time.Since(started)}
+}
+{{- end}}
+
func extractID(obj map[string]any) string {
for _, key := range []string{"id", "ID", "ticker", "event_ticker", "series_ticker", "key", "code", "uid", "uuid", "slug", "name"} {
if v, ok := obj[key]; ok {
diff --git a/internal/openapi/parser.go b/internal/openapi/parser.go
index 66aa9743..903ecd19 100644
--- a/internal/openapi/parser.go
+++ b/internal/openapi/parser.go
@@ -900,6 +900,41 @@ func securitySchemeValue(ref *openapi3.SecuritySchemeRef) *openapi3.SecuritySche
return ref.Value
}
+// pathPriorityScore assigns a priority score to an API path so that
+// user-facing endpoints sort before admin/internal ones. Higher is better.
+// Scoring rules:
+// - Base score: 100
+// - Subtract 10 per path segment (depth penalty)
+// - Subtract 30 if any segment starts with admin, internal, system, or management
+// - Add 10 for short paths (2 or fewer segments)
+func pathPriorityScore(path string) int {
+ score := 100
+
+ trimmed := strings.TrimPrefix(path, "/")
+ if trimmed == "" {
+ return score + 10 // root path, short bonus
+ }
+ segments := strings.Split(trimmed, "/")
+ score -= 10 * len(segments)
+
+ if len(segments) <= 2 {
+ score += 10
+ }
+
+ lowPath := strings.ToLower(path)
+ for _, prefix := range []string{"admin", "internal", "system", "management"} {
+ for _, seg := range strings.Split(strings.TrimPrefix(lowPath, "/"), "/") {
+ // Match segments that start with the prefix (catches admin, admin.users, etc.)
+ if strings.HasPrefix(seg, prefix) {
+ score -= 30
+ break
+ }
+ }
+ }
+
+ return score
+}
+
func mapResources(doc *openapi3.T, out *spec.APISpec, basePath string) {
if doc == nil || out == nil || doc.Paths == nil {
return
@@ -912,7 +947,13 @@ func mapResources(doc *openapi3.T, out *spec.APISpec, basePath string) {
for path := range pathMap {
pathKeys = append(pathKeys, path)
}
- sort.Strings(pathKeys)
+ sort.SliceStable(pathKeys, func(i, j int) bool {
+ si, sj := pathPriorityScore(pathKeys[i]), pathPriorityScore(pathKeys[j])
+ if si != sj {
+ return si > sj
+ }
+ return pathKeys[i] < pathKeys[j]
+ })
commonPrefix := detectCommonPrefix(pathKeys, basePath)
// Auto-calibrate endpoint limit unless the user explicitly set it.
diff --git a/internal/openapi/parser_test.go b/internal/openapi/parser_test.go
index c872da54..3a81644a 100644
--- a/internal/openapi/parser_test.go
+++ b/internal/openapi/parser_test.go
@@ -2,9 +2,11 @@ package openapi
import (
"encoding/json"
+ "fmt"
"os"
"os/exec"
"path/filepath"
+ "sort"
"testing"
"github.com/getkin/kin-openapi/openapi3"
@@ -920,6 +922,62 @@ paths:
})
}
+func TestPathPriorityScore(t *testing.T) {
+ t.Parallel()
+
+ t.Run("admin paths score lower than user paths", func(t *testing.T) {
+ assert.Greater(t, pathPriorityScore("/users"), pathPriorityScore("/admin/users"))
+ assert.Greater(t, pathPriorityScore("/channels"), pathPriorityScore("/admin.conversations"))
+ assert.Greater(t, pathPriorityScore("/messages"), pathPriorityScore("/internal/metrics"))
+ assert.Greater(t, pathPriorityScore("/items"), pathPriorityScore("/system/health"))
+ assert.Greater(t, pathPriorityScore("/teams"), pathPriorityScore("/management/roles"))
+ })
+
+ t.Run("shallow paths score higher than deep paths", func(t *testing.T) {
+ assert.Greater(t, pathPriorityScore("/users"), pathPriorityScore("/users/{id}/posts/{postId}/comments"))
+ })
+
+ t.Run("short paths get bonus", func(t *testing.T) {
+ short := pathPriorityScore("/users")
+ long := pathPriorityScore("/a/b/c/d")
+ assert.Greater(t, short, long)
+ })
+}
+
+func TestPathPriorityScoreSortOrder(t *testing.T) {
+ t.Parallel()
+
+ // Build 600 paths: 100 admin.* paths and 500 user-facing paths.
+ var paths []string
+ for i := 0; i < 100; i++ {
+ paths = append(paths, fmt.Sprintf("/admin.resource%d/action", i))
+ }
+ for i := 0; i < 500; i++ {
+ paths = append(paths, fmt.Sprintf("/resource%d", i))
+ }
+
+ // Sort by priority score descending, alphabetical tiebreaker.
+ sort.SliceStable(paths, func(i, j int) bool {
+ si, sj := pathPriorityScore(paths[i]), pathPriorityScore(paths[j])
+ if si != sj {
+ return si > sj
+ }
+ return paths[i] < paths[j]
+ })
+
+ // With a 500-path cap, all admin paths should be in the tail (indices 500+).
+ const maxResources = 500
+ kept := paths[:maxResources]
+ dropped := paths[maxResources:]
+
+ for _, p := range dropped {
+ assert.Contains(t, p, "admin", "expected only admin paths to be dropped, but got: %s", p)
+ }
+ for _, p := range kept {
+ assert.NotContains(t, p, "admin", "expected no admin paths in kept set, but got: %s", p)
+ }
+}
+
func mustMarshalJSON(t *testing.T, v any) []byte {
t.Helper()
data, err := json.Marshal(v)
diff --git a/internal/pipeline/scorecard.go b/internal/pipeline/scorecard.go
index 1b033b6a..9ec15ce0 100644
--- a/internal/pipeline/scorecard.go
+++ b/internal/pipeline/scorecard.go
@@ -1125,6 +1125,39 @@ func loadOpenAPISpec(specPath string) (*openAPISpecInfo, error) {
}
}
+ // Swagger 2.0 fallback: check top-level securityDefinitions when OAS3
+ // components.securitySchemes is empty or missing.
+ if len(info.SecuritySchemes) == 0 {
+ if securityDefs, ok := raw["securityDefinitions"].(map[string]any); ok {
+ for schemeName, value := range securityDefs {
+ scheme := openAPISecurityScheme{Key: schemeName}
+ if fields, ok := value.(map[string]any); ok {
+ swType := strings.ToLower(asString(fields["type"]))
+ swIn := strings.ToLower(asString(fields["in"]))
+ swName := asString(fields["name"])
+
+ switch swType {
+ case "oauth2":
+ scheme.Type = "oauth2"
+ case "apikey":
+ scheme.Type = "apikey"
+ scheme.In = swIn
+ scheme.HeaderName = swName
+ // Detect Bearer-style API key in header with Authorization name.
+ if swIn == "header" && strings.EqualFold(swName, "Authorization") {
+ scheme.Type = "http"
+ scheme.Scheme = "bearer"
+ }
+ case "basic":
+ scheme.Type = "http"
+ scheme.Scheme = "basic"
+ }
+ }
+ info.SecuritySchemes[schemeName] = scheme
+ }
+ }
+ }
+
rootSecurity, rootHasSecurity := parseSecurityRequirementSet(raw["security"])
foundOperation := false
if paths, ok := raw["paths"].(map[string]any); ok {
diff --git a/internal/pipeline/scorecard_tier2_test.go b/internal/pipeline/scorecard_tier2_test.go
index cb6fff48..69aba6bd 100644
--- a/internal/pipeline/scorecard_tier2_test.go
+++ b/internal/pipeline/scorecard_tier2_test.go
@@ -1071,6 +1071,176 @@ func (c *Client) do() {
})
}
+func TestLoadOpenAPISpec_Swagger20SecurityDefinitions(t *testing.T) {
+ t.Run("swagger 2.0 apiKey in header with Authorization maps to bearer", func(t *testing.T) {
+ dir := t.TempDir()
+ specPath := filepath.Join(dir, "swagger.json")
+ writeScorecardFixture(t, dir, "swagger.json", `{
+ "swagger": "2.0",
+ "paths": {
+ "/api/chat": {
+ "get": {
+ "responses": { "200": { "description": "ok" } }
+ }
+ }
+ },
+ "securityDefinitions": {
+ "api_key": {
+ "type": "apiKey",
+ "in": "header",
+ "name": "Authorization"
+ }
+ },
+ "security": [
+ { "api_key": [] }
+ ]
+}`)
+
+ info, err := loadOpenAPISpec(specPath)
+ assert.NoError(t, err)
+ assert.Len(t, info.SecuritySchemes, 1)
+ scheme := info.SecuritySchemes["api_key"]
+ assert.Equal(t, "http", scheme.Type)
+ assert.Equal(t, "bearer", scheme.Scheme)
+ assert.Len(t, info.SecurityRequirements, 1)
+ })
+
+ t.Run("swagger 2.0 oauth2 maps correctly", func(t *testing.T) {
+ dir := t.TempDir()
+ specPath := filepath.Join(dir, "swagger.json")
+ writeScorecardFixture(t, dir, "swagger.json", `{
+ "swagger": "2.0",
+ "paths": {
+ "/api/data": {
+ "get": {
+ "responses": { "200": { "description": "ok" } }
+ }
+ }
+ },
+ "securityDefinitions": {
+ "oauth": {
+ "type": "oauth2",
+ "flow": "accessCode"
+ }
+ },
+ "security": [
+ { "oauth": [] }
+ ]
+}`)
+
+ info, err := loadOpenAPISpec(specPath)
+ assert.NoError(t, err)
+ assert.Len(t, info.SecuritySchemes, 1)
+ scheme := info.SecuritySchemes["oauth"]
+ assert.Equal(t, "oauth2", scheme.Type)
+ })
+
+ t.Run("swagger 2.0 basic auth maps to http basic", func(t *testing.T) {
+ dir := t.TempDir()
+ specPath := filepath.Join(dir, "swagger.json")
+ writeScorecardFixture(t, dir, "swagger.json", `{
+ "swagger": "2.0",
+ "paths": { "/api": {} },
+ "securityDefinitions": {
+ "basicAuth": {
+ "type": "basic"
+ }
+ },
+ "security": [
+ { "basicAuth": [] }
+ ]
+}`)
+
+ info, err := loadOpenAPISpec(specPath)
+ assert.NoError(t, err)
+ scheme := info.SecuritySchemes["basicAuth"]
+ assert.Equal(t, "http", scheme.Type)
+ assert.Equal(t, "basic", scheme.Scheme)
+ })
+
+ t.Run("OAS3 takes precedence over swagger 2.0 securityDefinitions", func(t *testing.T) {
+ dir := t.TempDir()
+ specPath := filepath.Join(dir, "mixed.json")
+ writeScorecardFixture(t, dir, "mixed.json", `{
+ "paths": {
+ "/api": {
+ "get": {
+ "responses": { "200": { "description": "ok" } }
+ }
+ }
+ },
+ "components": {
+ "securitySchemes": {
+ "bearerAuth": {
+ "type": "http",
+ "scheme": "bearer"
+ }
+ }
+ },
+ "securityDefinitions": {
+ "api_key": {
+ "type": "apiKey",
+ "in": "header",
+ "name": "X-API-Key"
+ }
+ },
+ "security": [
+ { "bearerAuth": [] }
+ ]
+}`)
+
+ info, err := loadOpenAPISpec(specPath)
+ assert.NoError(t, err)
+ // OAS3 should win: only bearerAuth, not api_key.
+ assert.Len(t, info.SecuritySchemes, 1)
+ _, hasBearerAuth := info.SecuritySchemes["bearerAuth"]
+ assert.True(t, hasBearerAuth)
+ _, hasAPIKey := info.SecuritySchemes["api_key"]
+ assert.False(t, hasAPIKey)
+ })
+
+ t.Run("spec with neither OAS3 nor swagger 2.0 security has empty schemes", func(t *testing.T) {
+ dir := t.TempDir()
+ specPath := filepath.Join(dir, "bare.json")
+ writeScorecardFixture(t, dir, "bare.json", `{
+ "paths": {
+ "/api": {}
+ }
+}`)
+
+ info, err := loadOpenAPISpec(specPath)
+ assert.NoError(t, err)
+ assert.Empty(t, info.SecuritySchemes)
+ assert.Empty(t, info.SecurityRequirements)
+ })
+
+ t.Run("swagger 2.0 apiKey without Authorization stays as apikey", func(t *testing.T) {
+ dir := t.TempDir()
+ specPath := filepath.Join(dir, "swagger.json")
+ writeScorecardFixture(t, dir, "swagger.json", `{
+ "swagger": "2.0",
+ "paths": { "/api": {} },
+ "securityDefinitions": {
+ "token": {
+ "type": "apiKey",
+ "in": "header",
+ "name": "X-API-Token"
+ }
+ },
+ "security": [
+ { "token": [] }
+ ]
+}`)
+
+ info, err := loadOpenAPISpec(specPath)
+ assert.NoError(t, err)
+ scheme := info.SecuritySchemes["token"]
+ assert.Equal(t, "apikey", scheme.Type)
+ assert.Equal(t, "header", scheme.In)
+ assert.Equal(t, "X-API-Token", scheme.HeaderName)
+ })
+}
+
func writeScorecardFixture(t *testing.T, root, relPath, content string) {
t.Helper()
diff --git a/internal/profiler/profiler.go b/internal/profiler/profiler.go
index c927b707..f3ac5618 100644
--- a/internal/profiler/profiler.go
+++ b/internal/profiler/profiler.go
@@ -59,6 +59,15 @@ type SyncableResource struct {
Path string
}
+// DependentResource describes a child resource that requires iterating a parent
+// to sync (e.g., /channels/{channelId}/messages depends on channels).
+type DependentResource struct {
+ Name string // child resource name, e.g. "messages"
+ ParentResource string // parent resource name, e.g. "channels"
+ ParentIDParam string // path param name, e.g. "channel_id"
+ Path string // full path template, e.g. "/channels/{channel_id}/messages"
+}
+
// APIProfile describes the shape of an API and what power-user features it warrants.
type APIProfile struct {
HighVolume bool
@@ -75,8 +84,9 @@ type APIProfile struct {
TotalEndpoints int
ReadRatio float64
- SyncableResources []SyncableResource
- SearchableFields map[string][]string
+ SyncableResources []SyncableResource
+ DependentSyncResources []DependentResource
+ SearchableFields map[string][]string
// SearchEndpointPath is the API path for live search (e.g., "/search", "/users/search").
// Empty if the API has no search endpoint.
@@ -107,7 +117,8 @@ func Profile(s *spec.APISpec) *APIProfile {
}
resourceNames := collectResourceNames(s.Resources)
- syncable := make(map[string]string) // resource name -> list endpoint path
+ syncable := make(map[string]string) // resource name -> list endpoint path
+ parameterized := make(map[string]string) // resource name -> parameterized list endpoint path (excluded from flat sync)
searchable := make(map[string]map[string]struct{})
listResources := make(map[string]struct{})
@@ -267,6 +278,12 @@ func Profile(s *spec.APISpec) *APIProfile {
// deterministic output regardless of Go map iteration order.
syncable[expandedName] = expandedPath
}
+ } else if strings.Contains(endpoint.Path, "{") {
+ // Parameterized paginated paths can't sync standalone — track
+ // them for dependent-resource detection below.
+ if _, ok := parameterized[resourceName]; !ok {
+ parameterized[resourceName] = endpoint.Path
+ }
} else {
// Paginated endpoints override the path set above — they have
// richer pagination support for full data retrieval.
@@ -366,6 +383,7 @@ func Profile(s *spec.APISpec) *APIProfile {
p.NeedsSearch = len(listResources) >= 3 && float64(searchEndpointCount)/float64(len(listResources)) < 0.5
p.SyncableResources = sortedSyncableResources(syncable)
+ p.DependentSyncResources = detectDependentResources(parameterized, syncable)
for resource, fields := range searchable {
p.SearchableFields[resource] = sortedKeys(fields)
}
@@ -845,6 +863,55 @@ func sortedKeys[V any](m map[string]V) []string {
return keys
}
+// detectDependentResources examines parameterized paths and identifies parent-child
+// relationships. For example, /channels/{channel_id}/messages becomes a dependent
+// resource of "channels" (only one level of nesting).
+func detectDependentResources(parameterized map[string]string, syncable map[string]string) []DependentResource {
+ var deps []DependentResource
+ for childName, path := range parameterized {
+ // Extract the first {param} from the path
+ start := strings.Index(path, "{")
+ end := strings.Index(path, "}")
+ if start < 0 || end < 0 || end <= start {
+ continue
+ }
+ paramName := path[start+1 : end]
+
+ // Derive the parent resource name by stripping trailing Id/_id from the param.
+ // e.g., "channel_id" -> "channel", "channelId" -> "channel"
+ parentCandidate := paramName
+ parentCandidate = strings.TrimSuffix(parentCandidate, "_id")
+ parentCandidate = strings.TrimSuffix(parentCandidate, "Id")
+ parentCandidate = strings.TrimSuffix(parentCandidate, "ID")
+ parentCandidate = strings.ToLower(parentCandidate)
+
+ // Check if a flat syncable resource exists matching the parent name
+ // (try both singular and common plural forms).
+ parentResource := ""
+ for _, candidate := range []string{parentCandidate, parentCandidate + "s", parentCandidate + "es"} {
+ if _, ok := syncable[candidate]; ok {
+ parentResource = candidate
+ break
+ }
+ }
+ if parentResource == "" {
+ continue
+ }
+
+ deps = append(deps, DependentResource{
+ Name: childName,
+ ParentResource: parentResource,
+ ParentIDParam: paramName,
+ Path: path,
+ })
+ }
+ // Sort for deterministic output
+ sort.Slice(deps, func(i, j int) bool {
+ return deps[i].Name < deps[j].Name
+ })
+ return deps
+}
+
// sortedSyncableResources converts a name->path map into a sorted slice of SyncableResource.
func sortedSyncableResources(m map[string]string) []SyncableResource {
names := make([]string, 0, len(m))
diff --git a/internal/profiler/profiler_test.go b/internal/profiler/profiler_test.go
index a76600dc..81173bdc 100644
--- a/internal/profiler/profiler_test.go
+++ b/internal/profiler/profiler_test.go
@@ -30,8 +30,20 @@ func TestProfileDiscord(t *testing.T) {
for i, sr := range profile.SyncableResources {
syncNames[i] = sr.Name
}
- assert.Contains(t, syncNames, "messages")
+ // Messages has a parameterized path (/channels/{channel_id}/messages) so it
+ // should NOT be in flat SyncableResources - it goes to DependentSyncResources.
+ assert.NotContains(t, syncNames, "messages")
assert.Contains(t, profile.SearchableFields["messages"], "content")
+
+ // Dependent resources should be detected for parameterized paths
+ depNames := make([]string, len(profile.DependentSyncResources))
+ for i, dr := range profile.DependentSyncResources {
+ depNames[i] = dr.Name
+ }
+ assert.Contains(t, depNames, "messages")
+ assert.Contains(t, depNames, "threads")
+ assert.Contains(t, depNames, "members")
+ assert.Contains(t, depNames, "roles")
}
func TestProfileMinimal(t *testing.T) {
@@ -666,3 +678,73 @@ func TestProfileSimpleListEndpointSyncable(t *testing.T) {
// query is POST-only so it should be excluded
assert.NotContains(t, syncNames, "query", "POST-only resource should not be syncable")
}
+
+func TestProfileDependentResources(t *testing.T) {
+ // A spec with /channels (flat) and /channels/{channelId}/messages (parameterized)
+ // should produce a DependentResource linking messages to channels.
+ s := &spec.APISpec{
+ Name: "messaging",
+ Resources: map[string]spec.Resource{
+ "channels": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/channels",
+ Response: spec.ResponseDef{Type: "array"},
+ Pagination: &spec.Pagination{CursorParam: "after", LimitParam: "limit"},
+ },
+ },
+ },
+ "messages": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/channels/{channelId}/messages",
+ Response: spec.ResponseDef{Type: "array"},
+ Pagination: &spec.Pagination{CursorParam: "after", LimitParam: "limit"},
+ },
+ },
+ },
+ },
+ }
+
+ profile := Profile(s)
+
+ // channels should be in flat SyncableResources
+ syncNames := make([]string, len(profile.SyncableResources))
+ for i, sr := range profile.SyncableResources {
+ syncNames[i] = sr.Name
+ }
+ assert.Contains(t, syncNames, "channels")
+ assert.NotContains(t, syncNames, "messages", "parameterized path should not be in flat sync")
+
+ // messages should be a dependent resource with channels as parent
+ require.Len(t, profile.DependentSyncResources, 1)
+ dep := profile.DependentSyncResources[0]
+ assert.Equal(t, "messages", dep.Name)
+ assert.Equal(t, "channels", dep.ParentResource)
+ assert.Equal(t, "channelId", dep.ParentIDParam)
+ assert.Equal(t, "/channels/{channelId}/messages", dep.Path)
+}
+
+func TestProfileDependentResources_NoParentNoDependent(t *testing.T) {
+ // If the parent resource doesn't exist as a flat syncable, no dependent is created.
+ s := &spec.APISpec{
+ Name: "orphan",
+ Resources: map[string]spec.Resource{
+ "messages": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/channels/{channelId}/messages",
+ Response: spec.ResponseDef{Type: "array"},
+ Pagination: &spec.Pagination{CursorParam: "after", LimitParam: "limit"},
+ },
+ },
+ },
+ },
+ }
+
+ profile := Profile(s)
+ assert.Empty(t, profile.DependentSyncResources, "no parent resource means no dependent detection")
+}
diff --git a/testdata/graphql/test.graphql b/testdata/graphql/test.graphql
new file mode 100644
index 00000000..0114558d
--- /dev/null
+++ b/testdata/graphql/test.graphql
@@ -0,0 +1,54 @@
+type Query {
+ issues(first: Int, after: String): IssueConnection
+ issue(id: String!): Issue
+ projects(first: Int, after: String): ProjectConnection
+ project(id: String!): Project
+}
+
+type Mutation {
+ issueCreate(input: IssueCreateInput!): Issue
+ issueUpdate(id: String!, input: IssueUpdateInput!): Issue
+ issueDelete(id: String!): Boolean
+}
+
+type IssueConnection {
+ nodes: [Issue]
+ pageInfo: PageInfo
+}
+
+type ProjectConnection {
+ nodes: [Project]
+ pageInfo: PageInfo
+}
+
+type PageInfo {
+ hasNextPage: Boolean!
+ endCursor: String
+}
+
+type Issue {
+ id: ID!
+ title: String!
+ description: String
+ status: String
+ createdAt: String
+ updatedAt: String
+}
+
+type Project {
+ id: ID!
+ name: String!
+ description: String
+ status: String
+}
+
+input IssueCreateInput {
+ title: String!
+ description: String
+}
+
+input IssueUpdateInput {
+ title: String
+ description: String
+ status: String
+}
← b436b081 fix(cli): add transitive reachability to dogfood dead functi
·
back to Cli Printing Press
·
fix(cli): enrich README and generate SKILL.md so printed CLI 1011df38 →