← back to Cli Printing Press
fix(cli): source store columns from response schema, not query params (#708)
9e039c0be787abf49242b668397155f6940525f7 · 2026-05-08 00:23:15 -0700 · Trevin Chow
* fix(cli): source store columns from response schema, not query params
collectResponseFields was misnamed — it iterated endpoint.Params
(query/path) and endpoint.Body instead of endpoint.Response, so every
paginated REST table received columns named after its GET filter knobs
(filter, sort, per_page, ...) and sync left them NULL forever because
the API never echoes filter inputs back as response fields.
Fix sources columns from s.Types[endpoint.Response.Item], registers
inline list-response and single-object response schemas in the parser
(namespaced by resource so two default-named GETs don't collide),
threads format hints through TypeField, and walks GET first with
non-GET fallback for write-only resources.
Fixes #698
* docs(cli): document store-column response-schema sourcing
Captures the #698 fix as a logic-error solution under docs/solutions/,
naming the durable invariant — response schema is the authority for
SQLite column shape; request Params are input filters — and cross-
referencing the structurally-similar prior fixes (MCP path-vs-query
conflation, inline auth bearer inference) so the next time someone
touches generator surfaces that read from spec.Endpoint, the rule is
one grep away.
Files touched
A docs/solutions/logic-errors/store-columns-sourced-from-request-params-instead-of-response-2026-05-08.mdM internal/generator/generator_test.goM internal/generator/schema_builder.goM internal/generator/schema_builder_test.goM internal/openapi/parser.goM internal/openapi/parser_test.goM internal/spec/spec.go
Diff
commit 9e039c0be787abf49242b668397155f6940525f7
Author: Trevin Chow <trevin@trevinchow.com>
Date: Fri May 8 00:23:15 2026 -0700
fix(cli): source store columns from response schema, not query params (#708)
* fix(cli): source store columns from response schema, not query params
collectResponseFields was misnamed — it iterated endpoint.Params
(query/path) and endpoint.Body instead of endpoint.Response, so every
paginated REST table received columns named after its GET filter knobs
(filter, sort, per_page, ...) and sync left them NULL forever because
the API never echoes filter inputs back as response fields.
Fix sources columns from s.Types[endpoint.Response.Item], registers
inline list-response and single-object response schemas in the parser
(namespaced by resource so two default-named GETs don't collide),
threads format hints through TypeField, and walks GET first with
non-GET fallback for write-only resources.
Fixes #698
* docs(cli): document store-column response-schema sourcing
Captures the #698 fix as a logic-error solution under docs/solutions/,
naming the durable invariant — response schema is the authority for
SQLite column shape; request Params are input filters — and cross-
referencing the structurally-similar prior fixes (MCP path-vs-query
conflation, inline auth bearer inference) so the next time someone
touches generator surfaces that read from spec.Endpoint, the rule is
one grep away.
---
...equest-params-instead-of-response-2026-05-08.md | 143 ++++++++++
internal/generator/generator_test.go | 100 ++++---
internal/generator/schema_builder.go | 260 ++++++++++--------
internal/generator/schema_builder_test.go | 214 +++++++++++++--
internal/openapi/parser.go | 125 ++++++---
internal/openapi/parser_test.go | 304 +++++++++++++++++++++
internal/spec/spec.go | 6 +
7 files changed, 952 insertions(+), 200 deletions(-)
diff --git a/docs/solutions/logic-errors/store-columns-sourced-from-request-params-instead-of-response-2026-05-08.md b/docs/solutions/logic-errors/store-columns-sourced-from-request-params-instead-of-response-2026-05-08.md
new file mode 100644
index 00000000..563f7e8a
--- /dev/null
+++ b/docs/solutions/logic-errors/store-columns-sourced-from-request-params-instead-of-response-2026-05-08.md
@@ -0,0 +1,143 @@
+---
+title: "Store table columns sourced from request query params instead of response schema"
+date: 2026-05-08
+category: logic-errors
+module: internal/generator/schema_builder
+problem_type: logic_error
+component: tooling
+symptoms:
+ - "Generated SQLite tables get columns named after a list endpoint's query parameters (filter, sort, per_page, page) instead of fields from the response payload"
+ - "Sync reports success and writes the JSON payload to the data column, but every typed column stays NULL forever because nothing populates it"
+ - "SQL queries against typed columns (WHERE state='open', ORDER BY created_at) return NULL or zero rows regardless of sync volume"
+ - "Affects every paginated REST API; visible only via direct SQLite inspection or the sql tool, never at the CLI surface"
+root_cause: logic_error
+resolution_type: code_fix
+severity: high
+related_components:
+ - internal/openapi
+ - internal/spec
+tags:
+ - schema-builder
+ - response-schema
+ - sqlite-columns
+ - openapi-parser
+ - generator-pipeline
+ - typefield-format
+---
+
+# Store table columns sourced from request query params instead of response schema
+
+## Problem
+
+`collectResponseFields` in `internal/generator/schema_builder.go` was misnamed — it iterated `endpoint.Params` (query/path parameters) and `endpoint.Body` (request body) instead of `endpoint.Response`. Every paginated REST resource ended up with a SQLite table whose typed columns mirrored its GET filter knobs rather than its response entity shape, which sync could never populate because filter inputs don't appear in response payloads.
+
+## Symptoms
+
+- For GitHub's `GET /issues`, the generated `issues` table emitted columns `filter, state, labels, sort, since, per_page, page` — exactly the GET query params, none of the actual response fields (`id, number, title, body, created_at, updated_at`).
+- After `sync`, the `data` JSON column held the real response, but typed columns were NULL on every row.
+- `SELECT state FROM issues WHERE state='open'` returned nothing because the column was uniformly NULL; the same query expressed as `SELECT json_extract(data, '$.state') FROM issues WHERE json_extract(data, '$.state') = 'open'` returned the right rows.
+- `sync_complete` events reported the right counts; the failure was schema-shape, not transport.
+- Same pattern reproduced across every printed CLI generated from any paginated REST API spec.
+
+## What Didn't Work
+
+**Fix only `schema_builder.go`, leave the parser alone.** This worked for specs whose response types were defined in `Components/Schemas` and already registered in `s.Types`. It silently failed for any spec that declared an inline list response (no `$ref`) — `s.Types[endpoint.Response.Item]` had no entry, so the resource fell back to `id/data/synced_at` with no typed columns at all. Required the parser to call `registerInlineSchemaType` so inline schemas fill the same `s.Types` map.
+
+**Inline registration logic written directly inside `mapResponse`.** The field-walking code was ~35 lines nearly identical to what `mapTypes` already did for Components.Schemas entries. Code review flagged the duplication. Extracted `buildTypeFields(schemaRef)` as a shared helper called from both paths.
+
+**Add an exists-check inside `mapTypes` to prevent collisions.** The goal was to let Components-defined types win over inline registrations. Adding the guard inside `mapTypes` required `mapTypes` to know about inline names, coupling the two code paths. The cleaner fix was to swap call order — `mapTypes` runs before `mapResources` (which calls `mapResponse`/`registerInlineSchemaType`), so Components types are already in `out.Types` when inline registration runs. The existing exists-check in `registerInlineSchemaType` then naturally yields without any new coupling.
+
+**Use a flat `endpointName + "Item"` for the inline-item synthetic name.** Two resources both with a default-named GET (`list`) both computed `ListItem`. The second resource's inline registration silently inherited the first resource's field shape. Adversarial code review caught this as a P1 cross-resource collision: it re-introduced the wrong-columns class for cross-resource cases. Fixed by namespacing: `targetResourceName + "_" + endpointName` as the fallback so names are distinct per resource.
+
+## Solution
+
+Three files changed in coordination:
+
+**`internal/spec/spec.go` — `TypeField` gains `Format`.**
+
+```go
+type TypeField struct {
+ Name string `yaml:"name" json:"name"`
+ Type string `yaml:"type" json:"type"`
+ Enum []string `yaml:"enum,omitempty" json:"enum,omitempty"`
+ Format string `yaml:"format,omitempty" json:"format,omitempty"` // new
+ Selection string `yaml:"selection,omitempty" json:"selection,omitempty"`
+}
+```
+
+This carries the OpenAPI `format` hint (`date-time`, `date`) from response schema through to `sqliteType`, enabling `DATETIME` instead of `TEXT` for temporal fields end-to-end.
+
+**`internal/openapi/parser.go` — parser registers inline response schemas.**
+
+- Extracted `buildTypeFields(schemaRef)`: walks object properties (flattening JSON:API shapes), filters underscore-prefixed names and Go-name collisions, emits `[]spec.TypeField` with `Format` populated. Called from both `mapTypes` and the inline registration path.
+- `registerInlineSchemaType(out, itemRef, fallbackName)` — when `itemRef` has no `$ref` and the slot is empty, registers the type into `out.Types` under the synthetic name `mapResponse` will later set on `endpoint.Response.Item`. Handles list-item schemas and single-object detail responses uniformly.
+- `mapResponse` signature gained `out *spec.APISpec`. All three response shape branches (`{data: array}` envelope, bare array, single object) now register inline schemas. Before this change the single-object branch registered nothing — detail-only resources lost typed columns entirely.
+- Inline-item synthetic names namespaced by resource: `targetResourceName + "_" + endpointName` as fallback, so two resources with default-named GETs cannot collide on a shared `ListItem` Types entry.
+- Swapped the call order in `Parse`: `mapTypes` now runs before `mapResources`, so Components-defined types populate `out.Types` first and inline registrations naturally yield to them.
+
+**`internal/generator/schema_builder.go` — `collectResponseFields` rewritten.**
+
+Before:
+
+```go
+for _, ep := range r.Endpoints {
+ if ep.Method != "GET" { continue }
+ for _, p := range ep.Params { ... } // request-side filter knobs
+ for _, p := range ep.Body { ... } // request body fields
+}
+// Plus a second pass over POST/PUT bodies "as they often mirror response shape"
+```
+
+After:
+
+```go
+typeName := ep.Response.Item // what the API returns
+typeDef, ok := s.Types[typeName] // registered by parser
+if !ok { continue }
+for _, f := range typeDef.Fields { ... }
+```
+
+GET endpoints walked first. Non-GET (POST/PUT) walked only when no GET endpoint contributed any fields, so write-only resources (event-emit, webhook ingestion) keep typed columns from their canonical create-response shape without letting wrapper-shaped responses pollute typed columns when a GET exists.
+
+Format-aware dedup: when the same field appears across endpoints with differing format hints, an entry with empty `Format` is upgraded when a later endpoint declares one — preventing list-vs-detail format drift from downgrading `DATETIME` to `TEXT` based on alphabetic endpoint-key order.
+
+Additional housekeeping: `baseTableColumns` promoted to package-level slice (shared by `BuildSchema` and `buildSubResourceTable`); `textFieldKeywords` promoted to package scope; `seenIndexes` map gates `_id` index emission to prevent duplicate `IndexDef.Name`; `isScalarTypeField`/`hasTypeField` helpers extracted; fields resolved once per resource and threaded into `computeDataGravity`/`collectTextFieldNamesFromFields` instead of being re-walked five times.
+
+## Why This Works
+
+The original `collectResponseFields` was misnamed — it was iterating the request side of the endpoint while its declared purpose was to derive the SQLite schema from what the API *returns*. The OpenAPI parser correctly populated both `endpoint.Response.Item` (the type name) and `s.Types[<TypeName>]` (the field list), but `schema_builder.go` never read either. Since sync writes the response payload to `data` and then extracts into typed columns via `lookupFieldValue(obj, columnName)`, a column whose name is `filter` or `sort` or `per_page` can never be populated: the API does not echo filter inputs back as top-level response fields.
+
+The fix closes the loop end-to-end: the parser registers every response item type — whether from `Components/Schemas` (`$ref`) or inline — into `s.Types` under the same name `endpoint.Response.Item` carries. The schema builder reads that entry. The column list now matches what sync actually stores.
+
+## Prevention
+
+**Invariant to internalize.** Response schema is the authority for SQLite column shape. Request `Params` are input filters sent *to* the API; they describe what the caller can say, not what the API returns. Any generator surface that describes the entity shape (column lists, FTS5 fields, typed indexes) must source from `endpoint.Response.Item` → `s.Types[name]`, never from `endpoint.Params` or `endpoint.Body`.
+
+**Write-time discipline in fixture tests.** Five existing generator tests had declared request `Params` and relied on the buggy column-from-params behavior. They passed precisely because the bug was in production. When writing generator tests that assert column derivation, declare `Response.Item` and the corresponding `Types` entry — not `Params`. A test that only populates `Params` and asserts column names is testing the wrong invariant and will silently validate a future regression.
+
+Correct fixture shape:
+
+```go
+resource.Endpoints["list"] = spec.Endpoint{
+ Method: "GET",
+ Response: spec.ResponseDef{Type: "array", Item: "Issue"},
+}
+s.Types["Issue"] = spec.TypeDef{Fields: []spec.TypeField{
+ {Name: "id", Type: "integer"},
+ {Name: "title", Type: "string"},
+ {Name: "created_at", Type: "string", Format: "date-time"},
+}}
+```
+
+**Pin the response-vs-request distinction with a regression test on every schema-builder change.** The fix added `TestBuildSchema_ColumnsFromResponseSchema` (request params don't leak), `TestBuildSchema_ParamResponseNameOverlap` (when names collide, the response field's *type* drives the column — fixture uses param=string, response=integer; asserts INTEGER), and `TestBuildSchema_NoResponseTypeFallback` (table-driven over both unregistered-type-name and empty-Item cases; both yield only base columns). Future changes to `collectResponseFields` should keep these green.
+
+**Golden coverage gap to close.** The golden suite has no fixture for `internal/store/store.go` (the generated SQLite DDL). A regression that reverts column sourcing would pass `scripts/golden.sh verify` silently. Adding a store.go golden — with a spec that has both request params and a distinct response shape — would catch this class of bug mechanically.
+
+**Bug shape is wider than this instance.** The same wrong-source pattern produced a related defect documented in `docs/solutions/logic-errors/mcp-handler-conflates-path-and-query-positional-params-2026-05-05.md` (MCP handler treating URL path placeholders as positional query args). When adding new generator surfaces that read from endpoints, explicitly identify which field of `spec.Endpoint` is the authoritative source and add a comment naming the invariant, as the rewritten `collectResponseFields` now does.
+
+## Related Issues
+
+- [#698](https://github.com/mvanhorn/cli-printing-press/issues/698) — the tracked issue for this fix.
+- [#689](https://github.com/mvanhorn/cli-printing-press/issues/689) — Kalshi sync correctness retro. Bug 4 in #689 ("novel-feature SQL hardcodes field names that don't match real API shape") shares the same philosophy: artifacts emitted without grounding in the response schema. Different surface (SQL authoring vs. column derivation), same root principle.
+- `docs/solutions/logic-errors/mcp-handler-conflates-path-and-query-positional-params-2026-05-05.md` — same bug class. Generator consumed the wrong-source data; bug survived because the wrong source partially overlapped the right source in common cases.
+- `docs/solutions/logic-errors/inline-authorization-param-bearer-inference-2026-05-05.md` — same file (`internal/openapi/parser.go`), same extension pattern (a `register*` / `infer*` helper that walks an inline OpenAPI structure to fill a derivation gap).
diff --git a/internal/generator/generator_test.go b/internal/generator/generator_test.go
index c57eae61..83be19b7 100644
--- a/internal/generator/generator_test.go
+++ b/internal/generator/generator_test.go
@@ -2284,17 +2284,24 @@ func TestGenerateStoreUpsertBatchDispatchesToTypedTable(t *testing.T) {
Method: "GET",
Path: "/campaigns",
Description: "List campaigns",
- Response: spec.ResponseDef{Type: "array"},
- Params: []spec.Param{
- {Name: "id", Type: "string"},
- {Name: "name", Type: "string"},
- {Name: "status", Type: "string"},
- {Name: "account_id", Type: "string"},
- },
+ // Column derivation reads the response shape (Types
+ // entry below), not request Params, so the fixture
+ // must declare Response.Item.
+ Response: spec.ResponseDef{Type: "array", Item: "Campaign"},
},
},
},
},
+ Types: map[string]spec.TypeDef{
+ "Campaign": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "string"},
+ {Name: "name", Type: "string"},
+ {Name: "status", Type: "string"},
+ {Name: "account_id", Type: "string"},
+ },
+ },
+ },
}
outputDir := filepath.Join(t.TempDir(), naming.CLI(apiSpec.Name))
@@ -2323,17 +2330,15 @@ func TestSyncDiscriminatorDispatchRoutesMixedItemsToTypedTables(t *testing.T) {
t.Parallel()
typedListEndpoint := func(path string) spec.Endpoint {
+ // Response.Item drives typed-column emission via the TypedEntity
+ // Types entry below. Without it the discriminator-routed tables
+ // would degrade to id/data/synced_at and the dispatch test would
+ // have nowhere to write rows.
return spec.Endpoint{
Method: "GET",
Path: path,
Description: "List typed entities",
- Response: spec.ResponseDef{Type: "array"},
- Params: []spec.Param{
- {Name: "id", Type: "string"},
- {Name: "type", Type: "string"},
- {Name: "name", Type: "string"},
- {Name: "created_at", Type: "string", Format: "date-time"},
- },
+ Response: spec.ResponseDef{Type: "array", Item: "TypedEntity"},
}
}
@@ -2380,6 +2385,14 @@ func TestSyncDiscriminatorDispatchRoutesMixedItemsToTypedTables(t *testing.T) {
{Name: "id", Type: "string"},
},
},
+ "TypedEntity": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "string"},
+ {Name: "type", Type: "string"},
+ {Name: "name", Type: "string"},
+ {Name: "created_at", Type: "string", Format: "date-time"},
+ },
+ },
},
}
@@ -2461,18 +2474,22 @@ func TestGenerateStoreBackfillsIndexedColumnsOnUpgrade(t *testing.T) {
Method: "GET",
Path: "/emails",
Description: "List emails",
- Response: spec.ResponseDef{Type: "array"},
- Params: []spec.Param{
- {Name: "id", Type: "string"},
- {Name: "email_id", Type: "string"},
- {Name: "name", Type: "string"},
- {Name: "description", Type: "string"},
- {Name: "created_at", Type: "string", Format: "date-time"},
- },
+ Response: spec.ResponseDef{Type: "array", Item: "Email"},
},
},
},
},
+ Types: map[string]spec.TypeDef{
+ "Email": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "string"},
+ {Name: "email_id", Type: "string"},
+ {Name: "name", Type: "string"},
+ {Name: "description", Type: "string"},
+ {Name: "created_at", Type: "string", Format: "date-time"},
+ },
+ },
+ },
}
outputDir := filepath.Join(t.TempDir(), naming.CLI(apiSpec.Name))
@@ -2510,19 +2527,23 @@ func TestGenerateStoreQuotesNumericTableAndDerivedIdentifiers(t *testing.T) {
Method: "GET",
Path: "/0",
Description: "List numeric resources",
- Response: spec.ResponseDef{Type: "array"},
- Params: []spec.Param{
- {Name: "id", Type: "string"},
- {Name: "replay_id", Type: "string"},
- {Name: "name", Type: "string"},
- {Name: "description", Type: "string"},
- {Name: "message", Type: "string"},
- {Name: "created_at", Type: "string", Format: "date-time"},
- },
+ Response: spec.ResponseDef{Type: "array", Item: "Numeric"},
},
},
},
},
+ Types: map[string]spec.TypeDef{
+ "Numeric": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "string"},
+ {Name: "replay_id", Type: "string"},
+ {Name: "name", Type: "string"},
+ {Name: "description", Type: "string"},
+ {Name: "message", Type: "string"},
+ {Name: "created_at", Type: "string", Format: "date-time"},
+ },
+ },
+ },
}
outputDir := filepath.Join(t.TempDir(), naming.CLI(apiSpec.Name))
@@ -7335,16 +7356,21 @@ func TestStoreSkipsDeadTablesForResourcesWithoutTypedUpsert(t *testing.T) {
Method: "GET",
Path: "/items",
Description: "List items",
- Params: []spec.Param{
- {Name: "id", Type: "string"},
- {Name: "name", Type: "string"},
- {Name: "status", Type: "string"},
- {Name: "created_at", Type: "string", Format: "date-time"},
- },
+ Response: spec.ResponseDef{Type: "array", Item: "Item"},
},
},
},
},
+ Types: map[string]spec.TypeDef{
+ "Item": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "string"},
+ {Name: "name", Type: "string"},
+ {Name: "status", Type: "string"},
+ {Name: "created_at", Type: "string", Format: "date-time"},
+ },
+ },
+ },
}
outputDir := filepath.Join(t.TempDir(), naming.CLI(apiSpec.Name))
diff --git a/internal/generator/schema_builder.go b/internal/generator/schema_builder.go
index a7113aa0..e04aff70 100644
--- a/internal/generator/schema_builder.go
+++ b/internal/generator/schema_builder.go
@@ -31,6 +31,15 @@ type IndexDef struct {
Unique bool
}
+// baseTableColumns are the three columns every domain table starts with.
+// Used both as the initial column set and as the "already emitted" guard
+// when extending the table with response-derived columns.
+var baseTableColumns = []ColumnDef{
+ {Name: "id", Type: "TEXT", PrimaryKey: true},
+ {Name: "data", Type: "JSON", NotNull: true},
+ {Name: "synced_at", Type: "DATETIME DEFAULT CURRENT_TIMESTAMP"},
+}
+
// BuildSchema generates domain-specific table definitions from the API spec.
// High-gravity entities (many endpoints, text fields, temporal fields) get
// full column extraction. Low-gravity entities get simple id+data tables.
@@ -49,38 +58,44 @@ func BuildSchema(s *spec.APISpec) []TableDef {
for _, name := range resourceNames {
resource := s.Resources[name]
- gravity := computeDataGravity(name, resource)
+ fields := collectResponseFields(s, resource)
+ gravity := computeDataGravity(resource, fields)
tableName := toSnakeCase(name)
table := TableDef{
- Name: tableName,
- Columns: []ColumnDef{
- {Name: "id", Type: "TEXT", PrimaryKey: true},
- {Name: "data", Type: "JSON", NotNull: true},
- {Name: "synced_at", Type: "DATETIME DEFAULT CURRENT_TIMESTAMP"},
- },
+ Name: tableName,
+ Columns: append([]ColumnDef(nil), baseTableColumns...),
}
if gravity >= 2 {
- fields := collectResponseFields(resource)
+ seenColumns := map[string]bool{}
+ for _, c := range baseTableColumns {
+ seenColumns[c.Name] = true
+ }
+ seenIndexes := map[string]bool{}
for _, f := range fields {
- if isScalarField(f) && f.Name != "id" {
- col := ColumnDef{
- Name: toSnakeCase(f.Name),
+ colName := toSnakeCase(f.Name)
+ if isScalarTypeField(f) && !seenColumns[colName] {
+ seenColumns[colName] = true
+ table.Columns = append(table.Columns, ColumnDef{
+ Name: colName,
Type: sqliteType(f.Type, f.Format),
- }
- table.Columns = append(table.Columns, col)
+ })
}
if strings.HasSuffix(strings.ToLower(f.Name), "_id") {
- table.Indexes = append(table.Indexes, IndexDef{
- Name: "idx_" + tableName + "_" + toSnakeCase(f.Name),
- TableName: tableName,
- Columns: toSnakeCase(f.Name),
- })
+ idxName := "idx_" + tableName + "_" + colName
+ if !seenIndexes[idxName] {
+ seenIndexes[idxName] = true
+ table.Indexes = append(table.Indexes, IndexDef{
+ Name: idxName,
+ TableName: tableName,
+ Columns: colName,
+ })
+ }
}
}
for _, temporal := range []string{"created_at", "updated_at"} {
- if hasField(fields, temporal) {
+ if hasTypeField(fields, temporal) {
table.Indexes = append(table.Indexes, IndexDef{
Name: "idx_" + tableName + "_" + temporal,
TableName: tableName,
@@ -90,7 +105,7 @@ func BuildSchema(s *spec.APISpec) []TableDef {
}
}
- textFields := collectTextFieldNames(resource)
+ textFields := collectTextFieldNamesFromFields(fields)
if len(textFields) >= 2 && gravity >= 2 {
table.FTS5 = true
table.FTS5Fields = textFields
@@ -140,12 +155,14 @@ func BuildSchema(s *spec.APISpec) []TableDef {
return tables
}
-// computeDataGravity scores 0-12 based on endpoint count, field count,
-// text fields, temporal fields, and FK references.
-func computeDataGravity(name string, r spec.Resource) int {
+// computeDataGravity scores 0-12 based on endpoint count, response field
+// count, text fields, temporal fields, and FK references. Caller passes
+// the resolved response fields (collectResponseFields) so the function
+// doesn't re-walk the spec; gravity scoring uses the same response shape
+// the rest of BuildSchema relies on.
+func computeDataGravity(r spec.Resource, fields []spec.TypeField) int {
score := 0
- // Endpoint count: 1pt per endpoint, max 4
epCount := len(r.Endpoints)
if epCount >= 4 {
score += 4
@@ -153,47 +170,35 @@ func computeDataGravity(name string, r spec.Resource) int {
score += epCount
}
- // Field count from all params/body across endpoints
- totalFields := 0
- for _, ep := range r.Endpoints {
- totalFields += len(ep.Params) + len(ep.Body)
- }
- if totalFields >= 10 {
+ if len(fields) >= 10 {
score += 2
- } else if totalFields >= 5 {
+ } else if len(fields) >= 5 {
score += 1
}
- // Text fields bonus
- textFields := collectTextFieldNames(r)
+ textFields := collectTextFieldNamesFromFields(fields)
if len(textFields) >= 3 {
score += 2
} else if len(textFields) >= 1 {
score += 1
}
- // Temporal fields bonus
- allFields := collectResponseFields(r)
temporalCount := 0
- for _, f := range allFields {
+ fkCount := 0
+ for _, f := range fields {
lower := strings.ToLower(f.Name)
if strings.HasSuffix(lower, "_at") || strings.Contains(lower, "date") || f.Format == "date-time" {
temporalCount++
}
+ if strings.HasSuffix(lower, "_id") {
+ fkCount++
+ }
}
if temporalCount >= 2 {
score += 2
} else if temporalCount >= 1 {
score += 1
}
-
- // FK references bonus
- fkCount := 0
- for _, f := range allFields {
- if strings.HasSuffix(strings.ToLower(f.Name), "_id") {
- fkCount++
- }
- }
if fkCount >= 2 {
score += 2
} else if fkCount >= 1 {
@@ -206,48 +211,75 @@ func computeDataGravity(name string, r spec.Resource) int {
return score
}
-// collectResponseFields gathers all field specs from GET endpoints.
-func collectResponseFields(r spec.Resource) []spec.Param {
- seen := make(map[string]bool)
- var fields []spec.Param
-
- for _, ep := range r.Endpoints {
- if ep.Method != "GET" {
- continue
- }
- for _, p := range ep.Params {
- if !seen[p.Name] {
- seen[p.Name] = true
- fields = append(fields, p)
+// collectResponseFields gathers per-item response fields for a resource's
+// endpoints, resolved via s.Types[endpoint.Response.Item]. Fields are
+// returned in the order they appear in the response type, deduplicated
+// across endpoints (list and detail share an item shape).
+//
+// Why response Types and not request Params/Body: query/path/body fields
+// are inputs sent to the API; sync stores what the API returns. Sourcing
+// columns from request-side fields means sync can never populate them.
+// When the response type is not registered in s.Types the resource yields
+// no fields and the caller leaves the table at id/data/synced_at —
+// falling back to request fields would re-emit columns sync can't fill.
+//
+// GET endpoints are walked first; non-GET (POST/PUT/PATCH) endpoints are
+// walked only if no GET endpoint contributed any fields. This handles
+// write-only resources (event-emit, webhook ingestion) whose POST/PUT
+// response is the canonical record shape, without letting wrapper-shaped
+// create-responses pollute typed columns when a GET endpoint exists.
+//
+// On dedup, an entry without a Format hint is upgraded if a later
+// endpoint declares the same field with a non-empty Format. Otherwise
+// list-vs-detail Format drift could downgrade a DATETIME column to TEXT
+// based on alphabetic endpoint-key order.
+func collectResponseFields(s *spec.APISpec, r spec.Resource) []spec.TypeField {
+ endpointKeys := make([]string, 0, len(r.Endpoints))
+ for k := range r.Endpoints {
+ endpointKeys = append(endpointKeys, k)
+ }
+ sort.Strings(endpointKeys)
+
+ collect := func(predicate func(method string) bool) []spec.TypeField {
+ seenIdx := make(map[string]int)
+ var fields []spec.TypeField
+ for _, key := range endpointKeys {
+ ep := r.Endpoints[key]
+ if !predicate(ep.Method) {
+ continue
}
- }
- for _, p := range ep.Body {
- if !seen[p.Name] {
- seen[p.Name] = true
- fields = append(fields, p)
+ typeName := ep.Response.Item
+ if typeName == "" {
+ continue
}
- }
- }
-
- // Also include body fields from POST/PUT as they often mirror response shape
- for _, ep := range r.Endpoints {
- if ep.Method == "GET" {
- continue
- }
- for _, p := range ep.Body {
- if !seen[p.Name] {
- seen[p.Name] = true
- fields = append(fields, p)
+ typeDef, ok := s.Types[typeName]
+ if !ok {
+ continue
+ }
+ for _, f := range typeDef.Fields {
+ if existing, ok := seenIdx[f.Name]; ok {
+ if fields[existing].Format == "" && f.Format != "" {
+ fields[existing].Format = f.Format
+ }
+ continue
+ }
+ seenIdx[f.Name] = len(fields)
+ fields = append(fields, f)
}
}
+ return fields
}
- return fields
+ if got := collect(func(m string) bool { return m == "GET" }); len(got) > 0 {
+ return got
+ }
+ return collect(func(m string) bool { return m != "GET" })
}
-// isScalarField returns true for string/int/bool/number fields (not objects/arrays).
-func isScalarField(p spec.Param) bool {
- switch strings.ToLower(p.Type) {
+// isScalarTypeField returns true for string/int/bool/number TypeFields
+// (not objects/arrays).
+func isScalarTypeField(f spec.TypeField) bool {
+ switch strings.ToLower(f.Type) {
case "string", "integer", "int", "boolean", "bool", "number", "float":
return true
default:
@@ -274,35 +306,38 @@ func sqliteType(goType, format string) string {
}
}
-// collectTextFieldNames finds fields likely to contain searchable text.
-func collectTextFieldNames(r spec.Resource) []string {
- textKeywords := map[string]bool{
- "title": true, "name": true, "description": true,
- "body": true, "content": true, "summary": true, "subject": true,
- "text": true, "message": true, "comment": true, "note": true,
- "notes": true, "tag": true, "tags": true, "label": true, "labels": true,
- "category": true, "categories": true, "metadata": true,
- }
+// textFieldKeywords flags response field names that should feed the FTS5
+// index. Defined at package scope so callers don't reallocate per call.
+var textFieldKeywords = map[string]bool{
+ "title": true, "name": true, "description": true,
+ "body": true, "content": true, "summary": true, "subject": true,
+ "text": true, "message": true, "comment": true, "note": true,
+ "notes": true, "tag": true, "tags": true, "label": true, "labels": true,
+ "category": true, "categories": true, "metadata": true,
+}
+// collectTextFieldNamesFromFields picks searchable scalar string fields
+// out of an already-resolved response field list. Keeps the FTS index
+// pinned to fields sync actually stores — request-side filter knobs like
+// `tags` or `labels` never appear here because they aren't in the
+// response schema.
+func collectTextFieldNamesFromFields(fields []spec.TypeField) []string {
seen := make(map[string]bool)
var result []string
-
- for _, ep := range r.Endpoints {
- allParams := append(ep.Params, ep.Body...)
- for _, p := range allParams {
- lower := strings.ToLower(p.Name)
- if textKeywords[lower] && !seen[lower] && isScalarField(p) {
- seen[lower] = true
- result = append(result, toSnakeCase(p.Name))
- }
+ for _, f := range fields {
+ lower := strings.ToLower(f.Name)
+ if !textFieldKeywords[lower] || seen[lower] || !isScalarTypeField(f) {
+ continue
}
+ seen[lower] = true
+ result = append(result, toSnakeCase(f.Name))
}
-
return result
}
-// hasField checks if a field with the given name exists in the list.
-func hasField(fields []spec.Param, name string) bool {
+// hasTypeField reports whether fields contains an entry whose name matches
+// the given snake_cased-or-lowered key.
+func hasTypeField(fields []spec.TypeField, name string) bool {
for _, f := range fields {
if toSnakeCase(f.Name) == name || strings.ToLower(f.Name) == name {
return true
@@ -312,28 +347,29 @@ func hasField(fields []spec.Param, name string) bool {
}
// buildSubResourceTable creates a table definition for a sub-resource with
-// a foreign key column referencing the parent table.
+// a foreign key column referencing the parent table. Sub-resources share
+// the base id/data/synced_at shape with top-level tables and add a
+// parent_id column between id and data.
func buildSubResourceTable(name string, r spec.Resource, parentTable string) TableDef {
tableName := toSnakeCase(name)
+ parentCol := parentTable + "_id"
- table := TableDef{
- Name: tableName,
- Columns: []ColumnDef{
- {Name: "id", Type: "TEXT", PrimaryKey: true},
- {Name: parentTable + "_id", Type: "TEXT", NotNull: true},
- {Name: "data", Type: "JSON", NotNull: true},
- {Name: "synced_at", Type: "DATETIME DEFAULT CURRENT_TIMESTAMP"},
- },
+ columns := make([]ColumnDef, 0, len(baseTableColumns)+1)
+ columns = append(columns, baseTableColumns[0]) // id
+ columns = append(columns, ColumnDef{Name: parentCol, Type: "TEXT", NotNull: true})
+ columns = append(columns, baseTableColumns[1:]...) // data, synced_at
+
+ return TableDef{
+ Name: tableName,
+ Columns: columns,
Indexes: []IndexDef{
{
- Name: "idx_" + tableName + "_" + parentTable + "_id",
+ Name: "idx_" + tableName + "_" + parentCol,
TableName: tableName,
- Columns: parentTable + "_id",
+ Columns: parentCol,
},
},
}
-
- return table
}
// sqlReservedWords is the set of SQL keywords that must be quoted when used
diff --git a/internal/generator/schema_builder_test.go b/internal/generator/schema_builder_test.go
index 9c1618f7..1390e7fb 100644
--- a/internal/generator/schema_builder_test.go
+++ b/internal/generator/schema_builder_test.go
@@ -62,54 +62,50 @@ func TestCollectTextFieldNames(t *testing.T) {
// Fields like tag/label/category/metadata should be picked up for FTS5
// alongside the core text fields. Motivated by the ESPN retro where
// "notes" (event tags) were unsearchable until manually added.
- mkResource := func(paramNames ...string) spec.Resource {
- params := make([]spec.Param, 0, len(paramNames))
- for _, n := range paramNames {
- params = append(params, spec.Param{Name: n, Type: "string"})
- }
- return spec.Resource{
- Endpoints: map[string]spec.Endpoint{
- "get": {Params: params},
- },
+ mkFields := func(names ...string) []spec.TypeField {
+ fields := make([]spec.TypeField, 0, len(names))
+ for _, n := range names {
+ fields = append(fields, spec.TypeField{Name: n, Type: "string"})
}
+ return fields
}
tests := []struct {
name string
- params []string
+ fields []string
wantIncl []string
wantExcl []string
}{
{
name: "picks up core text fields",
- params: []string{"title", "description", "body"},
+ fields: []string{"title", "description", "body"},
wantIncl: []string{"title", "description", "body"},
},
{
name: "picks up tag-family fields",
- params: []string{"name", "tag", "tags", "label", "labels"},
+ fields: []string{"name", "tag", "tags", "label", "labels"},
wantIncl: []string{"name", "tag", "tags", "label", "labels"},
},
{
name: "picks up category and metadata fields",
- params: []string{"title", "category", "categories", "metadata"},
+ fields: []string{"title", "category", "categories", "metadata"},
wantIncl: []string{"title", "category", "categories", "metadata"},
},
{
name: "picks up notes and note",
- params: []string{"note", "notes"},
+ fields: []string{"note", "notes"},
wantIncl: []string{"note", "notes"},
},
{
name: "ignores non-text fields",
- params: []string{"id", "created_at", "price"},
+ fields: []string{"id", "created_at", "price"},
wantExcl: []string{"id", "created_at", "price"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
- got := collectTextFieldNames(mkResource(tt.params...))
+ got := collectTextFieldNamesFromFields(mkFields(tt.fields...))
for _, want := range tt.wantIncl {
assert.Contains(t, got, want)
}
@@ -119,3 +115,189 @@ func TestCollectTextFieldNames(t *testing.T) {
})
}
}
+
+// TestBuildSchema_ColumnsFromResponseSchema pins that domain-table columns
+// come from the GET endpoint's response schema (looked up via
+// APISpec.Types[endpoint.Response.Item]) and never from request-side query
+// or path parameters. Without this pin, a regression where columns mirror
+// filter/sort/pagination params silently breaks every SQL-backed novel
+// command, since sync can't populate columns the response doesn't contain.
+func TestBuildSchema_ColumnsFromResponseSchema(t *testing.T) {
+ s := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "issues": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/issues",
+ Params: []spec.Param{
+ {Name: "filter", Type: "string"},
+ {Name: "labels", Type: "string"},
+ {Name: "sort", Type: "string"},
+ {Name: "since", Type: "string", Format: "date-time"},
+ {Name: "per_page", Type: "integer"},
+ {Name: "page", Type: "integer"},
+ },
+ Response: spec.ResponseDef{Type: "array", Item: "Issue"},
+ },
+ },
+ },
+ },
+ Types: map[string]spec.TypeDef{
+ "Issue": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "integer"},
+ {Name: "number", Type: "integer"},
+ {Name: "title", Type: "string"},
+ {Name: "body", Type: "string"},
+ {Name: "state", Type: "string"},
+ {Name: "created_at", Type: "string", Format: "date-time"},
+ {Name: "updated_at", Type: "string", Format: "date-time"},
+ },
+ },
+ },
+ }
+
+ issues := findTable(BuildSchema(s), "issues")
+ if !assert.NotNil(t, issues, "issues table should be emitted") {
+ return
+ }
+
+ cols := map[string]string{}
+ for _, c := range issues.Columns {
+ cols[c.Name] = c.Type
+ }
+
+ for _, want := range []string{"number", "title", "body", "state", "created_at", "updated_at"} {
+ assert.Contains(t, cols, want, "expected column %q from response schema", want)
+ }
+ for _, leak := range []string{"filter", "labels", "sort", "since", "per_page", "page"} {
+ assert.NotContains(t, cols, leak, "request param %q must not appear as a column", leak)
+ }
+ assert.Equal(t, "DATETIME", cols["created_at"])
+ assert.Equal(t, "DATETIME", cols["updated_at"])
+}
+
+// TestBuildSchema_ParamResponseNameOverlap asserts that when a request param
+// and a response field share a name (common: "state"), the resulting column
+// reflects the response field's *type* — not the param's — because the param
+// is discarded entirely from column derivation. The fixture deliberately
+// gives the param a different type from the response field so a regression
+// where the param's type leaked into column emission would be caught.
+func TestBuildSchema_ParamResponseNameOverlap(t *testing.T) {
+ s := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "issues": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Params: []spec.Param{
+ // Request-side filter knob, declared as string.
+ {Name: "state", Type: "string"},
+ },
+ Response: spec.ResponseDef{Type: "array", Item: "Issue"},
+ },
+ },
+ },
+ },
+ Types: map[string]spec.TypeDef{
+ "Issue": {
+ Fields: []spec.TypeField{
+ {Name: "id", Type: "integer"},
+ {Name: "title", Type: "string"},
+ // Response-side: state is an integer here, distinct from
+ // the request-param string. Only the response type should
+ // drive the emitted column.
+ {Name: "state", Type: "integer"},
+ },
+ },
+ },
+ }
+
+ issues := findTable(BuildSchema(s), "issues")
+ if !assert.NotNil(t, issues) {
+ return
+ }
+
+ stateCols := []ColumnDef{}
+ for _, c := range issues.Columns {
+ if c.Name == "state" {
+ stateCols = append(stateCols, c)
+ }
+ }
+ assert.Len(t, stateCols, 1, "exactly one state column should exist")
+ if len(stateCols) == 1 {
+ assert.Equal(t, "INTEGER", stateCols[0].Type,
+ "state column type must come from the response field (integer), not the request param (string)")
+ }
+}
+
+// TestBuildSchema_NoResponseTypeFallback asserts that when the GET endpoint's
+// response item cannot be resolved against APISpec.Types — either because
+// Response.Item names a type that isn't registered, or because Response.Item
+// is empty (spec author left the response declaration off entirely) — the
+// table degrades to id/data/synced_at. Hallucinating columns from request
+// params would re-introduce the bug class the response-sourcing fix targets.
+func TestBuildSchema_NoResponseTypeFallback(t *testing.T) {
+ cases := []struct {
+ name string
+ response spec.ResponseDef
+ types map[string]spec.TypeDef
+ }{
+ {
+ name: "Response.Item names an unregistered type",
+ response: spec.ResponseDef{Type: "array", Item: "UnknownItem"},
+ types: map[string]spec.TypeDef{},
+ },
+ {
+ name: "Response.Item is empty (no response declared)",
+ response: spec.ResponseDef{},
+ types: map[string]spec.TypeDef{},
+ },
+ }
+
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ s := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "issues": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Params: []spec.Param{
+ {Name: "filter", Type: "string"},
+ {Name: "page", Type: "integer"},
+ },
+ Response: tc.response,
+ },
+ },
+ },
+ },
+ Types: tc.types,
+ }
+
+ issues := findTable(BuildSchema(s), "issues")
+ if !assert.NotNil(t, issues) {
+ return
+ }
+
+ names := make([]string, 0, len(issues.Columns))
+ for _, c := range issues.Columns {
+ names = append(names, c.Name)
+ }
+ assert.ElementsMatch(t, []string{"id", "data", "synced_at"}, names,
+ "unresolved response type must yield only the base columns; got %v", names)
+ })
+ }
+}
+
+// findTable returns nil when no match exists so callers can render
+// a clearer assertion failure than `tables[0]` panicking.
+func findTable(tables []TableDef, name string) *TableDef {
+ for i := range tables {
+ if tables[i].Name == name {
+ return &tables[i]
+ }
+ }
+ return nil
+}
diff --git a/internal/openapi/parser.go b/internal/openapi/parser.go
index f7d45e04..1f652934 100644
--- a/internal/openapi/parser.go
+++ b/internal/openapi/parser.go
@@ -314,8 +314,12 @@ func parse(data []byte, lenient bool) (*spec.APISpec, error) {
if baseURL != "" {
resourceBasePath = baseURLPath(baseURL)
}
- mapResources(doc, result, resourceBasePath)
+ // mapTypes runs before mapResources so Components.Schemas types are
+ // registered first. registerInlineSchemaType (called from mapResponse)
+ // has an exists-check, so Components-defined types win on name
+ // collisions; inline schemas only fill genuine gaps.
mapTypes(doc, result)
+ mapResources(doc, result, resourceBasePath)
// Post-parse sweep: if the spec has no authentication at all (not inferred
// from description keywords), mark every endpoint as NoAuth. The per-operation
@@ -1567,7 +1571,11 @@ func mapResources(doc *openapi3.T, out *spec.APISpec, basePath string) {
endpoint.Tier = pathTier
}
- endpoint.Response, endpoint.ResponsePath = mapResponse(op, endpointName)
+ // Namespace the inline-item synthetic name with the resource so
+ // two resources whose default GET endpoints both compute the
+ // same endpointName ("list") don't collide on a shared
+ // "ListItem" Types entry.
+ endpoint.Response, endpoint.ResponsePath = mapResponse(op, targetResourceName+"_"+endpointName, out)
if strings.ToUpper(method) == "GET" {
endpoint.Pagination = detectPagination(endpoint.Params, op)
}
@@ -2452,7 +2460,7 @@ func collectAllOfProperties(
return false
}
-func mapResponse(op *openapi3.Operation, fallbackName string) (spec.ResponseDef, string) {
+func mapResponse(op *openapi3.Operation, fallbackName string, out *spec.APISpec) (spec.ResponseDef, string) {
if op == nil || op.Responses == nil {
return spec.ResponseDef{}, ""
}
@@ -2471,26 +2479,32 @@ func mapResponse(op *openapi3.Operation, fallbackName string) (spec.ResponseDef,
if isObjectSchema(schema) {
if dataRef := schema.Properties["data"]; dataRef != nil && isArraySchema(schemaRefValue(dataRef)) {
itemRef := schemaRefValue(dataRef).Items
+ itemFallback := fallbackName + "Item"
+ registerInlineSchemaType(out, itemRef, itemFallback)
return spec.ResponseDef{
Type: "array",
- Item: schemaTypeName(itemRef, fallbackName+"Item"),
+ Item: schemaTypeName(itemRef, itemFallback),
Discriminator: mapResponseDiscriminator(itemRef),
}, "data"
}
}
if isArraySchema(schema) {
+ itemFallback := fallbackName + "Item"
+ registerInlineSchemaType(out, schema.Items, itemFallback)
return spec.ResponseDef{
Type: "array",
- Item: schemaTypeName(schema.Items, fallbackName+"Item"),
+ Item: schemaTypeName(schema.Items, itemFallback),
Discriminator: mapResponseDiscriminator(schema.Items),
}, ""
}
if isObjectSchema(schema) {
+ objFallback := fallbackName + "Response"
+ registerInlineSchemaType(out, schemaRef, objFallback)
return spec.ResponseDef{
Type: "object",
- Item: schemaTypeName(schemaRef, fallbackName+"Response"),
+ Item: schemaTypeName(schemaRef, objFallback),
Discriminator: mapResponseDiscriminator(schemaRef),
}, ""
}
@@ -2792,40 +2806,81 @@ func mapTypes(doc *openapi3.T, out *spec.APISpec) {
continue
}
- properties := map[string]*openapi3.SchemaRef{}
- if isJSONAPIResourceSchema(schema) {
- jsonAPIFlattenInto(schema, properties)
- } else {
- collectTypeProperties(schemaRef, properties, map[*openapi3.Schema]struct{}{})
- }
+ out.Types[goName] = spec.TypeDef{Fields: buildTypeFields(schemaRef)}
+ }
+}
- fieldNames := make([]string, 0, len(properties))
- for fieldName := range properties {
- fieldNames = append(fieldNames, fieldName)
- }
- sort.Strings(fieldNames)
+// buildTypeFields walks an object schema's properties (flattening JSON:API
+// resource shapes the same way mapTypes/mapResponse do elsewhere) and
+// returns the spec.TypeField list used by both Components.Schemas type
+// registration and inline-response item registration. Underscore-prefixed
+// property names and Go-name collisions are filtered out so the same field
+// set drives generated Go structs and SQLite column derivation.
+func buildTypeFields(schemaRef *openapi3.SchemaRef) []spec.TypeField {
+ schema := schemaRefValue(schemaRef)
+ if schema == nil {
+ return nil
+ }
+ properties := map[string]*openapi3.SchemaRef{}
+ if isJSONAPIResourceSchema(schema) {
+ jsonAPIFlattenInto(schema, properties)
+ } else {
+ collectTypeProperties(schemaRef, properties, map[*openapi3.Schema]struct{}{})
+ }
- fields := make([]spec.TypeField, 0, len(fieldNames))
- seenGoNames := map[string]bool{}
- for _, fieldName := range fieldNames {
- if strings.HasPrefix(fieldName, "_") {
- continue
- }
- goFieldName := toCamelCase(fieldName)
- if seenGoNames[goFieldName] {
- continue
- }
- seenGoNames[goFieldName] = true
- fieldSchema := schemaRefValue(properties[fieldName])
- fields = append(fields, spec.TypeField{
- Name: fieldName,
- Type: mapSchemaType(fieldSchema),
- Enum: schemaEnum(fieldSchema),
- })
+ fieldNames := make([]string, 0, len(properties))
+ for fieldName := range properties {
+ fieldNames = append(fieldNames, fieldName)
+ }
+ sort.Strings(fieldNames)
+
+ fields := make([]spec.TypeField, 0, len(fieldNames))
+ seenGoNames := map[string]bool{}
+ for _, fieldName := range fieldNames {
+ if strings.HasPrefix(fieldName, "_") {
+ continue
+ }
+ goFieldName := toCamelCase(fieldName)
+ if seenGoNames[goFieldName] {
+ continue
}
+ seenGoNames[goFieldName] = true
+ fieldSchema := schemaRefValue(properties[fieldName])
+ fields = append(fields, spec.TypeField{
+ Name: fieldName,
+ Type: mapSchemaType(fieldSchema),
+ Enum: schemaEnum(fieldSchema),
+ Format: schemaFormat(fieldSchema),
+ })
+ }
+ return fields
+}
- out.Types[goName] = spec.TypeDef{Fields: fields}
+// registerInlineSchemaType registers an inline response schema (item type
+// for list responses or full object type for single-object responses) into
+// out.Types under the synthetic name mapResponse uses for
+// endpoint.Response.Item. $ref-shaped schemas already land in Types via
+// mapTypes; this only fires when the schema is inline and the slot is
+// empty. No-op when out is nil (test-only callers).
+func registerInlineSchemaType(out *spec.APISpec, itemRef *openapi3.SchemaRef, fallbackName string) {
+ if out == nil || itemRef == nil || refComponentName(itemRef.Ref) != "" {
+ return
+ }
+ itemSchema := schemaRefValue(itemRef)
+ if itemSchema == nil || !isObjectSchema(itemSchema) {
+ return
+ }
+ typeName := schemaTypeName(itemRef, fallbackName)
+ if typeName == "" {
+ return
+ }
+ if out.Types == nil {
+ out.Types = map[string]spec.TypeDef{}
+ }
+ if _, exists := out.Types[typeName]; exists {
+ return
}
+ out.Types[typeName] = spec.TypeDef{Fields: buildTypeFields(itemRef)}
}
func mapResponseDiscriminator(schemaRef *openapi3.SchemaRef) *spec.ResponseDiscriminator {
diff --git a/internal/openapi/parser_test.go b/internal/openapi/parser_test.go
index feb2a64a..db47a5ce 100644
--- a/internal/openapi/parser_test.go
+++ b/internal/openapi/parser_test.go
@@ -120,6 +120,310 @@ components:
assert.Equal(t, []string{"workspace", "collection"}, typeField.Enum)
}
+// TestParseRegistersInlineListResponseItemTypes pins that inline list-
+// response item schemas land in APISpec.Types under the synthetic name
+// mapResponse stores in endpoint.Response.Item. Without this, a list
+// endpoint whose item schema is declared inline produces a Types miss and
+// the generated store table degrades to id/data/synced_at, losing every
+// typed column for that resource.
+func TestParseRegistersInlineListResponseItemTypes(t *testing.T) {
+ t.Parallel()
+
+ parsed, err := Parse([]byte(`
+openapi: 3.0.3
+info:
+ title: Inline Issues API
+ version: 1.0.0
+paths:
+ /issues:
+ get:
+ operationId: listIssues
+ parameters:
+ - name: filter
+ in: query
+ schema:
+ type: string
+ - name: state
+ in: query
+ schema:
+ type: string
+ responses:
+ "200":
+ description: ok
+ content:
+ application/json:
+ schema:
+ type: array
+ items:
+ type: object
+ properties:
+ id:
+ type: integer
+ title:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ format: date-time
+`))
+ require.NoError(t, err)
+
+ endpoint := parsed.Resources["issues"].Endpoints["list"]
+ require.NotEmpty(t, endpoint.Response.Item, "list endpoint should resolve a response item type name")
+
+ typeDef, ok := parsed.Types[endpoint.Response.Item]
+ require.True(t, ok, "inline response item schema must register into Types under %q",
+ endpoint.Response.Item)
+
+ fieldsByName := map[string]spec.TypeField{}
+ for _, f := range typeDef.Fields {
+ fieldsByName[f.Name] = f
+ }
+
+ for _, want := range []string{"id", "title", "state", "created_at"} {
+ _, ok := fieldsByName[want]
+ assert.True(t, ok, "expected response field %q registered under inline item type", want)
+ }
+
+ // Format hint must propagate so DATETIME columns survive end-to-end.
+ assert.Equal(t, "date-time", fieldsByName["created_at"].Format,
+ "created_at format must be carried through TypeField for DATETIME mapping")
+
+ // Request-side query parameters must NOT bleed into the response type.
+ for _, leak := range []string{"filter"} {
+ _, leaked := fieldsByName[leak]
+ assert.False(t, leaked, "request parameter %q must not appear in response Type", leak)
+ }
+}
+
+// TestParseInlineItemTypesNamespacedByResource pins that two resources
+// whose default GET endpoints share an endpointName ("list") and both
+// declare inline (no-$ref, no-title) array-item schemas get distinct
+// Types entries. Without resource-namespacing, both registrations would
+// land on the same synthetic name and the second resource would silently
+// inherit the first's response shape — re-introducing the wrong-columns
+// bug class for cross-resource cases.
+func TestParseInlineItemTypesNamespacedByResource(t *testing.T) {
+ t.Parallel()
+
+ parsed, err := Parse([]byte(`
+openapi: 3.0.3
+info:
+ title: Two Resources
+ version: 1.0.0
+paths:
+ /issues:
+ get:
+ responses:
+ "200":
+ description: ok
+ content:
+ application/json:
+ schema:
+ type: array
+ items:
+ type: object
+ properties:
+ issue_field:
+ type: string
+ /users:
+ get:
+ responses:
+ "200":
+ description: ok
+ content:
+ application/json:
+ schema:
+ type: array
+ items:
+ type: object
+ properties:
+ user_field:
+ type: string
+`))
+ require.NoError(t, err)
+
+ issuesItem := parsed.Resources["issues"].Endpoints["list"].Response.Item
+ usersItem := parsed.Resources["users"].Endpoints["list"].Response.Item
+ require.NotEmpty(t, issuesItem)
+ require.NotEmpty(t, usersItem)
+ assert.NotEqual(t, issuesItem, usersItem,
+ "two resources with default-named GET endpoints must produce distinct synthetic item type names")
+
+ issuesType, ok := parsed.Types[issuesItem]
+ require.True(t, ok)
+ usersType, ok := parsed.Types[usersItem]
+ require.True(t, ok)
+
+ issuesNames := []string{}
+ for _, f := range issuesType.Fields {
+ issuesNames = append(issuesNames, f.Name)
+ }
+ usersNames := []string{}
+ for _, f := range usersType.Fields {
+ usersNames = append(usersNames, f.Name)
+ }
+ assert.Contains(t, issuesNames, "issue_field")
+ assert.NotContains(t, issuesNames, "user_field",
+ "issues item type must not contain users' fields")
+ assert.Contains(t, usersNames, "user_field")
+ assert.NotContains(t, usersNames, "issue_field",
+ "users item type must not contain issues' fields")
+}
+
+// TestParseRegistersInlineSingleObjectResponseTypes pins that detail-only
+// resources (GET /x/{id} with a single-object inline response) get their
+// per-item schema registered into Types. Without registration, BuildSchema
+// would degrade these tables to id/data/synced_at and lose typed columns
+// for any API that exposes only a detail endpoint.
+func TestParseRegistersInlineSingleObjectResponseTypes(t *testing.T) {
+ t.Parallel()
+
+ parsed, err := Parse([]byte(`
+openapi: 3.0.3
+info:
+ title: Detail Only
+ version: 1.0.0
+paths:
+ /widgets/{id}:
+ get:
+ parameters:
+ - name: id
+ in: path
+ required: true
+ schema: { type: string }
+ responses:
+ "200":
+ description: ok
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ id:
+ type: string
+ name:
+ type: string
+ created_at:
+ type: string
+ format: date-time
+`))
+ require.NoError(t, err)
+
+ endpoint := parsed.Resources["widgets"].Endpoints["get"]
+ require.NotEmpty(t, endpoint.Response.Item)
+
+ typeDef, ok := parsed.Types[endpoint.Response.Item]
+ require.True(t, ok, "single-object inline response should register a Types entry under %q", endpoint.Response.Item)
+
+ names := []string{}
+ for _, f := range typeDef.Fields {
+ names = append(names, f.Name)
+ }
+ assert.Contains(t, names, "name")
+ assert.Contains(t, names, "created_at")
+}
+
+// TestParseAndBuildSchemaSourcesColumnsFromResponse is the end-to-end
+// regression for the OpenAPI parser → BuildSchema seam. A list endpoint
+// with filter/sort/pagination query parameters must produce a SQLite
+// table whose columns mirror the response item schema, not the request-
+// side parameters — otherwise sync stores nothing in those columns and
+// SQL queries silently return NULL.
+func TestParseAndBuildSchemaSourcesColumnsFromResponse(t *testing.T) {
+ t.Parallel()
+
+ parsed, err := Parse([]byte(`
+openapi: 3.0.3
+info:
+ title: Issues
+ version: 1.0.0
+paths:
+ /issues:
+ get:
+ operationId: listIssues
+ parameters:
+ - name: filter
+ in: query
+ schema: { type: string }
+ - name: state
+ in: query
+ schema: { type: string }
+ - name: labels
+ in: query
+ schema: { type: string }
+ - name: sort
+ in: query
+ schema: { type: string }
+ - name: per_page
+ in: query
+ schema: { type: integer }
+ - name: page
+ in: query
+ schema: { type: integer }
+ responses:
+ "200":
+ description: ok
+ content:
+ application/json:
+ schema:
+ type: array
+ items:
+ $ref: "#/components/schemas/Issue"
+components:
+ schemas:
+ Issue:
+ type: object
+ properties:
+ id:
+ type: integer
+ number:
+ type: integer
+ title:
+ type: string
+ body:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ format: date-time
+ updated_at:
+ type: string
+ format: date-time
+`))
+ require.NoError(t, err)
+
+ tables := generator.BuildSchema(parsed)
+ var issues *generator.TableDef
+ for i := range tables {
+ if tables[i].Name == "issues" {
+ issues = &tables[i]
+ break
+ }
+ }
+ require.NotNil(t, issues, "issues table should be emitted from the parsed spec")
+
+ cols := map[string]string{}
+ for _, c := range issues.Columns {
+ cols[c.Name] = c.Type
+ }
+
+ // Response-derived columns must be present.
+ for _, want := range []string{"number", "title", "body", "state", "created_at", "updated_at"} {
+ assert.Contains(t, cols, want, "expected column %q sourced from Issue schema", want)
+ }
+
+ // Query-param leaks the bug used to cause:
+ for _, leak := range []string{"filter", "labels", "sort", "per_page", "page"} {
+ assert.NotContains(t, cols, leak, "request param %q must not appear as a column", leak)
+ }
+
+ // Format hint flows through OpenAPI parser → TypeField → sqliteType.
+ assert.Equal(t, "DATETIME", cols["created_at"], "date-time format must map to DATETIME column")
+}
+
func TestParseMapsAllOfRequestBodyFields(t *testing.T) {
t.Parallel()
diff --git a/internal/spec/spec.go b/internal/spec/spec.go
index bdd323a7..6fe78f1b 100644
--- a/internal/spec/spec.go
+++ b/internal/spec/spec.go
@@ -1086,6 +1086,12 @@ type TypeField struct {
Name string `yaml:"name" json:"name"`
Type string `yaml:"type" json:"type"`
Enum []string `yaml:"enum,omitempty" json:"enum,omitempty"`
+ // Format mirrors the OpenAPI `format` hint for the field (date-time,
+ // date, email, uri, …). Carried through so SQLite column derivation
+ // can map date/date-time response fields to DATETIME instead of TEXT.
+ // Empty for fields with no format declared and for internal YAML specs
+ // that never set it.
+ Format string `yaml:"format,omitempty" json:"format,omitempty"`
// Selection is an optional GraphQL sub-selection rendered when this field
// is used in a generated GraphQL query. It lets wrapper specs keep the Go
// field simple (for example, totalPriceSet as json.RawMessage) while still
← b196bd81 fix(cli): close MCP sql tool exfiltration vector with allowl
·
back to Cli Printing Press
·
fix(cli): shard sub-resource tables per parent on collision 10230542 →