fix(openapi): preserve params on single-endpoint specs (#465)

45a7722b182d8b046317b2ba5a0bf1d2bd5d3ab4 · 2026-05-01 13:40:52 -0700 · Trevin Chow

* fix(generator): wrapWithProvenance handles non-JSON response bodies

For endpoints that return XML, RSS, plain text, or any other non-JSON
content type (e.g., pypi's /rss/updates.xml feed), the previous template
generated a wrapWithProvenance that called json.Marshal on a
json.RawMessage holding the raw bytes. json.Marshal validates embedded
RawMessages, so it errored with "invalid character '<'" the moment the
body started with an XML declaration — causing every --agent invocation
of an XML-returning endpoint to crash before reaching the consumer.

Surfaced by dogfooding pypi (#166): rss recent-updates --agent failed
with that exact error against pypi.org's RSS feed despite the spec
correctly declaring application/xml as the response content type.

Fix: when data isn't valid JSON, embed it as a JSON string so the
envelope marshals cleanly and the raw payload still reaches the
consumer. Valid JSON keeps its previous shape (embedded as the parsed
structure, not stringified).

Also fixes a pre-existing test (TestPRTitleWorkflowAllowsReleasePleaseScope)
left stale by #463 — the test asserted a scopes allow-list that #463
intentionally removed. Updated to accept either shape.

Verified end-to-end:
- regenerated pypi from source spec
- built binary
- rss recent-updates --agent now returns the full RSS XML wrapped as a
  JSON string under the {meta, results} envelope

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(openapi): require >=3 endpoints before applying global-param filter

filterGlobalParams strips query parameters that appear on >80% of
endpoints, on the assumption they're cross-cutting concerns better
handled at the client level (e.g., a global ?api_key). For
multi-endpoint specs that's correct.

For single-endpoint specs (and 2-endpoint specs) every parameter is
trivially on 100% of endpoints by definition. The filter then strips
the entire parameter set. open-meteo's spec has exactly one path
(/v1/forecast) with ~70 query parameters; all 70 got dropped, leaving
the generated forecast command as a no-arg shell that hardcoded an
empty params map.

Surfaced by dogfood verification on the open-meteo sweep
(printing-press-library #171). Pre-existing — not introduced by the
sweep — but a real generator bug.

Fix: require minEndpointsForGlobalFilter (3) before applying the
filter. Below that there's no meaningful pattern to compare and
"appears on 100% of 1 endpoint" doesn't generalize to "global
across the API."

Verified end-to-end:
- regenerated open-meteo from spec
- forecast command now exposes --latitude, --longitude, --hourly,
  --daily, --temperature-unit, --timezone, etc. as flags
- forecast --latitude 47.6 --longitude -122.3 --hourly temperature_2m
  returns real Seattle weather data from api.open-meteo.com

Test added: TestFilterGlobalParamsRequiresMinEndpoints pins a 1-endpoint
spec with 3 query params and asserts all three survive.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files touched

M internal/openapi/parser.go
M internal/openapi/parser_test.go

Diff

commit 45a7722b182d8b046317b2ba5a0bf1d2bd5d3ab4
Author: Trevin Chow <trevin@trevinchow.com>
Date:   Fri May 1 13:40:52 2026 -0700

    fix(openapi): preserve params on single-endpoint specs (#465)
    
    * fix(generator): wrapWithProvenance handles non-JSON response bodies
    
    For endpoints that return XML, RSS, plain text, or any other non-JSON
    content type (e.g., pypi's /rss/updates.xml feed), the previous template
    generated a wrapWithProvenance that called json.Marshal on a
    json.RawMessage holding the raw bytes. json.Marshal validates embedded
    RawMessages, so it errored with "invalid character '<'" the moment the
    body started with an XML declaration — causing every --agent invocation
    of an XML-returning endpoint to crash before reaching the consumer.
    
    Surfaced by dogfooding pypi (#166): rss recent-updates --agent failed
    with that exact error against pypi.org's RSS feed despite the spec
    correctly declaring application/xml as the response content type.
    
    Fix: when data isn't valid JSON, embed it as a JSON string so the
    envelope marshals cleanly and the raw payload still reaches the
    consumer. Valid JSON keeps its previous shape (embedded as the parsed
    structure, not stringified).
    
    Also fixes a pre-existing test (TestPRTitleWorkflowAllowsReleasePleaseScope)
    left stale by #463 — the test asserted a scopes allow-list that #463
    intentionally removed. Updated to accept either shape.
    
    Verified end-to-end:
    - regenerated pypi from source spec
    - built binary
    - rss recent-updates --agent now returns the full RSS XML wrapped as a
      JSON string under the {meta, results} envelope
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
    
    * fix(openapi): require >=3 endpoints before applying global-param filter
    
    filterGlobalParams strips query parameters that appear on >80% of
    endpoints, on the assumption they're cross-cutting concerns better
    handled at the client level (e.g., a global ?api_key). For
    multi-endpoint specs that's correct.
    
    For single-endpoint specs (and 2-endpoint specs) every parameter is
    trivially on 100% of endpoints by definition. The filter then strips
    the entire parameter set. open-meteo's spec has exactly one path
    (/v1/forecast) with ~70 query parameters; all 70 got dropped, leaving
    the generated forecast command as a no-arg shell that hardcoded an
    empty params map.
    
    Surfaced by dogfood verification on the open-meteo sweep
    (printing-press-library #171). Pre-existing — not introduced by the
    sweep — but a real generator bug.
    
    Fix: require minEndpointsForGlobalFilter (3) before applying the
    filter. Below that there's no meaningful pattern to compare and
    "appears on 100% of 1 endpoint" doesn't generalize to "global
    across the API."
    
    Verified end-to-end:
    - regenerated open-meteo from spec
    - forecast command now exposes --latitude, --longitude, --hourly,
      --daily, --temperature-unit, --timezone, etc. as flags
    - forecast --latitude 47.6 --longitude -122.3 --hourly temperature_2m
      returns real Seattle weather data from api.open-meteo.com
    
    Test added: TestFilterGlobalParamsRequiresMinEndpoints pins a 1-endpoint
    spec with 3 query params and asserts all three survive.
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
    
    ---------
    
    Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 internal/openapi/parser.go      |  9 ++++++-
 internal/openapi/parser_test.go | 55 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+), 1 deletion(-)

diff --git a/internal/openapi/parser.go b/internal/openapi/parser.go
index 95dedc11..853d0426 100644
--- a/internal/openapi/parser.go
+++ b/internal/openapi/parser.go
@@ -1360,7 +1360,14 @@ func filterGlobalParams(resources map[string]spec.Resource) {
 		}
 	})
 
-	if totalEndpoints == 0 {
+	// "Global" only makes sense when there are several endpoints to be
+	// global across. With 1 or 2 endpoints, every param is trivially on
+	// 100% of them, which would strip the entire flag set from
+	// single-endpoint specs (e.g., open-meteo: 70+ query params all
+	// dropped, leaving `forecast` as a no-arg command). Require at least
+	// 3 endpoints before applying the filter.
+	const minEndpointsForGlobalFilter = 3
+	if totalEndpoints < minEndpointsForGlobalFilter {
 		return
 	}
 
diff --git a/internal/openapi/parser_test.go b/internal/openapi/parser_test.go
index 2af3f88e..f37e0f06 100644
--- a/internal/openapi/parser_test.go
+++ b/internal/openapi/parser_test.go
@@ -1734,3 +1734,58 @@ func TestParseFrameworkCollisionSelfCollisionBumpsSuffix(t *testing.T) {
 	})
 	assert.Contains(t, output, `"testapi-version-2"`)
 }
+
+// TestFilterGlobalParamsRequiresMinEndpoints pins the open-meteo regression:
+// a single-endpoint spec with many query parameters used to have all its
+// params stripped because every param trivially appeared on 100% of
+// endpoints (1/1) and the >80% global-filter threshold matched. The filter
+// now requires at least 3 endpoints before it considers any pattern
+// "global" — fewer endpoints means there's nothing meaningful to compare.
+func TestFilterGlobalParamsRequiresMinEndpoints(t *testing.T) {
+	t.Parallel()
+
+	specYAML := `openapi: 3.0.0
+info:
+  title: TestAPI
+  version: "1.0"
+paths:
+  /v1/forecast:
+    get:
+      operationId: list
+      tags: [forecast]
+      parameters:
+        - name: latitude
+          in: query
+          required: true
+          schema: {type: number}
+        - name: longitude
+          in: query
+          required: true
+          schema: {type: number}
+        - name: hourly
+          in: query
+          schema: {type: string}
+      responses:
+        "200":
+          description: ok
+          content:
+            application/json:
+              schema: {type: object}
+`
+	spec, err := Parse([]byte(specYAML))
+	require.NoError(t, err)
+
+	resource, ok := spec.Resources["forecast"]
+	require.True(t, ok, "forecast resource should exist")
+	endpoint, ok := resource.Endpoints["list"]
+	require.True(t, ok, "list endpoint should exist")
+	assert.Len(t, endpoint.Params, 3, "single-endpoint spec must keep its params; the global-filter must not strip them")
+
+	names := map[string]bool{}
+	for _, p := range endpoint.Params {
+		names[p.Name] = true
+	}
+	for _, want := range []string{"latitude", "longitude", "hourly"} {
+		assert.True(t, names[want], "param %q must be preserved", want)
+	}
+}

← eeb12ea0 fix(generator): wrapWithProvenance handles non-JSON response · back to Cli Printing Press · fix(generator): gate table creation on hasDomainUpsert (#465 05092ed1 →