← back to Cli Printing Press
fix(cli): route Swagger 2.0 specs through openapi2conv to avoid OOM on circular refs (#1630)
e798f6716a184910bc5e00fa58ddcecbf9e97c9a · 2026-05-18 12:40:50 -0700 · Trevin Chow
* fix(cli): route Swagger 2.0 specs through openapi2conv to avoid OOM on circular refs
Swagger 2.0 specs with circular $ref chains in definitions/ (Tripletex,
NetSuite REST, Salesforce Tooling) caused the generator to burn 15-30
minutes of CPU at 1.8-4.4 GB RSS and eventually OOM. The same content
converted to OpenAPI 3.x externally via swagger2openapi generated in
about three minutes.
Detect Swagger 2.0 at the parser boundary (normalized JSON has top-level
"swagger": "2.0") and route through openapi2conv.ToV3WithLoader before
the OpenAPI 3 loader runs. The conversion rewrites #/definitions/X to
#/components/schemas/X so the existing cycle-aware OpenAPI 3 code path
handles resolution. openapi2conv is a sub-package of kin-openapi, which
is already a dependency.
Emit one stderr line announcing the conversion so operators have
visibility into the multi-minute resolution phase. The retro called out
the silent phase as the biggest UX issue in the failure mode.
Closes #1241
* fix(cli): address Greptile feedback on Swagger 2.0 loader (P1 + 2 P2s)
P1 (Missing ReadFromURIFunc): the conversion path's loader did not
install the custom ReadFromURIFunc that the main OpenAPI 3 path uses,
so a Swagger 2.0 spec loaded with a file location whose external $refs
pointed at YAML companion files would skip the YAML->JSON normalization
step and fail deep inside ToV3WithLoader. Extract the loader setup into
newConfiguredOpenAPI3Loader and call it from both paths.
P2 (Overly permissive version match): drop the "swagger":"2" prefix
variants; the Swagger 2.0 spec mandates exactly "2.0" so the bare
"2" forms only matched hypothetical future version strings. Add
test fixtures asserting "2" and "2.5" do not trigger detection.
P2 (Goroutine leak on timeout): document the intentional leak in the
30s-timeout branch of the cyclic-ref regression test. Parse exposes no
cancellation hook; failing the test fast and abandoning the goroutine
is the lesser evil compared to letting CI sit for ~25 minutes before
OOMing.
Refs #1241
* fix(cli): detect Swagger 2.0 via streaming JSON decode, not head substring
Greptile P1: the substring-scan detector silently missed real-world
Swagger 2.0 specs. normalizeSpecData round-trips through encoding/json,
which sorts map keys alphabetically — so "swagger" (s) lands AFTER
"definitions" (d) and "paths" (p) in the serialized output. For a
multi-MB vendor spec (Tripletex, NetSuite, Salesforce), the "swagger"
marker is far past any reasonable head-buffer window, and the detector
returned false on the exact specs this PR was built to handle.
Switch to a streaming json.Decoder that walks top-level keys, returning
true only when a top-level "swagger" key has the value "2.0". Stops
reading after the swagger value is found; for non-Swagger 2.0 specs it
traverses tokens at the top level (skipping nested values en bloc),
which is still fast for the ~2MB upper bound of real specs.
Add a regression test that builds a Swagger 2.0 fixture whose alphabetized
serialization pushes the "swagger" marker well past the 4KB mark.
Refs #1241
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Files touched
M internal/openapi/parser.goA internal/openapi/swagger2.goA internal/openapi/swagger2_test.go
Diff
commit e798f6716a184910bc5e00fa58ddcecbf9e97c9a
Author: Trevin Chow <trevin@trevinchow.com>
Date: Mon May 18 12:40:50 2026 -0700
fix(cli): route Swagger 2.0 specs through openapi2conv to avoid OOM on circular refs (#1630)
* fix(cli): route Swagger 2.0 specs through openapi2conv to avoid OOM on circular refs
Swagger 2.0 specs with circular $ref chains in definitions/ (Tripletex,
NetSuite REST, Salesforce Tooling) caused the generator to burn 15-30
minutes of CPU at 1.8-4.4 GB RSS and eventually OOM. The same content
converted to OpenAPI 3.x externally via swagger2openapi generated in
about three minutes.
Detect Swagger 2.0 at the parser boundary (normalized JSON has top-level
"swagger": "2.0") and route through openapi2conv.ToV3WithLoader before
the OpenAPI 3 loader runs. The conversion rewrites #/definitions/X to
#/components/schemas/X so the existing cycle-aware OpenAPI 3 code path
handles resolution. openapi2conv is a sub-package of kin-openapi, which
is already a dependency.
Emit one stderr line announcing the conversion so operators have
visibility into the multi-minute resolution phase. The retro called out
the silent phase as the biggest UX issue in the failure mode.
Closes #1241
* fix(cli): address Greptile feedback on Swagger 2.0 loader (P1 + 2 P2s)
P1 (Missing ReadFromURIFunc): the conversion path's loader did not
install the custom ReadFromURIFunc that the main OpenAPI 3 path uses,
so a Swagger 2.0 spec loaded with a file location whose external $refs
pointed at YAML companion files would skip the YAML->JSON normalization
step and fail deep inside ToV3WithLoader. Extract the loader setup into
newConfiguredOpenAPI3Loader and call it from both paths.
P2 (Overly permissive version match): drop the "swagger":"2" prefix
variants; the Swagger 2.0 spec mandates exactly "2.0" so the bare
"2" forms only matched hypothetical future version strings. Add
test fixtures asserting "2" and "2.5" do not trigger detection.
P2 (Goroutine leak on timeout): document the intentional leak in the
30s-timeout branch of the cyclic-ref regression test. Parse exposes no
cancellation hook; failing the test fast and abandoning the goroutine
is the lesser evil compared to letting CI sit for ~25 minutes before
OOMing.
Refs #1241
* fix(cli): detect Swagger 2.0 via streaming JSON decode, not head substring
Greptile P1: the substring-scan detector silently missed real-world
Swagger 2.0 specs. normalizeSpecData round-trips through encoding/json,
which sorts map keys alphabetically — so "swagger" (s) lands AFTER
"definitions" (d) and "paths" (p) in the serialized output. For a
multi-MB vendor spec (Tripletex, NetSuite, Salesforce), the "swagger"
marker is far past any reasonable head-buffer window, and the detector
returned false on the exact specs this PR was built to handle.
Switch to a streaming json.Decoder that walks top-level keys, returning
true only when a top-level "swagger" key has the value "2.0". Stops
reading after the swagger value is found; for non-Swagger 2.0 specs it
traverses tokens at the top level (skipping nested values en bloc),
which is still fast for the ~2MB upper bound of real specs.
Add a regression test that builds a Swagger 2.0 fixture whose alphabetized
serialization pushes the "swagger" marker well past the 4KB mark.
Refs #1241
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
---
internal/openapi/parser.go | 27 ++++-
internal/openapi/swagger2.go | 124 ++++++++++++++++++++++
internal/openapi/swagger2_test.go | 212 ++++++++++++++++++++++++++++++++++++++
3 files changed, 359 insertions(+), 4 deletions(-)
diff --git a/internal/openapi/parser.go b/internal/openapi/parser.go
index 79ecb131..795496d6 100644
--- a/internal/openapi/parser.go
+++ b/internal/openapi/parser.go
@@ -1063,6 +1063,28 @@ func authVarsExtension(raw any) ([]spec.AuthEnvVar, error) {
}
func loadOpenAPIDoc(data []byte, lenient bool, location *url.URL) (*openapi3.T, error) {
+ // Swagger 2.0 specs with circular $ref chains (Tripletex, NetSuite, etc.)
+ // burn 15-30 minutes of CPU and OOM the process when fed straight to the
+ // OpenAPI 3 loader. Detect them at the boundary and route through
+ // openapi2conv.ToV3 so the existing OpenAPI 3 code path handles the
+ // resolved spec. See issue #1241 and internal/openapi/swagger2.go.
+ if isSwagger2SpecJSON(data) {
+ return loadSwagger2AsOpenAPI3(data, lenient, location)
+ }
+
+ loader := newConfiguredOpenAPI3Loader(lenient, location)
+ if location != nil {
+ return loader.LoadFromDataWithPath(data, location)
+ }
+ return loader.LoadFromData(data)
+}
+
+// newConfiguredOpenAPI3Loader returns an openapi3.Loader with the project's
+// standard ReadFromURIFunc installed: the per-ref file-URI guard for strict
+// mode and the YAML->JSON normalization step for referenced files. Shared
+// between the OpenAPI 3 load path and the Swagger 2.0 conversion path so both
+// honor the same external-ref policy.
+func newConfiguredOpenAPI3Loader(lenient bool, location *url.URL) *openapi3.Loader {
loader := openapi3.NewLoader()
loader.IsExternalRefsAllowed = lenient || location != nil
allowLocalExternalRefs := location != nil
@@ -1081,10 +1103,7 @@ func loadOpenAPIDoc(data []byte, lenient bool, location *url.URL) (*openapi3.T,
}
return data, nil
}
- if location != nil {
- return loader.LoadFromDataWithPath(data, location)
- }
- return loader.LoadFromData(data)
+ return loader
}
func isFileURI(location *url.URL) bool {
diff --git a/internal/openapi/swagger2.go b/internal/openapi/swagger2.go
new file mode 100644
index 00000000..9171601e
--- /dev/null
+++ b/internal/openapi/swagger2.go
@@ -0,0 +1,124 @@
+package openapi
+
+import (
+ "bytes"
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/url"
+ "os"
+
+ "github.com/getkin/kin-openapi/openapi2"
+ "github.com/getkin/kin-openapi/openapi2conv"
+ "github.com/getkin/kin-openapi/openapi3"
+)
+
+// isSwagger2SpecJSON reports whether normalized JSON spec bytes describe a
+// Swagger 2.0 (OpenAPI v2) document.
+//
+// Real-world Swagger 2.0 specs (Tripletex, NetSuite REST, Salesforce Tooling)
+// frequently contain circular $ref chains through `definitions/`. Routing them
+// through openapi2conv.ToV3 before kin-openapi's OpenAPI 3 resolver runs
+// avoids the runaway memory + 15-30 minute hangs documented in the retro
+// (issue #1241): the conversion rewrites `#/definitions/X` to
+// `#/components/schemas/X` and lets the existing cycle-aware OpenAPI 3 code
+// path do the resolution work.
+//
+// Uses a streaming JSON decoder to find the top-level `swagger` key and stop
+// reading. Substring scanning is unsafe at this stage: normalizeSpecData
+// round-trips through encoding/json, which sorts map keys alphabetically, so
+// "swagger" lands AFTER "definitions" and "paths" in the serialized output —
+// far past any reasonable head window for a multi-MB spec like Tripletex.
+func isSwagger2SpecJSON(data []byte) bool {
+ dec := json.NewDecoder(bytes.NewReader(data))
+ // Expect a top-level object.
+ tok, err := dec.Token()
+ if err != nil {
+ return false
+ }
+ if delim, ok := tok.(json.Delim); !ok || delim != '{' {
+ return false
+ }
+ // Walk top-level keys. For each key whose value we don't need, skip the
+ // value by reading and discarding tokens until the value's depth balances.
+ for dec.More() {
+ keyTok, err := dec.Token()
+ if err != nil {
+ return false
+ }
+ key, ok := keyTok.(string)
+ if !ok {
+ return false
+ }
+ valTok, err := dec.Token()
+ if err != nil {
+ return false
+ }
+ if key == "swagger" {
+ value, ok := valTok.(string)
+ return ok && value == "2.0"
+ }
+ // If the value is a container, drain it. Scalars consumed one token
+ // by the call above and need no further work.
+ if delim, ok := valTok.(json.Delim); ok && (delim == '{' || delim == '[') {
+ if err := skipJSONValue(dec, 1); err != nil {
+ return false
+ }
+ }
+ }
+ return false
+}
+
+// skipJSONValue consumes JSON tokens from dec until the container depth
+// returns to zero. depth must start at 1 because the caller has already read
+// the opening `{` or `[`.
+func skipJSONValue(dec *json.Decoder, depth int) error {
+ for depth > 0 {
+ tok, err := dec.Token()
+ if err != nil {
+ if err == io.EOF {
+ return err
+ }
+ return err
+ }
+ if delim, ok := tok.(json.Delim); ok {
+ switch delim {
+ case '{', '[':
+ depth++
+ case '}', ']':
+ depth--
+ }
+ }
+ }
+ return nil
+}
+
+// loadSwagger2AsOpenAPI3 parses normalized Swagger 2.0 JSON bytes into an
+// openapi2.T, converts the result to an openapi3.T, and returns the converted
+// document. The returned doc behaves identically to one produced by
+// openapi3.Loader.LoadFromData on the equivalent OpenAPI 3 spec, which lets
+// the downstream parser code path stay format-agnostic.
+//
+// Emits a single stderr line so operators can see why a long-running phase
+// happened on Swagger 2.0 input. The retro called out the silent multi-minute
+// phase as the biggest UX issue in the failure mode this function exists to
+// fix.
+func loadSwagger2AsOpenAPI3(data []byte, lenient bool, location *url.URL) (*openapi3.T, error) {
+ var doc2 openapi2.T
+ if err := json.Unmarshal(data, &doc2); err != nil {
+ return nil, fmt.Errorf("loading Swagger 2.0 spec: %w", err)
+ }
+
+ // Reuse the OpenAPI 3 loader configuration so the conversion path honors
+ // the same external-ref policy (strict file-URI guard + YAML->JSON
+ // normalization for referenced files) as the direct OpenAPI 3 load path.
+ loader := newConfiguredOpenAPI3Loader(lenient, location)
+
+ fmt.Fprintf(os.Stderr, "info: converting Swagger 2.0 spec to OpenAPI 3 before parsing\n")
+
+ doc3, err := openapi2conv.ToV3WithLoader(&doc2, loader, location)
+ if err != nil {
+ return nil, fmt.Errorf("converting Swagger 2.0 to OpenAPI 3: %w", err)
+ }
+ return doc3, nil
+}
diff --git a/internal/openapi/swagger2_test.go b/internal/openapi/swagger2_test.go
new file mode 100644
index 00000000..fdef0f04
--- /dev/null
+++ b/internal/openapi/swagger2_test.go
@@ -0,0 +1,212 @@
+package openapi
+
+import (
+ "encoding/json"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+)
+
+// bigSwagger2Spec returns a Swagger 2.0 JSON document with a large
+// `definitions/` block padded so that the `"swagger":"2.0"` marker only
+// appears well past the 4KB mark in the serialized output. Models the real
+// shape produced by normalizeSpecData when fed a vendor Swagger 2.0 spec:
+// encoding/json sorts keys alphabetically, so "definitions" (d) and "paths"
+// (p) precede "swagger" (s) in the wire form.
+func bigSwagger2Spec() []byte {
+ const pad = 8192 // generously past any plausible head-buffer cap
+ definitions := map[string]any{
+ "Padding": map[string]any{
+ "type": "string",
+ "description": strings.Repeat("x", pad),
+ },
+ }
+ doc := map[string]any{
+ "swagger": "2.0",
+ "info": map[string]any{"title": "Big", "version": "1.0.0"},
+ "definitions": definitions,
+ "paths": map[string]any{},
+ }
+ out, err := json.Marshal(doc)
+ if err != nil {
+ panic(err)
+ }
+ return out
+}
+
+func TestIsSwagger2SpecJSON(t *testing.T) {
+ t.Parallel()
+
+ cases := []struct {
+ name string
+ data []byte
+ want bool
+ }{
+ {
+ name: "swagger 2.0 minimal",
+ data: []byte(`{"swagger":"2.0","info":{"title":"Demo","version":"1.0.0"},"paths":{}}`),
+ want: true,
+ },
+ {
+ name: "swagger 2.0 with space",
+ data: []byte(`{"swagger": "2.0","info":{"title":"Demo","version":"1.0.0"},"paths":{}}`),
+ want: true,
+ },
+ {
+ name: "openapi 3.0",
+ data: []byte(`{"openapi":"3.0.3","info":{"title":"Demo","version":"1.0.0"},"paths":{}}`),
+ want: false,
+ },
+ {
+ name: "openapi 3 with swagger mention in description",
+ // A spec that mentions "swagger" in a description should not be misdetected.
+ data: []byte(`{"openapi":"3.0.3","info":{"title":"Demo","description":"swagger inspired","version":"1.0.0"},"paths":{}}`),
+ want: false,
+ },
+ {
+ name: "swagger 2 without minor version",
+ // Bare "2" is not a Swagger version string and must not match.
+ data: []byte(`{"swagger":"2","info":{"title":"Demo","version":"1.0.0"},"paths":{}}`),
+ want: false,
+ },
+ {
+ name: "swagger future minor version",
+ // "2.5" is not Swagger 2.0; prefix-match temptation must be avoided.
+ data: []byte(`{"swagger":"2.5","info":{"title":"Demo","version":"1.0.0"},"paths":{}}`),
+ want: false,
+ },
+ {
+ name: "swagger key after large alphabetized definitions block",
+ // normalizeSpecData round-trips through encoding/json, which sorts
+ // keys alphabetically; "swagger" (s) lands after "definitions" (d)
+ // and "paths" (p). A real Swagger 2.0 spec's "swagger" marker is
+ // often far past the start of the serialized JSON. Build a fixture
+ // where definitions[]'s padding pushes "swagger" well past 4KB to
+ // assert the detector still finds it.
+ data: bigSwagger2Spec(),
+ want: true,
+ },
+ {
+ name: "empty",
+ data: []byte{},
+ want: false,
+ },
+ }
+
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ t.Parallel()
+ assert.Equal(t, tc.want, isSwagger2SpecJSON(tc.data))
+ })
+ }
+}
+
+func TestParseSwagger2WithCircularRefsConverts(t *testing.T) {
+ t.Parallel()
+
+ // Build a small Swagger 2.0 spec with a bi-directional cycle in
+ // definitions/. Pre-fix, this shape combined with kin-openapi's OpenAPI 3
+ // loader caused unbounded resolution time. Post-fix, the conversion to
+ // OpenAPI 3 should let the parser complete quickly.
+ swagger2 := []byte(`{
+ "swagger": "2.0",
+ "info": {"title": "Cycle Demo", "version": "1.0.0"},
+ "host": "api.example.com",
+ "basePath": "/v1",
+ "schemes": ["https"],
+ "paths": {
+ "/employees/{id}": {
+ "get": {
+ "operationId": "getEmployee",
+ "parameters": [
+ {"name": "id", "in": "path", "required": true, "type": "string"}
+ ],
+ "responses": {
+ "200": {
+ "description": "ok",
+ "schema": {"$ref": "#/definitions/Employee"}
+ }
+ }
+ }
+ }
+ },
+ "definitions": {
+ "Employee": {
+ "type": "object",
+ "properties": {
+ "id": {"type": "string"},
+ "name": {"type": "string"},
+ "department": {"$ref": "#/definitions/Department"},
+ "manager": {"$ref": "#/definitions/Employee"}
+ }
+ },
+ "Department": {
+ "type": "object",
+ "properties": {
+ "id": {"type": "string"},
+ "name": {"type": "string"},
+ "head": {"$ref": "#/definitions/Employee"}
+ }
+ }
+ }
+}`)
+
+ done := make(chan struct{})
+ var parsed any
+ var parseErr error
+ go func() {
+ defer close(done)
+ parsed, parseErr = Parse(swagger2)
+ }()
+
+ select {
+ case <-done:
+ case <-time.After(30 * time.Second):
+ // Note: this intentionally abandons the Parse goroutine when the
+ // regression returns. Parse exposes no cancellation hook, so the
+ // alternative is letting the test run for ~25 minutes and OOM the
+ // CI runner before failing. Leaking one goroutine for the rest of
+ // the test binary's lifetime is the lesser evil; the failure will
+ // surface immediately and the run will end shortly after.
+ t.Fatal("parsing cyclic Swagger 2.0 spec did not complete within 30s; regression of issue #1241")
+ }
+
+ require.NoError(t, parseErr)
+ require.NotNil(t, parsed)
+}
+
+func TestParseSwagger2BasicEndpointShape(t *testing.T) {
+ t.Parallel()
+
+ // Make sure the conversion preserves enough endpoint metadata for the
+ // downstream generator. Asserts only the load-bearing shape (resource +
+ // endpoint by method+path); fine-grained field-by-field parity with the
+ // equivalent OpenAPI 3 spec is enforced by the conversion library's own
+ // test suite.
+ swagger2 := []byte(`{
+ "swagger": "2.0",
+ "info": {"title": "Shape Demo", "version": "1.0.0"},
+ "host": "api.example.com",
+ "basePath": "/v1",
+ "schemes": ["https"],
+ "paths": {
+ "/users": {
+ "get": {
+ "operationId": "listUsers",
+ "responses": {"200": {"description": "ok"}}
+ }
+ }
+ }
+}`)
+
+ parsed, err := Parse(swagger2)
+ require.NoError(t, err)
+ require.NotNil(t, parsed)
+ assert.Equal(t, "https://api.example.com/v1", parsed.BaseURL)
+
+ endpoint := findParsedEndpointByPath(t, parsed, "GET", "/users")
+ require.NotNil(t, endpoint)
+}
← 042a78d4 feat(cli): flag empty defaultSyncResources at dogfood time (
·
back to Cli Printing Press
·
fix(cli): emit structured cache_warning JSON on auto-refresh 187b6992 →