← back to Cli Printing Press
feat(cli): wrapper-only catalog entries for reverse-engineered APIs (#177)
096950fd8bca3d8f1f1375dd93670019e29ea3f1 · 2026-04-11 23:53:05 -0700 · Trevin Chow
Some APIs (Google Flights, Kayak) have no official spec but are reachable
through community-maintained reverse-engineered libraries. Add a
wrapper_libraries field to catalog Entry and allow spec_url to be omitted
when that list is populated. Seed with google-flights (krisukox Go wrapper +
fli Python subprocess) and kayak (embedded-HTML extract).
The main skill's Phase 1 catalog check now branches on entry type: spec-based
entries behave as before; wrapper-only entries surface the library options to
the user and skip spec search entirely.
Addresses #168 item 4.
Files touched
A catalog/google-flights.yamlA catalog/kayak.yamlM internal/catalog/catalog.goM internal/catalog/catalog_test.goM internal/cli/catalog.goM internal/cli/catalog_test.goM skills/printing-press/SKILL.md
Diff
commit 096950fd8bca3d8f1f1375dd93670019e29ea3f1
Author: Trevin Chow <trevin@trevinchow.com>
Date: Sat Apr 11 23:53:05 2026 -0700
feat(cli): wrapper-only catalog entries for reverse-engineered APIs (#177)
Some APIs (Google Flights, Kayak) have no official spec but are reachable
through community-maintained reverse-engineered libraries. Add a
wrapper_libraries field to catalog Entry and allow spec_url to be omitted
when that list is populated. Seed with google-flights (krisukox Go wrapper +
fli Python subprocess) and kayak (embedded-HTML extract).
The main skill's Phase 1 catalog check now branches on entry type: spec-based
entries behave as before; wrapper-only entries surface the library options to
the user and skip spec search entirely.
Addresses #168 item 4.
---
catalog/google-flights.yaml | 28 +++++++++++++
catalog/kayak.yaml | 25 +++++++++++
internal/catalog/catalog.go | 81 +++++++++++++++++++++++++++++++-----
internal/catalog/catalog_test.go | 89 ++++++++++++++++++++++++++++++++++++++++
internal/cli/catalog.go | 21 +++++++++-
internal/cli/catalog_test.go | 5 ++-
skills/printing-press/SKILL.md | 19 ++++++++-
7 files changed, 253 insertions(+), 15 deletions(-)
diff --git a/catalog/google-flights.yaml b/catalog/google-flights.yaml
new file mode 100644
index 00000000..6ec45f36
--- /dev/null
+++ b/catalog/google-flights.yaml
@@ -0,0 +1,28 @@
+name: google-flights
+display_name: Google Flights
+description: Flight search via Google Flights. No public API — reached through community-maintained reverse-engineered wrappers.
+category: other
+tier: community
+verified_date: "2026-04-11"
+homepage: https://www.google.com/travel/flights
+notes: |
+ Google does not publish a Flights API. The wrapper libraries below reverse-engineer
+ the internal protobuf protocol that powers the website. Prefer the native Go wrapper
+ for printed CLIs so the resulting binary has no Python runtime dependency. Fall back
+ to the subprocess option when the Go wrapper is missing coverage the user needs.
+
+ When this entry is returned for a combo-CLI source, the skill should surface the
+ wrapper options to the user rather than searching for an OpenAPI spec.
+wrapper_libraries:
+ - name: krisukox/google-flights-api
+ url: https://github.com/krisukox/google-flights-api
+ language: go
+ license: MIT
+ integration_mode: native
+ notes: Pure Go, importable. Preferred default — produces a single-binary CLI with no runtime deps.
+ - name: punitarani/fli
+ url: https://github.com/punitarani/fli
+ language: python
+ license: MIT
+ integration_mode: subprocess
+ notes: Python library with broader feature coverage (multi-leg, cabin class filters). Invoke as a subprocess and parse JSON output. Requires Python 3.10+ at runtime.
diff --git a/catalog/kayak.yaml b/catalog/kayak.yaml
new file mode 100644
index 00000000..f960a764
--- /dev/null
+++ b/catalog/kayak.yaml
@@ -0,0 +1,25 @@
+name: kayak
+display_name: Kayak
+description: Flight, hotel, and car rental aggregator. No public API and no maintained community wrappers — reached by scraping embedded JSON from server-rendered pages.
+category: other
+tier: community
+verified_date: "2026-04-11"
+homepage: https://www.kayak.com
+notes: |
+ Kayak blocks automated API access and has no official wrapper. The reliable path is
+ to fetch the `/direct` and search result pages and extract the embedded JSON payload
+ that the page's own JavaScript consumes to render the results — a single HTTP GET
+ per query, no browser automation required.
+
+ Treat this as html-scrape implementation backing. The generator should emit a thin
+ HTTP client with regex/JSON extraction rather than trying to produce a spec-driven
+ CLI.
+wrapper_libraries:
+ - name: embedded-json-extract
+ url: https://www.kayak.com/direct
+ language: go
+ integration_mode: html-scrape
+ notes: |
+ Fetch the page, locate the inline `window.__INITIAL_STATE__` or equivalent
+ JSON blob, and decode it. Token-cheap compared to headless browser capture,
+ but brittle — re-verify the selector when Kayak redesigns the page.
diff --git a/internal/catalog/catalog.go b/internal/catalog/catalog.go
index ba44da58..debb9cf4 100644
--- a/internal/catalog/catalog.go
+++ b/internal/catalog/catalog.go
@@ -62,6 +62,29 @@ type KnownAlt struct {
Language string `yaml:"language"`
}
+// WrapperLibrary describes a community-maintained library that wraps an API
+// without an official OpenAPI spec. The generator uses these as implementation
+// backing when the primary source is reverse-engineered rather than spec-driven.
+//
+// IntegrationMode values:
+// - native: library is in Go and can be imported directly
+// - subprocess: library is in another language and must be invoked as a subprocess
+// - html-scrape: no library — the entry documents a reverse-engineering technique
+type WrapperLibrary struct {
+ Name string `yaml:"name"`
+ URL string `yaml:"url"`
+ Language string `yaml:"language"`
+ License string `yaml:"license,omitempty"`
+ IntegrationMode string `yaml:"integration_mode"`
+ Notes string `yaml:"notes,omitempty"`
+}
+
+var validIntegrationModes = map[string]struct{}{
+ "native": {},
+ "subprocess": {},
+ "html-scrape": {},
+}
+
type Entry struct {
Name string `yaml:"name"`
DisplayName string `yaml:"display_name"`
@@ -87,6 +110,16 @@ type Entry struct {
// ProxyRoutes maps path prefixes to backend service names for proxy-envelope APIs.
// Only relevant when ClientPattern is "proxy-envelope".
ProxyRoutes map[string]string `yaml:"proxy_routes,omitempty"`
+ // WrapperLibraries lists reverse-engineered community libraries the generator
+ // can use as implementation backing when no official spec exists. When this
+ // list is non-empty, spec_url and spec_format are optional.
+ WrapperLibraries []WrapperLibrary `yaml:"wrapper_libraries,omitempty"`
+}
+
+// IsWrapperOnly reports whether this entry represents an API reached through
+// community wrapper libraries rather than an official spec.
+func (e *Entry) IsWrapperOnly() bool {
+ return e.SpecURL == "" && len(e.WrapperLibraries) > 0
}
func ParseEntry(data []byte) (*Entry, error) {
@@ -169,17 +202,45 @@ func (e *Entry) Validate() error {
if _, ok := validCategories[e.Category]; !ok {
return fmt.Errorf("category must be one of: %s", strings.Join(PublicCategories(), ", "))
}
- if e.SpecURL == "" {
- return fmt.Errorf("spec_url is required")
- }
- if !strings.HasPrefix(e.SpecURL, "https://") {
- return fmt.Errorf(`spec_url must start with "https://"`)
- }
- if e.SpecFormat == "" {
- return fmt.Errorf("spec_format is required")
+ wrapperOnly := len(e.WrapperLibraries) > 0 && e.SpecURL == ""
+ if !wrapperOnly {
+ if e.SpecURL == "" {
+ return fmt.Errorf("spec_url is required (or populate wrapper_libraries for a wrapper-only entry)")
+ }
+ if !strings.HasPrefix(e.SpecURL, "https://") {
+ return fmt.Errorf(`spec_url must start with "https://"`)
+ }
+ if e.SpecFormat == "" {
+ return fmt.Errorf("spec_format is required")
+ }
+ if _, ok := validSpecFormats[e.SpecFormat]; !ok {
+ return fmt.Errorf("spec_format must be one of: yaml, json")
+ }
+ } else if e.SpecFormat != "" {
+ if _, ok := validSpecFormats[e.SpecFormat]; !ok {
+ return fmt.Errorf("spec_format must be one of: yaml, json")
+ }
}
- if _, ok := validSpecFormats[e.SpecFormat]; !ok {
- return fmt.Errorf("spec_format must be one of: yaml, json")
+
+ for i, w := range e.WrapperLibraries {
+ if w.Name == "" {
+ return fmt.Errorf("wrapper_libraries[%d]: name is required", i)
+ }
+ if w.URL == "" {
+ return fmt.Errorf("wrapper_libraries[%d]: url is required", i)
+ }
+ if !strings.HasPrefix(w.URL, "https://") {
+ return fmt.Errorf(`wrapper_libraries[%d]: url must start with "https://"`, i)
+ }
+ if w.Language == "" {
+ return fmt.Errorf("wrapper_libraries[%d]: language is required", i)
+ }
+ if w.IntegrationMode == "" {
+ return fmt.Errorf("wrapper_libraries[%d]: integration_mode is required", i)
+ }
+ if _, ok := validIntegrationModes[w.IntegrationMode]; !ok {
+ return fmt.Errorf("wrapper_libraries[%d]: integration_mode must be one of: native, subprocess, html-scrape", i)
+ }
}
if e.Tier == "" {
return fmt.Errorf("tier is required")
diff --git a/internal/catalog/catalog_test.go b/internal/catalog/catalog_test.go
index e2b2caf3..cfae7a9c 100644
--- a/internal/catalog/catalog_test.go
+++ b/internal/catalog/catalog_test.go
@@ -246,6 +246,95 @@ func TestOptionalFieldsOmittedValid(t *testing.T) {
assert.Empty(t, entry.ClientPattern)
}
+func TestWrapperOnlyEntryValid(t *testing.T) {
+ entry := Entry{
+ Name: "google-flights",
+ DisplayName: "Google Flights",
+ Description: "Flight search via reverse-engineered wrapper libraries",
+ Category: "other",
+ Tier: "community",
+ WrapperLibraries: []WrapperLibrary{
+ {
+ Name: "krisukox/google-flights-api",
+ URL: "https://github.com/krisukox/google-flights-api",
+ Language: "go",
+ License: "MIT",
+ IntegrationMode: "native",
+ },
+ },
+ }
+ assert.NoError(t, entry.Validate())
+ assert.True(t, entry.IsWrapperOnly())
+}
+
+func TestWrapperEntryRequiresIntegrationMode(t *testing.T) {
+ entry := Entry{
+ Name: "test-wrapper",
+ DisplayName: "Test",
+ Description: "Test",
+ Category: "other",
+ Tier: "community",
+ WrapperLibraries: []WrapperLibrary{
+ {Name: "lib", URL: "https://github.com/example/lib", Language: "go"},
+ },
+ }
+ err := entry.Validate()
+ require.Error(t, err)
+ assert.Contains(t, err.Error(), "integration_mode is required")
+}
+
+func TestWrapperEntryRejectsInvalidIntegrationMode(t *testing.T) {
+ entry := Entry{
+ Name: "test-wrapper",
+ DisplayName: "Test",
+ Description: "Test",
+ Category: "other",
+ Tier: "community",
+ WrapperLibraries: []WrapperLibrary{
+ {Name: "lib", URL: "https://github.com/example/lib", Language: "go", IntegrationMode: "ffi"},
+ },
+ }
+ err := entry.Validate()
+ require.Error(t, err)
+ assert.Contains(t, err.Error(), "integration_mode must be one of")
+}
+
+func TestWrapperEntryRequiresHTTPSURL(t *testing.T) {
+ entry := Entry{
+ Name: "test-wrapper",
+ DisplayName: "Test",
+ Description: "Test",
+ Category: "other",
+ Tier: "community",
+ WrapperLibraries: []WrapperLibrary{
+ {Name: "lib", URL: "http://example.com", Language: "go", IntegrationMode: "native"},
+ },
+ }
+ err := entry.Validate()
+ require.Error(t, err)
+ assert.Contains(t, err.Error(), `url must start with "https://"`)
+}
+
+func TestSpecURLRequiredWhenNoWrapperLibraries(t *testing.T) {
+ entry := Entry{
+ Name: "test-api",
+ DisplayName: "Test",
+ Description: "Test",
+ Category: "other",
+ Tier: "community",
+ }
+ err := entry.Validate()
+ require.Error(t, err)
+ assert.Contains(t, err.Error(), "spec_url is required")
+}
+
+func TestEmbeddedCatalogParsesWrapperOnlyEntries(t *testing.T) {
+ entry, err := LookupFS(catalogfs.FS, "google-flights")
+ require.NoError(t, err)
+ assert.True(t, entry.IsWrapperOnly())
+ assert.NotEmpty(t, entry.WrapperLibraries)
+}
+
func TestPublicCategoriesExcludeExample(t *testing.T) {
categories := PublicCategories()
assert.NotContains(t, categories, "example")
diff --git a/internal/cli/catalog.go b/internal/cli/catalog.go
index ded7f735..3d577309 100644
--- a/internal/cli/catalog.go
+++ b/internal/cli/catalog.go
@@ -108,8 +108,12 @@ func newCatalogShowCmd() *cobra.Command {
fmt.Printf("Description: %s\n", entry.Description)
fmt.Printf("Category: %s\n", entry.Category)
fmt.Printf("Tier: %s\n", entry.Tier)
- fmt.Printf("Spec URL: %s\n", entry.SpecURL)
- fmt.Printf("Spec Format: %s\n", entry.SpecFormat)
+ if entry.IsWrapperOnly() {
+ fmt.Printf("Mode: wrapper-only (no official spec)\n")
+ } else {
+ fmt.Printf("Spec URL: %s\n", entry.SpecURL)
+ fmt.Printf("Spec Format: %s\n", entry.SpecFormat)
+ }
if entry.OpenAPIVersion != "" {
fmt.Printf("OpenAPI: %s\n", entry.OpenAPIVersion)
}
@@ -131,6 +135,19 @@ func newCatalogShowCmd() *cobra.Command {
if entry.VerifiedDate != "" {
fmt.Printf("Verified: %s\n", entry.VerifiedDate)
}
+ if len(entry.WrapperLibraries) > 0 {
+ fmt.Printf("\nWrapper Libraries:\n")
+ for _, w := range entry.WrapperLibraries {
+ fmt.Printf(" - %s (%s, %s)\n", w.Name, w.Language, w.IntegrationMode)
+ fmt.Printf(" %s\n", w.URL)
+ if w.License != "" {
+ fmt.Printf(" License: %s\n", w.License)
+ }
+ if w.Notes != "" {
+ fmt.Printf(" Notes: %s\n", strings.TrimSpace(w.Notes))
+ }
+ }
+ }
return nil
},
diff --git a/internal/cli/catalog_test.go b/internal/cli/catalog_test.go
index 8196fc3b..1507606e 100644
--- a/internal/cli/catalog_test.go
+++ b/internal/cli/catalog_test.go
@@ -41,7 +41,10 @@ func TestCatalogListJSON(t *testing.T) {
for _, e := range entries {
assert.NotEmpty(t, e.Name)
- assert.NotEmpty(t, e.SpecURL)
+ // Every entry must have either a spec_url or wrapper_libraries.
+ if e.SpecURL == "" {
+ assert.NotEmpty(t, e.WrapperLibraries, "entry %s has no spec_url and no wrapper_libraries", e.Name)
+ }
}
}
diff --git a/skills/printing-press/SKILL.md b/skills/printing-press/SKILL.md
index 398d741e..5d2e7277 100644
--- a/skills/printing-press/SKILL.md
+++ b/skills/printing-press/SKILL.md
@@ -514,13 +514,28 @@ Before starting research, check if the API has a built-in catalog entry:
printing-press catalog show <api> --json 2>/dev/null
```
-If the catalog has an entry for this API, present the user with a choice:
+If the catalog has an entry for this API, branch on the entry type:
+
+**Spec-based entry** (`spec_url` populated) — present the user with a choice:
> "<API> is in the built-in catalog (spec: <spec_url>). Use the catalog config to skip discovery, or run full discovery?"
- If catalog config: use the spec_url from the catalog entry, skip the research/discovery phase
- If full discovery: proceed with the normal research workflow
-- If the catalog doesn't have this API: proceed normally without mentioning the catalog
+
+**Wrapper-only entry** (no `spec_url`, `wrapper_libraries` populated) — this is a reverse-engineered API that has no official spec but has known community libraries the generator can use as implementation backing. Do not try to resolve or sniff a spec. Instead, surface the wrapper options to the user via `AskUserQuestion`:
+
+> "<API> has no official spec. The catalog knows about these community-maintained implementations:"
+
+Present each `wrapper_libraries` entry as a selectable option with language, integration mode, and notes. Example for `google-flights`:
+- **krisukox/google-flights-api** (Go, native, MIT) — Pure Go, importable; single-binary CLI with no runtime deps.
+- **punitarani/fli** (Python, subprocess, MIT) — broader feature coverage (multi-leg, cabin class); requires Python 3.10+ at runtime.
+
+Capture the user's choice and record it in `$API_RUN_DIR/state.json` under an `implementation` field: `{ "library": "<name>", "url": "<url>", "integration_mode": "native|subprocess|html-scrape" }`. Phase 3 generation reads this to decide whether to `go get` a wrapper, emit a subprocess shell-out, or emit HTML-scrape code. Skip the spec-analysis step entirely — there is no spec.
+
+**No catalog hit** — proceed normally without mentioning the catalog.
+
+**Adding new wrapper-only APIs:** drop a YAML file in `catalog/` with `wrapper_libraries` populated and rebuild the binary. No skill changes needed.
Write one build-driving brief, not a stack of phase essays.
← 23ccc603 fix(skills): add combo-CLI priority gate to prevent source i
·
back to Cli Printing Press
·
fix(cli): address remaining ESPN retro findings — verify hin 9da1cabd →