← back to Cli Printing Press
fix(cli): address retro findings from yahoo-finance run (#174) (#175)
8357850dd103e6bb041e51890e3e73c0240f38e7 · 2026-04-11 23:36:51 -0700 · Trevin Chow
* fix(cli): address retro findings from yahoo-finance run (#174)
Implements all 5 work units from the Printing Press retro for the
yahoo-finance session:
WU-1: Crowd-sniff endpoints inherit the most recent base URL observed in
their source file's scan order. New FilterByHost() drops endpoints whose
origin host doesn't match the user-provided --base-url, eliminating
cross-API contamination when npm packages glue together multiple APIs
(Polygon, FRED, Tiingo, etc. inside a Yahoo Finance wrapper). Falls open
for endpoints with no origin signal.
WU-2: New auth type `session_handshake` covers the crumb/CSRF pattern used
by Yahoo Finance and similarly reverse-engineered APIs. The generator
emits a session.go helper (cookie jar, bootstrap+token fetch, disk
persistence, invalidation on configurable status codes) alongside the
client when this type is set. Schema adds BootstrapURL, SessionTokenURL,
TokenFormat/JSONPath, TokenParamName/In, InvalidateOnStatus, and
SessionTTLHours. Zero hand-patching of client.go required for the next
crumb-protected API we generate.
WU-3: Dogfood's novel-features matcher is now three passes — exact,
prefix (hyphen-boundary), alias — instead of literal-only leaf match.
Flags are stripped from planned commands before leaf extraction.
Research.json schema gains optional `aliases: []string` per feature.
Features renamed during implementation no longer false-negative.
WU-4: Dry-run previews now include the auth material that would be
attached to the live request (Authorization header or query token),
resolved from cache only — no network fetch during --dry-run. Users can
verify imported sessions without firing a real request.
WU-5: Workflows and insight scorers only count command files whose
constructor is actually registered in root.go. Removing dead code no
longer drops the score, and orphan files no longer inflate it. Falls
open when root.go isn't parseable (older CLIs).
Tests: 3 new test files, 15+ targeted cases covering the retro scenarios.
Pre-existing test failures (TestVersionConsistencyAcrossFiles,
TestGenerateFromOpenAPICompiles/stytch) verified on main before these
changes — unrelated to this diff.
Retro: docs/retros/2026-04-11-yahoo-finance-retro.md
Issue: https://github.com/mvanhorn/cli-printing-press/issues/174
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): restore two pre-existing test failures on main
1. Version consistency: marketplace.json's plugins[0] entry was missing
the version field after the recent marketplace decoupling commit.
Added "version": "1.3.2" — release-please config already targets
$.plugins[0].version, so this stays in sync automatically going
forward.
2. goLiteral template helper: a nil default rendered as the literal
string "<nil>" (fmt default), which Go's parser rejects. Now renders
as the keyword `nil`. Fixes stytch fixture (and any future spec
with unset defaults on search body fields) from failing to compile.
Both failures existed on main before the WU-1..5 retro work; fixing them
together unblocks a clean `go test ./...`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): correct marketplace.json version handling
Plugin versions live only in plugin.json. marketplace.json's
$.metadata.version describes the marketplace format itself, not any
individual plugin entry — a per-plugin version field was previously
removed from marketplace.json for exactly this reason.
Reverts the marketplace.json addition from 5e98834 (which papered over
the test failure by re-coupling the files) and instead:
- Drops the $.plugins[0].version entry from release-please-config.json
so release-please stops trying to bump a field we don't keep
- Rewrites TestVersionConsistencyAcrossFiles to only assert plugin.json
↔ version.go consistency, which is the only intended coupling
- Adds TestMarketplaceJSONHasNoPluginVersion to prevent the field
from being re-added by a future well-meaning edit or release tool
misconfiguration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Files touched
A docs/retros/2026-04-11-yahoo-finance-retro.mdM internal/cli/crowd_sniff.goM internal/cli/release_test.goM internal/crowdsniff/aggregate.goA internal/crowdsniff/filter_test.goM internal/crowdsniff/patterns.goM internal/crowdsniff/types.goM internal/generator/generator.goA internal/generator/session_handshake_test.goM internal/generator/templates/client.go.tmplA internal/generator/templates/session_handshake.go.tmplM internal/pipeline/dogfood.goA internal/pipeline/novel_features_matcher_test.goM internal/pipeline/research.goM internal/pipeline/scorecard.goA internal/pipeline/scorecard_registered_test.goM internal/spec/spec.goM release-please-config.jsonA testdata/session-handshake.yaml
Diff
commit 8357850dd103e6bb041e51890e3e73c0240f38e7
Author: Trevin Chow <trevin@trevinchow.com>
Date: Sat Apr 11 23:36:51 2026 -0700
fix(cli): address retro findings from yahoo-finance run (#174) (#175)
* fix(cli): address retro findings from yahoo-finance run (#174)
Implements all 5 work units from the Printing Press retro for the
yahoo-finance session:
WU-1: Crowd-sniff endpoints inherit the most recent base URL observed in
their source file's scan order. New FilterByHost() drops endpoints whose
origin host doesn't match the user-provided --base-url, eliminating
cross-API contamination when npm packages glue together multiple APIs
(Polygon, FRED, Tiingo, etc. inside a Yahoo Finance wrapper). Falls open
for endpoints with no origin signal.
WU-2: New auth type `session_handshake` covers the crumb/CSRF pattern used
by Yahoo Finance and similarly reverse-engineered APIs. The generator
emits a session.go helper (cookie jar, bootstrap+token fetch, disk
persistence, invalidation on configurable status codes) alongside the
client when this type is set. Schema adds BootstrapURL, SessionTokenURL,
TokenFormat/JSONPath, TokenParamName/In, InvalidateOnStatus, and
SessionTTLHours. Zero hand-patching of client.go required for the next
crumb-protected API we generate.
WU-3: Dogfood's novel-features matcher is now three passes — exact,
prefix (hyphen-boundary), alias — instead of literal-only leaf match.
Flags are stripped from planned commands before leaf extraction.
Research.json schema gains optional `aliases: []string` per feature.
Features renamed during implementation no longer false-negative.
WU-4: Dry-run previews now include the auth material that would be
attached to the live request (Authorization header or query token),
resolved from cache only — no network fetch during --dry-run. Users can
verify imported sessions without firing a real request.
WU-5: Workflows and insight scorers only count command files whose
constructor is actually registered in root.go. Removing dead code no
longer drops the score, and orphan files no longer inflate it. Falls
open when root.go isn't parseable (older CLIs).
Tests: 3 new test files, 15+ targeted cases covering the retro scenarios.
Pre-existing test failures (TestVersionConsistencyAcrossFiles,
TestGenerateFromOpenAPICompiles/stytch) verified on main before these
changes — unrelated to this diff.
Retro: docs/retros/2026-04-11-yahoo-finance-retro.md
Issue: https://github.com/mvanhorn/cli-printing-press/issues/174
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): restore two pre-existing test failures on main
1. Version consistency: marketplace.json's plugins[0] entry was missing
the version field after the recent marketplace decoupling commit.
Added "version": "1.3.2" — release-please config already targets
$.plugins[0].version, so this stays in sync automatically going
forward.
2. goLiteral template helper: a nil default rendered as the literal
string "<nil>" (fmt default), which Go's parser rejects. Now renders
as the keyword `nil`. Fixes stytch fixture (and any future spec
with unset defaults on search body fields) from failing to compile.
Both failures existed on main before the WU-1..5 retro work; fixing them
together unblocks a clean `go test ./...`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(cli): correct marketplace.json version handling
Plugin versions live only in plugin.json. marketplace.json's
$.metadata.version describes the marketplace format itself, not any
individual plugin entry — a per-plugin version field was previously
removed from marketplace.json for exactly this reason.
Reverts the marketplace.json addition from 5e98834 (which papered over
the test failure by re-coupling the files) and instead:
- Drops the $.plugins[0].version entry from release-please-config.json
so release-please stops trying to bump a field we don't keep
- Rewrites TestVersionConsistencyAcrossFiles to only assert plugin.json
↔ version.go consistency, which is the only intended coupling
- Adds TestMarketplaceJSONHasNoPluginVersion to prevent the field
from being re-added by a future well-meaning edit or release tool
misconfiguration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---
docs/retros/2026-04-11-yahoo-finance-retro.md | 213 ++++++++++++++
internal/cli/crowd_sniff.go | 32 +++
internal/cli/release_test.go | 36 ++-
internal/crowdsniff/aggregate.go | 70 +++++
internal/crowdsniff/filter_test.go | 128 +++++++++
internal/crowdsniff/patterns.go | 98 ++++++-
internal/crowdsniff/types.go | 22 +-
internal/generator/generator.go | 20 ++
internal/generator/session_handshake_test.go | 136 +++++++++
internal/generator/templates/client.go.tmpl | 89 +++++-
.../generator/templates/session_handshake.go.tmpl | 307 +++++++++++++++++++++
internal/pipeline/dogfood.go | 88 +++++-
internal/pipeline/novel_features_matcher_test.go | 129 +++++++++
internal/pipeline/research.go | 6 +
internal/pipeline/scorecard.go | 81 ++++++
internal/pipeline/scorecard_registered_test.go | 112 ++++++++
internal/spec/spec.go | 18 +-
release-please-config.json | 5 -
testdata/session-handshake.yaml | 37 +++
19 files changed, 1581 insertions(+), 46 deletions(-)
diff --git a/docs/retros/2026-04-11-yahoo-finance-retro.md b/docs/retros/2026-04-11-yahoo-finance-retro.md
new file mode 100644
index 00000000..d0758603
--- /dev/null
+++ b/docs/retros/2026-04-11-yahoo-finance-retro.md
@@ -0,0 +1,213 @@
+# Printing Press Retro: yahoo-finance
+
+## Session Stats
+- API: yahoo-finance (undocumented, reverse-engineered endpoints on `query1/query2.finance.yahoo.com`)
+- Spec source: hand-authored internal YAML (crowd-sniff output was too contaminated; sniff via browser not possible — IP 429-blocked)
+- Scorecard: 87/100 (Grade A)
+- Verify pass rate: 100% (mock), 84% before polish
+- Fix loops: 2 (generator issue with dry-run + auth handoff; file-rename needed for dogfood heuristic)
+- Manual code edits: substantial — added ~300 lines of crumb+cookie session handling to client.go, plus 700 lines of transcendence commands (portfolio, watchlist, digest, compare, sparkline, sql, fx, options-chain, auth login-chrome)
+- Features built from scratch: 9 transcendence commands (expected — these are P2 per the skill contract)
+
+## Findings
+
+### F1. Crowd-sniff doesn't filter endpoints by domain (Bug / Template gap)
+
+- **What happened:** Ran `printing-press crowd-sniff --api yahoo-finance --base-url https://query1.finance.yahoo.com`. Tool produced 27 endpoints across 18 "resources" — but most were from other financial APIs. `/fred/series/observations` (FRED), `/v3/reference/tickers` (Polygon), `/tiingo/daily/{id}` (Tiingo), `/v2/stocks/trades/latest` (Alpaca), `/query` (Alpha Vantage), `/api/v1/company-news` (Finnhub) all ended up in the spec. The crowd picks up endpoints from npm packages like `yfin-wrapper` that aggregate multiple financial APIs — and nothing scopes the extraction to the stated `--base-url`.
+- **Scorer correct?** N/A (not a scoring issue; it's a generation-input-quality issue)
+- **Root cause:** `internal/crowdsniff/specgen.go` aggregates endpoints from package source code without checking whether each endpoint's origin URL (as hardcoded in the wrapper source) matches the `baseURL` the user passed. The npm parser extracts every string that looks like an API path regardless of which base URL the wrapper code actually calls it on.
+- **Cross-API check:** Would recur on any API whose community SDKs also touch other APIs. Examples: every financial aggregator (Alpha Vantage wrappers often also call Polygon/FRED), weather aggregators (Open-Meteo wrappers often also call NWS/OpenWeather), social aggregators. Rough frequency: whenever the community ecosystem is "glue code across multiple providers," contamination is severe. Tight ecosystems (Stripe, Twilio) are clean.
+- **Frequency:** subclass:`aggregator-adjacent-apis` — at least 20% of APIs we'd build CLIs for.
+- **Fallback if Printing Press doesn't fix:** Claude rejects the spec manually and hand-writes one (what I did). Adds 15-30 minutes and depends on Claude noticing the contamination. Easy to miss if crowd-sniff output looks plausible.
+- **Worth a Printing Press fix?** Yes.
+- **Inherent or fixable:** Fixable. The npm extraction already sees the HTTP client call site; it can read the base URL from the same function scope and drop mismatches.
+- **Durable fix:** In the npm endpoint extractor, when an HTTP call is detected, also resolve the base URL (look for the closest enclosing `const BASE = '...'` or `baseURL` variable, or the argument to the HTTP client constructor). Compare to the user's `--base-url` host. If domains don't match, emit the endpoint at `tier: community-sdk-other-origin` so the builder can skip it by default (or accept it with `--include-other-origins`).
+- **Test:**
+ - positive: crowd-sniff `--api yahoo-finance --base-url query1.finance.yahoo.com` against current npm ecosystem produces no `/fred/...` or `/v3/reference/tickers` paths
+ - negative: crowd-sniff `--api finnhub --base-url finnhub.io` still picks up `/api/v1/company-news`
+- **Evidence:** `research/yahoo-finance-crowd-spec.yaml` shows 11 of 27 endpoints clearly belong to other APIs. Kept for audit.
+
+### F2. No `session_handshake` auth type for APIs that require a pre-request crumb/CSRF fetch (Template gap)
+
+- **What happened:** Yahoo Finance requires: `GET fc.yahoo.com/` → persist A1/B1 cookies → `GET query2.finance.yahoo.com/v1/test/getcrumb` → pass `?crumb=<value>` on every subsequent request. The generator's supported auth types (`api_key`, `oauth2`, `bearer_token`, `cookie`, `composed`, `none`) don't model this two-step handshake with a token expiry and automatic refresh. I had to hand-patch `internal/client/client.go` with ~250 lines: cookie jar, session.json persistence, `ensureCrumb()`, crumb invalidation on 401/403, and query-param injection.
+- **Scorer correct?** N/A.
+- **Root cause:** `internal/spec/spec.go` `AuthConfig` doesn't have fields to describe "bootstrap URL", "token endpoint URL", "token parameter name/location", or "invalidate on status codes".
+- **Cross-API check:** This isn't just Yahoo. The same pattern is used by:
+ - Sites with CSRF-protected JSON APIs (Facebook, many e-commerce backends)
+ - Many unofficial/reverse-engineered APIs on consumer sites (Walmart, Best Buy, Whole Foods)
+ - Some streaming services (Spotify token-from-page, Netflix, Twitch)
+- **Frequency:** subclass:`csrf-token-pattern` — probably 10-15% of APIs we'd reverse-engineer via sniff.
+- **Fallback if Printing Press doesn't fix:** Claude hand-writes the handshake every time. ~250 lines of boilerplate, easy to get wrong (cookie persistence, invalidation rules, thread-safety).
+- **Worth a Printing Press fix?** Yes. Hand-writing this is the single biggest chunk of work-that-shouldn't-have-been-necessary in this run.
+- **Inherent or fixable:** Fixable with a new auth type.
+- **Durable fix:** Add `session_handshake` auth type with these fields:
+ ```yaml
+ auth:
+ type: session_handshake
+ bootstrap_url: "https://fc.yahoo.com/" # optional — GET to seed cookies
+ token_url: "https://query2.finance.yahoo.com/v1/test/getcrumb"
+ token_format: text # or json
+ token_json_path: "" # if json, JSONPath to extract
+ token_param_name: crumb # query param name
+ token_param_in: query # query or header
+ invalidate_on_status: [401, 403] # codes that trigger re-handshake
+ ttl_hours: 24 # persist session this long
+ ```
+ The generator's client template then emits the cookie jar, session.json persistence under `~/.config/<cli>/session.json`, an `ensureToken()` method, and invalidation retry logic. Zero hand-patching.
+- **Test:**
+ - positive: generate with `type: session_handshake` and observe the client has cookie jar + ensureToken() + crumb query-param injection; dry-run output shows `?crumb=<token>`
+ - negative: generate with `type: bearer_token` and confirm no session.json persistence logic appears
+- **Evidence:** Hand-patched `client.go` diff in the working dir; this run's working dir has the pattern.
+
+### F3. Dogfood "novel features built" detection is brittle (Scorer bug)
+
+- **What happened:** Dogfood compared `novel_features` from `research.json` against actual CLI commands. It matched exact command paths. My `auth login-chrome` wasn't matched against planned `auth login --chrome`; my built `portfolio gains` wasn't matched against planned `portfolio dividends` (different feature but thematically related); my `options-chain --moneyness otm` wasn't matched against planned `options --moneyness otm`. Result: reported "2/8 survived" even though 7+ were built, just renamed during implementation.
+- **Scorer correct?** No — the CLI genuinely has these features (verified by dry-run: 19/19 commands work). The scorer's literal string match is too narrow.
+- **Root cause:** Dogfood matches `planned.command` (from research.json) to actual `<cli> help <command>` output verbatim. No fuzzy matching, no aliasing, no feature-intent lookup.
+- **Cross-API check:** This recurs on every CLI where implementation naturally diverges from planning — which is nearly every run. Rename `foo --bar` to `foo-bar` during implementation and the scorer reports the feature as missing.
+- **Frequency:** most APIs — the planning/implementation naming drift is universal.
+- **Fallback if Printing Press doesn't fix:** Claude notices the false-positive and overrides the scorer's verdict manually (what I did this run with ship-with-gaps).
+- **Worth a Printing Press fix?** Yes. This is a recurring credibility gap in the scorer.
+- **Inherent or fixable:** Fixable.
+- **Durable fix:** Two complementary changes:
+ 1. `research.json` gains an optional `aliases: []string` field per novel feature. Planner or implementer can record alternative command paths.
+ 2. Dogfood's matcher does three passes: (a) exact path match, (b) prefix match (`auth login-chrome` matches planned `auth login`), (c) alias match. Only after all three fail, report missing.
+- **Test:**
+ - positive: plan has `portfolio perf` alias `[portfolio performance, portfolio pnl]`; built command is `portfolio performance`; scorer reports built.
+ - negative: plan has `portfolio dividends` with no aliases; built `portfolio gains` ≠ `portfolio dividends`; scorer correctly reports missing.
+- **Evidence:** Shipcheck output: `Novel Features: 2/8 survived (WARN)` with 6 items that were in fact built, just named differently.
+
+### F4. Scorer penalizes dead-code removal (Scorer bug)
+
+- **What happened:** Deleted `internal/cli/search_query.go` (genuinely dead — duplicate of `search` command). Scorecard workflows dimension dropped from 10/10 to 8/10 and total from 88 to 87.
+- **Scorer correct?** No. Removing dead code is good. The scorer counts candidate files without checking whether those files register actual commands.
+- **Root cause:** The workflows scorer in `internal/pipeline/scorecard.go` (or wherever) counts files matching a pattern (probably `internal/cli/*.go`) rather than commands actually registered in `root.go`.
+- **Cross-API check:** Recurs every time polish removes dead files or when the generator's dead-code elimination fires.
+- **Frequency:** most APIs with polish pass.
+- **Fallback if Printing Press doesn't fix:** Claude ignores the -2 score delta (as I did). No real harm but signals wrong incentive (don't clean up dead code).
+- **Worth a Printing Press fix?** Yes, but low priority. Score signal matters.
+- **Inherent or fixable:** Fixable.
+- **Durable fix:** Workflows scorer should count commands registered via `rootCmd.AddCommand(new...Cmd(...))` calls, not files. Parse `root.go`, build a set of command constructor names, count those.
+- **Test:**
+ - positive: delete a command file whose constructor IS registered → workflows score drops
+ - negative: delete a dead file whose constructor is NOT in root.go → workflows score unchanged
+- **Evidence:** Polish agent output: "Scorecard Workflows dropped 10→8 solely because we deleted a dead file (search_query.go) — the scorer counts workflow-candidate files, not actual commands."
+
+### F5. Dry-run short-circuits before auth/session preparation (Bug)
+
+- **What happened:** The generated `client.do()` has this structure: early-return if `DryRun`, THEN run auth setup, THEN make request. My hand-added crumb logic initially sat in the "after dry-run" branch, so dry-run output didn't show the crumb query param. A user running `--dry-run` to verify their imported session would see no evidence of the crumb. Fixed locally by moving crumb resolution before the dry-run branch (and skipping the network fetch in dry-run mode if no cached crumb).
+- **Scorer correct?** N/A.
+- **Root cause:** The client template (`internal/generator/templates/client.go.tmpl`) emits `if c.DryRun { return c.dryRun(...) }` as the first branch in `do()`, before the auth header is computed. For bearer-token and API-key auth this happens to be fine because `authHeader()` runs later and is non-network. For session-handshake or oauth2 (where auth resolution could require a network call), this matters — and dry-run should at least show what auth material WOULD be attached.
+- **Cross-API check:** Relevant for any auth type that materially changes the outgoing request shape. bearer_token / api_key — silent ok. oauth2 — currently silent, should ideally show "Authorization: Bearer <cached|refresh-required>". session_handshake (once added) — must show the crumb.
+- **Frequency:** every API where dry-run is used to verify auth wiring, which should be every API (it's a documented feature of the CLI).
+- **Fallback if Printing Press doesn't fix:** Claude patches the generated client (as I did).
+- **Worth a Printing Press fix?** Yes.
+- **Inherent or fixable:** Fixable.
+- **Durable fix:** Restructure `client.do()` template to:
+ 1. Compute auth material (header or query token) up-front using only cached values (never trigger a network fetch during dry-run)
+ 2. If dry-run, pass the computed auth material into `dryRun()` so it surfaces in the preview output
+ 3. Only the non-dry-run branch triggers `ensureToken()` etc.
+- **Test:**
+ - positive: `cli auth login-chrome --crumb abc` + `cli quote AAPL --dry-run` shows `?crumb=abc` in stderr preview
+ - negative: clean session, `cli quote AAPL --dry-run` shows no crumb (doesn't trigger network to fetch one) and notes "no cached session; live run will bootstrap"
+- **Evidence:** Had to hand-patch `dryRunWithCrumb()` and reorder the branch in client.go this run.
+
+## Prioritized Improvements
+
+### P1 — High priority
+
+| Finding | Title | Component | Frequency | Fallback Reliability | Complexity | Guards |
+|---------|-------|-----------|-----------|---------------------|------------|--------|
+| F1 | Crowd-sniff domain filtering | `internal/crowdsniff/` | 20% of APIs (aggregator-adjacent) | Low — contamination is subtle and easy to accept if Claude isn't paying attention | medium | Only apply when `--base-url` is provided; keep permissive mode for "omnibus CLI" intent |
+| F2 | `session_handshake` auth type | `internal/spec/spec.go`, `internal/generator/templates/client.go.tmpl` | 10-15% of reverse-engineered APIs | Low — ~250 lines of correct boilerplate is a lot to ask Claude to repeat | large | Only activates when `auth.type: session_handshake`; existing types unchanged |
+| F3 | Dogfood novel-features fuzzy matching | `internal/pipeline/dogfood.go` + research.json schema | Most APIs | Medium — false negatives are annoying but not blocking | medium | None — broadly safe change |
+
+### P2 — Medium priority
+
+| Finding | Title | Component | Frequency | Fallback Reliability | Complexity | Guards |
+|---------|-------|-----------|-----------|---------------------|------------|--------|
+| F5 | Dry-run shows auth material | `internal/generator/templates/client.go.tmpl` | Every API | High when bug matters (session auth), low otherwise | small | Only the dryRun() signature changes; no-op for bearer/api_key |
+
+### P3 — Low priority
+
+| Finding | Title | Component | Frequency | Fallback Reliability | Complexity | Guards |
+|---------|-------|-----------|-----------|---------------------|------------|--------|
+| F4 | Workflows scorer counts commands not files | `internal/pipeline/scorecard.go` | Most APIs with polish pass | High — small score delta, no behavior impact | small | None |
+
+### Skip
+
+None — all findings apply beyond this one API.
+
+## Work Units
+
+### WU-1: Crowd-sniff domain filtering (F1)
+- **Goal:** Crowd-sniff's npm endpoint extractor drops endpoints whose apparent origin host doesn't match the `--base-url` the user supplied.
+- **Target:** `internal/crowdsniff/npm.go` (and/or `specgen.go`) — wherever an HTTP-call-like expression is recognized and turned into an endpoint record.
+- **Acceptance criteria:**
+ - positive test: a fake npm package that declares `const BASE = 'https://other-api.com'` and calls `fetch(BASE + '/foo')` does NOT appear in output when user passed `--base-url https://query1.finance.yahoo.com`
+ - negative test: a fake npm package that calls `fetch('https://query1.finance.yahoo.com/v8/finance/chart/AAPL')` DOES appear when `--base-url` matches
+ - integration test: rerun against a yahoo-finance-like fixture; previously-contaminated endpoints (`/fred/*`, `/v3/reference/*`, `/tiingo/*`) no longer present
+- **Scope boundary:** Only endpoint-source filtering. Don't change the GitHub code-search extractor in this WU. Don't change the spec schema.
+- **Dependencies:** None
+- **Complexity:** medium
+
+### WU-2: Add `session_handshake` auth type (F2)
+- **Goal:** Spec authors can declare a session-handshake auth type; the generator emits a client with cookie jar, token bootstrap, disk persistence, invalidation, and query/header token injection — no hand-patching required.
+- **Target:**
+ - Schema: `internal/spec/spec.go` `AuthConfig` (new fields)
+ - Template: `internal/generator/templates/client.go.tmpl` (new conditional branch)
+ - Template: `internal/generator/templates/config.go.tmpl` (session file path)
+ - Generator: `internal/generator/` (wire the new branch)
+- **Acceptance criteria:**
+ - positive test: spec with `auth.type: session_handshake, token_url: ..., token_param_name: crumb, ttl_hours: 24` generates a client that (a) has a cookie jar, (b) persists `~/.config/<cli>/session.json`, (c) fetches the token before first data call, (d) adds `?crumb=<token>` on every data request, (e) invalidates on 401/403 and retries once
+ - negative test: spec with `auth.type: bearer_token` generates the same client code that exists today (no regression)
+ - positive test: generated CLI also includes `auth login-<method>` subcommand for importing a pre-obtained session (Chrome-cookie style)
+- **Scope boundary:** Don't replace `cookie` or `composed` auth types. Don't implement automatic browser cookie harvesting — user brings a JSON file, like the yahoo-finance implementation.
+- **Dependencies:** None, but benefits from F5 (dry-run auth preview)
+- **Complexity:** large
+
+### WU-3: Dogfood novel-features fuzzy matching (F3)
+- **Goal:** Novel features survive when implementation naturally renames or prefixes commands.
+- **Target:** `internal/pipeline/dogfood.go` novel-features matcher; optionally extend research.json schema with an `aliases` field.
+- **Acceptance criteria:**
+ - positive test: planned `auth login --chrome`, built `auth login-chrome` → scorer reports built (prefix match)
+ - positive test: planned `portfolio perf` with `aliases: [portfolio performance]`, built `portfolio performance` → scorer reports built (alias match)
+ - negative test: planned `portfolio dividends`, built `portfolio gains` (unrelated feature) → scorer reports missing (correct)
+- **Scope boundary:** Don't add semantic similarity or LLM matching. Keep the match deterministic: exact → prefix → alias → miss.
+- **Dependencies:** None
+- **Complexity:** medium
+
+### WU-4: Dry-run shows auth material (F5)
+- **Goal:** `--dry-run` output includes the auth material that would be attached to the request (header or query param), using cached values only — never triggers a network call to fetch a fresh token.
+- **Target:** `internal/generator/templates/client.go.tmpl` — restructure `do()` so auth is computed up-front and passed to both the dry-run and live branches.
+- **Acceptance criteria:**
+ - positive test: bearer_token client with `--dry-run` prints `Authorization: Bearer ****<last4>` in stderr preview
+ - positive test: session_handshake client with cached crumb and `--dry-run` prints the crumb query param
+ - negative test: session_handshake client with NO cached crumb and `--dry-run` prints `(no cached session; live run will bootstrap)` and does NOT make a network call to fetch one
+- **Scope boundary:** Don't change the dry-run output format — just populate it more fully.
+- **Dependencies:** Benefits WU-2 but not strictly required by it.
+- **Complexity:** small
+
+### WU-5: Workflows scorer counts commands not files (F4)
+- **Goal:** Scorer's workflows dimension reflects actually-registered commands, not file count.
+- **Target:** `internal/pipeline/scorecard.go` — specifically the workflows dimension calculation.
+- **Acceptance criteria:**
+ - positive test: delete a file `internal/cli/foo.go` whose `newFooCmd` IS registered in root.go → score drops
+ - negative test: delete a file `internal/cli/dead.go` whose constructor is NOT registered → score unchanged
+ - negative test: orphan constructor (file exists, never added to root) → not counted
+- **Scope boundary:** Only the workflows dimension. Don't touch other dimensions.
+- **Dependencies:** None
+- **Complexity:** small
+
+## Anti-patterns
+- **Rubber-stamping crowd-sniff output.** Produced 27 endpoints including Polygon, FRED, Tiingo, Alpaca, Alpha Vantage, and Finnhub paths. Always inspect and domain-filter manually until F1 lands.
+- **Writing session-handshake auth by hand every time.** We've now done this enough (Redfin, Zillow-style, and now Yahoo Finance) that it's clearly a missing primitive, not a one-off.
+- **Renaming commands during Phase 3 without updating research.json aliases.** The scorer penalizes you for this even though the feature is built.
+
+## What the Printing Press Got Right
+- **The crumb-and-cookie hand-patch *worked* on the first try when properly restructured.** The existing client template is clean enough that threading a new auth type through is mechanical.
+- **Adaptive rate limiter.** Out-of-the-box handling of 429s was already excellent. My hand-additions to crumb/cookie interact cleanly with it.
+- **Verify mock server handled 9 varied endpoint shapes and gave 100% pass rate.** Strong signal that the generated code paths are correct.
+- **Slug-keyed library layout** (`library/yahoo-finance/` with binary `yahoo-finance-pp-cli`) avoided the `-pp-cli` naming confusion when mixing binaries and directories.
+- **Lock/promote ergonomics.** `printing-press lock promote --cli X --dir Y` did the right thing in one shot — stage, swap, manifest, release. Very clean.
+- **Polish agent autonomy.** Polish worker ran without intervention, fixed 9 example gaps + 4 dry-run bugs + 1 dead file + renamed 10 files, returned delta structured.
diff --git a/internal/cli/crowd_sniff.go b/internal/cli/crowd_sniff.go
index 80e54146..9e6beaee 100644
--- a/internal/cli/crowd_sniff.go
+++ b/internal/cli/crowd_sniff.go
@@ -117,6 +117,27 @@ func runCrowdSniff(ctx context.Context, apiName, baseURL, outputPath string, asJ
return fmt.Errorf("base URL must use HTTPS: %s", resolvedBaseURL)
}
+ // Filter out endpoints whose origin doesn't match the resolved base URL.
+ // Community wrappers that glue together multiple APIs (common for financial,
+ // weather, and aggregator domains) would otherwise contaminate the spec
+ // with endpoints from Polygon, FRED, Tiingo, etc. when targeting Yahoo
+ // Finance — observed in retro #174.
+ targetHost := hostFromBaseURL(resolvedBaseURL)
+ if targetHost != "" {
+ kept, dropped := crowdsniff.FilterByHost(aggregated, targetHost)
+ if len(dropped) > 0 {
+ fmt.Fprintf(stderr, "filtered %d endpoint(s) from other origins (target host: %s)\n", len(dropped), targetHost)
+ for _, d := range dropped {
+ fmt.Fprintf(stderr, " dropped: %s %s (origin(s): %s)\n", d.Method, d.Path, strings.Join(d.OriginBaseURLs, ", "))
+ }
+ }
+ aggregated = kept
+ }
+
+ if len(aggregated) == 0 {
+ return fmt.Errorf("no endpoints remained after host filtering for %q (target: %s)", apiName, targetHost)
+ }
+
apiSpec, err := crowdsniff.BuildSpec(apiName, resolvedBaseURL, aggregated, mergedAuth)
if err != nil {
return fmt.Errorf("building spec: %w", err)
@@ -203,3 +224,14 @@ func isHTTPS(rawURL string) bool {
}
return strings.EqualFold(parsed.Scheme, "https")
}
+
+// hostFromBaseURL returns the lowercase host of a URL (no scheme, no port).
+// Used by host-based endpoint filtering.
+func hostFromBaseURL(rawURL string) string {
+ parsed, err := url.Parse(rawURL)
+ if err != nil {
+ return ""
+ }
+ host := parsed.Hostname()
+ return strings.ToLower(host)
+}
diff --git a/internal/cli/release_test.go b/internal/cli/release_test.go
index 03f31165..2e48c88a 100644
--- a/internal/cli/release_test.go
+++ b/internal/cli/release_test.go
@@ -51,10 +51,17 @@ func TestReleasePleaseAnnotationExists(t *testing.T) {
}
func TestVersionConsistencyAcrossFiles(t *testing.T) {
- // All version surfaces should match. release-please keeps them
- // in sync, but this catches manual edits that drift.
+ // The plugin's version lives in exactly two places:
+ // - .claude-plugin/plugin.json ($.version)
+ // - internal/version/version.go (Version const, ldflags target)
+ // release-please keeps both in lockstep; this test catches manual drift.
+ //
+ // marketplace.json intentionally does NOT carry a per-plugin version —
+ // its $.metadata.version describes the marketplace format itself, not
+ // any individual plugin entry. If either of those separate versions
+ // ever needs to be asserted, add its own test; do not re-couple them
+ // here.
- // Read plugin.json version
pluginData, err := os.ReadFile("../../.claude-plugin/plugin.json")
require.NoError(t, err)
@@ -63,21 +70,26 @@ func TestVersionConsistencyAcrossFiles(t *testing.T) {
}
require.NoError(t, json.Unmarshal(pluginData, &plugin))
- // Read marketplace.json version
+ assert.Equal(t, plugin.Version, version.Version,
+ "plugin.json and version.go hardcoded version must match")
+}
+
+func TestMarketplaceJSONHasNoPluginVersion(t *testing.T) {
+ // Guard against a reviewer (or release-please misconfiguration) re-adding
+ // a per-plugin version field to marketplace.json. Plugin versions live
+ // only in plugin.json; this file catalogs plugins, not their versions.
marketData, err := os.ReadFile("../../.claude-plugin/marketplace.json")
require.NoError(t, err)
var market struct {
- Plugins []struct {
- Version string `json:"version"`
- } `json:"plugins"`
+ Plugins []map[string]any `json:"plugins"`
}
require.NoError(t, json.Unmarshal(marketData, &market))
require.NotEmpty(t, market.Plugins)
- // All three should match
- assert.Equal(t, plugin.Version, market.Plugins[0].Version,
- "plugin.json and marketplace.json versions must match")
- assert.Equal(t, plugin.Version, version.Version,
- "plugin.json and version.go hardcoded version must match")
+ for i, p := range market.Plugins {
+ if _, has := p["version"]; has {
+ t.Errorf("marketplace.json plugins[%d] should not declare a version (belongs in plugin.json only)", i)
+ }
+ }
}
diff --git a/internal/crowdsniff/aggregate.go b/internal/crowdsniff/aggregate.go
index 8d90acf1..313111e0 100644
--- a/internal/crowdsniff/aggregate.go
+++ b/internal/crowdsniff/aggregate.go
@@ -48,6 +48,7 @@ func Aggregate(results []SourceResult) ([]AggregatedEndpoint, []string) {
bestTier string
sources map[string]struct{} // distinct source names
params map[string]paramEntry
+ origins map[string]struct{} // distinct origin base URLs contributed by sources
}
index := make(map[endpointKey]*accumulator)
@@ -67,6 +68,7 @@ func Aggregate(results []SourceResult) ([]AggregatedEndpoint, []string) {
acc = &accumulator{
sources: make(map[string]struct{}),
params: make(map[string]paramEntry),
+ origins: make(map[string]struct{}),
}
index[key] = acc
order = append(order, key)
@@ -76,6 +78,9 @@ func Aggregate(results []SourceResult) ([]AggregatedEndpoint, []string) {
acc.bestTier = ep.SourceTier
}
acc.sources[ep.SourceName] = struct{}{}
+ if ep.OriginBaseURL != "" {
+ acc.origins[ep.OriginBaseURL] = struct{}{}
+ }
// Union-merge params: prefer metadata from higher-tier source.
for _, p := range ep.Params {
@@ -103,12 +108,77 @@ func Aggregate(results []SourceResult) ([]AggregatedEndpoint, []string) {
if len(acc.params) > 0 {
ep.Params = sortedParams(acc.params)
}
+ if len(acc.origins) > 0 {
+ ep.OriginBaseURLs = make([]string, 0, len(acc.origins))
+ for o := range acc.origins {
+ ep.OriginBaseURLs = append(ep.OriginBaseURLs, o)
+ }
+ sort.Strings(ep.OriginBaseURLs)
+ }
aggregated = append(aggregated, ep)
}
return aggregated, deduplicateStrings(baseURLs)
}
+// FilterByHost drops aggregated endpoints whose origin base URL hosts don't
+// match the target host. Endpoints with no known origin (nothing detected in
+// source file context) are kept — we fall open when signal is absent rather
+// than silently losing candidate endpoints.
+//
+// Returns (kept, dropped) so callers can report how much noise was filtered
+// and let users inspect the drops if needed.
+func FilterByHost(endpoints []AggregatedEndpoint, targetHost string) (kept []AggregatedEndpoint, dropped []AggregatedEndpoint) {
+ targetHost = strings.ToLower(strings.TrimSpace(targetHost))
+ if targetHost == "" {
+ return endpoints, nil
+ }
+ for _, ep := range endpoints {
+ if len(ep.OriginBaseURLs) == 0 {
+ // No origin signal — keep (permissive default).
+ kept = append(kept, ep)
+ continue
+ }
+ matched := false
+ for _, origin := range ep.OriginBaseURLs {
+ if hostFromURL(origin) == targetHost {
+ matched = true
+ break
+ }
+ }
+ if matched {
+ kept = append(kept, ep)
+ } else {
+ dropped = append(dropped, ep)
+ }
+ }
+ return kept, dropped
+}
+
+// hostFromURL extracts the lowercased host from a URL, stripping scheme,
+// port, and path. Returns empty string on parse failure.
+func hostFromURL(rawURL string) string {
+ s := strings.TrimSpace(rawURL)
+ if s == "" {
+ return ""
+ }
+ // Strip scheme
+ if idx := strings.Index(s, "://"); idx >= 0 {
+ s = s[idx+3:]
+ }
+ // Strip path / query / fragment
+ for _, sep := range []byte{'/', '?', '#'} {
+ if i := strings.IndexByte(s, sep); i >= 0 {
+ s = s[:i]
+ }
+ }
+ // Strip port
+ if i := strings.IndexByte(s, ':'); i >= 0 {
+ s = s[:i]
+ }
+ return strings.ToLower(s)
+}
+
// NormalizePath unifies parameter syntax and replaces concrete values with placeholders.
// Step 1: Convert :id, {user_id}, <id>, $id → {id}
// Step 2: Replace UUIDs, numeric IDs, long hashes → {id}/{uuid}/{hash}
diff --git a/internal/crowdsniff/filter_test.go b/internal/crowdsniff/filter_test.go
new file mode 100644
index 00000000..d4534570
--- /dev/null
+++ b/internal/crowdsniff/filter_test.go
@@ -0,0 +1,128 @@
+package crowdsniff
+
+import (
+ "testing"
+)
+
+// TestGrepEndpoints_StampsOriginFromBaseURL verifies that endpoints inherit
+// the most recent base URL observed in the source file scan order.
+func TestGrepEndpoints_StampsOriginFromBaseURL(t *testing.T) {
+ src := `
+const baseUrl = "https://query1.finance.yahoo.com";
+client.get("/v8/finance/chart/AAPL");
+`
+ eps, _ := GrepEndpoints(src, "yahoo-test", TierCommunitySDK)
+ if len(eps) == 0 {
+ t.Fatalf("expected at least one endpoint, got none")
+ }
+ if eps[0].OriginBaseURL != "https://query1.finance.yahoo.com" {
+ t.Errorf("expected OriginBaseURL to be stamped, got %q", eps[0].OriginBaseURL)
+ }
+}
+
+// TestGrepEndpoints_MultipleOriginsInOneFile verifies that when a file
+// redeclares its base URL (common in aggregator wrappers), subsequent
+// endpoints inherit the new origin — not the old one. Uses only recognized
+// call shapes (client.get/axios.get) since the extractors don't match
+// arbitrary identifiers like `yahoo.get`.
+func TestGrepEndpoints_MultipleOriginsInOneFile(t *testing.T) {
+ src := `
+const baseUrl = "https://query1.finance.yahoo.com";
+client.get("/v8/finance/chart/AAPL");
+const apiBase = "https://api.polygon.io";
+axios.get("/v3/reference/tickers");
+const baseUri = "https://api.stlouisfed.org";
+client.get("/fred/series/observations");
+`
+ eps, _ := GrepEndpoints(src, "aggregator", TierCommunitySDK)
+ if len(eps) < 3 {
+ t.Fatalf("expected >=3 endpoints extracted, got %d", len(eps))
+ }
+
+ byPath := make(map[string]string)
+ for _, e := range eps {
+ byPath[e.Path] = e.OriginBaseURL
+ }
+
+ cases := map[string]string{
+ "/v8/finance/chart/AAPL": "https://query1.finance.yahoo.com",
+ "/v3/reference/tickers": "https://api.polygon.io",
+ "/fred/series/observations": "https://api.stlouisfed.org",
+ }
+ for path, wantOrigin := range cases {
+ if got, found := byPath[path]; !found {
+ t.Errorf("endpoint %s not extracted", path)
+ } else if got != wantOrigin {
+ t.Errorf("endpoint %s: expected origin %q, got %q", path, wantOrigin, got)
+ }
+ }
+}
+
+// TestFilterByHost_DropsCrossAPIContamination is the retro F1 acceptance test —
+// reproduces the exact yahoo-finance contamination scenario and verifies
+// FilterByHost cleans it up.
+func TestFilterByHost_DropsCrossAPIContamination(t *testing.T) {
+ endpoints := []AggregatedEndpoint{
+ {Method: "GET", Path: "/v8/finance/chart/{id}", OriginBaseURLs: []string{"https://query1.finance.yahoo.com"}},
+ {Method: "GET", Path: "/v7/finance/quote", OriginBaseURLs: []string{"https://query1.finance.yahoo.com"}},
+ {Method: "GET", Path: "/v3/reference/tickers", OriginBaseURLs: []string{"https://api.polygon.io"}},
+ {Method: "GET", Path: "/fred/series/observations", OriginBaseURLs: []string{"https://api.stlouisfed.org"}},
+ {Method: "GET", Path: "/tiingo/daily/{id}", OriginBaseURLs: []string{"https://api.tiingo.com"}},
+ }
+
+ kept, dropped := FilterByHost(endpoints, "query1.finance.yahoo.com")
+
+ if len(kept) != 2 {
+ t.Errorf("expected 2 kept (Yahoo endpoints), got %d", len(kept))
+ }
+ if len(dropped) != 3 {
+ t.Errorf("expected 3 dropped (Polygon/FRED/Tiingo), got %d", len(dropped))
+ }
+}
+
+// TestFilterByHost_FallsOpenForUnknownOrigin verifies endpoints with no origin
+// detected are kept (we don't have signal to drop them, so be permissive).
+func TestFilterByHost_FallsOpenForUnknownOrigin(t *testing.T) {
+ endpoints := []AggregatedEndpoint{
+ {Method: "GET", Path: "/v7/finance/quote", OriginBaseURLs: []string{"https://query1.finance.yahoo.com"}},
+ {Method: "GET", Path: "/unknown/origin", OriginBaseURLs: nil}, // no origin signal
+ }
+ kept, dropped := FilterByHost(endpoints, "query1.finance.yahoo.com")
+ if len(kept) != 2 {
+ t.Errorf("expected both kept (one match + one unknown), got %d kept", len(kept))
+ }
+ if len(dropped) != 0 {
+ t.Errorf("expected no drops, got %d", len(dropped))
+ }
+}
+
+// TestFilterByHost_NoTargetNoFilter verifies that passing an empty target host
+// is a no-op — preserves backwards-compatible behavior for callers that don't
+// care about origin filtering.
+func TestFilterByHost_NoTargetNoFilter(t *testing.T) {
+ endpoints := []AggregatedEndpoint{
+ {Method: "GET", Path: "/v7/finance/quote", OriginBaseURLs: []string{"https://query1.finance.yahoo.com"}},
+ {Method: "GET", Path: "/v3/reference/tickers", OriginBaseURLs: []string{"https://api.polygon.io"}},
+ }
+ kept, dropped := FilterByHost(endpoints, "")
+ if len(kept) != 2 || len(dropped) != 0 {
+ t.Errorf("expected no-op with empty target: got %d kept, %d dropped", len(kept), len(dropped))
+ }
+}
+
+// TestHostFromURL exercises host extraction for URL shapes we'll see in the wild.
+func TestHostFromURL(t *testing.T) {
+ cases := map[string]string{
+ "https://query1.finance.yahoo.com": "query1.finance.yahoo.com",
+ "https://query1.finance.yahoo.com/": "query1.finance.yahoo.com",
+ "https://api.polygon.io:443/v3/reference/tickers": "api.polygon.io",
+ "HTTPS://API.Notion.COM/v1/pages": "api.notion.com",
+ "no-scheme-at-all": "no-scheme-at-all",
+ "": "",
+ }
+ for in, want := range cases {
+ if got := hostFromURL(in); got != want {
+ t.Errorf("hostFromURL(%q) = %q, want %q", in, got, want)
+ }
+ }
+}
diff --git a/internal/crowdsniff/patterns.go b/internal/crowdsniff/patterns.go
index ebc1e019..e9fad6b2 100644
--- a/internal/crowdsniff/patterns.go
+++ b/internal/crowdsniff/patterns.go
@@ -38,9 +38,16 @@ var (
// GrepEndpoints scans source code content for API endpoint patterns.
// It returns discovered endpoints and base URL candidates.
+//
+// As the scan walks lines in order, each base URL match updates a "current
+// origin" that subsequent endpoints inherit via OriginBaseURL. When a source
+// file wraps multiple unrelated APIs (e.g. a finance aggregator that calls
+// Polygon, FRED, and Tiingo), this lets downstream filtering drop endpoints
+// whose origin doesn't match the user's target host.
func GrepEndpoints(content, sourceName, sourceTier string) ([]DiscoveredEndpoint, []string) {
var endpoints []DiscoveredEndpoint
var baseURLs []string
+ currentOrigin := "" // most recently observed base URL in this file's scan order
lines := strings.Split(content, "\n")
@@ -50,27 +57,114 @@ func GrepEndpoints(content, sourceName, sourceTier string) ([]DiscoveredEndpoint
continue
}
- // Extract base URL candidates.
+ // Extract base URL candidates and update the running origin. Extractors
+ // called below stamp endpoints with `currentOrigin` so callers can
+ // filter by host later.
if matches := baseURLPattern.FindStringSubmatch(line); len(matches) > 1 {
baseURLs = append(baseURLs, matches[1])
+ currentOrigin = matches[1]
}
// Try to extract method + path from method call patterns.
eps := extractMethodCallEndpoints(line, sourceName, sourceTier)
+ stampOrigin(eps, currentOrigin)
endpoints = append(endpoints, eps...)
- // Try to extract from fetch() calls.
+ // Try to extract from fetch() calls. Fetch URLs may contain an absolute
+ // host — when they do, record it as the endpoint's origin directly so
+ // inline-URL contamination is caught even without a preceding baseURL
+ // variable.
eps = extractFetchEndpoints(line, sourceName, sourceTier)
+ stampOriginFetch(eps, currentOrigin)
endpoints = append(endpoints, eps...)
// Try to extract paths from URL path literals when near an HTTP method.
eps = extractContextualPathEndpoints(line, sourceName, sourceTier)
+ stampOrigin(eps, currentOrigin)
endpoints = append(endpoints, eps...)
}
return deduplicateEndpoints(endpoints), deduplicateStrings(baseURLs)
}
+// stampOrigin assigns the current base URL context to every endpoint that
+// lacks one. Endpoints may already carry an origin extracted from an inline
+// absolute URL (see stampOriginFetch); those are preserved.
+func stampOrigin(eps []DiscoveredEndpoint, origin string) {
+ if origin == "" {
+ return
+ }
+ for i := range eps {
+ if eps[i].OriginBaseURL == "" {
+ eps[i].OriginBaseURL = origin
+ }
+ }
+}
+
+// stampOriginFetch handles fetch-style extractions where the path argument is
+// sometimes an absolute URL (`fetch("https://other-api.com/foo")`). When so,
+// the scheme+host becomes the endpoint's own origin — more precise than the
+// enclosing file's baseURL variable. Otherwise falls back to stampOrigin.
+func stampOriginFetch(eps []DiscoveredEndpoint, fileOrigin string) {
+ for i := range eps {
+ if eps[i].OriginBaseURL != "" {
+ continue
+ }
+ if inline := extractInlineOrigin(eps[i].Path); inline != "" {
+ eps[i].OriginBaseURL = inline
+ // The path should be just the path component now, but
+ // extractContextualPathEndpoints stripped the scheme already via
+ // urlPathLiteral. If Path still carries a scheme (from fetch
+ // parser), normalize it.
+ if strings.HasPrefix(eps[i].Path, "http://") || strings.HasPrefix(eps[i].Path, "https://") {
+ if u := parseURLSilent(eps[i].Path); u != "" {
+ eps[i].Path = u
+ }
+ }
+ continue
+ }
+ if fileOrigin != "" {
+ eps[i].OriginBaseURL = fileOrigin
+ }
+ }
+}
+
+// extractInlineOrigin returns the scheme+host portion of an absolute URL path
+// argument, or empty string if the argument is already a bare path.
+func extractInlineOrigin(path string) string {
+ if !strings.HasPrefix(path, "http://") && !strings.HasPrefix(path, "https://") {
+ return ""
+ }
+ // Split at the first slash after the scheme://host to isolate the origin.
+ // Use strings.IndexByte rather than net/url to avoid pulling in the full
+ // parser for every match.
+ idx := strings.Index(path, "://")
+ if idx < 0 {
+ return ""
+ }
+ rest := path[idx+3:]
+ slash := strings.IndexByte(rest, '/')
+ if slash < 0 {
+ return path
+ }
+ return path[:idx+3+slash]
+}
+
+// parseURLSilent returns just the path portion of an absolute URL, or empty
+// string on failure.
+func parseURLSilent(absURL string) string {
+ idx := strings.Index(absURL, "://")
+ if idx < 0 {
+ return ""
+ }
+ rest := absURL[idx+3:]
+ slash := strings.IndexByte(rest, '/')
+ if slash < 0 {
+ return "/"
+ }
+ return rest[slash:]
+}
+
// extractMethodCallEndpoints handles patterns like:
//
// this.get("/v1/users"), client.post("/users")
diff --git a/internal/crowdsniff/types.go b/internal/crowdsniff/types.go
index a7c7b4c6..76dc06fe 100644
--- a/internal/crowdsniff/types.go
+++ b/internal/crowdsniff/types.go
@@ -18,11 +18,12 @@ type DiscoveredParam struct {
// DiscoveredEndpoint represents a single API endpoint found by a source.
type DiscoveredEndpoint struct {
- Method string // HTTP method (GET, POST, etc.)
- Path string // URL path (e.g., "/v1/users", "/users/:id")
- Params []DiscoveredParam // query parameters extracted from SDK code
- SourceTier string // one of the Tier* constants
- SourceName string // e.g., "@notionhq/client", "github-code-search"
+ Method string // HTTP method (GET, POST, etc.)
+ Path string // URL path (e.g., "/v1/users", "/users/:id")
+ Params []DiscoveredParam // query parameters extracted from SDK code
+ SourceTier string // one of the Tier* constants
+ SourceName string // e.g., "@notionhq/client", "github-code-search"
+ OriginBaseURL string // nearest base URL observed in the same source file (e.g., "https://api.notion.com"); empty if not detectable
}
// DiscoveredAuth represents an authentication pattern detected in SDK source code.
@@ -45,9 +46,10 @@ type SourceResult struct {
// AggregatedEndpoint is a deduplicated endpoint with cross-source metadata.
type AggregatedEndpoint struct {
- Method string
- Path string // normalized
- Params []DiscoveredParam // union-merged params across sources, sorted by name
- SourceTier string // highest tier across sources
- SourceCount int // number of distinct sources
+ Method string
+ Path string // normalized
+ Params []DiscoveredParam // union-merged params across sources, sorted by name
+ SourceTier string // highest tier across sources
+ SourceCount int // number of distinct sources
+ OriginBaseURLs []string // distinct base URLs observed in source files that contributed this endpoint; used for host filtering
}
diff --git a/internal/generator/generator.go b/internal/generator/generator.go
index f6cf6ca9..e41ad38e 100644
--- a/internal/generator/generator.go
+++ b/internal/generator/generator.go
@@ -175,6 +175,15 @@ func New(s *spec.APISpec, outputDir string) *Generator {
return strings.Join(safe, sep)
},
"goLiteral": func(v any) string {
+ // A nil default — common when the spec declares a field with no
+ // default value — must render as the Go keyword `nil`, not `<nil>`
+ // (which is what `fmt.Sprintf("%v", nil)` produces and which the
+ // Go compiler rejects). Without this branch, search.go and other
+ // generated files emit invalid syntax for spec fields with missing
+ // defaults.
+ if v == nil {
+ return "nil"
+ }
switch val := v.(type) {
case string:
return fmt.Sprintf("%q", val)
@@ -620,6 +629,17 @@ func (g *Generator) Generate() error {
return fmt.Errorf("rendering auth: %w", err)
}
+ // For session_handshake auth, emit the session manager helper alongside
+ // the client. This was previously hand-patched in every CLI that used a
+ // crumb/CSRF-token pattern (yahoo-finance and any future reverse-engineered
+ // API with anti-CSRF on JSON endpoints). See retro issue #174 WU-2.
+ if g.Spec.Auth.Type == "session_handshake" {
+ sessionPath := filepath.Join("internal", "client", "session.go")
+ if err := g.renderTemplate("session_handshake.go.tmpl", sessionPath, g.Spec); err != nil {
+ return fmt.Errorf("rendering session manager: %w", err)
+ }
+ }
+
// MCP server: generate cmd/{name}-pp-mcp/ entry point and internal/mcp/ package
if g.VisionSet.MCP || true { // Always generate MCP for now
mcpDirs := []string{
diff --git a/internal/generator/session_handshake_test.go b/internal/generator/session_handshake_test.go
new file mode 100644
index 00000000..b92d2ad4
--- /dev/null
+++ b/internal/generator/session_handshake_test.go
@@ -0,0 +1,136 @@
+package generator
+
+import (
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/mvanhorn/cli-printing-press/internal/spec"
+)
+
+// TestSessionHandshakeGeneration verifies the generator emits a working
+// session.go helper and wires it into the client when auth.type is
+// "session_handshake". This is the WU-2 acceptance test (retro issue #174).
+func TestSessionHandshakeGeneration(t *testing.T) {
+ sp := &spec.APISpec{
+ Name: "demo",
+ Version: "1.0.0",
+ Description: "test",
+ BaseURL: "https://query1.example.com",
+ Auth: spec.AuthConfig{
+ Type: "session_handshake",
+ BootstrapURL: "https://bootstrap.example.com/",
+ SessionTokenURL: "https://query2.example.com/v1/getcrumb",
+ TokenFormat: "text",
+ TokenParamName: "crumb",
+ TokenParamIn: "query",
+ In: "query",
+ Header: "crumb",
+ InvalidateOnStatus: []int{401, 403},
+ SessionTTLHours: 24,
+ },
+ Config: spec.ConfigSpec{Format: "toml", Path: "~/.config/demo-pp-cli/config.toml"},
+ Resources: map[string]spec.Resource{
+ "quote": {
+ Description: "Quotes",
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/v7/finance/quote",
+ Description: "Get quotes",
+ Params: []spec.Param{{
+ Name: "symbols", Type: "string", Required: true,
+ }},
+ },
+ },
+ },
+ },
+ }
+
+ dir := t.TempDir()
+ g := New(sp, dir)
+ if err := g.Generate(); err != nil {
+ t.Fatalf("Generate: %v", err)
+ }
+
+ // session.go must exist alongside client.go
+ sessionPath := filepath.Join(dir, "internal", "client", "session.go")
+ if _, err := os.Stat(sessionPath); err != nil {
+ t.Fatalf("expected %s to exist, got error: %v", sessionPath, err)
+ }
+
+ sessionContent, err := os.ReadFile(sessionPath)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ // Sanity checks on the emitted helper
+ wants := []string{
+ `"https://bootstrap.example.com/"`, // BootstrapURL substituted
+ `"https://query2.example.com/v1/getcrumb"`,
+ `401: true`, // InvalidateOnStatus rendered
+ `403: true`,
+ `24 * time.Hour`, // TTL rendered
+ `type SessionManager struct`,
+ `func (m *SessionManager) EnsureToken()`,
+ `func (m *SessionManager) Invalidate()`,
+ `func (m *SessionManager) ImportSession(`,
+ }
+ for _, w := range wants {
+ if !strings.Contains(string(sessionContent), w) {
+ t.Errorf("session.go missing expected substring %q", w)
+ }
+ }
+
+ // client.go must reference the SessionManager
+ clientPath := filepath.Join(dir, "internal", "client", "client.go")
+ clientContent, err := os.ReadFile(clientPath)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if !strings.Contains(string(clientContent), "Session *SessionManager") {
+ t.Error("client.go missing Session field")
+ }
+ if !strings.Contains(string(clientContent), "c.Session.EnsureToken()") {
+ t.Error("client.go doesn't call EnsureToken() on the live path")
+ }
+ if !strings.Contains(string(clientContent), "c.Session.ShouldInvalidate(") {
+ t.Error("client.go doesn't check ShouldInvalidate on responses")
+ }
+ if !strings.Contains(string(clientContent), "c.Session.Invalidate()") {
+ t.Error("client.go doesn't invalidate on status-code match")
+ }
+}
+
+// TestSessionHandshakeNotEmittedForOtherAuth verifies the session helper is
+// NOT emitted for non-session auth types — no file bloat for bearer_token CLIs.
+func TestSessionHandshakeNotEmittedForOtherAuth(t *testing.T) {
+ sp := &spec.APISpec{
+ Name: "demo",
+ Version: "1.0.0",
+ Description: "test",
+ BaseURL: "https://api.example.com",
+ Auth: spec.AuthConfig{Type: "bearer_token", Header: "Authorization", EnvVars: []string{"DEMO_TOKEN"}},
+ Config: spec.ConfigSpec{Format: "toml", Path: "~/.config/demo-pp-cli/config.toml"},
+ Resources: map[string]spec.Resource{
+ "users": {
+ Description: "Users",
+ Endpoints: map[string]spec.Endpoint{
+ "list": {Method: "GET", Path: "/users", Description: "list"},
+ },
+ },
+ },
+ }
+
+ dir := t.TempDir()
+ g := New(sp, dir)
+ if err := g.Generate(); err != nil {
+ t.Fatalf("Generate: %v", err)
+ }
+
+ sessionPath := filepath.Join(dir, "internal", "client", "session.go")
+ if _, err := os.Stat(sessionPath); err == nil {
+ t.Errorf("session.go should NOT exist for bearer_token auth (got file at %s)", sessionPath)
+ }
+}
diff --git a/internal/generator/templates/client.go.tmpl b/internal/generator/templates/client.go.tmpl
index 871b6bed..cfe3aed6 100644
--- a/internal/generator/templates/client.go.tmpl
+++ b/internal/generator/templates/client.go.tmpl
@@ -31,6 +31,9 @@ type Client struct {
NoCache bool
cacheDir string
limiter *adaptiveLimiter
+{{- if eq .Auth.Type "session_handshake"}}
+ Session *SessionManager
+{{- end}}
}
// adaptiveLimiter provides proactive rate limiting with adaptive ceiling discovery.
@@ -183,6 +186,19 @@ func (e *APIError) Error() string {
func New(cfg *config.Config, timeout time.Duration, rateLimit float64) *Client {
homeDir, _ := os.UserHomeDir()
cacheDir := filepath.Join(homeDir, ".cache", "{{.Name}}-pp-cli")
+{{- if eq .Auth.Type "session_handshake"}}
+ sess := newSessionManager(timeout)
+ httpClient := &http.Client{Timeout: timeout}
+ sess.AttachTo(httpClient)
+ return &Client{
+ BaseURL: strings.TrimRight(cfg.BaseURL, "/"),
+ Config: cfg,
+ HTTPClient: httpClient,
+ cacheDir: cacheDir,
+ limiter: newAdaptiveLimiter(rateLimit),
+ Session: sess,
+ }
+{{- else}}
return &Client{
BaseURL: strings.TrimRight(cfg.BaseURL, "/"),
Config: cfg,
@@ -190,6 +206,7 @@ func New(cfg *config.Config, timeout time.Duration, rateLimit float64) *Client {
cacheDir: cacheDir,
limiter: newAdaptiveLimiter(rateLimit),
}
+{{- end}}
}
// RateLimit returns the current effective rate limit in req/s. Returns 0 if disabled.
@@ -294,9 +311,31 @@ func (c *Client) do(method, path string, params map[string]string, body any, hea
bodyBytes = b
}
+ // Resolve auth material before the dry-run branch so --dry-run can preview
+ // exactly what would be sent. Uses only cached credentials; a token that
+ // requires a network refresh will be re-fetched on the live request path,
+ // not during dry-run.
+ authHeader, err := c.authHeader()
+ if err != nil {
+ return nil, 0, err
+ }
+{{- if eq .Auth.Type "session_handshake"}}
+ // For session-handshake auth, authHeader() only returns the cached token
+ // (or empty). On the live path, ensure a fresh token if none is cached.
+ // Dry-run deliberately skips this — we want previews to reflect what's
+ // cached, never trigger a network fetch.
+ if !c.DryRun && authHeader == "" && c.Session != nil {
+ tok, terr := c.Session.EnsureToken()
+ if terr != nil {
+ return nil, 0, terr
+ }
+ authHeader = tok
+ }
+{{- end}}
+
// Build the request for dry-run display or actual execution
if c.DryRun {
- return c.dryRun(method, targetURL, path, params, bodyBytes, headerOverrides)
+ return c.dryRun(method, targetURL, path, params, bodyBytes, headerOverrides, authHeader)
}
const maxRetries = 3
@@ -305,6 +344,18 @@ func (c *Client) do(method, path string, params map[string]string, body any, hea
for attempt := 0; attempt <= maxRetries; attempt++ {
// Proactive rate limiting — wait before sending
c.limiter.Wait()
+{{- if eq .Auth.Type "session_handshake"}}
+
+ // Re-resolve session token each iteration — may have been invalidated
+ // by a previous attempt returning 401/403.
+ if authHeader == "" && c.Session != nil {
+ tok, terr := c.Session.EnsureToken()
+ if terr != nil {
+ return nil, 0, terr
+ }
+ authHeader = tok
+ }
+{{- end}}
{{- if eq .ClientPattern "proxy-envelope"}}
// Build the proxy envelope
@@ -351,10 +402,6 @@ func (c *Client) do(method, path string, params map[string]string, body any, hea
}
{{- end}}
- authHeader, err := c.authHeader()
- if err != nil {
- return nil, 0, err
- }
if authHeader != "" {
{{- if and .Auth .Auth.In (eq .Auth.In "query")}}
// API key goes in query parameter
@@ -409,6 +456,18 @@ func (c *Client) do(method, path string, params map[string]string, body any, hea
lastErr = apiErr
continue
}
+{{- if eq .Auth.Type "session_handshake"}}
+
+ // Session-handshake invalidation: some status codes indicate the token
+ // was rejected. Clear the cache and retry once; EnsureToken() at the
+ // top of the next loop iteration will re-bootstrap.
+ if c.Session != nil && c.Session.ShouldInvalidate(resp.StatusCode) && attempt < maxRetries && authHeader != "" {
+ c.Session.Invalidate()
+ authHeader = "" // force re-fetch on next iteration
+ lastErr = apiErr
+ continue
+ }
+{{- end}}
// Server error - retry with backoff
if resp.StatusCode >= 500 && attempt < maxRetries {
@@ -426,7 +485,10 @@ func (c *Client) do(method, path string, params map[string]string, body any, hea
return nil, 0, lastErr
}
-func (c *Client) dryRun(method, targetURL, path string, params map[string]string, body []byte, headerOverrides map[string]string) (json.RawMessage, int, error) {
+// dryRun prints the outgoing request exactly as the live path would send it,
+// using the auth material already resolved in `do()`. Never triggers a network
+// call — the caller is responsible for passing cached auth material only.
+func (c *Client) dryRun(method, targetURL, path string, params map[string]string, body []byte, headerOverrides map[string]string, authHeader string) (json.RawMessage, int, error) {
{{- if eq .ClientPattern "proxy-envelope"}}
// Show the proxy envelope that would be sent
envelope := proxyEnvelope{
@@ -449,6 +511,13 @@ func (c *Client) dryRun(method, targetURL, path string, params map[string]string
}
}
}
+{{- if and .Auth .Auth.In (eq .Auth.In "query")}}
+ // Auth material travels as a query parameter on the live request — surface
+ // it in the preview so users can verify wiring.
+ if authHeader != "" {
+ fmt.Fprintf(os.Stderr, " ?%s=%s\n", "{{if .Auth.Header}}{{.Auth.Header}}{{else}}api_key{{end}}", maskToken(authHeader))
+ }
+{{- end}}
if body != nil {
var pretty json.RawMessage
if json.Unmarshal(body, &pretty) == nil {
@@ -459,13 +528,11 @@ func (c *Client) dryRun(method, targetURL, path string, params map[string]string
}
}
{{- end}}
- authHeader, err := c.authHeader()
- if err != nil {
- return nil, 0, err
- }
+{{- if not (and .Auth .Auth.In (eq .Auth.In "query"))}}
if authHeader != "" {
- fmt.Fprintf(os.Stderr, " Authorization: %s\n", maskToken(authHeader))
+ fmt.Fprintf(os.Stderr, " %s: %s\n", "{{if .Auth.Header}}{{.Auth.Header}}{{else}}Authorization{{end}}", maskToken(authHeader))
}
+{{- end}}
fmt.Fprintf(os.Stderr, "\n(dry run - no request sent)\n")
return json.RawMessage(`{"dry_run": true}`), 0, nil
}
diff --git a/internal/generator/templates/session_handshake.go.tmpl b/internal/generator/templates/session_handshake.go.tmpl
new file mode 100644
index 00000000..0a1140f1
--- /dev/null
+++ b/internal/generator/templates/session_handshake.go.tmpl
@@ -0,0 +1,307 @@
+// Copyright 2026 trevin-chow. Licensed under Apache-2.0. See LICENSE.
+// Generated by CLI Printing Press (https://github.com/mvanhorn/cli-printing-press). DO NOT EDIT.
+
+// Session-handshake auth implementation for {{.Name}}.
+//
+// Flow:
+// 1. GET {{.Auth.BootstrapURL}} — seed cookies (if bootstrap URL is configured)
+// 2. GET {{.Auth.SessionTokenURL}} — receive anti-CSRF token
+// 3. Attach token to every request via {{if eq .Auth.TokenParamIn "header"}}header {{.Auth.TokenParamName}}{{else}}query param {{.Auth.TokenParamName}}{{end}}
+// 4. On {{range .Auth.InvalidateOnStatus}}{{.}} {{end}}responses, invalidate and re-fetch
+//
+// The manager persists to ~/.config/{{.Name}}-pp-cli/session.json so the token
+// survives across invocations (TTL: {{if .Auth.SessionTTLHours}}{{.Auth.SessionTTLHours}}{{else}}24{{end}} hours).
+// Cookies from the jar are scoped to the requesting host automatically via
+// net/http/cookiejar.
+
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "net/http/cookiejar"
+ "net/url"
+ "os"
+ "path/filepath"
+ "strings"
+ "sync"
+ "time"
+)
+
+const (
+ sessionBootstrapURL = "{{.Auth.BootstrapURL}}"
+ sessionTokenURL = "{{.Auth.SessionTokenURL}}"
+ sessionTokenFormat = "{{if .Auth.TokenFormat}}{{.Auth.TokenFormat}}{{else}}text{{end}}"
+ sessionTokenJSONPath = "{{.Auth.TokenJSONPath}}"
+ sessionUserAgent = "Mozilla/5.0 (compatible; {{.Name}}-pp-cli)"
+)
+
+var sessionTTL = {{if .Auth.SessionTTLHours}}{{.Auth.SessionTTLHours}}{{else}}24{{end}} * time.Hour
+
+// sessionInvalidationStatuses are HTTP status codes that trigger token
+// invalidation. When the server returns one of these, the cached token is
+// cleared so the next request re-bootstraps.
+var sessionInvalidationStatuses = map[int]bool{
+{{- if .Auth.InvalidateOnStatus}}
+{{- range .Auth.InvalidateOnStatus}}
+ {{.}}: true,
+{{- end}}
+{{- else}}
+ 401: true,
+ 403: true,
+{{- end}}
+}
+
+// SessionManager owns the cookie jar + cached token and coordinates the
+// handshake. Safe for concurrent use by the client's retry loop.
+type SessionManager struct {
+ mu sync.Mutex
+ client *http.Client
+ jar http.CookieJar
+ token string
+}
+
+// newSessionManager constructs a SessionManager with a cookie jar attached to
+// its HTTP client. Any cached session on disk is loaded immediately.
+func newSessionManager(timeout time.Duration) *SessionManager {
+ jar, _ := cookiejar.New(nil)
+ m := &SessionManager{
+ jar: jar,
+ client: &http.Client{Timeout: timeout, Jar: jar},
+ }
+ m.loadFromDisk()
+ return m
+}
+
+// AttachTo hooks this SessionManager's cookie jar onto the shared client so
+// every data request reuses the bootstrapped cookies.
+func (m *SessionManager) AttachTo(c *http.Client) {
+ if c.Jar == nil {
+ c.Jar = m.jar
+ }
+}
+
+// Token returns the cached token without triggering a network call. Empty
+// string when no session is established or the cache was invalidated.
+func (m *SessionManager) Token() string {
+ m.mu.Lock()
+ defer m.mu.Unlock()
+ return m.token
+}
+
+// EnsureToken returns a valid token, bootstrapping a fresh session if needed.
+// The network calls happen under the manager's lock — concurrent callers wait
+// for a single in-flight handshake.
+func (m *SessionManager) EnsureToken() (string, error) {
+ m.mu.Lock()
+ if m.token != "" {
+ t := m.token
+ m.mu.Unlock()
+ return t, nil
+ }
+ m.mu.Unlock()
+
+ // Step 1: bootstrap (optional — not every API needs a cookie seed).
+ if sessionBootstrapURL != "" {
+ req, err := http.NewRequest("GET", sessionBootstrapURL, nil)
+ if err != nil {
+ return "", fmt.Errorf("session bootstrap: %w", err)
+ }
+ req.Header.Set("User-Agent", sessionUserAgent)
+ resp, err := m.client.Do(req)
+ if err != nil {
+ return "", fmt.Errorf("session bootstrap: %w", err)
+ }
+ _, _ = io.Copy(io.Discard, resp.Body)
+ resp.Body.Close()
+ }
+
+ // Step 2: fetch token.
+ req, err := http.NewRequest("GET", sessionTokenURL, nil)
+ if err != nil {
+ return "", fmt.Errorf("session token fetch: %w", err)
+ }
+ req.Header.Set("User-Agent", sessionUserAgent)
+ if sessionTokenFormat == "json" {
+ req.Header.Set("Accept", "application/json")
+ }
+ resp, err := m.client.Do(req)
+ if err != nil {
+ return "", fmt.Errorf("session token fetch: %w", err)
+ }
+ defer resp.Body.Close()
+ body, err := io.ReadAll(resp.Body)
+ if err != nil {
+ return "", fmt.Errorf("reading token response: %w", err)
+ }
+ if resp.StatusCode >= 400 {
+ return "", fmt.Errorf("session token endpoint returned HTTP %d — IP may be rate-limited", resp.StatusCode)
+ }
+
+ var tok string
+ if sessionTokenFormat == "json" {
+ tok = extractJSONToken(body, sessionTokenJSONPath)
+ } else {
+ tok = strings.TrimSpace(string(body))
+ }
+ if tok == "" {
+ return "", fmt.Errorf("session token endpoint returned empty body")
+ }
+
+ m.mu.Lock()
+ m.token = tok
+ m.mu.Unlock()
+ m.saveToDisk()
+ return tok, nil
+}
+
+// ShouldInvalidate returns true when an HTTP status code indicates the token
+// was rejected and the session should be re-bootstrapped.
+func (m *SessionManager) ShouldInvalidate(statusCode int) bool {
+ return sessionInvalidationStatuses[statusCode]
+}
+
+// Invalidate clears the cached token so the next EnsureToken re-bootstraps.
+func (m *SessionManager) Invalidate() {
+ m.mu.Lock()
+ m.token = ""
+ m.mu.Unlock()
+}
+
+// ImportSession accepts cookies + a token captured from a real browser session
+// (e.g. Chrome DevTools) and treats them as a live session. Used when the
+// server-side handshake is rate-limited from the user's IP.
+func (m *SessionManager) ImportSession(cookieDomain string, cookies []*http.Cookie, token string) error {
+ if m.jar == nil {
+ return fmt.Errorf("no cookie jar attached")
+ }
+ u, err := url.Parse(cookieDomain)
+ if err != nil {
+ return fmt.Errorf("invalid cookie domain: %w", err)
+ }
+ m.jar.SetCookies(u, cookies)
+ m.mu.Lock()
+ m.token = token
+ m.mu.Unlock()
+ m.saveToDisk()
+ return nil
+}
+
+// --- Persistence helpers ---
+
+func (m *SessionManager) sessionFilePath() string {
+ home, err := os.UserHomeDir()
+ if err != nil {
+ return ""
+ }
+ return filepath.Join(home, ".config", "{{.Name}}-pp-cli", "session.json")
+}
+
+type sessionRecord struct {
+ Token string `json:"token"`
+ Cookies []jarCookie `json:"cookies"`
+ Saved time.Time `json:"saved"`
+}
+
+type jarCookie struct {
+ URL string `json:"url"`
+ Name string `json:"name"`
+ Value string `json:"value"`
+ Path string `json:"path"`
+ Expires time.Time `json:"expires,omitempty"`
+}
+
+func (m *SessionManager) loadFromDisk() {
+ path := m.sessionFilePath()
+ if path == "" {
+ return
+ }
+ data, err := os.ReadFile(path)
+ if err != nil {
+ return
+ }
+ var rec sessionRecord
+ if err := json.Unmarshal(data, &rec); err != nil {
+ return
+ }
+ if time.Since(rec.Saved) > sessionTTL {
+ return // stale; ignore
+ }
+ m.mu.Lock()
+ m.token = rec.Token
+ m.mu.Unlock()
+ for _, c := range rec.Cookies {
+ u, err := url.Parse(c.URL)
+ if err != nil || m.jar == nil {
+ continue
+ }
+ ck := &http.Cookie{
+ Name: c.Name,
+ Value: c.Value,
+ Path: c.Path,
+ Expires: c.Expires,
+ }
+ m.jar.SetCookies(u, []*http.Cookie{ck})
+ }
+}
+
+func (m *SessionManager) saveToDisk() {
+ path := m.sessionFilePath()
+ if path == "" || m.jar == nil {
+ return
+ }
+ m.mu.Lock()
+ tok := m.token
+ m.mu.Unlock()
+
+ var cookies []jarCookie
+ // Persist cookies for the token endpoint host (broadest plausible scope).
+ if u, err := url.Parse(sessionTokenURL); err == nil {
+ for _, ck := range m.jar.Cookies(u) {
+ cookies = append(cookies, jarCookie{
+ URL: u.Scheme + "://" + u.Host + "/",
+ Name: ck.Name,
+ Value: ck.Value,
+ Path: ck.Path,
+ Expires: ck.Expires,
+ })
+ }
+ }
+ rec := sessionRecord{Token: tok, Cookies: cookies, Saved: time.Now()}
+ data, err := json.MarshalIndent(rec, "", " ")
+ if err != nil {
+ return
+ }
+ _ = os.MkdirAll(filepath.Dir(path), 0o755)
+ _ = os.WriteFile(path, data, 0o600)
+}
+
+// extractJSONToken pulls a token from a JSON body by dot-separated path.
+// Example: ("data.token", `{"data":{"token":"abc"}}`) → "abc". Empty string on
+// parse failure or missing field.
+func extractJSONToken(body []byte, path string) string {
+ var obj any
+ if err := json.Unmarshal(body, &obj); err != nil {
+ return ""
+ }
+ if path == "" {
+ if s, ok := obj.(string); ok {
+ return s
+ }
+ return ""
+ }
+ cur := obj
+ for _, part := range strings.Split(path, ".") {
+ m, ok := cur.(map[string]any)
+ if !ok {
+ return ""
+ }
+ cur = m[part]
+ }
+ if s, ok := cur.(string); ok {
+ return s
+ }
+ return ""
+}
diff --git a/internal/pipeline/dogfood.go b/internal/pipeline/dogfood.go
index db926641..d017c620 100644
--- a/internal/pipeline/dogfood.go
+++ b/internal/pipeline/dogfood.go
@@ -192,11 +192,7 @@ func checkNovelFeatures(cliDir, researchDir string) NovelFeaturesCheckResult {
}
built := make([]NovelFeature, 0)
for _, nf := range research.NovelFeatures {
- // The command field may be a subcommand path like "issues stale".
- // Check if the leaf command name is registered.
- parts := strings.Fields(nf.Command)
- leaf := parts[len(parts)-1]
- if registeredCmds[leaf] {
+ if matchNovelFeature(nf, registeredCmds) {
result.Found++
built = append(built, nf)
} else {
@@ -213,6 +209,88 @@ func checkNovelFeatures(cliDir, researchDir string) NovelFeaturesCheckResult {
return result
}
+// matchNovelFeature runs the three-pass matcher — exact, prefix, alias —
+// against the set of registered command Use: names. Returns true if the
+// planned feature has a plausible corresponding built command.
+//
+// Why this is three passes:
+// - Exact: planned command leaf equals a registered Use: name. The happy
+// path (e.g. planned "portfolio perf", built "perf" subcommand).
+// - Prefix: during implementation, planners frequently hyphenate or
+// specialize the command. Planned "auth login --chrome" becomes built
+// "login-chrome"; planned "options --moneyness" becomes built
+// "options-chain". Either direction of prefix relationship counts.
+// - Alias: escape hatch for cases the matcher can't infer — planner records
+// alternate names in research.json's `aliases: []`.
+//
+// The matcher strips flag tokens (anything starting with "-") from the planned
+// command before extracting the leaf, because flags aren't command names.
+// "options --moneyness otm --max-dte 45" reduces to leaf "options".
+func matchNovelFeature(nf NovelFeature, registered map[string]bool) bool {
+ leaf := strings.ToLower(commandLeaf(nf.Command))
+ if leaf == "" {
+ return false
+ }
+
+ // Pass 1: exact
+ if registered[leaf] {
+ return true
+ }
+
+ // Pass 2: prefix — either direction. Built command may be a hyphenated
+ // specialization (planned "options" → built "options-chain") or the planner
+ // may have been more specific than the actual binding (planned "earnings-
+ // calendar" is substring-matched by built "earnings"). Only hyphen
+ // boundaries count as prefix to avoid substring false positives like
+ // "options" matching "op".
+ for use := range registered {
+ if strings.HasPrefix(use, leaf+"-") || strings.HasPrefix(leaf, use+"-") {
+ return true
+ }
+ }
+
+ // Pass 3: aliases declared in research.json. Each alias is matched with
+ // the same exact + prefix rules so planners don't need to list every
+ // hyphen variant.
+ for _, alias := range nf.Aliases {
+ aliasLeaf := strings.ToLower(commandLeaf(alias))
+ if aliasLeaf == "" {
+ continue
+ }
+ if registered[aliasLeaf] {
+ return true
+ }
+ for use := range registered {
+ if strings.HasPrefix(use, aliasLeaf+"-") || strings.HasPrefix(aliasLeaf, use+"-") {
+ return true
+ }
+ }
+ }
+
+ return false
+}
+
+// commandLeaf strips flag tokens (anything beginning with "-") from a command
+// string and returns the last remaining token. "auth login --chrome" → "login";
+// "options --moneyness otm" → "options" (subsequent tokens are flag values).
+//
+// The rule "stop at first flag" mirrors how users describe commands in plans —
+// the command itself appears before any flag annotation.
+func commandLeaf(cmd string) string {
+ tokens := strings.Fields(cmd)
+ base := make([]string, 0, len(tokens))
+ for _, t := range tokens {
+ if strings.HasPrefix(t, "-") {
+ break
+ }
+ base = append(base, t)
+ }
+ if len(base) == 0 {
+ return ""
+ }
+ return base[len(base)-1]
+}
+
// collectRegisteredCommands returns a set of cobra Use: names found in the
// CLI's internal/cli/*.go files.
func collectRegisteredCommands(dir string) map[string]bool {
diff --git a/internal/pipeline/novel_features_matcher_test.go b/internal/pipeline/novel_features_matcher_test.go
new file mode 100644
index 00000000..e2066945
--- /dev/null
+++ b/internal/pipeline/novel_features_matcher_test.go
@@ -0,0 +1,129 @@
+package pipeline
+
+import (
+ "testing"
+)
+
+// TestMatchNovelFeature covers the three-pass matcher — exact, prefix, alias —
+// exercising the yahoo-finance retro scenarios where the previous literal
+// matcher false-negatived features that were actually built.
+func TestMatchNovelFeature(t *testing.T) {
+ registered := map[string]bool{
+ "perf": true, // under `portfolio perf`
+ "gains": true,
+ "digest": true,
+ "login-chrome": true, // under `auth login-chrome`
+ "options": true,
+ "options-chain": true,
+ "sparkline": true,
+ "earnings": true, // registered but planned was "earnings-calendar"
+ }
+
+ cases := []struct {
+ name string
+ feature NovelFeature
+ want bool
+ pass string
+ }{
+ {
+ name: "exact match on leaf (portfolio perf → perf)",
+ feature: NovelFeature{Command: "portfolio perf"},
+ want: true,
+ pass: "exact",
+ },
+ {
+ name: "exact match after flag strip (digest --watchlist tech → digest)",
+ feature: NovelFeature{Command: "digest --watchlist tech"},
+ want: true,
+ pass: "exact",
+ },
+ {
+ name: "prefix match: planned auth login --chrome → built login-chrome",
+ feature: NovelFeature{Command: "auth login --chrome"},
+ want: true,
+ pass: "prefix",
+ },
+ {
+ name: "prefix match: planned options --moneyness → built options or options-chain",
+ feature: NovelFeature{Command: "options --moneyness otm --max-dte 45"},
+ want: true,
+ pass: "exact (options registered) or prefix (options-chain)",
+ },
+ {
+ name: "no match when truly absent",
+ feature: NovelFeature{Command: "insiders --recent 30d --net-buying"},
+ want: false,
+ },
+ {
+ name: "alias rescues a rename",
+ feature: NovelFeature{Command: "portfolio dividends", Aliases: []string{"portfolio gains"}},
+ want: true,
+ pass: "alias (gains built as renamed feature)",
+ },
+ {
+ name: "alias prefix match",
+ feature: NovelFeature{Command: "portfolio dividends", Aliases: []string{"auth login-chrome"}},
+ want: true,
+ pass: "alias exact (login-chrome registered)",
+ },
+ {
+ name: "empty command returns false",
+ feature: NovelFeature{Command: ""},
+ want: false,
+ },
+ {
+ name: "flag-only command returns false (nothing to match)",
+ feature: NovelFeature{Command: "--json"},
+ want: false,
+ },
+ {
+ name: "case insensitive",
+ feature: NovelFeature{Command: "PORTFOLIO PERF"},
+ want: true,
+ pass: "exact (case folded)",
+ },
+ }
+
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ got := matchNovelFeature(tc.feature, registered)
+ if got != tc.want {
+ t.Errorf("matchNovelFeature(%q, aliases=%v) = %v, want %v (pass: %s)",
+ tc.feature.Command, tc.feature.Aliases, got, tc.want, tc.pass)
+ }
+ })
+ }
+}
+
+// TestCommandLeaf covers the flag-stripping helper.
+func TestCommandLeaf(t *testing.T) {
+ cases := map[string]string{
+ "portfolio perf": "perf",
+ "auth login --chrome": "login",
+ "options --moneyness otm --max-dte 45": "options",
+ "digest": "digest",
+ "digest --watchlist tech": "digest",
+ "earnings-calendar --watchlist": "earnings-calendar",
+ "": "",
+ "--json": "",
+ " padded perf ": "perf",
+ }
+ for in, want := range cases {
+ if got := commandLeaf(in); got != want {
+ t.Errorf("commandLeaf(%q) = %q, want %q", in, got, want)
+ }
+ }
+}
+
+// TestMatchNovelFeature_PrefixDoesNotFalsePositive guards against over-eager
+// prefix matching. "op" should NOT match "options" — only hyphen-bounded
+// prefixes count.
+func TestMatchNovelFeature_PrefixDoesNotFalsePositive(t *testing.T) {
+ registered := map[string]bool{
+ "options": true,
+ }
+ nf := NovelFeature{Command: "op"}
+ if matchNovelFeature(nf, registered) {
+ t.Error("commandLeaf 'op' should not prefix-match 'options' (no hyphen boundary)")
+ }
+}
diff --git a/internal/pipeline/research.go b/internal/pipeline/research.go
index 186e5eb7..cda45815 100644
--- a/internal/pipeline/research.go
+++ b/internal/pipeline/research.go
@@ -43,6 +43,12 @@ type NovelFeature struct {
Command string `json:"command"`
Description string `json:"description"`
Rationale string `json:"rationale"`
+ // Aliases are optional alternate command paths that should be considered
+ // "this feature was built" during dogfood's novel-feature verification.
+ // Planners use this when a feature may ship under any of several command
+ // names (e.g. `["auth login-chrome", "auth browser-login"]`). Empty by
+ // default — the three-pass matcher still covers most natural drift.
+ Aliases []string `json:"aliases,omitempty"`
}
// CompetitorAnalysis holds intelligence gathered from a single competitor repo.
diff --git a/internal/pipeline/scorecard.go b/internal/pipeline/scorecard.go
index 2c12ccb6..1b033b6a 100644
--- a/internal/pipeline/scorecard.go
+++ b/internal/pipeline/scorecard.go
@@ -808,6 +808,67 @@ func scoreVision(dir string) int {
return score
}
+// registeredCommandFiles returns the set of cli/*.go filenames whose command
+// constructor is referenced by root.go. Files without a registered constructor
+// should not inflate workflow/insight scores even if they match prefix or
+// behavioral heuristics — they're orphans, dead code, or half-built commands
+// that the user cannot actually invoke.
+//
+// Returns an empty map if root.go is missing or parsing yields no matches so
+// callers can fall open to the prior heuristic behavior (older or partial CLI
+// trees where the registration graph isn't parseable).
+func registeredCommandFiles(cliDir string) map[string]bool {
+ rootContent := readFileContent(filepath.Join(cliDir, "root.go"))
+ if rootContent == "" {
+ return map[string]bool{}
+ }
+
+ // Match every `newXxxCmd(` invocation — but not definitions. root.go may
+ // contain helper function declarations (e.g. `func newRootCmd()`) that we
+ // must not count as registrations. Strip `func Name(` declaration heads
+ // before scanning so only call-sites contribute to the ctor set.
+ funcDeclRe := regexp.MustCompile(`(?m)^func\s+\w+\s*\(`)
+ scanContent := funcDeclRe.ReplaceAllString(rootContent, "")
+
+ ctorRe := regexp.MustCompile(`\bnew([A-Z][A-Za-z0-9_]*)Cmd\s*\(`)
+ matches := ctorRe.FindAllStringSubmatch(scanContent, -1)
+ if len(matches) == 0 {
+ return map[string]bool{}
+ }
+ ctors := make(map[string]bool, len(matches))
+ for _, m := range matches {
+ ctors["new"+m[1]+"Cmd"] = true
+ }
+
+ // Walk cli/*.go and map each file to the constructor it defines. Use a
+ // regexp for the declaration site to avoid depending on go/parser for one
+ // lookup (keeps the scorer dependency-free, which matters because it runs
+ // against third-party generated trees).
+ defRe := regexp.MustCompile(`^func\s+(new[A-Z][A-Za-z0-9_]*Cmd)\s*\(`)
+ entries, err := os.ReadDir(cliDir)
+ if err != nil {
+ return map[string]bool{}
+ }
+ result := make(map[string]bool)
+ for _, e := range entries {
+ if e.IsDir() || !strings.HasSuffix(e.Name(), ".go") {
+ continue
+ }
+ content := readFileContent(filepath.Join(cliDir, e.Name()))
+ for _, line := range strings.Split(content, "\n") {
+ sm := defRe.FindStringSubmatch(line)
+ if sm == nil {
+ continue
+ }
+ if ctors[sm[1]] {
+ result[e.Name()] = true
+ break
+ }
+ }
+ }
+ return result
+}
+
func scoreWorkflows(dir string) int {
cliDir := filepath.Join(dir, "internal", "cli")
entries, err := os.ReadDir(cliDir)
@@ -815,6 +876,13 @@ func scoreWorkflows(dir string) int {
return 0
}
+ // Build the set of files whose command constructor is actually registered in
+ // root.go. Files that define constructors never added to the command tree —
+ // whether orphaned, dead, or pending — should not inflate the score. This
+ // also prevents dead-code removal from dropping the score: a file whose
+ // constructor isn't registered isn't counted in the first place.
+ registeredFiles := registeredCommandFiles(cliDir)
+
// Some prefixes overlap with insightPrefixes intentionally — per Steinberger,
// analytics/insights ARE compound commands (the visionary research plan lists
// "analytics" alongside "backup" and "moderate" as workflow examples). A command
@@ -833,6 +901,14 @@ func scoreWorkflows(dir string) int {
continue
}
+ // If root.go registration is discoverable, require the file to define a
+ // registered constructor. Falls open when no registrations are found at
+ // all (older CLIs or partial builds) so we don't zero out scores
+ // unexpectedly.
+ if len(registeredFiles) > 0 && !registeredFiles[e.Name()] {
+ continue
+ }
+
name := strings.ToLower(e.Name())
// Detect workflow commands by filename pattern
@@ -891,6 +967,8 @@ func scoreInsight(dir string) int {
return 0
}
+ registeredFiles := registeredCommandFiles(cliDir)
+
insightPrefixes := []string{"health", "similar", "bottleneck", "trends", "patterns", "forecast",
"stats", "conflicts", "stale", "analytics", "busiest", "velocity",
"utilization", "coverage", "gaps", "noshow"}
@@ -903,6 +981,9 @@ func scoreInsight(dir string) int {
if infraCoreFiles[e.Name()] {
continue
}
+ if len(registeredFiles) > 0 && !registeredFiles[e.Name()] {
+ continue
+ }
name := strings.ToLower(e.Name())
// Signal 1: filename prefix match (supplementary — kept for backward compat)
diff --git a/internal/pipeline/scorecard_registered_test.go b/internal/pipeline/scorecard_registered_test.go
new file mode 100644
index 00000000..c5da568d
--- /dev/null
+++ b/internal/pipeline/scorecard_registered_test.go
@@ -0,0 +1,112 @@
+package pipeline
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+// TestRegisteredCommandFiles_OrphanIgnored verifies that scoreWorkflows and
+// scoreInsight no longer count files whose constructor is never registered in
+// root.go. This prevents dead-code removal from dropping the score and ensures
+// orphaned command files don't inflate it either.
+func TestRegisteredCommandFiles_OrphanIgnored(t *testing.T) {
+ dir := t.TempDir()
+ cliDir := filepath.Join(dir, "internal", "cli")
+ if err := os.MkdirAll(cliDir, 0o755); err != nil {
+ t.Fatal(err)
+ }
+
+ // root.go registers only `newStaleCmd`. `newDeadCmd` exists but isn't added.
+ writeFile(t, filepath.Join(cliDir, "root.go"), `package cli
+import "github.com/spf13/cobra"
+func newRootCmd() *cobra.Command {
+ rootCmd := &cobra.Command{Use: "x"}
+ rootCmd.AddCommand(newStaleCmd(nil))
+ return rootCmd
+}`)
+
+ writeFile(t, filepath.Join(cliDir, "stale.go"), `package cli
+import "github.com/spf13/cobra"
+func newStaleCmd(flags any) *cobra.Command {
+ return &cobra.Command{Use: "stale"}
+}`)
+
+ writeFile(t, filepath.Join(cliDir, "dead.go"), `package cli
+import "github.com/spf13/cobra"
+func newDeadCmd(flags any) *cobra.Command {
+ return &cobra.Command{Use: "dead"}
+}`)
+
+ // With only stale.go registered, workflows should count 1 (not 2).
+ registered := registeredCommandFiles(cliDir)
+ if !registered["stale.go"] {
+ t.Errorf("expected stale.go to be registered, got %v", registered)
+ }
+ if registered["dead.go"] {
+ t.Errorf("expected dead.go to NOT be registered (orphan), got %v", registered)
+ }
+ if registered["root.go"] {
+ t.Errorf("root.go itself should not be in registered set (it has no newXxxCmd)")
+ }
+}
+
+// TestRegisteredCommandFiles_FallsOpenOnMissingRoot verifies graceful handling
+// when root.go is missing or unparseable — older CLIs and partial trees must
+// still score, not return zero.
+func TestRegisteredCommandFiles_FallsOpenOnMissingRoot(t *testing.T) {
+ dir := t.TempDir()
+ cliDir := filepath.Join(dir, "internal", "cli")
+ if err := os.MkdirAll(cliDir, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ // No root.go at all.
+ writeFile(t, filepath.Join(cliDir, "stale.go"), `package cli
+func newStaleCmd() {}`)
+
+ registered := registeredCommandFiles(cliDir)
+ if len(registered) != 0 {
+ t.Errorf("expected empty map when root.go is missing, got %v", registered)
+ }
+}
+
+// TestScoreWorkflows_IgnoresOrphanFile is the integration-level guard — the
+// workflows dimension must not count a dead-code file just because its name
+// matches a workflow prefix.
+func TestScoreWorkflows_IgnoresOrphanFile(t *testing.T) {
+ dir := t.TempDir()
+ cliDir := filepath.Join(dir, "internal", "cli")
+ if err := os.MkdirAll(cliDir, 0o755); err != nil {
+ t.Fatal(err)
+ }
+
+ writeFile(t, filepath.Join(cliDir, "root.go"), `package cli
+import "github.com/spf13/cobra"
+func run() {
+ rootCmd := &cobra.Command{}
+ rootCmd.AddCommand(newStaleCmd(nil))
+}`)
+ // Registered workflow command
+ writeFile(t, filepath.Join(cliDir, "stale.go"), `package cli
+import "github.com/spf13/cobra"
+func newStaleCmd(flags any) *cobra.Command { return &cobra.Command{} }`)
+ // Orphan — filename matches workflow prefix, but not registered
+ writeFile(t, filepath.Join(cliDir, "search_query.go"), `package cli
+import "github.com/spf13/cobra"
+func newSearchQueryCmd(flags any) *cobra.Command { return &cobra.Command{} }`)
+
+ score := scoreWorkflows(dir)
+ // Exactly one registered workflow-prefix file → score 2 (per scoreWorkflows
+ // rubric: >=1 compound command → 2). The orphan search_query.go must not
+ // bump this to 4.
+ if score != 2 {
+ t.Errorf("expected score=2 (one registered workflow), got %d — orphan likely counted", score)
+ }
+}
+
+func writeFile(t *testing.T, path, content string) {
+ t.Helper()
+ if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+ t.Fatal(err)
+ }
+}
diff --git a/internal/spec/spec.go b/internal/spec/spec.go
index b9347eb3..d389dd9e 100644
--- a/internal/spec/spec.go
+++ b/internal/spec/spec.go
@@ -38,7 +38,7 @@ type RequiredHeader struct {
}
type AuthConfig struct {
- Type string `yaml:"type" json:"type"` // api_key, oauth2, bearer_token, cookie, composed, none
+ Type string `yaml:"type" json:"type"` // api_key, oauth2, bearer_token, cookie, composed, session_handshake, none
Header string `yaml:"header" json:"header"`
Format string `yaml:"format" json:"format"`
EnvVars []string `yaml:"env_vars" json:"env_vars"`
@@ -51,6 +51,22 @@ type AuthConfig struct {
CookieDomain string `yaml:"cookie_domain,omitempty" json:"cookie_domain,omitempty"` // domain to read browser cookies from (e.g. ".notion.so")
Cookies []string `yaml:"cookies,omitempty" json:"cookies,omitempty"` // named cookies to extract for composed auth (e.g. ["customerId", "authToken"])
Inferred bool `yaml:"inferred,omitempty" json:"inferred,omitempty"` // true when auth was inferred from spec description, not declared in securitySchemes
+
+ // Session-handshake fields. Used only when Type == "session_handshake".
+ // The pattern: GET BootstrapURL to seed cookies → GET TokenURL to receive
+ // an anti-CSRF token (the "crumb" on Yahoo Finance, similarly named on
+ // Walmart, some streaming APIs, Facebook's internal graph) → pass that
+ // token on every subsequent data request as TokenParamName in TokenParamIn.
+ // The generator emits a cookie jar, disk-persisted session file, and auto-
+ // invalidation on InvalidateOnStatus responses.
+ BootstrapURL string `yaml:"bootstrap_url,omitempty" json:"bootstrap_url,omitempty"` // optional GET to seed cookies before token fetch (e.g. "https://fc.yahoo.com/")
+ SessionTokenURL string `yaml:"session_token_url,omitempty" json:"session_token_url,omitempty"` // endpoint that returns the token (e.g. "https://query2.finance.yahoo.com/v1/test/getcrumb"); distinct from TokenURL (OAuth) to avoid conflation
+ TokenFormat string `yaml:"token_format,omitempty" json:"token_format,omitempty"` // "text" (raw body) or "json" (extract via TokenJSONPath); default "text"
+ TokenJSONPath string `yaml:"token_json_path,omitempty" json:"token_json_path,omitempty"` // when TokenFormat is "json", dot-path to the token field (e.g. "data.crumb")
+ TokenParamName string `yaml:"token_param_name,omitempty" json:"token_param_name,omitempty"` // parameter name to attach to requests (e.g. "crumb")
+ TokenParamIn string `yaml:"token_param_in,omitempty" json:"token_param_in,omitempty"` // "query" or "header"; default "query"
+ InvalidateOnStatus []int `yaml:"invalidate_on_status,omitempty" json:"invalidate_on_status,omitempty"` // HTTP status codes that should invalidate the cached token and re-bootstrap (e.g. [401, 403])
+ SessionTTLHours int `yaml:"session_ttl_hours,omitempty" json:"session_ttl_hours,omitempty"` // how long to trust a cached session (default 24)
}
type ConfigSpec struct {
diff --git a/release-please-config.json b/release-please-config.json
index af15b7cc..82efdd96 100644
--- a/release-please-config.json
+++ b/release-please-config.json
@@ -10,11 +10,6 @@
"path": ".claude-plugin/plugin.json",
"jsonpath": "$.version"
},
- {
- "type": "json",
- "path": ".claude-plugin/marketplace.json",
- "jsonpath": "$.plugins[0].version"
- },
{
"type": "generic",
"path": "internal/version/version.go"
diff --git a/testdata/session-handshake.yaml b/testdata/session-handshake.yaml
new file mode 100644
index 00000000..65c97216
--- /dev/null
+++ b/testdata/session-handshake.yaml
@@ -0,0 +1,37 @@
+name: yahoofin-demo
+description: "Demo spec for session_handshake auth type"
+version: "1.0.0"
+base_url: "https://query1.finance.yahoo.com"
+
+auth:
+ type: session_handshake
+ bootstrap_url: "https://fc.yahoo.com/"
+ session_token_url: "https://query2.finance.yahoo.com/v1/test/getcrumb"
+ token_format: text
+ token_param_name: crumb
+ token_param_in: query
+ in: query
+ header: crumb
+ invalidate_on_status: [401, 403]
+ session_ttl_hours: 24
+
+config:
+ format: toml
+ path: "~/.config/yahoofin-demo-pp-cli/config.toml"
+
+resources:
+ quote:
+ description: "Quotes"
+ endpoints:
+ list:
+ method: GET
+ path: "/v7/finance/quote"
+ description: "Get quotes for symbols"
+ params:
+ - name: symbols
+ type: string
+ required: true
+ description: "Comma-separated ticker symbols"
+ response:
+ type: array
+ item: Quote
← cd93b172 fix(cli)!: decouple printing-press-library into standalone m
·
back to Cli Printing Press
·
fix(skills): add combo-CLI priority gate to prevent source i 23ccc603 →