← back to Cli Printing Press
fix(cli): Cal.com retro #334 — 4 of 5 P1 machine fixes (per-endpoint headers, novel_features, auth set-token, PII scrub) (#341)
c74dcd458c94640c6688f9f4fc5fb15a8af5bb2f · 2026-04-27 10:09:01 -0700 · Trevin Chow
* fix(cli): publish manifest preserves novel_features (cal-com retro #334 U2)
writeCLIManifestForPublish silently shipped manifests with empty
novel_features when run from the printing-press skill flow. Two
compounding bugs caused this:
1. LoadResearch lookup mismatch: the call passed state.PipelineDir()
which resolves to <RunRoot>/pipeline/research.json, but the skill
writes research.json to <RunRoot>/research.json directly. The
pipeline/ subdir is reserved for printing-press print's phase
artifacts. File-not-found was silent, novel_features stayed empty.
2. Carry-forward block at publish.go:208-219 preserved MCP and Auth
fields from the existing manifest but did NOT preserve
existing.NovelFeatures. So even when generate populated novel
features at codegen time, publish stripped them.
Fix is two-part defense in depth:
- internal/pipeline/research.go: new loadResearchForState helper tries
RunRoot/research.json first (skill convention, dominant case), falls
back to PipelineDir/research.json (print convention).
- internal/pipeline/publish.go: carry-forward block now preserves
existing.NovelFeatures. The research-loaded values still override
carry-forward when present, because post-dogfood data is the source
of truth.
Six tests cover the matrix: skill-flow research, print-flow research,
carry-forward only, research-overrides-carry-forward, empty-research-
keeps-carry-forward, and the genuinely-empty case.
Refs #334 (WU-2)
* fix(cli): auth set-token clears legacy auth_header (cal-com retro #334 U4)
When config.toml had a populated `auth_header` field — the common case
after regenerating a CLI that was previously generated by an older
version — `auth set-token <new-token>` saved the new token to
`access_token` but had no effect on the active credential. Subsequent
API calls returned 401 because Config.AuthHeader() reads `auth_header`
first and never falls through to `Bearer + access_token`.
Fix is one line in the set-token handler: clear `cfg.AuthHeaderVal`
before SaveTokens so the resolver correctly returns the new bearer
token. Silent clear (no log line) — a masked-tail variant could leak
token bytes through scripted dogfood that captures stderr.
Targeted internal/generator/templates/auth_simple.go.tmpl exclusively.
auth.go.tmpl and auth_browser.go.tmpl don't define set-token (OAuth and
cookie flows respectively).
Generator unit test asserts the cleared-line is emitted in the rendered
auth.go AND that it appears before the SaveTokens call. Golden fixture
updated to reflect the new template output (8-line addition, no other
changes).
Refs #334 (WU-4)
* fix(cli): thread per-endpoint headers through resolveRead path (cal-com retro #334 U1)
Per-endpoint required headers (e.g. cal-api-version, Stripe-Version) were
silently dropped on store-backed reads. The template built the
headerOverrides map correctly per endpoint, but only wired it into the
non-store branch (c.GetWithHeaders). The store branch called
resolveRead(c, flags, ..., path, params) which never accepted headers
and forwarded c.Get(path, params) — header-less.
Result: cal-com bookings get returned the v1-fallback empty array
(HTTP 200, empty data) silently. Same shape would hit Stripe and
GitHub on store-backed reads. Issue #135 (CLOSED) supposedly fixed
this in the parser; the consumer was never updated.
Fix is option (a) from the plan: thread headers through. Picked over
option (b) (global path-prefix map in client.do()) because the parser
keys per-endpoint headers by template path (/v2/bookings/{uid}/cancel)
and the client sees substituted paths at runtime — longest-prefix
matching can't bridge {uid} vs UID-123. Option (a) keeps the headers
baked in at codegen time per endpoint as a literal map; nothing to look
up at runtime.
Signature changes:
- paginatedGet (helpers.go.tmpl): new headers parameter; interface
requires GetWithHeaders instead of Get
- resolveRead, resolvePaginatedRead (data_source.go.tmpl): new headers
parameter; forwarded to c.GetWithHeaders / paginatedGet
Caller updates:
- command_endpoint.go.tmpl (3 sites): pass headerOverrides when
Endpoint.HeaderOverrides exists, else nil
- command_promoted.go.tmpl (3 sites): pass nil — promoted commands
don't currently carry per-endpoint header overrides
Doctor's root-path probe (c.Get("/", nil), added in PR #335) is
unaffected: it doesn't go through resolveRead, and the root path isn't
in any Endpoint.HeaderOverrides anyway.
Golden fixtures updated for the signature additions (4 files, ~11 line
changes total). Full test suite passes; harness verify clean for all 6
golden cases.
Refs #334 (WU-1), regression of #135
* fix(skills): PII pattern scanning before publish (cal-com retro #334 U3)
Two prior PII leaks shipped to the public printing-press-library repo
because the existing scrub model only catches API key VALUES (Layer 1
exact-value scan) and HAR auth headers. Real attendee names, emails,
account usernames, and other PII captured during live dogfood landed
in published manuscripts. Prose guidance ("Workspace & organization
PII redaction" section) failed twice in practice because it requires
the agent to remember and follow it during a long publish run.
Adds a mechanical defense layer to skills/printing-press/references/
secret-protection.md — a new "PII pattern scanning" section between
the prose redaction guidance and session-state cleanup. Two-tier
design with deliberately-asymmetric policies:
Tier 1 (auto-redact silently): vendor-prefix-anchored bearer tokens
(Bearer cal_live_*, sk_live_*, ghp_*, xoxp-*, etc.). Pattern set has
near-zero false-positive rate — vendor prefixes plus restrictive
character classes make accidental matches structurally implausible.
Tier 2 (warn with batched user prompt): generic emails, generic bearer
tokens (no vendor prefix), capitalized first+last name patterns. Cost
asymmetry inverted from the original design draft: false-positive
auto-redaction permanently corrupts a manuscript artifact, while
false-positive warning costs only a prompt. Warn is the safer default
for any pattern with non-trivial false-positive risk.
Allowlist (suppresses Tier 2 matches) is built at scan time from three
sources: spec operation summaries / tag names / parameter descriptions
(catches "Event Types", "Booking Links" automatically), the printed
CLI's --help tree (catches command group names), and a small static
list of universal terms. Spec-derived suppression scales naturally with
new APIs; no hand-curated allowlist to maintain.
File scope: content-sniff (NULL-byte detection) all text-readable
files, not extension-list. A future .yaml proof or .har variant gets
swept regardless.
Pre-scrub copy preserved at <staging>.pre-pii-scrub/ as a recovery
checkpoint if a redaction was wrong.
skills/printing-press-publish/SKILL.md cross-links to the new section
in its existing "Secret & PII Protection" block, marked mandatory.
Refs #334 (WU-3)
Files touched
A docs/plans/2026-04-27-001-fix-printing-press-p1-machine-fixes-from-cal-com-retro-plan.mdM internal/generator/auth_optional_test.goM internal/generator/templates/auth_simple.go.tmplM internal/generator/templates/command_endpoint.go.tmplM internal/generator/templates/command_promoted.go.tmplM internal/generator/templates/data_source.go.tmplM internal/generator/templates/helpers.go.tmplM internal/pipeline/publish.goM internal/pipeline/publish_test.goM internal/pipeline/research.goM skills/printing-press-publish/SKILL.mdM skills/printing-press/references/secret-protection.mdM testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/auth.goM testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/helpers.goM testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_list.goM testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_tasks_list-project.goM testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/promoted_public.go
Diff
commit c74dcd458c94640c6688f9f4fc5fb15a8af5bb2f
Author: Trevin Chow <trevin@trevinchow.com>
Date: Mon Apr 27 10:09:01 2026 -0700
fix(cli): Cal.com retro #334 — 4 of 5 P1 machine fixes (per-endpoint headers, novel_features, auth set-token, PII scrub) (#341)
* fix(cli): publish manifest preserves novel_features (cal-com retro #334 U2)
writeCLIManifestForPublish silently shipped manifests with empty
novel_features when run from the printing-press skill flow. Two
compounding bugs caused this:
1. LoadResearch lookup mismatch: the call passed state.PipelineDir()
which resolves to <RunRoot>/pipeline/research.json, but the skill
writes research.json to <RunRoot>/research.json directly. The
pipeline/ subdir is reserved for printing-press print's phase
artifacts. File-not-found was silent, novel_features stayed empty.
2. Carry-forward block at publish.go:208-219 preserved MCP and Auth
fields from the existing manifest but did NOT preserve
existing.NovelFeatures. So even when generate populated novel
features at codegen time, publish stripped them.
Fix is two-part defense in depth:
- internal/pipeline/research.go: new loadResearchForState helper tries
RunRoot/research.json first (skill convention, dominant case), falls
back to PipelineDir/research.json (print convention).
- internal/pipeline/publish.go: carry-forward block now preserves
existing.NovelFeatures. The research-loaded values still override
carry-forward when present, because post-dogfood data is the source
of truth.
Six tests cover the matrix: skill-flow research, print-flow research,
carry-forward only, research-overrides-carry-forward, empty-research-
keeps-carry-forward, and the genuinely-empty case.
Refs #334 (WU-2)
* fix(cli): auth set-token clears legacy auth_header (cal-com retro #334 U4)
When config.toml had a populated `auth_header` field — the common case
after regenerating a CLI that was previously generated by an older
version — `auth set-token <new-token>` saved the new token to
`access_token` but had no effect on the active credential. Subsequent
API calls returned 401 because Config.AuthHeader() reads `auth_header`
first and never falls through to `Bearer + access_token`.
Fix is one line in the set-token handler: clear `cfg.AuthHeaderVal`
before SaveTokens so the resolver correctly returns the new bearer
token. Silent clear (no log line) — a masked-tail variant could leak
token bytes through scripted dogfood that captures stderr.
Targeted internal/generator/templates/auth_simple.go.tmpl exclusively.
auth.go.tmpl and auth_browser.go.tmpl don't define set-token (OAuth and
cookie flows respectively).
Generator unit test asserts the cleared-line is emitted in the rendered
auth.go AND that it appears before the SaveTokens call. Golden fixture
updated to reflect the new template output (8-line addition, no other
changes).
Refs #334 (WU-4)
* fix(cli): thread per-endpoint headers through resolveRead path (cal-com retro #334 U1)
Per-endpoint required headers (e.g. cal-api-version, Stripe-Version) were
silently dropped on store-backed reads. The template built the
headerOverrides map correctly per endpoint, but only wired it into the
non-store branch (c.GetWithHeaders). The store branch called
resolveRead(c, flags, ..., path, params) which never accepted headers
and forwarded c.Get(path, params) — header-less.
Result: cal-com bookings get returned the v1-fallback empty array
(HTTP 200, empty data) silently. Same shape would hit Stripe and
GitHub on store-backed reads. Issue #135 (CLOSED) supposedly fixed
this in the parser; the consumer was never updated.
Fix is option (a) from the plan: thread headers through. Picked over
option (b) (global path-prefix map in client.do()) because the parser
keys per-endpoint headers by template path (/v2/bookings/{uid}/cancel)
and the client sees substituted paths at runtime — longest-prefix
matching can't bridge {uid} vs UID-123. Option (a) keeps the headers
baked in at codegen time per endpoint as a literal map; nothing to look
up at runtime.
Signature changes:
- paginatedGet (helpers.go.tmpl): new headers parameter; interface
requires GetWithHeaders instead of Get
- resolveRead, resolvePaginatedRead (data_source.go.tmpl): new headers
parameter; forwarded to c.GetWithHeaders / paginatedGet
Caller updates:
- command_endpoint.go.tmpl (3 sites): pass headerOverrides when
Endpoint.HeaderOverrides exists, else nil
- command_promoted.go.tmpl (3 sites): pass nil — promoted commands
don't currently carry per-endpoint header overrides
Doctor's root-path probe (c.Get("/", nil), added in PR #335) is
unaffected: it doesn't go through resolveRead, and the root path isn't
in any Endpoint.HeaderOverrides anyway.
Golden fixtures updated for the signature additions (4 files, ~11 line
changes total). Full test suite passes; harness verify clean for all 6
golden cases.
Refs #334 (WU-1), regression of #135
* fix(skills): PII pattern scanning before publish (cal-com retro #334 U3)
Two prior PII leaks shipped to the public printing-press-library repo
because the existing scrub model only catches API key VALUES (Layer 1
exact-value scan) and HAR auth headers. Real attendee names, emails,
account usernames, and other PII captured during live dogfood landed
in published manuscripts. Prose guidance ("Workspace & organization
PII redaction" section) failed twice in practice because it requires
the agent to remember and follow it during a long publish run.
Adds a mechanical defense layer to skills/printing-press/references/
secret-protection.md — a new "PII pattern scanning" section between
the prose redaction guidance and session-state cleanup. Two-tier
design with deliberately-asymmetric policies:
Tier 1 (auto-redact silently): vendor-prefix-anchored bearer tokens
(Bearer cal_live_*, sk_live_*, ghp_*, xoxp-*, etc.). Pattern set has
near-zero false-positive rate — vendor prefixes plus restrictive
character classes make accidental matches structurally implausible.
Tier 2 (warn with batched user prompt): generic emails, generic bearer
tokens (no vendor prefix), capitalized first+last name patterns. Cost
asymmetry inverted from the original design draft: false-positive
auto-redaction permanently corrupts a manuscript artifact, while
false-positive warning costs only a prompt. Warn is the safer default
for any pattern with non-trivial false-positive risk.
Allowlist (suppresses Tier 2 matches) is built at scan time from three
sources: spec operation summaries / tag names / parameter descriptions
(catches "Event Types", "Booking Links" automatically), the printed
CLI's --help tree (catches command group names), and a small static
list of universal terms. Spec-derived suppression scales naturally with
new APIs; no hand-curated allowlist to maintain.
File scope: content-sniff (NULL-byte detection) all text-readable
files, not extension-list. A future .yaml proof or .har variant gets
swept regardless.
Pre-scrub copy preserved at <staging>.pre-pii-scrub/ as a recovery
checkpoint if a redaction was wrong.
skills/printing-press-publish/SKILL.md cross-links to the new section
in its existing "Secret & PII Protection" block, marked mandatory.
Refs #334 (WU-3)
---
...ess-p1-machine-fixes-from-cal-com-retro-plan.md | 403 +++++++++++++++++++++
internal/generator/auth_optional_test.go | 42 +++
internal/generator/templates/auth_simple.go.tmpl | 8 +
.../generator/templates/command_endpoint.go.tmpl | 6 +-
.../generator/templates/command_promoted.go.tmpl | 6 +-
internal/generator/templates/data_source.go.tmpl | 21 +-
internal/generator/templates/helpers.go.tmpl | 13 +-
internal/pipeline/publish.go | 43 ++-
internal/pipeline/publish_test.go | 197 ++++++++++
internal/pipeline/research.go | 17 +
skills/printing-press-publish/SKILL.md | 23 ++
.../printing-press/references/secret-protection.md | 164 +++++++++
.../printing-press-golden/internal/cli/auth.go | 8 +
.../printing-press-golden/internal/cli/helpers.go | 13 +-
.../internal/cli/projects_list.go | 2 +-
.../internal/cli/projects_tasks_list-project.go | 2 +-
.../internal/cli/promoted_public.go | 2 +-
17 files changed, 929 insertions(+), 41 deletions(-)
diff --git a/docs/plans/2026-04-27-001-fix-printing-press-p1-machine-fixes-from-cal-com-retro-plan.md b/docs/plans/2026-04-27-001-fix-printing-press-p1-machine-fixes-from-cal-com-retro-plan.md
new file mode 100644
index 00000000..1f4952e2
--- /dev/null
+++ b/docs/plans/2026-04-27-001-fix-printing-press-p1-machine-fixes-from-cal-com-retro-plan.md
@@ -0,0 +1,403 @@
+---
+title: "fix: Printing Press P1 machine fixes from Cal.com retro #334"
+type: fix
+status: active
+date: 2026-04-27
+origin: https://github.com/mvanhorn/cli-printing-press/issues/334
+deepened: 2026-04-27
+---
+
+# fix: Printing Press P1 machine fixes from Cal.com retro #334
+
+## Overview
+
+Cal.com Run 3 surfaced 12 findings; ten landed in the Do bucket. WU-10 was already fixed by PR #332 ahead of this plan. This plan covers the remaining five P1 work units (WU-1 through WU-5).
+
+This plan was rewritten after document review surfaced three factual errors in the first draft (WU-2 misdiagnosed the root cause, WU-4 named a template that doesn't have the relevant handler, WU-1 picked a design that can't actually look up templated paths at runtime). The diagnostic work is recorded in **Key Technical Decisions** so the implementer doesn't have to re-derive it.
+
+The five units touch separate subsystems and can ship in any order, but three of them (U1, U4, U5) modify generator templates that the still-in-flight allrecipes-retro plan (`docs/plans/2026-04-26-002-feat-printing-press-p1-machine-fixes-plan.md`) is also touching — the dependency table calls out the conflict surface so reviewers can sequence merges.
+
+## Problem Frame
+
+The Cal.com regenerate against the live API hit five distinct machine defects that required hand-patching before the printed CLI worked:
+
+1. `bookings get --data-source live` returned `{"bookings":[],"totalCount":0}` despite 5 real bookings on the account. The `cal-api-version: 2024-08-13` header never landed on the wire because the store-backed read path (`{{- if .HasStore}}`) calls `resolveRead(c, flags, ..., path, params)` which has no `headers` parameter, while only the non-store branch wires up `c.GetWithHeaders(path, params, headerOverrides)`. The template builds the `headerOverrides` map correctly per-endpoint at codegen time; only the store-side caller drops it.
+2. `printing-press publish validate` failed the transcendence check ("no novel features recorded") on every CLI through publish even when `research.json` had 12 verified `novel_features_built`. Diagnosed during plan review: `writeCLIManifestForPublish` calls `LoadResearch(state.PipelineDir())` which reads `<RunRoot>/pipeline/research.json` — but the printing-press skill writes research.json to `<RunRoot>/research.json` directly (the `pipeline/` subdirectory is reserved for `printing-press print`'s phase artifacts, not the skill flow). The file-not-found is silent. Compounding this, the carry-forward block at `publish.go:208-219` preserves MCP and Auth fields from the existing manifest but does NOT preserve `existing.NovelFeatures`, so even when generate populated novel_features at codegen time, publish strips them.
+3. Five proof files contained real attendee names, emails, and the account username. Library is public — would have shipped real PII for the second time (PR #52 likely had the same issue and went unnoticed). Existing scrub catches API key VALUES via "Exact-value scan" plus inline prose "Workspace & organization PII redaction" guidance, but the prose path requires a human to follow it during run-time and has now demonstrably failed twice.
+4. `auth set-token <token>` silently had no effect when `config.toml` already had `auth_header` populated (the common regen scenario). API calls returned 401 because `Config.AuthHeader()` reads `auth_header` first. Diagnosed during plan review: the set-token handler lives ONLY in `auth_simple.go.tmpl` (`auth.go.tmpl` is the OAuth template — no set-token there).
+5. `sync --full` ingested only 5 of 11 Cal.com resources. Errors are silent (sync exits 0 with summary errors). Failed resources have row count 0 in the local store. Cal.com uses 4 distinct envelope shapes; the runtime extractor in the generated CLI knows only `data.<resource>: [...]`. The OpenAPI spec already contains response-schema info that could drive per-resource extraction, but the parser doesn't surface it onto `profiler.SyncableResource` and the sync template doesn't read it.
+
+Each failure mode is silent enough that several survived prior cal-com runs (including PR #52). The retro elevates them to P1 because they affect generation reliability across the full catalog, not Cal.com alone.
+
+## Requirements Trace
+
+- R1. Per-endpoint required headers detected by the OpenAPI parser must be sent on every request, including store-backed reads. Verified end-to-end against Cal.com bookings (covers U1, fixes regression of #135).
+- R2. `lock promote` must produce a `.printing-press.json` with `novel_features` populated whenever `research.json` has `novel_features_built` populated, regardless of where research.json lives within the run directory (covers U2).
+- R3. The publish skill must scrub PII captured during dogfood — emails, bearer-token tails, capitalized name patterns — before uploading to a public repository. The scrub must default to safe behavior (warn) over information-loss behavior (auto-redact) for ambiguous patterns (covers U3).
+- R4. `auth set-token <token>` must change the active credential. Subsequent API calls must use the new token, regardless of whether `config.toml` had a legacy `auth_header` field. The fix must not regress golden harness fixtures (covers U4).
+- R5. `sync --full` must ingest all envelope shapes the spec declares (`data:[]`, `data:<object>`, `data:<key>:[...]`, nested grouped shapes), not only `data.<resource>: [...]`. Falls back to current heuristic when the spec is genuinely ambiguous, with a warning (covers U5).
+
+**Origin:** [Issue #334](https://github.com/mvanhorn/cli-printing-press/issues/334) and [retro document](https://files.catbox.moe/itg03h.md). Local copy at `/tmp/printing-press/retro/20260427-005908-cal-com-retro.md`.
+
+## Scope Boundaries
+
+- This plan does NOT address WU-6 (operationId-derived path components), WU-7 (body-field --json flags), WU-8 (publish package run_id selection), or WU-9 (legacy NOT NULL migration). Those are P2 items in the retro and warrant separate plans.
+- This plan does NOT touch WU-10 (auth_protocol scorer) — already fixed by PR #332.
+- This plan does NOT redesign the storage or sync architecture. Envelope handling adds spec-driven extraction without changing how rows are persisted or how the resources table is keyed.
+- This plan does NOT introduce a new MCP server, new CLI subcommand, or change the publish PR template beyond the PII scrub.
+- This plan does NOT add generic "silent-failure detection" to dogfood (the meta-fix that would catch the next WU-1-shaped bug — see Deferred to Follow-Up Work). Surfacing this honestly because the retro called it out as the higher-leverage move.
+
+### Deferred to Follow-Up Work
+
+- **Dogfood silent-success detection** (e.g., flag HTTP 200 + empty list when the dogfood matrix or research.json declares the account has N>0 of the resource). Would have caught U1 and U5 BEFORE Cal.com Run 3 generated this retro. Separate plan, separate PR.
+- **Manifest-merge helper that walks struct tags** to carry non-empty fields forward by default in `writeCLIManifestForPublish`. Eliminates the class of bug where adding a new manifest field requires manual sync between two write sites. The two-write-site architecture is the deeper root cause of the U2 class — fixing it generally is out of scope here; U2 patches the specific symptom.
+
+## Context & Research
+
+### Relevant Code and Patterns
+
+- `internal/generator/templates/command_endpoint.go.tmpl` lines 148-204: builds `headerOverrides := map[string]string{ ... }` correctly per-endpoint at codegen time, then forks on `{{- if .HasStore}}`. The non-store branch correctly passes `headerOverrides` to `c.GetWithHeaders`. The store branch calls `resolveRead(c, flags, ..., path, params)` and drops the map. U1 fix lives here and in `data_source.go.tmpl`.
+- `internal/generator/templates/data_source.go.tmpl` line 88: `resolveRead(c *client.Client, flags *rootFlags, resourceType string, isList bool, path string, params map[string]string) (json.RawMessage, DataProvenance, error)`. Same gap on `resolvePaginatedRead`. The signature change is the U1 mechanism.
+- `internal/generator/templates/client.go.tmpl` lines 247-457: `c.Get` → `c.GetWithHeaders(path, params, nil)` → `do(method, path, params, body, headerOverrides)`. The merge-into-request loop already handles caller-supplied headers correctly (lines 453-459). Once `resolveRead` accepts and forwards a headers parameter, the existing template branches on lines 199-201 stay byte-identical.
+- `internal/openapi/parser.go` lines 482-656: `detectRequiredHeaders` returns `(globalRequired []spec.RequiredHeader, perEndpointHeaders map[string]map[string]string)`. `applyHeaderOverrides` (line 620) distributes per-endpoint values onto each `Endpoint.HeaderOverrides`. The per-endpoint override is ALREADY available at codegen time as a literal map in the template — no new parser data is needed for U1.
+- `internal/pipeline/publish.go` lines 195-275 (`writeCLIManifestForPublish`): the carry-forward block (lines 207-219) preserves MCP and Auth fields but NOT NovelFeatures. The LoadResearch call (lines 265-274) tries `<RunRoot>/pipeline/research.json` via `state.PipelineDir()`. U2 fix lives here.
+- `internal/pipeline/research.go` line 259: `func LoadResearch(pipelineDir string) (*ResearchResult, error)` reads `<dir>/research.json`. Caller passes the wrong directory; U2 fixes the resolution.
+- `internal/pipeline/paths.go` line 92: `RunPipelineDir(runID)` returns `<RunRoot>/pipeline/`. The skill writes research.json to `<RunRoot>/research.json` (the run root itself, not the pipeline subdir). Two parallel conventions; U2 must reconcile.
+- `internal/pipeline/climanifest.go` line 47-65: `CLIManifest.NovelFeatures []NovelFeatureManifest` field already exists with `omitempty`. Marshal logic is fine; only the population path is missing for the publish/promote case.
+- `internal/generator/templates/auth_simple.go.tmpl` lines 31, 57, 69-90: `newAuthSetTokenCmd` lives here. `auth.go.tmpl` is OAuth-only — no set-token. U4 must target `auth_simple.go.tmpl`.
+- `internal/generator/templates/config.go.tmpl` lines 91-128 + 183-190: `Config.AuthHeader()` reads `AuthHeaderVal` first; `SaveTokens(...)` does not touch `AuthHeaderVal`. U4 needs to set `cfg.AuthHeaderVal = ""` before the SaveTokens call (or extend SaveTokens with a clear-legacy flag).
+- `skills/printing-press/references/secret-protection.md`: actual file structure is "Exact-value scan before archiving" / "Strip auth from HAR captures before archiving" / "API key handling during the run" / "Workspace & organization PII redaction" / "Session state cleanup". There are NO "Layer 1/Layer 2" headings — adding "Layer 3" would orphan the labels. U3 must restructure with new section headings or add a properly-named sibling.
+- `skills/printing-press-publish/SKILL.md` Step 6 is "Package", and the secret-handling content is in an unnumbered section near line 784 ("Secret & PII Protection"). U3's invocation site lives in that section, not Step 6.
+- `internal/profiler/profiler.go` lines 58-61: `SyncableResource struct { Name string; Path string }`. No envelope-shape field. U5 needs to add one and populate from parser output.
+- `internal/spec/spec.go`: `Endpoint.Response` exists (`*ResponseDef`) but the sync template never reads it (`grep` returns 0 hits). U5 must extend BOTH the data-flow (parser → spec.APISpec → profiler) AND the sync template's consumer code.
+- `internal/generator/templates/sync.go.tmpl` line 446 (`extractPageItems`) and line 352 (`db.UpsertBatch(resource, items)`): the runtime envelope walker. Heuristic falls back to "exactly one array key" when `data.<resource>` doesn't match. U5 emits per-resource extractor code that calls `UpsertBatch` directly with shape-specific extraction.
+- `internal/generator/templates/graphql_sync.go.tmpl` is the GraphQL sync template (separate from `sync.go.tmpl`). GraphQL specs use `{data: {<query-name>: [...]}}` shape — a fifth pattern. U5 either includes GraphQL or explicitly defers it.
+
+### Institutional Learnings
+
+- **Issue #135 closed prematurely** with the parser fix landed but the template consumer half-wired. WU-1 surfacing again as the same regression confirms the anti-pattern: "we fixed the data structure but not the consumer." Mitigation in this plan: each WU includes an explicit verification step that exercises the consumer end-to-end, not just the structural change.
+- **PR #335 (an hour ago)** modified `internal/generator/templates/doctor.go.tmpl` to make doctor probe through `flags.newClient()` and `c.Get/c.GetWithHeaders` — meaning doctor reachability now flows through `do()` and inherits U1's per-endpoint header injection. Confirm during U1 implementation that doctor's root-path probe (`c.Get("/", nil)`) correctly receives no per-path headers (its target is the root, not a versioned endpoint) — this is the right behavior; the API-version header should only fire for versioned paths.
+- **Silent failures slip past dogfood** (cal-com `bookings get` returned HTTP 200 with empty list). Four of the five bugs in this plan share this shape: structural OK, payload empty/wrong. Without dogfood improvements (deferred above), each future occurrence will still ship undetected.
+- **PII scrub already failed twice publicly** despite inline guidance in `secret-protection.md`. The "warn-and-rubber-stamp" failure mode after false-positive fatigue is the dominant risk — U3's design must make the safe choice the default and the unsafe choice the explicit override, not the other way around.
+
+### External References
+
+None required. The work is internal templates, parser, pipeline, and skill changes — no new dependencies, no protocol research.
+
+## Key Technical Decisions
+
+- **U1 design — option (a) thread headers through `resolveRead`, NOT option (b) global path-prefix map in `client.do()`.** The first draft of this plan recommended (b) on the grounds that it covers more call sites. Document review surfaced that (b) cannot work as designed: the parser keys per-endpoint headers by template path (`/v2/bookings/{uid}/cancel`), but at runtime the client sees substituted paths (`/v2/bookings/UID-123/cancel`). Longest-prefix matching against a literal `{uid}` placeholder will not match the substituted path. Option (a) sidesteps the entire path-matching problem because `command_endpoint.go.tmpl` already builds `headerOverrides` at codegen time per endpoint as a baked-in literal map — there is nothing to look up at runtime; the headers are already in scope at the call site. The template change is three lines (signature + caller); `resolveRead`/`resolvePaginatedRead`/`paginatedGet` get a `headers map[string]string` parameter that they forward to `c.GetWithHeaders`. Novel-feature commands using `c.Get` directly are not auto-injected — they're hand-written code where the author controls the call site, and explicit `c.GetWithHeaders` is the right pattern there.
+- **U2 fix — both the carry-forward block AND LoadResearch path resolution.** Defense in depth. The LoadResearch path mismatch (`<RunRoot>/pipeline/research.json` vs `<RunRoot>/research.json`) is the proximate cause; fix it by trying the run-root path first and falling back to the pipeline-dir path (or vice versa — order doesn't matter as long as both are checked). Independently, add `m.NovelFeatures = existing.NovelFeatures` to the carry-forward block so the publish-time rewrite never silently strips a populated field. Either fix alone closes the visible bug; both together close the class of bug where research.json convention drifts in the future.
+- **U3 inversion of auto-redact policy — warn for everything by default; auto-redact ONLY when matching vendor-prefix anchors.** The first draft auto-redacted emails (permissive pattern, irreversible mutation) and warned on names (constrained pattern, easy to rubber-stamp). Document review (adversarial F5) called this cost-asymmetry inverted. New policy: emails get a warn-with-suggested-redaction (user one-key-confirms in interactive mode); names get the same warn flow with allowlist suppression; only patterns with a vendor-specific prefix anchor (`Bearer cal_live_*`, `Bearer sk_live_*`, `Bearer ghp_*`, `xoxp-*`, etc.) get auto-redacted because the false-positive rate on those is near-zero. The interactive prompt gates on "any non-anchored finding" — so a user with no PII hits sees no prompt, a user with mixed findings sees one prompt with all of them.
+- **U3 allowlist — derived from spec content, not hand-curated.** Document review (security F1) called the hand-curated allowlist structurally incomplete. Instead: at scrub time, build the allowlist from the spec's operation summaries, tag names, parameter descriptions, and the printed CLI's command names (`<cli> --help` walked recursively). A capitalized phrase that appears in any of those is suppressed. This catches "Event Types", "Booking Links", "Webhook Triggers" automatically without manual maintenance, and grows naturally as new APIs ship.
+- **U3 file scope — content-sniff non-binary text files, not extension-list.** Document review (security F2) noted `.yaml`, HAR variants, and future formats would slip through an extension-list approach. Use `file --mime-type` (Unix `file` command) or a small Go helper to detect text/* files in the staged dir; sweep all of them.
+- **U4 target template — `auth_simple.go.tmpl` exclusively.** `auth.go.tmpl` (OAuth) has no set-token handler. The auth-template-selection logic in `internal/generator/generator.go` chooses among three templates based on auth type; only `auth_simple.go.tmpl` is in scope here. If a future OAuth template grows a set-token handler it can apply the same fix.
+- **U4 deprecation log — silent clear OR TTY-gated.** The first draft proposed a stderr log line including a masked token tail (`****<tail>`). Document review (security F3) noted the log line could leak tail bytes through proof captures (scripted dogfood often tees stderr). Two safe options: (1) silently clear `auth_header` with no log line — the user can run `doctor` to see the new auth source; (2) emit the log line only when `os.Stderr` is a TTY (`golang.org/x/term.IsTerminal(int(os.Stderr.Fd()))`). Picked option (1) because it's the simpler safe default and the user can always run doctor for visibility. If implementation reveals the silent clear is too surprising, switch to (2).
+- **U5 GraphQL coverage — included via `graphql_sync.go.tmpl` modification.** Document review (adversarial F4) noted GraphQL has its own sync template that the first draft missed. Including it here keeps the fix coherent (one PR, one merge) rather than splitting envelope-shape work across two plans. The GraphQL shape is `{data: {<query-name>: [...]}}` which fits the vocabulary as `wrapped_at_data:<query-name>` — same shape primitive, different key. Validation includes regenerating a GraphQL fixture (Linear or similar) alongside Cal.com.
+- **U5 shape vocabulary — `array_at_data`, `single_at_data`, `wrapped_at_data:<key>`, `nested:<group>,<list>`, `unknown`.** Five shapes. `wrapped_at_data:<key>` accepts arbitrary keys (covers Cal.com bookings, GraphQL queries, Stripe-with-pagination using `data.<key>` patterns). `nested:<group>,<list>` for two-level group shapes (Cal.com event-types). `unknown` falls back to current heuristic. Vocabulary correctness is validated against four specs during implementation: Cal.com (REST, all four shapes), Stripe (REST with pagination metadata), Linear (GraphQL), and a synthetic fixture. If validation surfaces a shape outside the vocabulary, the implementation can extend the enum — the fallback to `unknown` ensures existing CLIs never regress.
+- **No new flags or CLI surface.** All five units are internal mechanism fixes; the printed CLI's user-facing API is unchanged. Bounds regression risk.
+- **Tests live next to code under test.** Each unit's test file is in the same package as the change, following existing repo convention.
+
+## Open Questions
+
+### Resolved During Planning
+
+- *Should U1 use option (a) or (b)?* — Option (a). See Key Technical Decisions. Document review changed this answer from the first draft.
+- *What is the actual root cause of U2's visible bug?* — The LoadResearch path mismatch (skill writes to `<RunRoot>/research.json`, code reads `<RunRoot>/pipeline/research.json`) compounded with the carry-forward block dropping `existing.NovelFeatures`. Both must be fixed.
+- *Where does U4's set-token handler live?* — Only `auth_simple.go.tmpl`. `auth.go.tmpl` is OAuth-only.
+- *Does the existing `secret-protection.md` have a layered scrub model?* — No. U3 either restructures with explicit headings or adds a sibling section to "Workspace & organization PII redaction" without claiming a layer-numbering it doesn't have.
+- *Does WU-3's auto-redact email policy have an irreversibility risk?* — Yes. Inverted to warn-by-default; only vendor-prefix anchors auto-redact.
+- *Does U5 need to cover GraphQL?* — Yes, via `graphql_sync.go.tmpl`. Same vocabulary, parallel template change.
+
+### Deferred to Implementation
+
+- *U1: should the `headers` parameter be `nil`-default for callers that don't have headers?* — Probably yes (matches the existing `c.GetWithHeaders(path, params, nil)` pattern in the non-store branch). Confirm during implementation.
+- *U2: which lookup order — RunRoot first then pipeline/, or pipeline/ first then RunRoot?* — Probably RunRoot first (skill convention is the dominant case). Confirm by checking how many existing plan-driven CLIs in the catalog rely on the pipeline/ convention.
+- *U3: exact regex for "vendor-prefix anchored" auto-redact tier* — Start with: `Bearer (sk_live_|sk_test_|cal_live_|cal_test_|ghp_|gho_|xoxp-|xoxb-)[A-Za-z0-9_\-]{8,}`. Tune during implementation by sweeping the catalog for vendor patterns.
+- *U3: how to invoke `file --mime-type` portably (macOS `file` vs GNU `file`)?* — Probably a small Go helper that reads the first 512 bytes and checks for binary content (`bytes.IndexByte(data, 0) != -1`). Decide during implementation.
+- *U5: how granular should the response-shape representation be on `SyncableResource`?* — A struct with shape enum + optional path components. Exact field set confirmed when reviewing the four validation specs during implementation.
+- *U5: what does the parser do when a response schema is genuinely undocumented (a `application/json` response with no `schema`)?* — Records `unknown`; sync template falls through to current heuristic. Same as today's behavior for that subset.
+
+---
+
+## Implementation Units
+
+- U1. **Add `headers` parameter to `resolveRead`/`resolvePaginatedRead`/`paginatedGet`; thread per-endpoint `headerOverrides` through the store-backed read path**
+
+**Goal:** Restore per-endpoint header injection on store-backed reads. APIs like Cal.com, Stripe, GitHub send the correct API-version header on every request without hand-patching.
+
+**Requirements:** R1
+
+**Dependencies:** None.
+
+**Files:**
+- Modify: `internal/generator/templates/data_source.go.tmpl` — add `headers map[string]string` parameter to `resolveRead` and `resolvePaginatedRead`; forward to `c.GetWithHeaders`/`paginatedGet`.
+- Modify: `internal/generator/templates/sync.go.tmpl` — `paginatedGet` helper signature gets `headers` parameter (wire it from callers; end users use `c.GetWithHeaders`).
+- Modify: `internal/generator/templates/command_endpoint.go.tmpl` lines 196-201 — store-backed branches pass `headerOverrides` (which is already declared above on line 149 when `.Endpoint.HeaderOverrides` is truthy) or `nil` when there are no overrides.
+- Test: `internal/generator/generator_test.go` — golden test asserting that an endpoint with per-endpoint `cal-api-version` headers generates a CLI whose store-backed handler passes the headers map all the way through to `c.GetWithHeaders`.
+
+**Approach:**
+- The `headerOverrides` literal map is already in scope at the template emit site for endpoints with per-endpoint headers. The template currently only uses it on the non-store branch. The store branch ignores the variable, leaving it unused (Go compile error if `.Endpoint.HeaderOverrides` is set but the branch ignores the var — verify the template guards correctly).
+- After the change, the store branch becomes:
+ - With overrides: `data, prov, err := resolveRead(c, flags, "<resource>", isList, path, params, headerOverrides)`
+ - Without overrides: `data, prov, err := resolveRead(c, flags, "<resource>", isList, path, params, nil)`
+- `resolveRead` forwards: in the `case "live":` branch, change `c.Get(path, params)` to `c.GetWithHeaders(path, params, headers)`. In the `default: // "auto"` branch, same swap. The local cache fallback path (`resolveLocal`) doesn't make HTTP requests so it doesn't need headers.
+- `paginatedGet` similarly forwards. The cursor-loop iterations all use the same headers (the per-endpoint version doesn't change between pages).
+- Doctor's root-path reachability probe (`c.Get("/", nil)` per PR #335) doesn't go through `resolveRead`; it stays as-is. The probe SHOULD NOT receive a per-endpoint version header (the root path isn't a versioned endpoint).
+- Novel-feature commands using `c.Get` directly are unchanged — hand-written code uses `c.GetWithHeaders` explicitly when needed.
+
+**Patterns to follow:**
+- Existing `headerOverrides` declaration block in `command_endpoint.go.tmpl` lines 148-153.
+- Existing `c.GetWithHeaders(path, params, headerOverrides)` call in the non-store branch (line 197).
+
+**Test scenarios:**
+- Happy path: An OpenAPI endpoint with `parameters: [{name: "Stripe-Version", in: header, required: true, schema: {default: "2024-01-01"}}]` and a syncable resource generates a CLI whose store-backed `<resource> get` call places `Stripe-Version: 2024-01-01` on the wire. Verify via a test fake intercepting `do()`.
+- Happy path: A non-store-backed endpoint with the same per-endpoint header continues to work (existing path unchanged).
+- Edge case: An endpoint with NO per-endpoint headers and a syncable resource generates `resolveRead(..., nil)` — confirm via golden output that the `nil` is emitted rather than an empty map literal (cosmetic but matters for diff churn).
+- Edge case: An endpoint with per-endpoint headers AND pagination — `resolvePaginatedRead` and the underlying `paginatedGet` BOTH forward the headers; verify on a paginated cal-com endpoint (e.g., bookings get with `take`+pagination).
+- Integration: Regenerate cal-com fixture WITHOUT the hand-written `internal/client/calcom_versions.go` workaround. Confirm `bookings get --dry-run --json` shows `cal-api-version: 2024-08-13` in the request preview AND `bookings get --data-source live` returns real bookings against the live API.
+- Negative: An API with no per-endpoint headers (e.g., a no-version REST API) regenerates byte-identical to current output (golden harness).
+- Doctor probe: `cal-com-pp-cli doctor` continues to send the existing `RequiredHeaders` (global) but does NOT send `cal-api-version` on the root-path probe, because the root path isn't in `Endpoint.HeaderOverrides`.
+
+**Verification:**
+- `go test ./internal/generator/...` and `./internal/openapi/...` pass.
+- `scripts/golden.sh verify` passes (no fixture changes for non-versioned APIs).
+- Cal.com regenerate (separate validation) confirms `bookings get` returns real bookings without the workaround.
+
+---
+
+- U2. **Promote populates `novel_features` reliably: fix LoadResearch path resolution AND preserve `NovelFeatures` in the carry-forward block**
+
+**Goal:** `lock promote` writes `.printing-press.json` with `novel_features` whenever `research.json` has `novel_features_built` populated, regardless of whether research.json lives in `<RunRoot>/` (skill convention) or `<RunRoot>/pipeline/` (printing-press print convention).
+
+**Requirements:** R2
+
+**Dependencies:** None.
+
+**Files:**
+- Modify: `internal/pipeline/publish.go` — in the carry-forward block (lines 207-219), add `m.NovelFeatures = existing.NovelFeatures`. In the LoadResearch block (lines 265-274), try `state.PipelineDir()` first, fall back to `state.RunRoot()` (or vice versa — see Open Questions).
+- Modify: `internal/pipeline/state.go` — add a `RunRoot()` helper if it doesn't already exist (returns the parent of `PipelineDir()`).
+- Test: `internal/pipeline/publish_test.go` — happy paths for both research.json locations, plus the carry-forward-only path.
+
+**Approach:**
+- Carry-forward block change: one line. After the existing `m.AuthEnvVars = existing.AuthEnvVars`, add `m.NovelFeatures = existing.NovelFeatures`. This handles the case where generate populated novel_features (via `WriteManifestForGenerate`'s `p.NovelFeatures` path) and publish should preserve them.
+- LoadResearch resolution: refactor to a small helper `loadResearchForState(state *PipelineState) (*ResearchResult, error)` that tries `state.RunRoot()/research.json` first (skill-flow convention), then `state.PipelineDir()/research.json` (print-flow convention). If both fail, return the second error (more useful diagnostic for print-mode users).
+- Even with the carry-forward fix, the LoadResearch fix is necessary because plan-driven CLIs that DIDN'T go through `WriteManifestForGenerate` won't have novel_features in `existing.NovelFeatures` at all — they need the research.json read to work.
+- When `loadResearchForState` succeeds and `NovelFeaturesBuilt` is non-empty, OVERRIDE `m.NovelFeatures` with the loaded values (research.json is the source of truth post-dogfood). When it fails or `NovelFeaturesBuilt` is nil/empty, KEEP the carry-forward value.
+
+**Patterns to follow:**
+- Existing carry-forward block (lines 207-219) for MCPBinary/MCPToolCount — same pattern: read existing data, mutate manifest.
+- `internal/pipeline/dogfood.go` `checkNovelFeatures` for the research.json read pattern.
+
+**Test scenarios:**
+- Happy path A (skill flow): research.json at `<RunRoot>/research.json` with 12 entries in `novel_features_built`. Promote. Manifest has 12 novel_features.
+- Happy path B (print flow): research.json at `<RunRoot>/pipeline/research.json`. Promote. Manifest has 12 novel_features.
+- Happy path C (carry-forward): research.json missing entirely. The `existing.NovelFeatures` (from WriteManifestForGenerate's earlier write) has 12 entries. Promote. Manifest preserves the 12.
+- Edge case: research.json present but `novel_features_built` empty array. Carry-forward has values. Promote keeps the carry-forward values (don't overwrite with empty research data).
+- Edge case: research.json present AND existing manifest both have entries but different. Research wins (post-dogfood is authoritative).
+- Edge case: research.json malformed. Function logs warning, falls through to carry-forward. No panic.
+- Integration: After promote, run `publish validate`. Transcendence check returns PASS for happy paths A/B/C; FAIL with the existing "no novel features recorded" message when ALL sources are empty (true no-transcendence case).
+
+**Verification:**
+- `go test ./internal/pipeline/...` passes including new tests.
+- A regenerate + promote of cal-com (manual smoke) writes a manifest with 12 novel_features visible; `publish validate --dir ~/printing-press/library/cal-com` returns `transcendence: PASS` without manual editing.
+
+---
+
+- U3. **PII scrubber added to publish flow: warn-by-default with vendor-anchored auto-redact, spec-derived allowlist, content-sniffed file scope**
+
+**Goal:** Block PII captured during dogfood from leaking into the public library repo. Scrub fails safe (warn over auto-redact) so a regression in the regex set or allowlist cannot corrupt manuscripts irreversibly.
+
+**Requirements:** R3
+
+**Dependencies:** None.
+
+**Files:**
+- Modify: `skills/printing-press/references/secret-protection.md` — add a new section "PII pattern scanning" as a sibling to the existing "Workspace & organization PII redaction" section (no Layer-numbering rename, since the file doesn't currently use that scheme). Document the regex patterns, the warn-vs-auto-redact policy split, the spec-derived allowlist mechanism, and the content-sniff file scope.
+- Modify: `skills/printing-press-publish/SKILL.md` — in the unnumbered "Secret & PII Protection" section near line 784, add an invocation of the new PII scan after the existing scrub steps.
+- Test: this is a skill-prose change; testability is via worked examples documented inline in `secret-protection.md`. Document positive and negative examples for each regex pattern.
+
+**Approach:**
+- Two-tier scrub:
+ - **Auto-redact tier** (low false-positive, high precision): vendor-prefix-anchored bearer tokens. Patterns: `Bearer (sk_live_|sk_test_|cal_live_|cal_test_|ghp_|gho_|xoxp-|xoxb-|pat-|github_pat_)[A-Za-z0-9_\-]{8,}`. Replace with `Bearer <REDACTED:vendor-token>`. Single Python regex sub call, no user prompt — these patterns have near-zero false-positive rate.
+ - **Warn tier** (any false-positive risk): generic emails, capitalized name patterns, generic bearer tails (`Bearer [A-Za-z0-9._\-+/=]{20,}` not matching the vendor-anchored set). Each finding reported with file + line + matched text. User decides per finding via the platform's blocking question tool. Recovery is built-in: original staged copy is preserved at `/tmp/<staging>.pre-pii-scrub/` so the user can re-stage if they want a different scrub policy.
+- Allowlist (suppresses warn-tier findings): build at scrub time from:
+ - The spec's operation summaries, descriptions, tag names, parameter descriptions
+ - The printed CLI's command tree (`<cli> --help` walked recursively, capitalized two-word phrases extracted)
+ - A small static list of universal terms ("Open Source", "Pull Request", "Bearer Token", "API Key", "Access Token")
+ - Match a finding against the allowlist by exact string AND by case-insensitive contains; if either hits, suppress.
+- File scope: walk the staged dir, for each file detect text vs binary by reading the first 512 bytes and checking for null bytes (`bytes.IndexByte(data[:n], 0) != -1` → binary, skip). Sweep all text files regardless of extension.
+- Sweep ordering: existing exact-value scan → existing HAR strip → new PII scan (auto-redact tier first, then warn tier). Warn tier runs LAST so the user sees only the surviving findings after auto-redaction.
+
+**Patterns to follow:**
+- Existing exact-value scan in `secret-protection.md` lines 18-47 — same structure: bash heredoc + Python regex sub.
+- Existing "Workspace & organization PII redaction" section as a sibling reference for prose voice.
+- Document review's adversarial F5 cost-asymmetry insight informs the "warn-by-default for everything except vendor-anchored" policy.
+
+**Test scenarios:**
+- Happy path (auto-redact tier): Stage a CLI with `Bearer sk_live_abc123def456ghi789` in `proofs/foo.md`. Run publish. Pattern auto-redacts to `Bearer <REDACTED:vendor-token>`. No user prompt for this finding.
+- Happy path (warn tier): Stage a CLI with `henryopenclaw@gmail.com` in a proof file. Scrub flags it via warn tier with file + line. User picks "redact" → replaces with `<REDACTED:email>`. User picks "keep" → original preserved with audit-trail note.
+- Allowlist: Stage a CLI whose spec has "Event Types" in tag names. Scrub finds "Event Types" in a manifest description, matches against derived allowlist, suppresses the warning.
+- Allowlist: Stage a CLI with "Open Source" in README. Universal-list match suppresses.
+- Edge case: A proof with both `Bearer sk_live_abc...` AND `Henry Claw` AND `henryopenclaw@gmail.com`. Vendor token auto-redacted silently. Email and name reported via warn tier — one prompt with both findings, user decides each.
+- Edge case: Scrub finds nothing (clean staged dir). No prompt, no warning, publish proceeds.
+- Edge case: A `.yaml` file in `discovery/` contains an email. Content-sniff detects text, sweep includes it. Email flagged.
+- Recovery: User runs publish, sees warn-tier findings, picks "abort to review". Original staged copy is at `/tmp/<staging>.pre-pii-scrub/`. User inspects, modifies scrub policy if needed, re-runs publish.
+- Negative: After publish completes with all findings addressed, the staged dir contains `<REDACTED:...>` substitutions but no live PII or live secrets.
+
+**Verification:**
+- The scrub logic in `secret-protection.md` is testable via the documented worked examples — a sample proof file before and after scrubbing.
+- `skills/printing-press-publish/SKILL.md` references the new section in the right sequence.
+- Re-publish cal-com (manual smoke) — vendor tokens auto-redacted in proofs, emails/names warn-flagged, no surprise leaks.
+
+---
+
+- U4. **Fix `auth set-token` to clear legacy `auth_header` and write to `access_token` (target `auth_simple.go.tmpl` only; silent clear)**
+
+**Goal:** `auth set-token <token>` actually changes the active credential. Subsequent API calls use the new token regardless of whether `config.toml` had a legacy `auth_header` field.
+
+**Requirements:** R4
+
+**Dependencies:** None.
+
+**Files:**
+- Modify: `internal/generator/templates/auth_simple.go.tmpl` lines 69-90 (`newAuthSetTokenCmd` handler) — set `cfg.AuthHeaderVal = ""` before calling `cfg.SaveTokens(...)`. Silent clear (no stderr log).
+- Modify: `internal/generator/templates/config.go.tmpl` — no behavior change required. The existing `Config.AuthHeader()` resolver order is correct once `AuthHeaderVal` is cleared. Confirm by inspection.
+- Test: `internal/generator/generator_test.go` — golden test asserting the generated `auth.go`'s set-token handler clears `AuthHeaderVal` before save. Plus a golden-fixture verification pass to confirm the change doesn't break existing fixtures.
+
+**Approach:**
+- In the `newAuthSetTokenCmd` handler in `auth_simple.go.tmpl`, modify the existing call sequence:
+ ```
+ cfg.AuthHeaderVal = "" // clear legacy auth_header so AuthHeader() falls through to access_token
+ cfg.SaveTokens("", "", args[0], "", cfg.TokenExpiry)
+ ```
+- No deprecation log line (silent clear). The user can run `doctor` to see the new auth source. Document review (security F3) noted that any masked-token log line risks leaking tail bytes through scripted dogfood that captures stderr; silent clear is the safest default.
+- `Config.AuthHeader()` resolver order stays unchanged: `if AuthHeaderVal != "" return AuthHeaderVal; else return "Bearer " + AccessToken`. Once AuthHeaderVal is cleared, the new token wins via the fallback path.
+- Auth template selection in `internal/generator/generator.go` (`renderAuthFiles`) chooses `auth_simple.go.tmpl` for the default auth case; `auth.go.tmpl` is OAuth-only (no set-token) and `auth_browser.go.tmpl` covers cookie/composed/persisted-query (no set-token in scope). Only `auth_simple.go.tmpl` needs modification.
+
+**Patterns to follow:**
+- Existing `cfg.SaveTokens(...)` call in `auth_simple.go.tmpl`.
+- `Config.AuthHeader()` resolver order in `config.go.tmpl`.
+
+**Test scenarios:**
+- Happy path: `config.toml` has `auth_header = 'Bearer OLD'` and `access_token = ''`. Run `set-token NEW`. Assert post-call config: `auth_header = ''`, `access_token = 'NEW'`. Subsequent `Config.AuthHeader()` returns `Bearer NEW`. No stderr output (silent clear).
+- Happy path: `config.toml` has only `access_token = 'OLD'` (clean state). Run `set-token NEW`. Assert: `access_token = 'NEW'`, `auth_header` remains empty. Same code path, no special handling.
+- Edge case: `config.toml` has both `auth_header = 'Bearer OLD1'` AND `access_token = 'OLD2'`. Run `set-token NEW`. Assert: `auth_header = ''`, `access_token = 'NEW'`.
+- Edge case: User had set `auth_header` manually with a custom prefix like `'Token foo'`. After `set-token`, their custom prefix is cleared. Documented as expected — set-token is the canonical write path. User can re-set `auth_header` manually if they need a non-Bearer prefix.
+- Golden-fixture verification: Run `scripts/golden.sh verify`. If the fixture's expected `auth.go` golden differs (because the new template emits an additional line), update the golden in the same commit and verify the diff is exactly the `cfg.AuthHeaderVal = ""` line — no other unexpected changes.
+- Integration: After the generator change, regenerate cal-com fixture, run `cal-com-pp-cli auth set-token NEW; doctor`. Doctor reports auth from `access_token`. API call uses `Bearer NEW`.
+- Negative: An empty `set-token ""` returns the existing usage error and does NOT clear `auth_header`. Existing arg validation handles this; verify it still does.
+
+**Verification:**
+- `go test ./internal/generator/...` passes including new test.
+- `scripts/golden.sh verify` passes (with the documented fixture update for the new clear-line if needed).
+- Manual smoke: regenerate cal-com, run set-token + doctor + a real API call. Confirm the new token is in the Authorization header.
+
+---
+
+- U5. **Sync emits per-resource envelope extractors driven by spec response shape (REST + GraphQL)**
+
+**Goal:** `sync --full` ingests responses with `data:[]`, `data:<single-object>`, `data:<key>:[...]`, and nested grouped shapes correctly across both REST and GraphQL CLIs, not only `data.<resource>: [...]`. Failed extractions surface as actionable errors, not silent zero-row stores.
+
+**Requirements:** R5
+
+**Dependencies:** None.
+
+**Files:**
+- Modify: `internal/openapi/parser.go` — record per-resource response envelope shape during parsing. Extract from the response schema's `application/json` content.
+- Modify: `internal/graphql/parser.go` — record envelope shape for GraphQL responses (always `wrapped_at_data:<query-name>` for list queries).
+- Modify: `internal/spec/spec.go` — add a `ResponseEnvelope` field to the parsed resource representation (or extend `Endpoint` if the field belongs there).
+- Modify: `internal/profiler/profiler.go` — `SyncableResource` struct gets an `Envelope ResponseEnvelope` field; populated from the parser output.
+- Modify: `internal/generator/templates/sync.go.tmpl` — emit per-resource extractor logic driven by the recorded shape. Fall back to current heuristic (`extractPageItems`) when shape is `unknown`.
+- Modify: `internal/generator/templates/graphql_sync.go.tmpl` — same per-resource extractor pattern for GraphQL.
+- Test: `internal/openapi/parser_test.go` and `internal/graphql/parser_test.go` — assert recorded shapes for fixture specs covering all five shape variants.
+- Test: `internal/generator/generator_test.go` — golden test that the sync templates emit the right extractor for each shape variant.
+
+**Approach:**
+- Define the shape vocabulary as a small Go type:
+ ```
+ type ResponseEnvelope struct {
+ Shape ShapeKind // array_at_data, single_at_data, wrapped_at_data, nested, unknown
+ Keys []string // for wrapped_at_data: [single key]; for nested: [group, list]
+ }
+ ```
+- During OpenAPI parsing, for each list endpoint that's a sync candidate, walk the response schema:
+ - `data` is `type: array` → `ResponseEnvelope{Shape: ArrayAtData}`
+ - `data` is `type: object` with no obvious list field → `ResponseEnvelope{Shape: SingleAtData}`
+ - `data` is `type: object` with one `type: array` property `<key>` → `ResponseEnvelope{Shape: WrappedAtData, Keys: ["<key>"]}`
+ - `data` is `type: object` containing a `type: array` of objects each with one `type: array` field → `ResponseEnvelope{Shape: Nested, Keys: ["<group>", "<list>"]}`
+ - Otherwise → `ResponseEnvelope{Shape: Unknown}`
+- During GraphQL parsing, for each query selected as syncable, record `ResponseEnvelope{Shape: WrappedAtData, Keys: ["<query-name>"]}`.
+- The profiler propagates the envelope from each parsed `Endpoint.Response` onto the corresponding `SyncableResource`.
+- The sync templates branch on `.Envelope.Shape` to emit the right extractor:
+ - `ArrayAtData`: `var items []json.RawMessage; json.Unmarshal(env.Data, &items)`
+ - `SingleAtData`: `var item json.RawMessage; json.Unmarshal(env.Data, &item); items := []json.RawMessage{item}`
+ - `WrappedAtData`: `var w struct{ Items []json.RawMessage `json:"<key>"` }; json.Unmarshal(env.Data, &w); items := w.Items`
+ - `Nested`: walk `[<group>][i].<list>` and concatenate
+ - `Unknown`: emit the existing `extractPageItems` heuristic call (zero behavior change for current CLIs)
+- All five shape extractors fall through to `db.UpsertBatch(resource, items)` — only the upstream extraction differs.
+- Input validation on `Keys`: enforce that path components are valid JSON keys (no `.`, no `[`, no shell metacharacters) before baking them into emitted Go code. Document review (security top-3 #3) flagged that adversarial spec keys could otherwise compromise the emitted extractor.
+
+**Patterns to follow:**
+- The current heuristic walker (`extractPageItems` in `internal/generator/templates/sync.go.tmpl:446`) — its envelope-walking logic becomes the `Unknown` shape's fallback path.
+- `client.go.tmpl`'s template directives that consume parser output (`range .RequiredHeaders`) — same pattern: parser produces structured data, template walks it.
+- Existing GraphQL handling in `graphql_sync.go.tmpl` for the parallel template change.
+
+**Test scenarios:**
+- Happy path A (REST `wrapped_at_data:<resource>`): Fixture spec declares `data: {bookings: [...]}` for `/bookings`. Parser records `WrappedAtData, Keys=["bookings"]`. Generated sync extracts via that key. Run sync against a stub returning `{"data": {"bookings": [{"id":1},{"id":2}]}}`. Assert 2 rows.
+- Happy path B (REST `single_at_data`): Fixture spec declares `data: <object>` for `/me`. Parser records `SingleAtData`. Generated sync extracts the single object as a one-item slice. Assert 1 row.
+- Happy path C (REST `array_at_data`): Fixture spec declares `data: [...]` for `/teams`. Parser records `ArrayAtData`. Generated sync extracts the array directly. Assert N rows.
+- Happy path D (REST `nested`): Fixture spec declares the nested `eventTypeGroups[].eventTypes` shape for `/event-types`. Parser records `Nested, Keys=["eventTypeGroups", "eventTypes"]`. Generated sync walks groups, concatenates items. Assert sum.
+- Happy path E (GraphQL): GraphQL fixture with `query Bookings { bookings { id title } }`. Parser records `WrappedAtData, Keys=["bookings"]`. Generated GraphQL sync extracts via `data.bookings`. Assert N rows.
+- Edge case (unknown): Spec is genuinely ambiguous (response schema absent or doesn't match any pattern). Parser records `Unknown`. Generator emits `extractPageItems` fallback. Existing CLIs regenerate byte-identically.
+- Edge case (validation): A spec with a malicious key like `.;rm -rf` or `__proto__` — parser rejects (returns `Unknown` plus a warning), template emits the safe fallback. No injection into emitted Go code.
+- Integration: Regenerate cal-com fixture. Run `sync --full`. Assert all 11 resources sync without "missing id for X" errors. Assert sync_summary's `errored` count is 0 for resources whose envelope shape is now correctly handled.
+- Integration GraphQL: Regenerate Linear (or another GraphQL fixture). Run sync. Assert resources ingest correctly.
+- Negative: An API with only `data.<resource>: [...]` shape (current default) regenerates byte-identical to current output (golden harness).
+
+**Verification:**
+- `go test ./internal/openapi/...`, `./internal/graphql/...`, `./internal/profiler/...`, `./internal/generator/...` all pass.
+- `scripts/golden.sh verify` passes for non-Cal.com APIs (no fixture changes for currently-working shapes).
+- Cal.com regenerate (separate validation) syncs all 11 resources without errors.
+- Linear/GraphQL regenerate (separate validation) syncs without regression.
+
+## System-Wide Impact
+
+- **Interaction graph:** U1 changes `resolveRead`/`resolvePaginatedRead`/`paginatedGet` signatures — every store-backed handler across every printed CLI is regenerated. Test coverage must verify both store-backed and direct-API paths still work for unaffected APIs (golden harness handles non-Cal.com cases; Cal.com regen verifies the affected case). U5 changes the sync templates — every store-backed CLI regenerates; `Unknown` shape preserves current behavior so no regression.
+- **Error propagation:** U2 must not panic when `research.json` is missing or malformed. The path-resolution fix tries both locations; on miss, the carry-forward path takes over silently. U4 keeps existing input validation (empty token rejected) intact. U3's auto-redact runs without prompting; warn-tier findings present a single batched prompt — connection failure or interrupt aborts the publish, leaves the staged dir for review.
+- **State lifecycle risks:** U2 mutates the manifest after carry-forward; if `novel_features_built` is malformed (entries missing required fields), skip the bad entries rather than aborting. U4 clears `auth_header` — users with intentional custom prefixes will lose that override (acceptable per Key Technical Decisions). U3 preserves a pre-scrub copy at `/tmp/` for recovery; if the user aborts mid-prompt, that copy lets them re-stage.
+- **API surface parity:** U1's headers parameter applies to all HTTP verbs in the store-backed branch (`resolveRead` is GET-only by design; mutations always go through `c.GetWithHeaders`/`c.PostWithHeaders` directly with their per-call headers). Doctor's root-path probe correctly receives no per-endpoint headers because the root path isn't in `Endpoint.HeaderOverrides`.
+- **Integration coverage:** Per-unit golden tests cover the structural changes. End-to-end regenerate-and-test of cal-com (manually before merge) proves the integrated fixes work against the live API.
+- **Unchanged invariants:** The `CLIManifest` struct shape, the `Config.AuthHeader()` resolver order, the `do()` dispatch pattern (auth → required headers → header overrides), the existing `secret-protection.md` "Workspace & organization PII redaction" prose section, and the existing `extractPageItems` heuristic all stay in place. New behavior is additive in every unit. The `c.Get` and `c.GetWithHeaders` public surfaces are unchanged.
+- **Concurrent-PR risk surface:** This plan modifies `client.go.tmpl`/`data_source.go.tmpl`/`command_endpoint.go.tmpl`/`sync.go.tmpl`/`auth_simple.go.tmpl`/`graphql_sync.go.tmpl`/`publish.go`/`secret-protection.md`/`publish/SKILL.md`. The still-in-flight allrecipes-retro plan (`docs/plans/2026-04-26-002-feat-printing-press-p1-machine-fixes-plan.md`) modifies `doctor.go.tmpl`/`html_extract.go.tmpl`/`helpers.go.tmpl`/`scorecard.go`. No file overlap — but both plans touch the same generator surface. Coordinate merge order to avoid unnecessary regen churn.
+
+## Risks & Dependencies
+
+| Risk | Mitigation |
+|------|------------|
+| U1's `headers` parameter cascade could miss a less-common code path (e.g., a future cache-aware helper) and silently drop headers for that path. | Make `headers` non-optional in the function signatures (caller must pass `nil` or a map), so the Go compiler flags any new caller that forgot the parameter. |
+| U2's carry-forward path could over-populate manifests for plan-driven CLIs where research.json was intentionally absent. | The carry-forward path only fires when `existing.NovelFeatures` is non-empty; the LoadResearch path only fires when `NovelFeaturesBuilt` is non-empty. Plan-driven CLIs without either will keep `m.NovelFeatures` empty (current behavior preserved). |
+| U3's spec-derived allowlist could include sensitive terms (e.g., a spec description mentions a real customer name). | Allowlist is INPUTS to suppression, not outputs to publication. A spec that names "Acme Corp" causes "Acme Corp" findings to be suppressed in proofs — but the spec itself is a separate scrub target. The exact-value scan on the spec is unchanged; spec-derived allowlist only relaxes warn-tier suppression, not auto-redact. |
+| U3's content-sniff could miss text files with a leading null byte (e.g., UTF-16 BOM that starts with `\x00`). | Detect BOM-prefixed UTF encodings as text. Document the limitation in `secret-protection.md` and recommend manual review for unusual encodings. |
+| U4's silent clear could surprise users who rely on the deprecation log line for visibility. | Document in the publish-time release notes; users can check `cfg.AuthHeaderVal` via `doctor --json` to see the current state. |
+| U5's per-resource extraction could miss edge cases the current heuristic catches. | Unknown shapes fall through to the heuristic. The change is strictly additive: known shapes get a precise extractor; unknown shapes preserve current behavior. |
+| U5's input validation on Keys could be incomplete and let injection through. | Use a strict allowlist regex on Keys (`^[a-zA-Z_][a-zA-Z0-9_]*$`); reject anything else with a parser warning and `Unknown` shape fallback. |
+| Regenerating the entire fixture catalog after this PR could surface unexpected churn. | Each unit has a golden-harness verification step; the consolidated `scripts/golden.sh verify` should pass before merge. Document any intentional fixture diffs in the PR. |
+| Concurrent merges with the allrecipes-retro plan (#333/#335) could create rebase pain even though no files overlap. | Coordinate merge order; whichever lands second runs the golden harness against the merged state before final approval. |
+
+## Documentation / Operational Notes
+
+- Update CHANGELOG (or release notes) with a `fix(cli):` entry per unit. Patch-level version bump (no breaking changes to printed CLI surface).
+- U3's PII-scrub section in `secret-protection.md` should be cross-linked from `skills/printing-press-publish/SKILL.md` so users see it during publish.
+- After merge, regenerate cal-com end-to-end (without the workaround `internal/client/calcom_versions.go`) to validate the integrated fixes. Document the regen in a follow-up note if anything else surfaces.
+- File a follow-up issue for the deferred meta-fixes (silent-failure detection in dogfood; generic manifest-merge helper). Link from this plan's PR description.
+
+## Sources & References
+
+- **Origin issue:** [#334](https://github.com/mvanhorn/cli-printing-press/issues/334)
+- **Origin retro doc:** [https://files.catbox.moe/itg03h.md](https://files.catbox.moe/itg03h.md), local copy at `/tmp/printing-press/retro/20260427-005908-cal-com-retro.md`
+- **Related closed issue:** [#135 — retro(cli): Cal.com Run 2](https://github.com/mvanhorn/cli-printing-press/issues/135) (per-endpoint headers; this plan addresses the regression)
+- **Related landed PR:** [#332 — fix(cli): score auth prefixes from config](https://github.com/mvanhorn/cli-printing-press/pull/332) (resolves WU-10 ahead of this plan)
+- **Related landed PR:** [#335 — fix(cli): printing-press P1 machine fixes (issue #333)](https://github.com/mvanhorn/cli-printing-press/pull/335) (allrecipes retro fixes; touches doctor.go.tmpl and others, no file overlap with this plan)
+- **Concurrent plan:** `docs/plans/2026-04-26-002-feat-printing-press-p1-machine-fixes-plan.md` (allrecipes retro, also modifying generator templates)
+- **Related code:** `internal/generator/templates/data_source.go.tmpl`, `internal/generator/templates/command_endpoint.go.tmpl`, `internal/generator/templates/client.go.tmpl`, `internal/generator/templates/sync.go.tmpl`, `internal/generator/templates/graphql_sync.go.tmpl`, `internal/generator/templates/auth_simple.go.tmpl`, `internal/generator/templates/config.go.tmpl`, `internal/openapi/parser.go`, `internal/graphql/parser.go`, `internal/spec/spec.go`, `internal/profiler/profiler.go`, `internal/pipeline/publish.go`, `internal/pipeline/research.go`, `internal/pipeline/state.go`, `internal/pipeline/climanifest.go`, `internal/pipeline/lock.go`, `skills/printing-press/references/secret-protection.md`, `skills/printing-press-publish/SKILL.md`
diff --git a/internal/generator/auth_optional_test.go b/internal/generator/auth_optional_test.go
index 2cf7077b..62555a17 100644
--- a/internal/generator/auth_optional_test.go
+++ b/internal/generator/auth_optional_test.go
@@ -3,6 +3,7 @@ package generator
import (
"os"
"path/filepath"
+ "strings"
"testing"
"github.com/mvanhorn/cli-printing-press/v2/internal/spec"
@@ -144,3 +145,44 @@ func TestAuthConfig_Optional_ZeroValue(t *testing.T) {
a.Optional = true
require.True(t, a.Optional)
}
+
+// TestSetToken_ClearsLegacyAuthHeader verifies the generated set-token handler
+// clears cfg.AuthHeaderVal before saving. Without this clear, a pre-existing
+// auth_header in config.toml (the common regenerate scenario) shadows the
+// newly-saved access_token via Config.AuthHeader()'s resolver order, and
+// set-token silently has no effect on the active credential.
+//
+// Verification is a substring match on the generated source: the rendered
+// auth.go must contain `cfg.AuthHeaderVal = ""` immediately before the
+// SaveTokens call. The presence of the line is the contract; the exact
+// behavior is exercised end-to-end by the generated CLI's own set-token
+// command at runtime.
+//
+// Refs cal-com retro #334 WU-4.
+func TestSetToken_ClearsLegacyAuthHeader(t *testing.T) {
+ t.Parallel()
+
+ apiSpec := minimalSpec("clear-legacy-header")
+ apiSpec.Auth.EnvVars = []string{"CLEAR_LEGACY_HEADER_API_KEY"}
+
+ outputDir := filepath.Join(t.TempDir(), "clear-legacy-header-pp-cli")
+ require.NoError(t, New(apiSpec, outputDir).Generate())
+
+ authSrc, err := os.ReadFile(filepath.Join(outputDir, "internal", "cli", "auth.go"))
+ require.NoError(t, err)
+ src := string(authSrc)
+
+ // The clear must appear inside the set-token handler (not elsewhere).
+ require.Contains(t, src, "func newAuthSetTokenCmd",
+ "set-token handler must be emitted for the default auth template")
+ require.Contains(t, src, `cfg.AuthHeaderVal = ""`,
+ "set-token must clear legacy auth_header before SaveTokens; without this, a pre-existing auth_header shadows the new token")
+
+ // Confirm ordering: the clear comes before SaveTokens in the same handler.
+ clearIdx := strings.Index(src, `cfg.AuthHeaderVal = ""`)
+ saveIdx := strings.Index(src, "cfg.SaveTokens(")
+ require.NotEqual(t, -1, clearIdx, "auth_header clear must be present")
+ require.NotEqual(t, -1, saveIdx, "SaveTokens call must be present")
+ require.Less(t, clearIdx, saveIdx,
+ "the auth_header clear must occur BEFORE SaveTokens — otherwise SaveTokens persists with the stale auth_header still set")
+}
diff --git a/internal/generator/templates/auth_simple.go.tmpl b/internal/generator/templates/auth_simple.go.tmpl
index c7d51c40..8cb81351 100644
--- a/internal/generator/templates/auth_simple.go.tmpl
+++ b/internal/generator/templates/auth_simple.go.tmpl
@@ -78,6 +78,14 @@ func newAuthSetTokenCmd(flags *rootFlags) *cobra.Command {
return configErr(err)
}
+ // Clear any legacy auth_header so AuthHeader() falls through to
+ // "Bearer " + AccessToken with the new token. Without this, a
+ // pre-existing auth_header value (common after regenerate) shadows
+ // the newly-saved access_token and set-token silently has no effect.
+ // Silent clear (no log line): a masked-tail variant could leak
+ // token bytes through scripted dogfood that captures stderr.
+ cfg.AuthHeaderVal = ""
+
// Save the token directly via the config's save mechanism
if err := cfg.SaveTokens("", "", args[0], "", cfg.TokenExpiry); err != nil {
return configErr(fmt.Errorf("saving token: %w", err))
diff --git a/internal/generator/templates/command_endpoint.go.tmpl b/internal/generator/templates/command_endpoint.go.tmpl
index 8c7806cc..6167e3e0 100644
--- a/internal/generator/templates/command_endpoint.go.tmpl
+++ b/internal/generator/templates/command_endpoint.go.tmpl
@@ -165,7 +165,7 @@ func new{{camel .FuncPrefix}}{{camel .EndpointName}}Cmd(flags *rootFlags) *cobra
"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
{{- end}}
{{- end}}
- }, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
+ }, {{if .Endpoint.HeaderOverrides}}headerOverrides{{else}}nil{{end}}, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
{{- else}}
data, err := paginatedGet(c, path, map[string]string{
{{- range $i, $p := .Endpoint.Params}}
@@ -176,7 +176,7 @@ func new{{camel .FuncPrefix}}{{camel .EndpointName}}Cmd(flags *rootFlags) *cobra
"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
{{- end}}
{{- end}}
- }, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
+ }, {{if .Endpoint.HeaderOverrides}}headerOverrides{{else}}nil{{end}}, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
{{- end}}
{{- else}}
params := map[string]string{}
@@ -191,7 +191,7 @@ func new{{camel .FuncPrefix}}{{camel .EndpointName}}Cmd(flags *rootFlags) *cobra
{{- end}}
{{- end}}
{{- if .HasStore}}
- data, prov, err := resolveRead(c, flags, "{{lower .ResourceName}}", {{if .Endpoint.Pagination}}true{{else}}false{{end}}, path, params)
+ data, prov, err := resolveRead(c, flags, "{{lower .ResourceName}}", {{if .Endpoint.Pagination}}true{{else}}false{{end}}, path, params, {{if .Endpoint.HeaderOverrides}}headerOverrides{{else}}nil{{end}})
{{- else}}
{{- if .Endpoint.HeaderOverrides}}
data, err := c.GetWithHeaders(path, params, headerOverrides)
diff --git a/internal/generator/templates/command_promoted.go.tmpl b/internal/generator/templates/command_promoted.go.tmpl
index 39df0ee9..dedbc10e 100644
--- a/internal/generator/templates/command_promoted.go.tmpl
+++ b/internal/generator/templates/command_promoted.go.tmpl
@@ -111,7 +111,7 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
{{- end}}
{{- end}}
- }, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
+ }, nil, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
{{- else}}
data, err := paginatedGet(c, path, map[string]string{
{{- range $i, $p := .Endpoint.Params}}
@@ -122,7 +122,7 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
{{- end}}
{{- end}}
- }, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
+ }, nil, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
{{- end}}
{{- else}}
params := map[string]string{}
@@ -137,7 +137,7 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
{{- end}}
{{- end}}
{{- if .HasStore}}
- data, prov, err := resolveRead(c, flags, "{{lower .ResourceName}}", {{if .Endpoint.Pagination}}true{{else}}false{{end}}, path, params)
+ data, prov, err := resolveRead(c, flags, "{{lower .ResourceName}}", {{if .Endpoint.Pagination}}true{{else}}false{{end}}, path, params, nil)
{{- else}}
data, err := c.Get(path, params)
{{- end}}
diff --git a/internal/generator/templates/data_source.go.tmpl b/internal/generator/templates/data_source.go.tmpl
index 663a41c1..2e010f4d 100644
--- a/internal/generator/templates/data_source.go.tmpl
+++ b/internal/generator/templates/data_source.go.tmpl
@@ -85,21 +85,26 @@ func attachFreshness(prov DataProvenance, flags *rootFlags) DataProvenance {
// - isList: true for list endpoints, false for get-by-ID endpoints
// - path: the API path (e.g., "/links" or "/links/abc123")
// - params: query parameters for the API call
-func resolveRead(c *client.Client, flags *rootFlags, resourceType string, isList bool, path string, params map[string]string) (json.RawMessage, DataProvenance, error) {
+// - headers: per-endpoint required headers (e.g. cal-api-version, Stripe-Version)
+// baked in by the command template at codegen time. Pass nil when the endpoint
+// declares no per-endpoint header overrides. Without this parameter, store-backed
+// reads on per-endpoint-versioned APIs silently get the wrong response shape
+// (cal-com retro #334 F1).
+func resolveRead(c *client.Client, flags *rootFlags, resourceType string, isList bool, path string, params map[string]string, headers map[string]string) (json.RawMessage, DataProvenance, error) {
switch flags.dataSource {
case "local":
data, prov, err := resolveLocal(resourceType, isList, path, params, "user_requested")
return data, attachFreshness(prov, flags), err
case "live":
- data, err := c.Get(path, params)
+ data, err := c.GetWithHeaders(path, params, headers)
if err != nil {
return nil, DataProvenance{}, err
}
return data, attachFreshness(DataProvenance{Source: "live"}, flags), nil
default: // "auto"
- data, err := c.Get(path, params)
+ data, err := c.GetWithHeaders(path, params, headers)
if err == nil {
writeThroughCache(resourceType, data)
return data, attachFreshness(DataProvenance{Source: "live"}, flags), nil
@@ -118,22 +123,24 @@ func resolveRead(c *client.Client, flags *rootFlags, resourceType string, isList
}
// resolvePaginatedRead dispatches a paginated GET request to either the live API
-// or local store. When local, skips pagination and returns all synced data.
-func resolvePaginatedRead(c *client.Client, flags *rootFlags, resourceType string, path string, params map[string]string, fetchAll bool, cursorParam, nextCursorPath, hasMoreField string) (json.RawMessage, DataProvenance, error) {
+// or local store. When local, skips pagination and returns all synced data. The
+// headers argument carries per-endpoint required headers; pass nil when the
+// endpoint declares no overrides.
+func resolvePaginatedRead(c *client.Client, flags *rootFlags, resourceType string, path string, params map[string]string, headers map[string]string, fetchAll bool, cursorParam, nextCursorPath, hasMoreField string) (json.RawMessage, DataProvenance, error) {
switch flags.dataSource {
case "local":
data, prov, err := resolveLocal(resourceType, true, path, params, "user_requested")
return data, attachFreshness(prov, flags), err
case "live":
- data, err := paginatedGet(c, path, params, fetchAll, cursorParam, nextCursorPath, hasMoreField)
+ data, err := paginatedGet(c, path, params, headers, fetchAll, cursorParam, nextCursorPath, hasMoreField)
if err != nil {
return nil, DataProvenance{}, err
}
return data, attachFreshness(DataProvenance{Source: "live"}, flags), nil
default: // "auto"
- data, err := paginatedGet(c, path, params, fetchAll, cursorParam, nextCursorPath, hasMoreField)
+ data, err := paginatedGet(c, path, params, headers, fetchAll, cursorParam, nextCursorPath, hasMoreField)
if err == nil {
writeThroughCache(resourceType, data)
return data, attachFreshness(DataProvenance{Source: "live"}, flags), nil
diff --git a/internal/generator/templates/helpers.go.tmpl b/internal/generator/templates/helpers.go.tmpl
index 7a8a7876..e99c2937 100644
--- a/internal/generator/templates/helpers.go.tmpl
+++ b/internal/generator/templates/helpers.go.tmpl
@@ -353,10 +353,13 @@ func replacePathParam(path, name, value string) string {
}
{{- end}}
-// paginatedGet fetches pages and concatenates array results.
+// paginatedGet fetches pages and concatenates array results. The headers
+// argument carries per-endpoint required headers (e.g. cal-api-version) that
+// must be sent on every page request, including the first; pass nil when the
+// endpoint has no per-endpoint header overrides.
func paginatedGet(c interface {
- Get(path string, params map[string]string) (json.RawMessage, error)
-}, path string, params map[string]string, fetchAll bool, cursorParam, nextCursorPath, hasMoreField string) (json.RawMessage, error) {
+ GetWithHeaders(path string, params map[string]string, headers map[string]string) (json.RawMessage, error)
+}, path string, params map[string]string, headers map[string]string, fetchAll bool, cursorParam, nextCursorPath, hasMoreField string) (json.RawMessage, error) {
// Clean zero-value params
clean := map[string]string{}
for k, v := range params {
@@ -366,7 +369,7 @@ func paginatedGet(c interface {
}
if !fetchAll {
- return c.Get(path, clean)
+ return c.GetWithHeaders(path, clean, headers)
}
// Fetch all pages
@@ -380,7 +383,7 @@ func paginatedGet(c interface {
fmt.Fprintf(os.Stderr, `{"event":"page_fetch","page":%d}`+"\n", page)
}
- data, err := c.Get(path, clean)
+ data, err := c.GetWithHeaders(path, clean, headers)
if err != nil {
return nil, err
}
diff --git a/internal/pipeline/publish.go b/internal/pipeline/publish.go
index 349a9cff..dfb79a00 100644
--- a/internal/pipeline/publish.go
+++ b/internal/pipeline/publish.go
@@ -204,7 +204,10 @@ func writeCLIManifestForPublish(state *PipelineState, dir string) error {
}
// Carry forward metadata from the generated manifest when publish-time
- // parsing is unavailable or lossy for the original spec format.
+ // parsing is unavailable or lossy for the original spec format. NovelFeatures
+ // is carried forward as a defensive fallback in case the loadResearchForState
+ // lookup below misses (e.g., research.json moved or absent); when both are
+ // available, research.json wins as the post-dogfood source of truth.
if existingData, err := os.ReadFile(filepath.Join(dir, CLIManifestFilename)); err == nil {
var existing CLIManifest
if json.Unmarshal(existingData, &existing) == nil {
@@ -214,6 +217,7 @@ func writeCLIManifestForPublish(state *PipelineState, dir string) error {
m.MCPReady = existing.MCPReady
m.AuthType = existing.AuthType
m.AuthEnvVars = existing.AuthEnvVars
+ m.NovelFeatures = existing.NovelFeatures
}
}
@@ -262,26 +266,35 @@ func writeCLIManifestForPublish(state *PipelineState, dir string) error {
m.Description = entry.Description
}
- // Load novel features from research.json if available. When research.json
- // can't be located (typically because state.RunID is empty in a fallback
- // scenario, or the run's pipeline dir was GCed), emit a debug breadcrumb
- // to stderr so the silent dropout from earlier versions is no longer
- // silent. The breadcrumb names the path that was attempted; the caller
- // (lock promote, publish) shows it but does not treat it as an error.
- pipelineDir := state.PipelineDir()
- if research, err := LoadResearch(pipelineDir); err == nil && research.NovelFeaturesBuilt != nil {
+ // Load novel features from research.json if available. Looks in both
+ // RunRoot/research.json (printing-press skill convention) and
+ // RunRoot/pipeline/research.json (printing-press print convention) — both
+ // must be attempted because the skill flow and print flow write research
+ // to different locations. When research.json has populated NovelFeaturesBuilt,
+ // it overrides the carry-forward value from the existing manifest because
+ // post-dogfood data is the source of truth. When neither location holds
+ // research.json, emit a debug breadcrumb so the silent dropout from earlier
+ // versions is no longer silent — the breadcrumb names both attempted paths;
+ // the caller (lock promote, publish) shows it but does not treat it as an error.
+ research, researchErr := loadResearchForState(state)
+ switch {
+ case researchErr != nil:
+ fmt.Fprintf(os.Stderr,
+ "debug: research.json not found at %s or %s; skipping novel_features enrichment "+
+ "(state.RunID=%q)\n",
+ filepath.Join(RunRoot(state.RunID), "research.json"),
+ filepath.Join(state.PipelineDir(), "research.json"),
+ state.RunID)
+ case research.NovelFeaturesBuilt != nil && len(*research.NovelFeaturesBuilt) > 0:
+ built := make([]NovelFeatureManifest, 0, len(*research.NovelFeaturesBuilt))
for _, nf := range *research.NovelFeaturesBuilt {
- m.NovelFeatures = append(m.NovelFeatures, NovelFeatureManifest{
+ built = append(built, NovelFeatureManifest{
Name: nf.Name,
Command: nf.Command,
Description: nf.Description,
})
}
- } else {
- fmt.Fprintf(os.Stderr,
- "debug: research.json not found at %s; skipping novel_features enrichment "+
- "(state.RunID=%q)\n",
- filepath.Join(pipelineDir, "research.json"), state.RunID)
+ m.NovelFeatures = built
}
return WriteCLIManifest(dir, m)
diff --git a/internal/pipeline/publish_test.go b/internal/pipeline/publish_test.go
index 2f59d9b7..2f926f9a 100644
--- a/internal/pipeline/publish_test.go
+++ b/internal/pipeline/publish_test.go
@@ -1,6 +1,7 @@
package pipeline
import (
+ "encoding/json"
"os"
"path/filepath"
"testing"
@@ -113,3 +114,199 @@ func TestCopyDirRejectsExternalSymlinks(t *testing.T) {
})
}
}
+
+// publishManifestEnvSetup wires PRINTING_PRESS_HOME/SCOPE/REPO_ROOT to a temp dir
+// so RunRoot()/PipelineDir()/PublishedLibraryRoot() resolve under the test sandbox.
+// Returns the temp root and a state seeded with the given run ID.
+func publishManifestEnvSetup(t *testing.T, runID string) (string, *PipelineState) {
+ t.Helper()
+ tmp := t.TempDir()
+ t.Setenv("PRINTING_PRESS_HOME", tmp)
+ t.Setenv("PRINTING_PRESS_SCOPE", "test-scope")
+ t.Setenv("PRINTING_PRESS_REPO_ROOT", tmp)
+ state := NewStateWithRun("test-api", filepath.Join(tmp, "working", "test-api-pp-cli"), runID, "test-scope")
+ require.NoError(t, os.MkdirAll(state.WorkingDir, 0o755))
+ return tmp, state
+}
+
+// readPublishedManifest reads the manifest from the given dir for assertions.
+func readPublishedManifest(t *testing.T, dir string) CLIManifest {
+ t.Helper()
+ data, err := os.ReadFile(filepath.Join(dir, CLIManifestFilename))
+ require.NoError(t, err)
+ var m CLIManifest
+ require.NoError(t, json.Unmarshal(data, &m))
+ return m
+}
+
+// writeResearchAt writes a ResearchResult to the given directory's research.json.
+func writeResearchAt(t *testing.T, dir string, r *ResearchResult) {
+ t.Helper()
+ require.NoError(t, os.MkdirAll(dir, 0o755))
+ data, err := json.MarshalIndent(r, "", " ")
+ require.NoError(t, err)
+ require.NoError(t, os.WriteFile(filepath.Join(dir, "research.json"), data, 0o644))
+}
+
+// TestWriteCLIManifestForPublish_NovelFeaturesFromSkillFlowResearch covers the
+// printing-press skill flow: research.json lives at <RunRoot>/research.json
+// (the run root itself, not the pipeline subdir). Before this fix, LoadResearch
+// only checked PipelineDir and silently missed this convention — manifest shipped
+// with empty novel_features and publish validate failed (cal-com retro #334 F2).
+func TestWriteCLIManifestForPublish_NovelFeaturesFromSkillFlowResearch(t *testing.T) {
+ _, state := publishManifestEnvSetup(t, "20260427-skill-flow")
+
+ // Skill flow: research.json at RunRoot, NOT under pipeline/.
+ built := []NovelFeature{
+ {Name: "One-shot booking", Command: "book", Description: "Compose slots-find + reserve + create + confirm."},
+ {Name: "Today's agenda", Command: "today", Description: "Read today's bookings from local store."},
+ }
+ writeResearchAt(t, RunRoot(state.RunID), &ResearchResult{
+ APIName: "test-api",
+ NovelFeaturesBuilt: &built,
+ })
+
+ require.NoError(t, writeCLIManifestForPublish(state, state.WorkingDir))
+
+ m := readPublishedManifest(t, state.WorkingDir)
+ require.Len(t, m.NovelFeatures, 2, "novel_features should be populated from RunRoot/research.json")
+ assert.Equal(t, "book", m.NovelFeatures[0].Command)
+ assert.Equal(t, "today", m.NovelFeatures[1].Command)
+}
+
+// TestWriteCLIManifestForPublish_NovelFeaturesFromPrintFlowResearch covers the
+// printing-press print flow: research.json lives at <RunRoot>/pipeline/research.json
+// alongside phase artifacts. The fallback path keeps print-flow CLIs working.
+func TestWriteCLIManifestForPublish_NovelFeaturesFromPrintFlowResearch(t *testing.T) {
+ _, state := publishManifestEnvSetup(t, "20260427-print-flow")
+
+ // Print flow: research.json under PipelineDir, NOT at RunRoot.
+ built := []NovelFeature{
+ {Name: "Conflicts", Command: "conflicts", Description: "Find overlaps."},
+ }
+ writeResearchAt(t, state.PipelineDir(), &ResearchResult{
+ APIName: "test-api",
+ NovelFeaturesBuilt: &built,
+ })
+
+ require.NoError(t, writeCLIManifestForPublish(state, state.WorkingDir))
+
+ m := readPublishedManifest(t, state.WorkingDir)
+ require.Len(t, m.NovelFeatures, 1, "novel_features should populate from PipelineDir/research.json")
+ assert.Equal(t, "conflicts", m.NovelFeatures[0].Command)
+}
+
+// TestWriteCLIManifestForPublish_NovelFeaturesPreservedFromCarryForward covers
+// the defense-in-depth path: research.json missing (deleted, not yet written),
+// but the existing manifest in the staging dir already has novel_features from
+// generate time. The carry-forward block must preserve them so publish doesn't
+// silently strip a populated field.
+func TestWriteCLIManifestForPublish_NovelFeaturesPreservedFromCarryForward(t *testing.T) {
+ _, state := publishManifestEnvSetup(t, "20260427-carry-forward")
+
+ // Pre-populate the staging dir's existing manifest with novel_features.
+ existing := CLIManifest{
+ SchemaVersion: 1,
+ APIName: "test-api",
+ CLIName: "test-api-pp-cli",
+ NovelFeatures: []NovelFeatureManifest{
+ {Name: "Today", Command: "today", Description: "Today's bookings."},
+ },
+ }
+ existingData, err := json.Marshal(existing)
+ require.NoError(t, err)
+ require.NoError(t, os.WriteFile(filepath.Join(state.WorkingDir, CLIManifestFilename), existingData, 0o644))
+
+ // No research.json anywhere. Publish should preserve the carry-forward value.
+ require.NoError(t, writeCLIManifestForPublish(state, state.WorkingDir))
+
+ m := readPublishedManifest(t, state.WorkingDir)
+ require.Len(t, m.NovelFeatures, 1, "carry-forward should preserve generate-time novel_features")
+ assert.Equal(t, "today", m.NovelFeatures[0].Command)
+}
+
+// TestWriteCLIManifestForPublish_NovelFeaturesResearchOverridesCarryForward
+// covers the precedence rule: when both research.json (post-dogfood) and the
+// existing manifest (generate-time) have novel_features, research wins because
+// post-dogfood verification is the source of truth.
+func TestWriteCLIManifestForPublish_NovelFeaturesResearchOverridesCarryForward(t *testing.T) {
+ _, state := publishManifestEnvSetup(t, "20260427-research-wins")
+
+ // Stale generate-time manifest with one feature.
+ existing := CLIManifest{
+ SchemaVersion: 1,
+ APIName: "test-api",
+ NovelFeatures: []NovelFeatureManifest{
+ {Name: "Stale", Command: "stale", Description: "Outdated."},
+ },
+ }
+ existingData, err := json.Marshal(existing)
+ require.NoError(t, err)
+ require.NoError(t, os.WriteFile(filepath.Join(state.WorkingDir, CLIManifestFilename), existingData, 0o644))
+
+ // Fresh post-dogfood research with two features (different from stale).
+ built := []NovelFeature{
+ {Name: "Book", Command: "book", Description: "One-shot booking."},
+ {Name: "Today", Command: "today", Description: "Today's agenda."},
+ }
+ writeResearchAt(t, RunRoot(state.RunID), &ResearchResult{
+ APIName: "test-api",
+ NovelFeaturesBuilt: &built,
+ })
+
+ require.NoError(t, writeCLIManifestForPublish(state, state.WorkingDir))
+
+ m := readPublishedManifest(t, state.WorkingDir)
+ require.Len(t, m.NovelFeatures, 2, "research should override carry-forward")
+ commands := []string{m.NovelFeatures[0].Command, m.NovelFeatures[1].Command}
+ assert.ElementsMatch(t, []string{"book", "today"}, commands, "research-loaded features replace stale carry-forward")
+}
+
+// TestWriteCLIManifestForPublish_EmptyResearchKeepsCarryForward covers the
+// edge case where research.json exists but novel_features_built is empty
+// (e.g., a run where no novel features survived dogfood). The empty research
+// must NOT clobber a populated carry-forward — the carry-forward represents
+// the most-recent verified data the system has.
+func TestWriteCLIManifestForPublish_EmptyResearchKeepsCarryForward(t *testing.T) {
+ _, state := publishManifestEnvSetup(t, "20260427-empty-research")
+
+ // Carry-forward has one feature.
+ existing := CLIManifest{
+ SchemaVersion: 1,
+ APIName: "test-api",
+ NovelFeatures: []NovelFeatureManifest{
+ {Name: "Today", Command: "today", Description: "Today's agenda."},
+ },
+ }
+ existingData, err := json.Marshal(existing)
+ require.NoError(t, err)
+ require.NoError(t, os.WriteFile(filepath.Join(state.WorkingDir, CLIManifestFilename), existingData, 0o644))
+
+ // Research has empty NovelFeaturesBuilt.
+ emptyBuilt := []NovelFeature{}
+ writeResearchAt(t, RunRoot(state.RunID), &ResearchResult{
+ APIName: "test-api",
+ NovelFeaturesBuilt: &emptyBuilt,
+ })
+
+ require.NoError(t, writeCLIManifestForPublish(state, state.WorkingDir))
+
+ m := readPublishedManifest(t, state.WorkingDir)
+ require.Len(t, m.NovelFeatures, 1, "empty research must not clobber populated carry-forward")
+ assert.Equal(t, "today", m.NovelFeatures[0].Command)
+}
+
+// TestWriteCLIManifestForPublish_NoResearchNoExistingManifest covers the
+// genuinely-empty case: no research.json, no prior manifest. The published
+// manifest has no novel_features (correct — there are none to publish).
+// publish validate's transcendence check will then fail with the existing
+// "no novel features recorded" message.
+func TestWriteCLIManifestForPublish_NoResearchNoExistingManifest(t *testing.T) {
+ _, state := publishManifestEnvSetup(t, "20260427-empty-everything")
+
+ // No research.json, no existing manifest in WorkingDir.
+ require.NoError(t, writeCLIManifestForPublish(state, state.WorkingDir))
+
+ m := readPublishedManifest(t, state.WorkingDir)
+ assert.Empty(t, m.NovelFeatures, "no novel_features when neither research nor prior manifest has any")
+}
diff --git a/internal/pipeline/research.go b/internal/pipeline/research.go
index 84eb5e37..08cece86 100644
--- a/internal/pipeline/research.go
+++ b/internal/pipeline/research.go
@@ -268,6 +268,23 @@ func LoadResearch(pipelineDir string) (*ResearchResult, error) {
return &r, nil
}
+// loadResearchForState reads research.json from the run directory, supporting
+// both write conventions: the printing-press skill writes to RunRoot/research.json
+// directly, while printing-press print writes to RunRoot/pipeline/research.json
+// alongside its phase artifacts. Tries the run-root path first (the dominant
+// case), falls back to pipeline-dir.
+//
+// The two-location lookup matters at promote/publish time, when downstream code
+// needs research.json to populate the published manifest's novel_features.
+// Without this fallback the publish-time read silently misses skill-flow runs
+// and ships manifests with empty novel_features (cal-com retro #334 F2).
+func loadResearchForState(state *PipelineState) (*ResearchResult, error) {
+ if r, err := LoadResearch(RunRoot(state.RunID)); err == nil {
+ return r, nil
+ }
+ return LoadResearch(state.PipelineDir())
+}
+
// WriteNovelFeaturesBuilt updates research.json with the verified list of
// novel features that survived the build. The original novel_features field
// is preserved as-is (the planned list); novel_features_built records what
diff --git a/skills/printing-press-publish/SKILL.md b/skills/printing-press-publish/SKILL.md
index 10959d03..c50f129e 100644
--- a/skills/printing-press-publish/SKILL.md
+++ b/skills/printing-press-publish/SKILL.md
@@ -825,6 +825,29 @@ If any issues are found, warn the user and ask whether to proceed. The user
makes the final call — they may have intentionally included something the scan
flagged (e.g., a test fixture with a fake key). Don't block silently.
+### PII pattern scanning (mandatory)
+
+Beyond the secret scans above, run the **PII pattern scanning** step from
+[../printing-press/references/secret-protection.md](../printing-press/references/secret-protection.md#pii-pattern-scanning)
+(section "PII pattern scanning"). This catches PII captured during live dogfood
+that the prose guidance missed — emails, real attendee names, account
+identifiers — before they ship to the public library repo.
+
+The scan has two tiers:
+- **Tier 1 (auto-redact silently):** vendor-prefix-anchored bearer tokens
+ (`Bearer cal_live_*`, `Bearer sk_live_*`, `Bearer ghp_*`, `xoxp-*`, etc.).
+ Near-zero false-positive rate.
+- **Tier 2 (warn, batched user prompt):** generic emails, generic bearer tokens,
+ capitalized first+last name patterns. Allowlist suppresses spec-derived API
+ vocabulary ("Event Types", "Booking Links") automatically.
+
+A pre-scrub copy of the staging directory is preserved at
+`<staging>.pre-pii-scrub/` so the user can recover from a wrong redaction.
+
+Two prior PII leaks shipped to the public library before this scan existed.
+The scan is the mechanical defense layer the prose guidance alone could not
+provide.
+
## Error Handling
- **`gh` not authenticated:** Detect in Step 1, tell user to run `gh auth login`
diff --git a/skills/printing-press/references/secret-protection.md b/skills/printing-press/references/secret-protection.md
index d8209416..b0c11e9a 100644
--- a/skills/printing-press/references/secret-protection.md
+++ b/skills/printing-press/references/secret-protection.md
@@ -111,6 +111,170 @@ workspace naturally produces.
manuscripts and test results. Any live test data quoted in the PR body must be
scrubbed of workspace PII. The library repo is public.
+## PII pattern scanning
+
+The prose guidance above tells the agent how to compose published artifacts safely.
+This section defines a mechanical sweep that runs before publishing to catch the
+PII the agent missed. Two prior PII leaks happened despite the prose guidance — a
+mechanical defense layer is required.
+
+**Run order:** exact-value scan (above) → HAR auth strip → PII pattern scanning.
+
+**File scope.** Sweep all text-readable files in the staging directory. Detect by
+content, not extension — a `.yaml` HAR variant or a `.txt` proof matters as much as
+a `.md`. Use this Go-equivalent helper to classify:
+
+```bash
+# Treat as text if no NULL byte in the first 8 KiB
+isText() { head -c 8192 "$1" | grep -L -q $'\x00' "$1" 2>/dev/null; }
+```
+
+(In Bash, `file --mime-type` works on macOS and GNU `file` portably enough.)
+
+### Tier 1: vendor-anchored auto-redact
+
+Auto-redact ONLY when the pattern includes a vendor-specific prefix anchor. These
+have near-zero false-positive rate. Auto-redact silently — no user prompt, no
+recovery prompt. Pattern set:
+
+```bash
+PII_AUTO_REDACT=(
+ 'bearer-stripe-live|Bearer sk_live_[A-Za-z0-9]{20,}|Bearer <REDACTED:stripe-live-token>'
+ 'bearer-stripe-test|Bearer sk_test_[A-Za-z0-9]{20,}|Bearer <REDACTED:stripe-test-token>'
+ 'bearer-cal-live|Bearer cal_live_[A-Za-z0-9]{20,}|Bearer <REDACTED:cal-live-token>'
+ 'bearer-cal-test|Bearer cal_test_[A-Za-z0-9]{20,}|Bearer <REDACTED:cal-test-token>'
+ 'bearer-github-pat|Bearer ghp_[A-Za-z0-9]{36,}|Bearer <REDACTED:github-pat>'
+ 'bearer-github-oauth|Bearer gho_[A-Za-z0-9]{36,}|Bearer <REDACTED:github-oauth>'
+ 'bearer-github-fine|Bearer github_pat_[A-Za-z0-9_]{60,}|Bearer <REDACTED:github-fine-grained-pat>'
+ 'slack-user-token|xoxp-[A-Za-z0-9-]{40,}|<REDACTED:slack-user-token>'
+ 'slack-bot-token|xoxb-[A-Za-z0-9-]{40,}|<REDACTED:slack-bot-token>'
+)
+
+for entry in "${PII_AUTO_REDACT[@]}"; do
+ IFS='|' read -r name regex tag <<< "$entry"
+ for f in $(find "$STAGING_DIR" -type f); do
+ isText "$f" || continue
+ if grep -qE "$regex" "$f" 2>/dev/null; then
+ perl -i -pe "s|$regex|$tag|g" "$f"
+ echo "Auto-redacted $name in ${f#$STAGING_DIR/}"
+ fi
+ done
+done
+```
+
+These patterns are vendor-anchored and the prefix character class is restrictive
+enough that the false-positive rate is effectively zero. Adding a new vendor
+pattern is safe — extend the list when shipping a new printed CLI for a vendor
+with a known token format.
+
+### Tier 2: warn-by-default with allowlist
+
+Everything else gets WARN behavior. The user sees the matches, decides whether to
+redact each one. This includes generic emails, generic bearer tokens (without a
+vendor prefix), and capitalized name patterns. Designed cost asymmetry:
+false-positive auto-redaction loses information irreversibly; false-positive
+warning costs only a prompt.
+
+```bash
+PII_WARN=(
+ 'email|[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}|email address'
+ 'bearer-generic|Bearer [A-Za-z0-9._\-+/=]{20,}|generic bearer token (no vendor prefix)'
+ 'name-pattern|\b[A-Z][a-z]{2,}\s+[A-Z][a-z]{2,}\b|capitalized first+last name'
+)
+```
+
+**Allowlist (suppress matches against this set):**
+
+Build at scan time from three sources:
+
+1. **Spec-derived terms.** Capitalized two-word phrases extracted from the spec's
+ operation summaries, descriptions, tag names, and parameter descriptions. This
+ catches "Event Types", "Booking Links", "Webhook Triggers" automatically — they
+ look like name patterns but are API vocabulary.
+
+2. **CLI-help-derived terms.** Walk `<cli> --help` recursively and extract
+ capitalized two-word phrases from command Short/Long fields. Catches command
+ group names that the spec might not mention by exact phrasing.
+
+3. **Universal allowlist.** A small static list of terms that appear across
+ APIs and would otherwise produce noise:
+
+ ```
+ "Open Source", "Pull Request", "Bearer Token", "API Key", "Access Token",
+ "Refresh Token", "GitHub Actions", "Cloud Run", "Service Account",
+ "Rate Limit", "Time Zone", "Web Hook", "Personal Access", "Application Token"
+ ```
+
+A match is suppressed if it matches any allowlist entry by:
+- Exact case-insensitive equality, OR
+- Case-insensitive `contains` against the allowlist entry
+
+Email pattern allowlist: any address whose domain matches `example.com`,
+`example.org`, `example.net`, `placeholder.com`, or starts with `noreply@`.
+
+### Tier 2 user interaction
+
+When Tier 2 finds matches, present them in a single batched prompt (NOT one
+per finding — that creates rubber-stamp fatigue). Per-finding format:
+
+```
+PII review (3 findings):
+
+ proofs/acceptance.md:42 email trevin@trevinchow.com
+ proofs/acceptance.md:48 email henryopenclaw@gmail.com
+ proofs/dogfood.md:108 name Henry Claw
+
+Action:
+ [a] Auto-redact all
+ [s] Skip all (proceed without redacting)
+ [q] Quit and review staging dir at /tmp/.../staging.pre-pii-scrub
+```
+
+Pre-scrub copy: before any Tier 1 or Tier 2 mutation runs, copy the staging
+directory to `<staging>.pre-pii-scrub/`. This gives the user a recoverable
+checkpoint if a redaction was wrong.
+
+### Why warn-by-default
+
+The first design draft auto-redacted emails (permissive pattern) and warned on
+names (constrained pattern). Two reviewers independently called this cost-
+asymmetry inverted: false-positive auto-redaction permanently corrupts a
+manuscript artifact, while false-positive warning costs only a prompt. The
+correct policy is the inverse — warn on patterns where any false positive would
+lose information; auto-redact only on vendor-prefix-anchored patterns where the
+false-positive rate is structurally near-zero.
+
+### Worked examples
+
+**Example 1: vendor token in a proof file.** Input proof file contains:
+```
+Authorization: Bearer cal_live_7d8911769f5c3d63811e36d95d873a16
+```
+Tier 1 matches `bearer-cal-live`. Auto-redacts to:
+```
+Authorization: Bearer <REDACTED:cal-live-token>
+```
+No user prompt.
+
+**Example 2: real attendee in a dogfood report.** Input proof file contains:
+```
+Returned 1 real booking (Henry Claw 15-min meeting, attendee henryopenclaw@gmail.com).
+```
+Tier 2 matches `email` and `name-pattern`. The email's domain (`gmail.com`) is
+not in the allowlist; "Henry Claw" is not in spec-derived terms. Both surface in
+the batched prompt. User picks `[a]` Auto-redact all → both redacted.
+
+**Example 3: API vocabulary that looks like a name.** Input README contains:
+```
+The `Event Types` resource lists every bookable meeting type.
+```
+Tier 2 `name-pattern` matches "Event Types". Spec-derived allowlist includes
+"Event Types" (extracted from `tag: "Event Types"` in the spec). Match is
+suppressed. No prompt.
+
+**Example 4: clean staged dir.** No matches in either tier. No prompt, publish
+proceeds.
+
## Session state cleanup
Session state files (`session-state.json`) contain browser cookies and auth tokens.
diff --git a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/auth.go b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/auth.go
index a2409194..12415903 100644
--- a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/auth.go
+++ b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/auth.go
@@ -66,6 +66,14 @@ func newAuthSetTokenCmd(flags *rootFlags) *cobra.Command {
return configErr(err)
}
+ // Clear any legacy auth_header so AuthHeader() falls through to
+ // "Bearer " + AccessToken with the new token. Without this, a
+ // pre-existing auth_header value (common after regenerate) shadows
+ // the newly-saved access_token and set-token silently has no effect.
+ // Silent clear (no log line): a masked-tail variant could leak
+ // token bytes through scripted dogfood that captures stderr.
+ cfg.AuthHeaderVal = ""
+
// Save the token directly via the config's save mechanism
if err := cfg.SaveTokens("", "", args[0], "", cfg.TokenExpiry); err != nil {
return configErr(fmt.Errorf("saving token: %w", err))
diff --git a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/helpers.go b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/helpers.go
index 1e418301..04d5620b 100644
--- a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/helpers.go
+++ b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/helpers.go
@@ -242,10 +242,13 @@ func replacePathParam(path, name, value string) string {
return strings.ReplaceAll(path, "{"+name+"}", value)
}
-// paginatedGet fetches pages and concatenates array results.
+// paginatedGet fetches pages and concatenates array results. The headers
+// argument carries per-endpoint required headers (e.g. cal-api-version) that
+// must be sent on every page request, including the first; pass nil when the
+// endpoint has no per-endpoint header overrides.
func paginatedGet(c interface {
- Get(path string, params map[string]string) (json.RawMessage, error)
-}, path string, params map[string]string, fetchAll bool, cursorParam, nextCursorPath, hasMoreField string) (json.RawMessage, error) {
+ GetWithHeaders(path string, params map[string]string, headers map[string]string) (json.RawMessage, error)
+}, path string, params map[string]string, headers map[string]string, fetchAll bool, cursorParam, nextCursorPath, hasMoreField string) (json.RawMessage, error) {
// Clean zero-value params
clean := map[string]string{}
for k, v := range params {
@@ -255,7 +258,7 @@ func paginatedGet(c interface {
}
if !fetchAll {
- return c.Get(path, clean)
+ return c.GetWithHeaders(path, clean, headers)
}
// Fetch all pages
@@ -269,7 +272,7 @@ func paginatedGet(c interface {
fmt.Fprintf(os.Stderr, `{"event":"page_fetch","page":%d}`+"\n", page)
}
- data, err := c.Get(path, clean)
+ data, err := c.GetWithHeaders(path, clean, headers)
if err != nil {
return nil, err
}
diff --git a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_list.go b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_list.go
index b40808a0..dd5e1fcf 100644
--- a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_list.go
+++ b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_list.go
@@ -45,7 +45,7 @@ func newProjectsListCmd(flags *rootFlags) *cobra.Command {
"status": fmt.Sprintf("%v", flagStatus),
"limit": fmt.Sprintf("%v", flagLimit),
"cursor": fmt.Sprintf("%v", flagCursor),
- }, flagAll, "cursor", "", "")
+ }, nil, flagAll, "cursor", "", "")
if err != nil {
return classifyAPIError(err)
}
diff --git a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_tasks_list-project.go b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_tasks_list-project.go
index cadb8191..6834cbc2 100644
--- a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_tasks_list-project.go
+++ b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/projects_tasks_list-project.go
@@ -50,7 +50,7 @@ func newProjectsTasksListProjectCmd(flags *rootFlags) *cobra.Command {
"priority": fmt.Sprintf("%v", flagPriority),
"limit": fmt.Sprintf("%v", flagLimit),
"cursor": fmt.Sprintf("%v", flagCursor),
- }, flagAll, "cursor", "", "")
+ }, nil, flagAll, "cursor", "", "")
if err != nil {
return classifyAPIError(err)
}
diff --git a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/promoted_public.go b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/promoted_public.go
index 1f6de6a5..349c30b4 100644
--- a/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/promoted_public.go
+++ b/testdata/golden/expected/generate-golden-api/printing-press-golden/internal/cli/promoted_public.go
@@ -26,7 +26,7 @@ func newPublicPromotedCmd(flags *rootFlags) *cobra.Command {
path := "/public/status"
params := map[string]string{}
- data, prov, err := resolveRead(c, flags, "public", false, path, params)
+ data, prov, err := resolveRead(c, flags, "public", false, path, params, nil)
if err != nil {
return classifyAPIError(err)
}
← b25c2b90 fix(cli): recover RunID in NewMinimalState so lock promote e
·
back to Cli Printing Press
·
feat(cli): printing-press machine fixes from food52 retro (# 5fa694a0 →