[object Object]

← back to Cli Printing Press

fix(cli): dogfood prefers bundled <dir>/spec.yaml over caller --spec (#1625)

17cd27742e434fbf691001dc98702494dde0d4cd · 2026-05-19 11:14:15 +0200 · Mark van de Ven

* fix(cli): dogfood prefers bundled <dir>/spec.yaml over caller --spec

When the caller invokes `printing-press dogfood --dir X --spec Y` and X
contains a spec archived by `publish package`, the archived spec is now
authoritative — caller's --spec is overridden. Fixes false-negative
Path Validity / Auth Protocol regressions on multi-spec manuscripts
runs (browser-sniff, crowd-sniff, hand-authored-on-top-of-official),
where the caller picks the upstream spec instead of the CLI's own.

Falls through to caller's --spec unchanged when no bundled spec exists.

Fixes #1620

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): typed DogfoodSpecSource const, drop ticket refs

Address Greptile review:
- Introduce DogfoodSpecSource type + DogfoodSpecSourceBundled /
  DogfoodSpecSourceCaller consts (AGENTS.md: categorical strings ->
  typed const at introduction). Matches MCPSurfaceState pattern.
- Remove ticket-number references from code comments (AGENTS.md: no
  ticket numbers in code comments).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(golden): refresh dogfood fixtures for bundled-spec auto-discovery

Two intentional behavior changes from the dogfood spec-resolution fix:

- dogfood-verdict-matrix: stderr gains the `dogfood: using spec ...
  (bundled)` line; fail-path-auth-dead.json gains spec_source field.
- generate-golden-api: same stderr + JSON additions. Also, auth_check
  and browser_session_check now run substantively (they were skipped
  previously because the test invokes `dogfood --dir X` without
  --spec; with bundled-spec auto-discovery the checks engage against
  <X>/spec.yaml as intended).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): clarify auth_check.detail when bundled spec has no recognized scheme

Now that dogfood resolves a bundled spec instead of short-circuiting on
no-spec, checkAuth's "expectedPrefix empty" branch is reachable with a
spec present. The old message "spec not provided or no bot/bearer/basic
scheme detected" contradicted spec_source="bundled" + a non-empty
spec_path in the report. Drop the "spec not provided" half — that case
no longer reaches this branch.

---------

Co-authored-by: Mark van de Ven <10142972+markvandeven@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Trevin Chow <trevin@trevinchow.com>

Files touched

Diff

commit 17cd27742e434fbf691001dc98702494dde0d4cd
Author: Mark van de Ven <markvandeven@users.noreply.github.com>
Date:   Tue May 19 11:14:15 2026 +0200

    fix(cli): dogfood prefers bundled <dir>/spec.yaml over caller --spec (#1625)
    
    * fix(cli): dogfood prefers bundled <dir>/spec.yaml over caller --spec
    
    When the caller invokes `printing-press dogfood --dir X --spec Y` and X
    contains a spec archived by `publish package`, the archived spec is now
    authoritative — caller's --spec is overridden. Fixes false-negative
    Path Validity / Auth Protocol regressions on multi-spec manuscripts
    runs (browser-sniff, crowd-sniff, hand-authored-on-top-of-official),
    where the caller picks the upstream spec instead of the CLI's own.
    
    Falls through to caller's --spec unchanged when no bundled spec exists.
    
    Fixes #1620
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
    
    * fix(cli): typed DogfoodSpecSource const, drop ticket refs
    
    Address Greptile review:
    - Introduce DogfoodSpecSource type + DogfoodSpecSourceBundled /
      DogfoodSpecSourceCaller consts (AGENTS.md: categorical strings ->
      typed const at introduction). Matches MCPSurfaceState pattern.
    - Remove ticket-number references from code comments (AGENTS.md: no
      ticket numbers in code comments).
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
    
    * test(golden): refresh dogfood fixtures for bundled-spec auto-discovery
    
    Two intentional behavior changes from the dogfood spec-resolution fix:
    
    - dogfood-verdict-matrix: stderr gains the `dogfood: using spec ...
      (bundled)` line; fail-path-auth-dead.json gains spec_source field.
    - generate-golden-api: same stderr + JSON additions. Also, auth_check
      and browser_session_check now run substantively (they were skipped
      previously because the test invokes `dogfood --dir X` without
      --spec; with bundled-spec auto-discovery the checks engage against
      <X>/spec.yaml as intended).
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
    
    * fix(cli): clarify auth_check.detail when bundled spec has no recognized scheme
    
    Now that dogfood resolves a bundled spec instead of short-circuiting on
    no-spec, checkAuth's "expectedPrefix empty" branch is reachable with a
    spec present. The old message "spec not provided or no bot/bearer/basic
    scheme detected" contradicted spec_source="bundled" + a non-empty
    spec_path in the report. Drop the "spec not provided" half — that case
    no longer reaches this branch.
    
    ---------
    
    Co-authored-by: Mark van de Ven <10142972+markvandeven@users.noreply.github.com>
    Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: Trevin Chow <trevin@trevinchow.com>
---
 internal/pipeline/dogfood.go                       |  90 +++++++++++-
 internal/pipeline/dogfood_test.go                  | 157 +++++++++++++++++++++
 .../fail-path-auth-dead.json                       |   1 +
 .../expected/dogfood-verdict-matrix/stderr.txt     |   1 +
 .../expected/generate-golden-api/dogfood.json      |   8 +-
 .../golden/expected/generate-golden-api/stderr.txt |   1 +
 6 files changed, 251 insertions(+), 7 deletions(-)

diff --git a/internal/pipeline/dogfood.go b/internal/pipeline/dogfood.go
index b063f8db..7e01771e 100644
--- a/internal/pipeline/dogfood.go
+++ b/internal/pipeline/dogfood.go
@@ -22,9 +22,19 @@ import (
 	"gopkg.in/yaml.v3"
 )
 
+// DogfoodSpecSource identifies which spec input RunDogfood actually loaded
+// when both a bundled <dir>/spec.* and a caller-passed --spec are reachable.
+type DogfoodSpecSource string
+
+const (
+	DogfoodSpecSourceBundled DogfoodSpecSource = "bundled"
+	DogfoodSpecSourceCaller  DogfoodSpecSource = "caller"
+)
+
 type DogfoodReport struct {
 	Dir                    string                       `json:"dir"`
 	SpecPath               string                       `json:"spec_path,omitempty"`
+	SpecSource             DogfoodSpecSource            `json:"spec_source,omitempty"`
 	Verdict                string                       `json:"verdict"`
 	PathCheck              PathCheckResult              `json:"path_check"`
 	AuthCheck              AuthCheckResult              `json:"auth_check"`
@@ -217,10 +227,20 @@ func RunDogfood(dir, specPath string, opts ...DogfoodOption) (*DogfoodReport, er
 		o(&cfg)
 	}
 
+	resolvedSpec, specSource, overriddenCaller := resolveDogfoodSpec(dir, specPath)
+	if resolvedSpec != "" {
+		fmt.Fprintf(os.Stderr, "dogfood: using spec %s (%s)\n", absOrSame(resolvedSpec), specSourceLabel(specSource))
+	}
+	if overriddenCaller != "" {
+		fmt.Fprintf(os.Stderr, "dogfood: caller --spec=%s overridden by bundled %s\n", overriddenCaller, absOrSame(resolvedSpec))
+	}
+	specPath = resolvedSpec
+
 	report := &DogfoodReport{
-		Dir:      dir,
-		SpecPath: specPath,
-		Verdict:  "PASS",
+		Dir:        dir,
+		SpecPath:   specPath,
+		SpecSource: specSource,
+		Verdict:    "PASS",
 	}
 
 	var spec *openAPISpec
@@ -759,6 +779,68 @@ func writeDogfoodResults(report *DogfoodReport, dir string) error {
 	return os.WriteFile(filepath.Join(dir, "dogfood-results.json"), data, 0o644)
 }
 
+// resolveDogfoodSpec picks the spec dogfood should actually load. The spec
+// bundled at <dir>/spec.{json,yaml,yml} by `publish package` is authoritative
+// for the printed CLI; preferring it over a caller-passed --spec avoids
+// false-negative Path Validity / Auth Protocol regressions when the caller
+// reaches into a multi-spec manuscripts directory and picks the upstream spec
+// instead of the internal one the CLI was actually generated from.
+//
+// Returns the resolved path, its DogfoodSpecSource, and — when bundled won
+// over a different caller path — the caller's overridden path for surfacing
+// in a warning. When neither a bundled nor a caller spec exists, all three
+// return values are empty and downstream checks that require a spec are
+// skipped exactly as before.
+func resolveDogfoodSpec(dir, callerSpec string) (resolved string, source DogfoodSpecSource, overriddenCaller string) {
+	bundled := findBundledDogfoodSpec(dir)
+	if bundled != "" {
+		if callerSpec != "" && !samePath(bundled, callerSpec) {
+			overriddenCaller = callerSpec
+		}
+		return bundled, DogfoodSpecSourceBundled, overriddenCaller
+	}
+	if callerSpec != "" {
+		return callerSpec, DogfoodSpecSourceCaller, ""
+	}
+	return "", "", ""
+}
+
+// findBundledDogfoodSpec returns the path to the spec archived alongside a
+// printed CLI by `publish package`, or "" if none is present. Search order
+// mirrors findArchivedSpec: spec.json (JSON inputs), spec.yaml, spec.yml.
+func findBundledDogfoodSpec(dir string) string {
+	for _, name := range []string{"spec.json", "spec.yaml", "spec.yml"} {
+		path := filepath.Join(dir, name)
+		if info, err := os.Stat(path); err == nil && !info.IsDir() {
+			return path
+		}
+	}
+	return ""
+}
+
+func samePath(a, b string) bool {
+	absA, errA := filepath.Abs(a)
+	absB, errB := filepath.Abs(b)
+	if errA != nil || errB != nil {
+		return filepath.Clean(a) == filepath.Clean(b)
+	}
+	return filepath.Clean(absA) == filepath.Clean(absB)
+}
+
+func absOrSame(p string) string {
+	if abs, err := filepath.Abs(p); err == nil {
+		return abs
+	}
+	return p
+}
+
+func specSourceLabel(source DogfoodSpecSource) string {
+	if source == DogfoodSpecSourceBundled {
+		return "bundled"
+	}
+	return "--spec"
+}
+
 func loadDogfoodOpenAPISpec(specPath string) (*openAPISpec, error) {
 	data, err := openapiparser.LoadSpecBytes(specPath, false, false)
 	if err != nil {
@@ -994,7 +1076,7 @@ func checkAuth(dir string, auth apispec.AuthConfig) AuthCheckResult {
 	}
 
 	if expectedPrefix == "" {
-		result.Detail = "spec not provided or no bot/bearer/basic scheme detected"
+		result.Detail = "no bot/bearer/basic scheme detected"
 		return result
 	}
 
diff --git a/internal/pipeline/dogfood_test.go b/internal/pipeline/dogfood_test.go
index 306c163f..07ed1b6f 100644
--- a/internal/pipeline/dogfood_test.go
+++ b/internal/pipeline/dogfood_test.go
@@ -1879,6 +1879,163 @@ func writeTestFile(t *testing.T, path string, content string) {
 	require.NoError(t, os.WriteFile(path, []byte(content), 0o644))
 }
 
+// --- resolveDogfoodSpec ---
+
+func TestResolveDogfoodSpec_PrefersBundledOverCallerSpec(t *testing.T) {
+	dir := t.TempDir()
+	bundled := filepath.Join(dir, "spec.yaml")
+	writeTestFile(t, bundled, "openapi: 3.0.0\n")
+
+	caller := filepath.Join(t.TempDir(), "upstream.yaml")
+	writeTestFile(t, caller, "openapi: 3.0.0\n")
+
+	resolved, source, overridden := resolveDogfoodSpec(dir, caller)
+	assert.Equal(t, bundled, resolved)
+	assert.Equal(t, DogfoodSpecSourceBundled, source)
+	assert.Equal(t, caller, overridden)
+}
+
+func TestResolveDogfoodSpec_NoOverrideWhenCallerPointsAtBundled(t *testing.T) {
+	dir := t.TempDir()
+	bundled := filepath.Join(dir, "spec.yaml")
+	writeTestFile(t, bundled, "openapi: 3.0.0\n")
+
+	resolved, source, overridden := resolveDogfoodSpec(dir, bundled)
+	assert.Equal(t, bundled, resolved)
+	assert.Equal(t, DogfoodSpecSourceBundled, source)
+	assert.Empty(t, overridden, "caller path equal to bundled should not be reported as overridden")
+}
+
+func TestResolveDogfoodSpec_FallsThroughWhenNoBundled(t *testing.T) {
+	dir := t.TempDir() // empty, no spec.* archived
+	caller := filepath.Join(t.TempDir(), "upstream.yaml")
+	writeTestFile(t, caller, "openapi: 3.0.0\n")
+
+	resolved, source, overridden := resolveDogfoodSpec(dir, caller)
+	assert.Equal(t, caller, resolved)
+	assert.Equal(t, DogfoodSpecSourceCaller, source)
+	assert.Empty(t, overridden)
+}
+
+func TestResolveDogfoodSpec_EmptyWhenNeitherPresent(t *testing.T) {
+	resolved, source, overridden := resolveDogfoodSpec(t.TempDir(), "")
+	assert.Empty(t, resolved)
+	assert.Empty(t, source)
+	assert.Empty(t, overridden)
+}
+
+func TestResolveDogfoodSpec_PrefersSpecJSONOverSpecYAML(t *testing.T) {
+	dir := t.TempDir()
+	jsonSpec := filepath.Join(dir, "spec.json")
+	yamlSpec := filepath.Join(dir, "spec.yaml")
+	writeTestFile(t, jsonSpec, `{"openapi":"3.0.0"}`)
+	writeTestFile(t, yamlSpec, "openapi: 3.0.0\n")
+
+	resolved, source, _ := resolveDogfoodSpec(dir, "")
+	assert.Equal(t, jsonSpec, resolved, "spec.json should win when both archive formats are present (mirrors findArchivedSpec)")
+	assert.Equal(t, DogfoodSpecSourceBundled, source)
+}
+
+// End-to-end: RunDogfood should score against the bundled spec when the caller
+// passes a different (smaller) spec.
+func TestRunDogfood_BundledSpecOverridesCallerSpec(t *testing.T) {
+	dir := t.TempDir()
+	require.NoError(t, os.MkdirAll(filepath.Join(dir, "internal", "cli"), 0o755))
+	require.NoError(t, os.MkdirAll(filepath.Join(dir, "internal", "client"), 0o755))
+	require.NoError(t, os.MkdirAll(filepath.Join(dir, "internal", "store"), 0o755))
+
+	writeTestFile(t, filepath.Join(dir, "internal", "cli", "root.go"), `package cli
+type rootFlags struct{}
+func initFlags(flags *rootFlags) { _ = flags }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "cli", "users_get.go"), `package cli
+func usersGet() { path := "/users/{id}"; _ = path }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "cli", "projects_get.go"), `package cli
+func projectsGet() { path := "/projects/{id}"; _ = path }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "client", "client.go"), `package client
+func authHeader(token string) string { return "Bearer " + token }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "store", "store.go"), "package store\n")
+
+	// Bundled spec: the CLI's own authoritative spec, two endpoints — matches
+	// what the CLI actually implements.
+	bundledSpec := filepath.Join(dir, "spec.json")
+	writeTestFile(t, bundledSpec, `{
+  "paths": {
+    "/users/{id}": {},
+    "/projects/{id}": {}
+  },
+  "components": {
+    "securitySchemes": {
+      "BearerAuth": { "type": "http", "scheme": "bearer" }
+    }
+  }
+}`)
+
+	// Caller's --spec: the upstream / partial spec with only one of the two
+	// endpoints. Today (pre-fix) this drives Path Validity to 1/2.
+	callerSpec := filepath.Join(t.TempDir(), "upstream.json")
+	writeTestFile(t, callerSpec, `{
+  "paths": {
+    "/users/{id}": {}
+  },
+  "components": {
+    "securitySchemes": {
+      "BearerAuth": { "type": "http", "scheme": "bearer" }
+    }
+  }
+}`)
+
+	report, err := RunDogfood(dir, callerSpec)
+	require.NoError(t, err)
+
+	assert.Equal(t, bundledSpec, report.SpecPath, "RunDogfood should record the bundled path it actually loaded")
+	assert.Equal(t, DogfoodSpecSourceBundled, report.SpecSource)
+	assert.Equal(t, 2, report.PathCheck.Tested, "should score against the bundled 2-endpoint spec, not the 1-endpoint caller spec")
+	assert.Equal(t, 2, report.PathCheck.Valid)
+}
+
+// When no spec is archived alongside the CLI, RunDogfood must still honor the
+// caller's --spec — no regression for legacy or orphan CLI directories that
+// pre-date publish package's spec-bundling.
+func TestRunDogfood_FallsBackToCallerSpecWhenNoBundle(t *testing.T) {
+	dir := t.TempDir()
+	require.NoError(t, os.MkdirAll(filepath.Join(dir, "internal", "cli"), 0o755))
+	require.NoError(t, os.MkdirAll(filepath.Join(dir, "internal", "client"), 0o755))
+	require.NoError(t, os.MkdirAll(filepath.Join(dir, "internal", "store"), 0o755))
+
+	writeTestFile(t, filepath.Join(dir, "internal", "cli", "root.go"), `package cli
+type rootFlags struct{}
+func initFlags(flags *rootFlags) { _ = flags }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "cli", "users_get.go"), `package cli
+func usersGet() { path := "/users/{id}"; _ = path }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "client", "client.go"), `package client
+func authHeader(token string) string { return "Bearer " + token }
+`)
+	writeTestFile(t, filepath.Join(dir, "internal", "store", "store.go"), "package store\n")
+
+	callerSpec := filepath.Join(t.TempDir(), "upstream.json")
+	writeTestFile(t, callerSpec, `{
+  "paths": { "/users/{id}": {} },
+  "components": {
+    "securitySchemes": {
+      "BearerAuth": { "type": "http", "scheme": "bearer" }
+    }
+  }
+}`)
+
+	report, err := RunDogfood(dir, callerSpec)
+	require.NoError(t, err)
+
+	assert.Equal(t, callerSpec, report.SpecPath)
+	assert.Equal(t, DogfoodSpecSourceCaller, report.SpecSource)
+	assert.Equal(t, 1, report.PathCheck.Tested)
+}
+
 // --- checkTestPresence ---
 
 func TestCheckTestPresence_PureLogicPackageWithoutTests(t *testing.T) {
diff --git a/testdata/golden/expected/dogfood-verdict-matrix/fail-path-auth-dead.json b/testdata/golden/expected/dogfood-verdict-matrix/fail-path-auth-dead.json
index 803652fb..a40979e1 100644
--- a/testdata/golden/expected/dogfood-verdict-matrix/fail-path-auth-dead.json
+++ b/testdata/golden/expected/dogfood-verdict-matrix/fail-path-auth-dead.json
@@ -78,6 +78,7 @@
     "skipped": true
   },
   "spec_path": "<ARTIFACT_DIR>/dogfood-verdict-matrix/fixtures/fail-path-auth-dead/spec.yaml",
+  "spec_source": "bundled",
   "test_presence": {
     "checked": 0
   },
diff --git a/testdata/golden/expected/dogfood-verdict-matrix/stderr.txt b/testdata/golden/expected/dogfood-verdict-matrix/stderr.txt
index a6f63c2a..edbd6632 100644
--- a/testdata/golden/expected/dogfood-verdict-matrix/stderr.txt
+++ b/testdata/golden/expected/dogfood-verdict-matrix/stderr.txt
@@ -1 +1,2 @@
+dogfood: using spec <ARTIFACT_DIR>/dogfood-verdict-matrix/fixtures/fail-path-auth-dead/spec.yaml (bundled)
 warning: no servers defined in spec; generated CLI will require base_url in config
diff --git a/testdata/golden/expected/generate-golden-api/dogfood.json b/testdata/golden/expected/generate-golden-api/dogfood.json
index 15ac1ee3..0f4c8cb0 100644
--- a/testdata/golden/expected/generate-golden-api/dogfood.json
+++ b/testdata/golden/expected/generate-golden-api/dogfood.json
@@ -1,12 +1,12 @@
 {
   "auth_check": {
-    "detail": "spec not provided; auth protocol check skipped",
-    "generated_format": "",
+    "detail": "no bot/bearer/basic scheme detected",
+    "generated_format": "unknown",
     "match": true,
     "spec_scheme": ""
   },
   "browser_session_check": {
-    "detail": "spec not provided; browser-session auth check skipped",
+    "detail": "browser-session auth not required",
     "pass": true,
     "required": false
   },
@@ -67,6 +67,8 @@
     "checked": 0,
     "skipped": true
   },
+  "spec_path": "<ARTIFACT_DIR>/generate-golden-api/printing-press-golden/spec.yaml",
+  "spec_source": "bundled",
   "test_presence": {
     "checked": 0
   },
diff --git a/testdata/golden/expected/generate-golden-api/stderr.txt b/testdata/golden/expected/generate-golden-api/stderr.txt
index 60a9fb00..27228cab 100644
--- a/testdata/golden/expected/generate-golden-api/stderr.txt
+++ b/testdata/golden/expected/generate-golden-api/stderr.txt
@@ -1,2 +1,3 @@
 warning: could not derive run_id from --research-dir; phase5 dogfood acceptance will refuse to write without it
 Generated printing-press-golden at <ARTIFACT_DIR>/generate-golden-api/printing-press-golden
+dogfood: using spec <ARTIFACT_DIR>/generate-golden-api/printing-press-golden/spec.yaml (bundled)

← ac3e15b4 fix(cli): sync pagination, auth aliases, narrative timing, a  ·  back to Cli Printing Press  ·  fix(cli): detect path params in browser-sniff URL grouper (# 2c3271fc →