← back to Cli Printing Press
test(cli): live-dogfood resolve-success and search error_path coverage (#583)
67c977d8881b798ec9f0459f20b12e881c88e0d5 · 2026-05-04 17:55:39 -0700 · Trevin Chow
* docs(cli): plan live-dogfood resolve-success test coverage
Plan for issue #579: extend internal/pipeline/live_dogfood_test.go
so the chained companion walk and search-aware error_path dispatch
from PR #577 are exercised end-to-end. No production-code changes;
adds rich-fixture builder + ~12 integration tests.
* test(cli): live-dogfood resolve-success and search error_path coverage
U1 — Surgical fix to writeLiveDogfoodFixture so the existing
acceptance-marker test exercises resolve-success end-to-end:
scrub `--limit 2` from widgets list --help Examples (companionSupportsLimit
operates on raw help, not flags-section, so any --limit token there
makes the resolver append --limit 1 and miss the bare-list branch);
return canonical {"results":[{"id":"123"}]} from `widgets list --json`;
assert widgets get happy_path = Pass.
U2 — New writeLiveDogfoodRichFixture exposes a multi-resource hierarchy
(widgets/gizmos/projects-tasks/failing-resource) with argv logging via
PRINTING_PRESS_TEST_ARGV_LOG. Five resolve-success integration tests
cover single-positional, chained-multi-positional, cache hit, companion
--limit, and negative-cache sentinel.
U3 — Adds widgets search/search-no-json/search-positional/delete to
the rich fixture. Per-command env vars
(PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE) drive mode dispatch without
cross-command pollution. Seven search/error_path integration tests
cover --json+empty, --json+fallback, no-json, positional <query>,
non-zero exit, invalid-JSON (only fail mode currently exercised), and
mutation-shape fallthrough.
Tests filter argv-log lines on `--json` substring to exclude --help
probes from companionSupportsLimit. Each test is a distinct top-level
function (no t.Run nesting) for AC-to-test traceability via -run flag.
Closes #579.
* fix(cli): apply ce-code-review findings (PS-001, T-01, T-02, T-03, T-04, M-02, M-03, C-01, C-02, C-03)
Cross-cutting test-quality and correctness fixes from /ce-code-review on PR #583:
- PS-001: strip "AC #3:" ticket-number reference from comment (AGENTS.md
Code & Comment Hygiene rule).
- C-01: correct cache-hit comment narration (alphabetical first widgets
sibling is widgets delete, not widgets describe — was wrong after U3
added widgets delete to the rich fixture).
- T-04: tighten negative-cache reason substrings to "list companion
failed at depth" / "list companion previously failed at depth" so a
hypothetical message like "previously had a transient failed
connection" no longer matches.
- T-02 / M-03 (cross-reviewer agreement): drop the dead `*"--json"*`
fallback arm from gizmos list shell case statement. The fallback arm
was masking a hypothetical companionSupportsLimit regression — bare
--json calls now fall through to the failure branch instead of
silently exiting 0.
- M-02: promote magic threshold (3) to named locals (walkerProbes,
resolverCallsWithCacheHit / resolverCallsWithSentinel) so the
arithmetic is self-documenting and a walker change that adds a probe
kind surfaces with a clear error.
- T-01: change `<=` to `==` on cache-hit and negative-cache assertions.
The bare upper bound silently passed if the walker reduced its own
probe count for an unrelated reason; equality catches both directions.
- T-03: pin the negative companion-leaf invariant in
TestRunLiveDogfoodResolveSuccessSinglePositional. Asserts that bare
`widgets search --json` (the resolver's argv shape if findListCompanion
ever flipped to picking search over list) does NOT appear in argv log.
- C-02: handle error_path probe shape `projects tasks list
__printing_press_invalid__` in the rich fixture. Was silently exiting
0 with plain text, producing a permanent FAIL in every rich-fixture
matrix walk that no test asserted on.
- C-03: handle the resolver's self-companion call shape `projects tasks
list --json` (4-arg, no resolved project-id). Fires when
findListCompanion picks projects tasks list as the companion for
itself; without this branch the walker silently skipped the bare list
probe.
Files touched
A docs/plans/2026-05-04-004-test-live-dogfood-resolve-success-coverage-plan.mdM internal/pipeline/live_dogfood_test.go
Diff
commit 67c977d8881b798ec9f0459f20b12e881c88e0d5
Author: Trevin Chow <trevin@trevinchow.com>
Date: Mon May 4 17:55:39 2026 -0700
test(cli): live-dogfood resolve-success and search error_path coverage (#583)
* docs(cli): plan live-dogfood resolve-success test coverage
Plan for issue #579: extend internal/pipeline/live_dogfood_test.go
so the chained companion walk and search-aware error_path dispatch
from PR #577 are exercised end-to-end. No production-code changes;
adds rich-fixture builder + ~12 integration tests.
* test(cli): live-dogfood resolve-success and search error_path coverage
U1 — Surgical fix to writeLiveDogfoodFixture so the existing
acceptance-marker test exercises resolve-success end-to-end:
scrub `--limit 2` from widgets list --help Examples (companionSupportsLimit
operates on raw help, not flags-section, so any --limit token there
makes the resolver append --limit 1 and miss the bare-list branch);
return canonical {"results":[{"id":"123"}]} from `widgets list --json`;
assert widgets get happy_path = Pass.
U2 — New writeLiveDogfoodRichFixture exposes a multi-resource hierarchy
(widgets/gizmos/projects-tasks/failing-resource) with argv logging via
PRINTING_PRESS_TEST_ARGV_LOG. Five resolve-success integration tests
cover single-positional, chained-multi-positional, cache hit, companion
--limit, and negative-cache sentinel.
U3 — Adds widgets search/search-no-json/search-positional/delete to
the rich fixture. Per-command env vars
(PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE) drive mode dispatch without
cross-command pollution. Seven search/error_path integration tests
cover --json+empty, --json+fallback, no-json, positional <query>,
non-zero exit, invalid-JSON (only fail mode currently exercised), and
mutation-shape fallthrough.
Tests filter argv-log lines on `--json` substring to exclude --help
probes from companionSupportsLimit. Each test is a distinct top-level
function (no t.Run nesting) for AC-to-test traceability via -run flag.
Closes #579.
* fix(cli): apply ce-code-review findings (PS-001, T-01, T-02, T-03, T-04, M-02, M-03, C-01, C-02, C-03)
Cross-cutting test-quality and correctness fixes from /ce-code-review on PR #583:
- PS-001: strip "AC #3:" ticket-number reference from comment (AGENTS.md
Code & Comment Hygiene rule).
- C-01: correct cache-hit comment narration (alphabetical first widgets
sibling is widgets delete, not widgets describe — was wrong after U3
added widgets delete to the rich fixture).
- T-04: tighten negative-cache reason substrings to "list companion
failed at depth" / "list companion previously failed at depth" so a
hypothetical message like "previously had a transient failed
connection" no longer matches.
- T-02 / M-03 (cross-reviewer agreement): drop the dead `*"--json"*`
fallback arm from gizmos list shell case statement. The fallback arm
was masking a hypothetical companionSupportsLimit regression — bare
--json calls now fall through to the failure branch instead of
silently exiting 0.
- M-02: promote magic threshold (3) to named locals (walkerProbes,
resolverCallsWithCacheHit / resolverCallsWithSentinel) so the
arithmetic is self-documenting and a walker change that adds a probe
kind surfaces with a clear error.
- T-01: change `<=` to `==` on cache-hit and negative-cache assertions.
The bare upper bound silently passed if the walker reduced its own
probe count for an unrelated reason; equality catches both directions.
- T-03: pin the negative companion-leaf invariant in
TestRunLiveDogfoodResolveSuccessSinglePositional. Asserts that bare
`widgets search --json` (the resolver's argv shape if findListCompanion
ever flipped to picking search over list) does NOT appear in argv log.
- C-02: handle error_path probe shape `projects tasks list
__printing_press_invalid__` in the rich fixture. Was silently exiting
0 with plain text, producing a permanent FAIL in every rich-fixture
matrix walk that no test asserted on.
- C-03: handle the resolver's self-companion call shape `projects tasks
list --json` (4-arg, no resolved project-id). Fires when
findListCompanion picks projects tasks list as the companion for
itself; without this branch the walker silently skipped the bare list
probe.
---
...t-live-dogfood-resolve-success-coverage-plan.md | 280 +++++++
internal/pipeline/live_dogfood_test.go | 869 ++++++++++++++++++++-
2 files changed, 1146 insertions(+), 3 deletions(-)
diff --git a/docs/plans/2026-05-04-004-test-live-dogfood-resolve-success-coverage-plan.md b/docs/plans/2026-05-04-004-test-live-dogfood-resolve-success-coverage-plan.md
new file mode 100644
index 00000000..2f64fdc6
--- /dev/null
+++ b/docs/plans/2026-05-04-004-test-live-dogfood-resolve-success-coverage-plan.md
@@ -0,0 +1,280 @@
+---
+title: "test: Live-dogfood resolve-success and search-aware error_path coverage"
+type: test
+status: active
+date: 2026-05-04
+deepened: 2026-05-04
+---
+
+# test: Live-dogfood resolve-success and search-aware error_path coverage
+
+## Summary
+
+Extend `internal/pipeline/live_dogfood_test.go` so the chained companion walk (parent plan U2) and search-aware error_path dispatch (parent plan U3) are exercised end-to-end against the fake binary instead of silently exercising the skip path. Add a parallel rich-fixture builder for multi-resource hierarchies, search variants, and mutation-shape commands; surgically update the existing fixture so `TestRunLiveDogfoodWritesAcceptanceMarkerOnPass` asserts real PASS rather than skip-with-overall-PASS. No production-code changes — PR #577's behavior is correct, only its integration coverage is incomplete.
+
+---
+
+## Problem Frame
+
+PR #577 shipped the WU-2 fix-up (chained companion walk, per-companion cache, search-aware error_path, quick-verdict gate) and the unit tests pass. The integration-level tests in `live_dogfood_test.go` silently exercise `resolveSkip` with reason `"no id parseable from companion at depth 0"` because the existing fake-binary fixture returns plain text (`widget 1`) for `widgets list --json`. Result: `TestRunLiveDogfoodWritesAcceptanceMarkerOnPass` passes because the full-level verdict tolerates skip, but the chained walk's success path is never exercised at integration level. Surfaced by `/ce-code-review` (testing reviewer T-01..T-04, cross-corroborated by maintainability T-02 and correctness testing-gaps).
+
+---
+
+## Requirements
+
+- R1. The existing acceptance-marker test exercises resolve-success — `widgets get` happy_path returns `LiveDogfoodStatusPass`, not `LiveDogfoodStatusSkip` (origin: issue #579 AC #3; verifies parent plan R3's implementation at integration level).
+- R2. A rich-fixture builder exposes a multi-resource hierarchy, search variants (both `--query` flag and positional `<query>` shape), a mutation command, and an opt-in argv-logging side channel (origin: issue #579 AC #1).
+- R3. Integration tests cover the five U2 resolve-success scenarios from issue #579: single-positional happy, chained multi-positional happy, cache hit, `companionSupportsLimit` exercise, negative-cache sentinel hit (origin: issue #579 AC #2 first list; verifies parent plan R3, R4 implementations at integration level).
+- R4. Integration tests cover the seven U3 search/error_path scenarios from issue #579: `--json` + empty results, `--json` + non-empty fallback results, no `--json` support, positional `<query>`, non-zero exit, search + invalid JSON (only fail mode currently exercised), mutation with no `--query`/`<query>` (origin: issue #579 AC #2 second list; verifies parent plan R5, R6 implementations at integration level).
+- R5. The argv-logging mechanism is shell-friendly, opt-in (no behavior change when env var unset), and isolated per test via `t.TempDir()`. Sequential matrix-walk ordering is the load-bearing serialization guarantee.
+- R6. Existing tests (`TestRunLiveDogfoodDetectsJSONParseFailure`, `TestRunLiveDogfoodErrorPathAcceptsExpectedNonZeroExit`, `TestRunLiveDogfoodAcceptanceRequiresManifestIdentity`, `TestRunLiveDogfoodExplicitBinaryNameMustExist`) continue to pass against the unchanged simple fixture.
+
+---
+
+## Scope Boundaries
+
+- No production-code changes in `internal/pipeline/live_dogfood.go`, `internal/pipeline/dogfood.go`, or `internal/generator/`. PR #577's behavior is correct; only its integration-test coverage is incomplete.
+- Depends on parent plan `docs/plans/2026-05-04-003-fix-live-dogfood-matrix-accuracy-plan.md` (PR #577) being merged. This plan adds integration-test coverage for already-landed production code; without #577, the assertions exercise behavior that doesn't exist yet.
+- No replacement of the existing `writeLiveDogfoodFixture` — the simple fixture's purpose (intentional regression coverage for malformed-JSON detection via `brokenJSONFixed: false`) is preserved.
+- No new test file. Extend `internal/pipeline/live_dogfood_test.go` rather than splitting integration tests into a separate file.
+- No Windows portability work. Existing tests skip on Windows (`runtime.GOOS == "windows"`); new tests follow the same convention.
+- No golden-fixture changes. Generator output is not affected by this work.
+
+### Deferred to Follow-Up Work
+
+- Cross-platform (Windows) live-dogfood test coverage: would require replacing shell-script fake binaries with Go-built ones across the entire test file. Out of scope here; tracked separately if/when it becomes a real requirement.
+
+---
+
+## Context & Research
+
+### Relevant Code and Patterns
+
+- `internal/pipeline/live_dogfood_test.go:577` — `writeLiveDogfoodFixture` is the existing fake-binary builder. Conditional shell-script generation keyed off `brokenJSONFixed`. Pattern to follow for `writeLiveDogfoodRichFixture`.
+- `internal/pipeline/live_dogfood_test.go:694` — `writeTestManifestForLiveDogfood` writes a minimal `CLIManifest` to the fixture dir. Reusable as-is.
+- `internal/pipeline/live_dogfood.go:252` — `buildSiblingMap` keys on `strings.Join(c.Path[:len(c.Path)-1], " ")`. For a root-level command at `Path = ["get"]`, the key is `""`. Confirms the root-level scenario shape.
+- `internal/pipeline/live_dogfood.go:266` — `findListCompanion` selects the first sibling in `companionLeaf` allowlist. Cross-API verbs (`list`, `all`, `index`, `query`, `find`, `search`, `discover`, `browse`, `recent`, `feed`) and cinema verbs (`popular`, `trending`, etc.).
+- `internal/pipeline/live_dogfood.go:320` — chained walk computation: `siblingKey = strings.Join(command.Path[:pathLen-nPlaceholders+i], " ")`. Implementer should re-read this for chain-test argv shape.
+- `internal/pipeline/dogfood.go:1779` — `extractFlagNames` and `extractFlagsSection` (PR #577 PERF-001). Used by `commandSupportsSearch` and `companionSupportsLimit`.
+- `cliutil.IsVerifyEnv` — established pattern for shell-side env-var sentinels (`PRINTING_PRESS_VERIFY=1`). Follow naming: `PRINTING_PRESS_TEST_ARGV_LOG`.
+
+### Institutional Learnings
+
+- AGENTS.md anti-pattern: "no API names in reusable artifacts." Rich fixture uses generic `widgets`/`projects`/`tasks`/`accounts` names, not API-specific brands.
+- AGENTS.md "Code & Comment Hygiene": no dates, incidents, or ticket numbers in code comments. Fixture comments document shape, not the issue that drove them.
+- Issue #579 was filed because `/ce-code-review` couldn't mechanically write the new shell scripts and orchestrate multi-resource fixtures — keep test scenarios specific enough that an implementer doesn't have to invent coverage.
+
+### External References
+
+None. This is purely test-infrastructure work using established Go testing patterns and POSIX shell.
+
+---
+
+## Key Technical Decisions
+
+- **Hybrid fixture strategy**: surgically update `writeLiveDogfoodFixture` (scrub `--limit 2` from the `widgets list --help` Examples block, delete the now-unreachable `widgets list --limit 2 --json` shell branch, add a clean `widgets list --json` branch returning `{"results":[{"id":"123"}]}`, and update the existing acceptance test's assertion) AND add a parallel `writeLiveDogfoodRichFixture` for multi-resource scenarios. Rationale: the simple fixture's purpose is intentional regression coverage for malformed-JSON detection (`brokenJSONFixed: false`); replacing it would couple unrelated concerns and break the unit-shaped surface the PR #577 retro itself called out as missing. The Examples-block scrub is required because `companionSupportsLimit` calls `extractFlagNames` on raw help text (not flags-section-scoped), so any `--limit` token anywhere in the help — including Examples — makes the resolver append `--limit 1` to the companion call and miss a bare `widgets list --json` branch. Drift risk between the two shell scripts is small (~30 lines of overlap) and surfaces as test failures the next time someone touches either.
+- **argv recording via env-var-pointed tempfile**: `PRINTING_PRESS_TEST_ARGV_LOG=$(mktemp)` set per-test; rich fixture's shell script appends `printf '%s\n' "$*" >> "$LOG"` on every invocation guarded by `[ -n "${PRINTING_PRESS_TEST_ARGV_LOG:-}" ]`. Rationale: the live-dogfood walker iterates the matrix sequentially, so writes are serialized by construction — no concurrency-atomicity guarantees are needed. A counter directory (mkdir-based) gives count but not content; a Go fake binary would require build ceremony for no portability gain over the shell-skip-on-Windows convention. Defensive env-var guard means tests that don't care don't have to set it. **Important — `--help` invocations also log**: `companionSupportsLimit` invokes `<companion> --help` as a separate subprocess before the actual companion call, and its argv is captured the same way. Tests asserting on companion-call counts must filter to `--json`-bearing argv lines (the actual companion call), not just the companion path.
+- **Per-command behavior-mode env vars** for U3 fixture variants: instead of one `PRINTING_PRESS_TEST_SEARCH_MODE` that affects every search-shape branch in a matrix walk, use distinct keys per command — e.g., `PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE=empty|fallback|invalid` only affects `widgets search`, `PRINTING_PRESS_TEST_WIDGETS_SEARCH_POSITIONAL_MODE` only affects `widgets search-positional`. Defaults to `empty` when unset. Rationale: the matrix walker probes ALL commands per run, so a single shared env var would cross-pollute branches inside one test, hiding real failures or producing spurious ones.
+- **Three implementation units** (not 2 or 4): U1 isolates the AC #3 fix as a small standalone commit (one help-text scrub + one shell-branch swap + one assertion update); U2 bundles the rich-fixture builder with its first five consumers (resolve-success scenarios) so the fixture is reviewable end-to-end; U3 layers the seven search/error_path scenarios on top. **U3 must execute strictly serial after U2** — both extend the same `writeLiveDogfoodRichFixture` shell-script literal, so parallel ce-work dispatch would produce a literal-merge conflict in the worktree-isolation flow. Rationale: a fixture-only commit (4-unit shape) lacks a consumer and is unreviewable in isolation; a single-test-commit shape (2-unit) creates a ~400-line append that crosses the adversarial-reviewer threshold and bundles two distinct test concerns.
+- **Test function naming convention**: U2's resolve-success tests use the `TestRunLiveDogfoodResolveSuccess*` prefix (e.g., `TestRunLiveDogfoodResolveSuccessSinglePositional`, `TestRunLiveDogfoodResolveSuccessChainedMultiPositional`, `TestRunLiveDogfoodResolveSuccessCacheHit`, `TestRunLiveDogfoodResolveSuccessCompanionLimit`, `TestRunLiveDogfoodResolveSuccessNegativeCacheSentinel`). U3's tests use `TestRunLiveDogfoodSearchErrorPath*` (e.g., `TestRunLiveDogfoodSearchErrorPathEmptyResults`, `TestRunLiveDogfoodSearchErrorPathFallbackResults`, etc.). Each scenario is a distinct top-level test function — no table-driven `t.Run` nesting — to keep AC-to-test traceability obvious in test output and `-run` flag use. Rationale: 13 new tests added to a file with 6 existing `TestRunLiveDogfood*` tests; without a convention, names drift and AC traceability is lost.
+- **"Only fail mode" is a tripwire, not a hard claim**: U3's "search + invalid JSON" scenario is the only fail mode currently produced by the search-shape error_path code in `live_dogfood.go:693-711`. Future production-code changes that add new fail branches (timeout, empty stdout under `--json`, schema-mismatch) require a corresponding new U3-shape integration test in the same change. Documented in U3's verification block as an invariant note so the assertion doesn't quietly become a coverage gap.
+- **Generic resource names** (`widgets`, `projects`, `tasks`, `accounts`): no API-specific brands. AGENTS.md "no API names in reusable artifacts" applies even to test fixtures because test code compounds across CLIs.
+
+---
+
+## Open Questions
+
+### Resolved During Planning
+
+- Fixture extension strategy: hybrid (parallel builder + minimal update). See Key Technical Decisions.
+- argv recording mechanism: env-var-pointed tempfile, with explicit caveat that `--help` probes also log (assertions filter on `--json` substring). See Key Technical Decisions.
+- Unit shape: 3 units; U3 strictly serial after U2 due to shared fixture literal. See Key Technical Decisions.
+- Root-level get scenario: dropped from R3. Production code's `resolveCommandPositionals` short-circuits at `pathLen < nPlaceholders+1`; for a top-level `get <id>` command at `Path=["get"]` with one placeholder, `1 < 2` fires and the resolver skips before consulting the sibling map. The empty-key sibling case is unreachable at integration level; the existing unit test `TestResolveCommandPositionalsSkipPaths` already covers the rejected shape. R3 reduced from 6 to 5 scenarios.
+- U3 fixture mode dispatch: per-command env var keys (`PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE`, `PRINTING_PRESS_TEST_WIDGETS_SEARCH_POSITIONAL_MODE`), not a single shared key. See Key Technical Decisions.
+- Test function naming convention: distinct top-level test functions with `TestRunLiveDogfoodResolveSuccess*` and `TestRunLiveDogfoodSearchErrorPath*` prefixes. See Key Technical Decisions.
+- Companion-leaf invariant: U2 and U3 tests must assert the chosen companion explicitly via argv-log substring (`widgets list --json` not just any companion path). Documented in U2's verification block.
+
+### Deferred to Implementation
+
+- Exact JSON shape returned by `projects list --json` and `projects tasks list <pid> --json`: pick the canonical-path shape (`.results[0].id`) for clarity unless a chain-walk test specifically exercises a different path. Implementer chooses during U2.
+- Whether to gate `widgets describe` (cache-hit sibling) under a fixture opt or always include it: include always — cheap to add a no-op branch, and tests that don't probe it ignore it.
+- Whether to record argv as `\t`-joined or newline-joined per invocation: pick newline-per-invocation for trivial parsing; argv tokens within a line joined with single spaces (matching `$*` shell behavior). Implementer can adjust if a test needs token-level precision.
+- Argv ordering for `--limit 1` (resolver may emit `--json --limit 1` or `--limit 1 --json`): implementer should verify against `live_dogfood.go` resolver logic when wiring U2's `companionSupportsLimit` test fixture branches.
+
+---
+
+## Implementation Units
+
+- U1. **Surgical update to existing fixture + acceptance-test assertion fix**
+
+**Goal:** Make `TestRunLiveDogfoodWritesAcceptanceMarkerOnPass` exercise the resolve-success path end-to-end, asserting `widgets get` happy_path returns `LiveDogfoodStatusPass`. AC #3 from issue #579.
+
+**Requirements:** R1, R6
+
+**Dependencies:** None.
+
+**Files:**
+- Modify: `internal/pipeline/live_dogfood_test.go` (scrub `--limit 2` from `widgets list --help` Examples block; delete the unreachable `widgets list --limit 2 --json` shell branch; add a `widgets list --json` branch returning canonical-path JSON; add one assertion in `TestRunLiveDogfoodWritesAcceptanceMarkerOnPass`)
+
+**Approach:**
+- In `writeLiveDogfoodFixture`'s `widgets list --help` block, remove the `Examples:` line containing `widgets list --limit 2`. This makes `extractFlagNames(help)` return no `limit` flag, so `companionSupportsLimit` returns false, so the resolver calls `widgets list --json` (no `--limit`).
+- Delete the existing shell branch matching `--limit 2 --json` (lines 657-660 in current source). It returned `{"widgets":[{"id":"1"}]}`, which is unreachable after the help scrub and was using a non-canonical JSON path (`.widgets[]` is not in the resolver's 7-path try-list anyway, so it would never have been parseable).
+- Add a new shell branch matching `widgets list --json` returning `{"results":[{"id":"123"}]}` — canonical path #1 (`.results[0].id`) so the resolver parses it on the first attempt.
+- In `TestRunLiveDogfoodWritesAcceptanceMarkerOnPass`, add an assertion that locates the `widgets get` happy_path test result via `Command == "widgets get" && Kind == LiveDogfoodTestHappy` and asserts `Status == LiveDogfoodStatusPass`. Existing overall-verdict and marker-file assertions stay.
+- Verify the existing `widgets get 123 --json` branch (already returns `{"id":"123"}`) and the no-json branch (returns `widget 123`) still work — they should.
+
+**Patterns to follow:**
+- Existing shell branch structure in `writeLiveDogfoodFixture`.
+- Existing `report.Tests[i].Command == "widgets get" && report.Tests[i].Kind == LiveDogfoodTestHappy` lookup pattern from `TestRunLiveDogfoodErrorPathAcceptsExpectedNonZeroExit:90-99`.
+
+**Test scenarios:**
+- Happy path — `widgets get` happy_path resolves via companion `widgets list --json` returning `{"results":[{"id":"123"}]}`, gets substituted to `widgets get 123`, runs against fixture's `widgets get 123` branch, returns `Status == LiveDogfoodStatusPass`. Existing overall verdict assertion still passes.
+- Edge case — `TestRunLiveDogfoodDetectsJSONParseFailure` (which uses `brokenJSONFixed: false`) continues to pass because the help scrub and new shell branch only affect `widgets list`, not `widgets broken`.
+- Regression — all other existing tests against `writeLiveDogfoodFixture` (`TestRunLiveDogfoodErrorPathAcceptsExpectedNonZeroExit`, `TestRunLiveDogfoodAcceptanceRequiresManifestIdentity`, `TestRunLiveDogfoodExplicitBinaryNameMustExist`) still pass.
+
+**Verification:**
+- `go test ./internal/pipeline/... -run TestRunLiveDogfoodWritesAcceptanceMarkerOnPass` passes with the new assertion.
+- `go test ./internal/pipeline/... -run TestRunLiveDogfoodDetectsJSONParseFailure` still passes (no regression on broken-JSON coverage).
+- `go test ./internal/pipeline/...` passes overall.
+
+---
+
+- U2. **Rich-fixture builder + U2 resolve-success integration tests**
+
+**Goal:** Build `writeLiveDogfoodRichFixture` exposing a multi-resource hierarchy with argv logging, and add the five U2 resolve-success integration tests from issue #579 AC #2 first list (root-level scenario dropped — see Open Questions).
+
+**Requirements:** R2, R3, R5
+
+**Dependencies:** None (independent of U1; both modify the same test file but target separate fixture builders and separate test functions, so merge-conflict surface is small).
+
+**Files:**
+- Modify: `internal/pipeline/live_dogfood_test.go` (add `writeLiveDogfoodRichFixture`; add five resolve-success test functions)
+
+**Approach:**
+
+Rich fixture exposes:
+- `widgets list` (no `--limit` declared in help — see U1 rationale on raw-help flag detection), `widgets get <id>`, `widgets describe <id>` (cache-hit sibling).
+- `widgets list-with-limit` (separate companion that DOES declare `--limit` in its help — for the `companionSupportsLimit` test exclusively, with its own `widgets get-with-limit <id>` sibling so its companion choice is unambiguous and isolated).
+- `projects list`, `projects tasks list <project-id>`, `projects tasks update <project-id> <task-id>` (multi-resource chain).
+- `failing-resource list` (returns exit non-zero for negative-cache sentinel test) and `failing-resource get <id>`, `failing-resource describe <id>` (two siblings sharing a failing companion).
+
+**Companion-leaf invariant**: every get-shape command in the rich fixture has exactly ONE allowlisted sibling, and that sibling's leaf name is intentionally chosen to win alphabetically against any other commands the fixture might add later. `widgets`-family siblings: only `widgets list` is allowlisted (`describe`, `get` are not in `crossAPIListVerbs`/`cinemaListVerbs`). `projects tasks`-family: only `projects tasks list`. `failing-resource`-family: only `failing-resource list`. Tests explicitly assert the chosen companion via argv-log substring (e.g., `widgets list --json` must appear, not just any `widgets *` companion call) so future fixture additions can't silently change which companion wins.
+
+argv logging:
+- Defensive shell guard: `[ -n "${PRINTING_PRESS_TEST_ARGV_LOG:-}" ] && printf '%s\n' "$*" >> "$PRINTING_PRESS_TEST_ARGV_LOG"`.
+- Tests opt in by setting `t.Setenv("PRINTING_PRESS_TEST_ARGV_LOG", filepath.Join(t.TempDir(), "argv.log"))` before invoking `RunLiveDogfood`.
+- **`--help` invocations log too**: `companionSupportsLimit` calls `<companion> --help` as a separate subprocess before the actual companion call. Tests asserting on companion-call counts must filter argv lines to those containing `--json` (the actual companion call), not just the companion path.
+
+Each command's `--help` block declares its flags accurately so `commandSupportsJSON`, `commandSupportsSearch`, `companionSupportsLimit`, and `extractPositionalPlaceholders` see the right surface. Per the U1 finding, `companionSupportsLimit` operates on raw help text — keep `--limit` out of help blocks for companions where the test wants the resolver to NOT append `--limit 1`.
+
+**Patterns to follow:**
+- `writeLiveDogfoodFixture` shell-script-template structure.
+- `t.Setenv` for env-var isolation per test (avoids cross-test pollution).
+- `t.TempDir()` for argv-log file isolation.
+
+**Test scenarios** (5 — root-level scenario dropped per Open Questions, each becomes a distinct top-level test function with `TestRunLiveDogfoodResolveSuccess*` prefix):
+
+- Happy path (`TestRunLiveDogfoodResolveSuccessSinglePositional`) — fixture exposes `widgets get <id>` with companion `widgets list --json` returning `{"results":[{"id":"42"}]}`. Probe substitutes id, runs `widgets get 42`, returns `LiveDogfoodStatusPass`. argv-log assertion: contains `widgets list --json` (companion-pin) and `widgets get 42` (probe).
+- Happy path (`TestRunLiveDogfoodResolveSuccessChainedMultiPositional`) — fixture exposes `projects tasks update <project-id> <task-id>`. Probe walks `projects list --json` → `{"results":[{"id":"P1"}]}`, then `projects tasks list P1 --json` → `{"results":[{"id":"T7"}]}`, then runs `projects tasks update P1 T7`. argv log records both companion calls in chained order plus the final probe call.
+- Happy path (`TestRunLiveDogfoodResolveSuccessCacheHit`) — fixture exposes `widgets get <id>` and `widgets describe <id>` sharing companion `widgets list`. Run live-dogfood once; assert `count(argv lines containing "widgets list --json") == 1` (filtered to exclude the `--help` probe), plus both `widgets get 42` and `widgets describe 42` probes appear in log.
+- Happy path (`TestRunLiveDogfoodResolveSuccessCompanionLimit`) — fixture's `widgets list-with-limit --help` declares a `--limit` flag in its Flags section. Resolver appends `--limit 1` to the companion call. Probe `widgets get-with-limit <id>` resolves via this companion. Assert argv log contains a line matching both `widgets list-with-limit` AND `--limit 1` AND `--json` (substring, order-agnostic since resolver may emit `--json --limit 1` or `--limit 1 --json`).
+- Edge case (`TestRunLiveDogfoodResolveSuccessNegativeCacheSentinel`) — fixture exposes `failing-resource get <id>` and `failing-resource describe <id>` sharing failing companion `failing-resource list` (exit 2 on `--json` call; exit 0 on `--help`). First probe runs `companionSupportsLimit` (`failing-resource list --help`, succeeds + cached) then the actual companion (`failing-resource list --json`, fails) — sentinel cached. Second probe hits the sentinel without re-invoking the actual companion. Assert `count(argv lines containing "failing-resource list" AND "--json") == 1` (the `--help` from probe 1 is filtered out and `companionSupportsLimit`'s help cache prevents a second `--help` call). Both probes return `LiveDogfoodStatusSkip` with reasons naming the failed companion (probe-1 reason includes `exit N`; probe-2 reason includes `previously failed`).
+
+**Verification:**
+- `go test ./internal/pipeline/...` passes including all five new test functions.
+- Each test's argv-log assertion explicitly pins the chosen companion (e.g., `widgets list --json` not just `widgets list`) so future fixture additions can't silently flip companion choice.
+- argv log assertions work reliably across consecutive test runs (no leakage between tests via `t.TempDir()` isolation).
+- Each test exits cleanly without leaving zombie subprocesses or temp files (Go test framework handles cleanup).
+
+---
+
+- U3. **U3 search-aware error_path integration tests**
+
+**Goal:** Add the seven search-shape and mutation-shape error_path tests from issue #579 AC #2 second list, layered on top of U2's rich fixture.
+
+**Requirements:** R4
+
+**Dependencies:** U2 (consumes `writeLiveDogfoodRichFixture` and its argv-logging convention). **Strictly serial after U2** — U3 extends the same `writeLiveDogfoodRichFixture` shell-script literal that U2 introduces. Parallel ce-work dispatch under worktree isolation would produce a literal-merge conflict in the multi-line raw string. Run sequentially.
+
+**Files:**
+- Modify: `internal/pipeline/live_dogfood_test.go` (extend `writeLiveDogfoodRichFixture` with `widgets search`, `widgets search-no-json`, `widgets search-positional`, `widgets delete` branches; add seven test functions)
+
+**Approach:**
+
+Rich-fixture additions for U3 (each command honors its OWN env-var key — see Key Technical Decisions on per-command mode dispatch):
+
+- `widgets search` with `--query` flag and `--json` flag (declared in Flags section of help). Behavior dispatch via `PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE`:
+ - unset or `empty` → exit 0 + `{"results":[]}`
+ - `fallback` → exit 0 + `{"results":[{"id":"recent-1"},{"id":"recent-2"}]}`
+ - `nonzero` → exit 4 + empty stdout
+ - `invalid` → exit 0 + `{not-json` (the only fail mode currently)
+- `widgets search-no-json` with `--query` flag but NO `--json` flag in help. Returns exit 0 + plain text `0 results found.` Used for the "no `--json` support" scenario.
+- `widgets search-positional` with positional `<query>` (no `--query` flag in help) and `--json` flag. Branch: `<q> --json` returns exit 0 + `{"results":[]}`. Drives the positional-query-arg-construction assertion.
+- `widgets delete <id>` (mutation-shape: no `--query` flag, no `<query>` positional, accepts id positional). Branch: `widgets delete __printing_press_invalid__` returns exit 2. Verifies mutation-shape commands fall through to the non-zero-required strategy.
+
+The fixture surface grows but each branch is a small shell conditional. Structure shell branches by command then by env-var mode then by argv shape.
+
+**Patterns to follow:**
+- U2's argv-logging convention (and its `--help` filtering caveat).
+- U2's companion-leaf invariant — none of U3's added commands should leak into the U2 widgets-family companion choice (`widgets search` IS in `crossAPIListVerbs`, but `widgets list` sorts alphabetically before `widgets search`, so `findListCompanion` still picks `widgets list` for `widgets get`/`widgets describe`. Tests should pin this via argv-log assertion).
+- `writeLiveDogfoodFixture`'s shell-branch-per-arg-shape structure.
+- `t.Setenv` for per-test env-var control. Each test sets exactly the env vars it needs; defaults handle the rest.
+
+**Test scenarios** (7 — each becomes a distinct top-level test function with `TestRunLiveDogfoodSearchErrorPath*` prefix):
+
+- Happy path (`TestRunLiveDogfoodSearchErrorPathEmptyResults`) — env vars unset (default empty). `widgets search --query __printing_press_invalid__ --json` returns exit 0 + `{"results":[]}`. error_path = `LiveDogfoodStatusPass`.
+- Happy path (`TestRunLiveDogfoodSearchErrorPathFallbackResults`) — `t.Setenv("PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE", "fallback")`. Same probe shape; fixture returns exit 0 + `{"results":[{"id":"recent-1"}]}`. error_path = `LiveDogfoodStatusPass` (recency-fallback APIs return content under unmatched queries).
+- Happy path (`TestRunLiveDogfoodSearchErrorPathNoJSONSupport`) — probe runs `widgets search-no-json --query __printing_press_invalid__` (no `--json`); fixture returns exit 0 + plain text. error_path = `LiveDogfoodStatusPass` (exit 0 is sufficient when `--json` wasn't supplied).
+- Happy path (`TestRunLiveDogfoodSearchErrorPathPositionalQuery`) — probe runs `widgets search-positional __printing_press_invalid__ --json`; fixture returns exit 0 + `{"results":[]}`. error_path = `LiveDogfoodStatusPass`. argv-log assertion: probe used positional argument, NOT `--query` flag.
+- Happy path (`TestRunLiveDogfoodSearchErrorPathNonZeroExit`) — `t.Setenv("PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE", "nonzero")`. Fixture returns exit 4. error_path = `LiveDogfoodStatusPass` (non-zero exit is also a valid "no match" signal for some APIs).
+- Error path (`TestRunLiveDogfoodSearchErrorPathInvalidJSON`) — `t.Setenv("PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE", "invalid")`. Fixture returns exit 0 + `{not-json` under `--json`. error_path = `LiveDogfoodStatusFail` with reason matching `"invalid JSON"`. **Tripwire**: this is the only fail mode currently produced by `live_dogfood.go:693-711`'s search-shape switch. If a future production change adds a new fail branch (timeout, empty stdout under `--json`, schema mismatch), add a corresponding new `TestRunLiveDogfoodSearchErrorPath*` test in the same change.
+- Mutation-shape (`TestRunLiveDogfoodSearchErrorPathMutationFallthrough`) — probe runs `widgets delete __printing_press_invalid__`; fixture returns exit 2. error_path = `LiveDogfoodStatusPass`. Verifies `commandSupportsSearch` correctly rejects mutation-shape commands (no `--query` flag, no `<query>` positional) and falls through to the existing non-zero-required strategy.
+
+**Verification:**
+- `go test ./internal/pipeline/...` passes including all seven new test functions.
+- All seven scenarios use the same `writeLiveDogfoodRichFixture` from U2 — no fixture proliferation.
+- `commandSupportsSearch` predicate correctness verified at integration level (search-shape commands take search strategy; mutation-shape commands take mutation strategy).
+
+---
+
+## System-Wide Impact
+
+- **Interaction graph:** No production code changes. New tests interact with `RunLiveDogfood` and `writeTestManifestForLiveDogfood` (existing pattern). Argv-log env var (`PRINTING_PRESS_TEST_ARGV_LOG`) is a test-only side channel — never set in production code paths.
+- **Error propagation:** Test failures surface via `assert`/`require` in standard Go testing pattern. argv-log read errors should `require.NoError` to fail tests fast; missing log files when expected indicate test setup bugs, not production issues.
+- **State lifecycle risks:** None. Each test owns its `t.TempDir()` and `t.Setenv` scope. Go test framework cleans up subprocess and tempfile state.
+- **API surface parity:** No public-API changes. `writeLiveDogfoodRichFixture` is package-internal (lowercase initial). Argv-log env-var name (`PRINTING_PRESS_TEST_ARGV_LOG`) follows the established `PRINTING_PRESS_*` naming convention but is documented only in test code, not in user-facing docs.
+- **Integration coverage:** This plan IS the integration coverage. Unit tests in PR #577 already exist for the helpers (`extractFirstIDFromJSON`, `commandSupportsSearch`, `buildSiblingMap`, `findListCompanion`, `substitutePositionals`); this plan adds the missing end-to-end integration layer.
+- **Unchanged invariants:** `writeLiveDogfoodFixture` interface (`(t *testing.T, brokenJSONFixed bool)`) is unchanged. Existing tests using it (`TestRunLiveDogfoodDetectsJSONParseFailure`, `TestRunLiveDogfoodErrorPathAcceptsExpectedNonZeroExit`, `TestRunLiveDogfoodAcceptanceRequiresManifestIdentity`, `TestRunLiveDogfoodExplicitBinaryNameMustExist`) continue to pass. Production code in `live_dogfood.go` and `dogfood.go` is unchanged.
+
+---
+
+## Risks & Dependencies
+
+| Risk | Mitigation |
+|------|------------|
+| Argv-log writes interleave under parallel subprocess invocation | Live-dogfood walker iterates the matrix sequentially, so writes are serialized by construction. Forbid `t.Parallel()` in this test file (no current test uses it; future contributors must not add it without redesigning the argv-log mechanism). |
+| Drift between simple fixture and rich fixture as features land in one but not the other | Drift surfaces as test failures the next time someone touches either. Acceptable cost; both fixtures live in the same test file ~100 lines apart, making review at touch-time straightforward. |
+| Rich-fixture shell script becomes hard to read as branches multiply | Structure shell branches by command then by env-var mode then by argv shape (matches existing `writeLiveDogfoodFixture` pattern). If branch count grows past ~50, consider splitting into per-command helper builders. Not a concern at this plan's scope (~25 branches across 8 commands). |
+| Per-command env-var dispatch (`PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE` etc.) creates a proliferation of env vars | Each env var has a defensive default (`empty` when unset), is documented in the rich-fixture comment, and is scoped to exactly one command's behavior. Tests `t.Setenv` only the variables they need. The cost is one extra env var per command-with-multiple-modes, which is a small price for not having shared-mode pollution across the matrix walk. |
+| Companion-leaf alphabetical-sort dependence (e.g., `widgets list` wins over `widgets search`) silently breaks if a future fixture rename or addition flips the order | Tests pin the chosen companion explicitly via argv-log substring assertion. Document the constraint in the rich-fixture comment so a reviewer adding new commands sees the invariant. |
+| Companion `--help` invocations from `companionSupportsLimit` show up in argv log alongside actual companion calls | Tests filter argv lines on `--json` substring (the actual companion call), not the bare companion path. Documented as a Key Technical Decision and reinforced in U2's verification block. |
+| `companionSupportsLimit` operates on raw help text (not flags-section-scoped, unlike `commandSupportsSearch`), making fixture help blocks load-bearing for any `--limit` token anywhere in the help | Documented as Key Technical Decision (U1's help-text scrub motivation). For companions where the test wants `--limit 1` appended, declare `--limit` in the Flags section. For companions where the test wants the bare call, keep `--limit` out of the help entirely (including Examples). The asymmetry between `companionSupportsLimit` and `commandSupportsSearch` is a latent inconsistency in PR #577's production code; this plan codifies it into fixture conventions rather than fixing it. |
+
+---
+
+## Documentation / Operational Notes
+
+- No user-facing doc updates. The argv-log env var is test-internal and not documented in user-facing material.
+- No skill or generator-template changes. AGENTS.md skill-authoring rule ("update SKILL.md when machine change alters what an agent should do") doesn't apply — no machine change here.
+
+---
+
+## Sources & References
+
+- **Origin issue:** [issue #579](https://github.com/mvanhorn/cli-printing-press/issues/579) — Test coverage gap surfaced by `/ce-code-review` on PR #577.
+- **Parent sub-issue:** [issue #573](https://github.com/mvanhorn/cli-printing-press/issues/573) — WU-2 from retro #571.
+- **Parent retro:** [issue #571](https://github.com/mvanhorn/cli-printing-press/issues/571) — Movie Goat retro that surfaced the live-dogfood matrix accuracy gaps.
+- **Implementing PR:** [PR #577](https://github.com/mvanhorn/cli-printing-press/pull/577) — WU-2 fix-up (camelCase ID, chained companion walk, search-aware error_path, quick-verdict gate). The production code being tested.
+- **Parent plan:** `docs/plans/2026-05-04-003-fix-live-dogfood-matrix-accuracy-plan.md` — R3, R4, R5, R6 trace through to this plan's R3, R4.
+- **Run artifact:** `/tmp/compound-engineering/ce-code-review/20260504-141500-907bca25/` — `testing.json` carries per-finding evidence from the original code review pass.
+- Related code: `internal/pipeline/live_dogfood.go`, `internal/pipeline/live_dogfood_test.go`, `internal/pipeline/dogfood.go`.
diff --git a/internal/pipeline/live_dogfood_test.go b/internal/pipeline/live_dogfood_test.go
index f48dfc1f..2734c16b 100644
--- a/internal/pipeline/live_dogfood_test.go
+++ b/internal/pipeline/live_dogfood_test.go
@@ -5,6 +5,7 @@ import (
"os"
"path/filepath"
"runtime"
+ "strings"
"testing"
"time"
@@ -59,6 +60,14 @@ func TestRunLiveDogfoodWritesAcceptanceMarkerOnPass(t *testing.T) {
require.NoError(t, err)
require.Equal(t, "PASS", report.Verdict, report.Tests)
+ // widgets get happy_path must exercise the resolve-success chain
+ // (companion widgets list returns parseable JSON, resolver substitutes the
+ // id, get probe runs and passes), not silently skip on companion-parse
+ // failure as it did before the fixture fix.
+ widgetsGetHappy := findResultByCommandKind(report, "widgets get", LiveDogfoodTestHappy)
+ require.NotNil(t, widgetsGetHappy, "expected widgets get happy_path test result in report")
+ assert.Equal(t, LiveDogfoodStatusPass, widgetsGetHappy.Status, widgetsGetHappy.Reason)
+
data, err := os.ReadFile(markerPath)
require.NoError(t, err)
var marker Phase5GateMarker
@@ -613,7 +622,7 @@ Usage:
fixture-pp-cli widgets list [flags]
Examples:
- fixture-pp-cli widgets list --limit 2
+ fixture-pp-cli widgets list --json
Flags:
--json Output JSON
@@ -654,8 +663,8 @@ HELP
fi
if [ "$1" = "widgets" ] && [ "$2" = "list" ]; then
- if [ "${3:-}" = "--limit" ] && [ "${4:-}" = "2" ] && [ "${5:-}" = "--json" ]; then
- echo '{"widgets":[{"id":"1"}]}'
+ if [ "${3:-}" = "--json" ]; then
+ echo '{"results":[{"id":"123"}]}'
exit 0
fi
echo 'widget 1'
@@ -701,3 +710,857 @@ func writeTestManifestForLiveDogfood(t *testing.T, dir string) {
AuthType: "none",
}))
}
+
+// writeLiveDogfoodRichFixture builds a fake binary with multi-resource
+// command families (widgets, gizmos, projects/tasks, failing-resource) plus
+// search/delete commands. Each family's purpose is named in its section
+// header below; the test names that consume them follow the same naming.
+//
+// IMPORTANT: companionSupportsLimit operates on RAW help text (not
+// flags-section-scoped), so any --limit token anywhere in a companion's
+// help — Examples included — makes the resolver append --limit 1.
+// Companions where the test expects a bare call must keep --limit out of
+// the help entirely; companions where the test expects --limit 1 must
+// declare --limit in Flags.
+func writeLiveDogfoodRichFixture(t *testing.T) (dir string, binaryName string) {
+ t.Helper()
+
+ dir = t.TempDir()
+ binaryName = "fixture-pp-cli"
+ writeTestManifestForLiveDogfood(t, dir)
+
+ binPath := filepath.Join(dir, binaryName)
+ script := `#!/bin/sh
+set -u
+
+# Argv logging side channel. Every invocation appends its argv (space-joined)
+# when PRINTING_PRESS_TEST_ARGV_LOG is set. Defaults to no-op so tests that
+# don't care about argv tracking work unchanged.
+if [ -n "${PRINTING_PRESS_TEST_ARGV_LOG:-}" ]; then
+ printf '%s\n' "$*" >> "$PRINTING_PRESS_TEST_ARGV_LOG"
+fi
+
+if [ "$1" = "agent-context" ]; then
+ cat <<'JSON'
+{
+ "commands": [
+ {"name":"widgets","subcommands":[
+ {"name":"list"},
+ {"name":"get"},
+ {"name":"describe"},
+ {"name":"search"},
+ {"name":"search-no-json"},
+ {"name":"search-positional"},
+ {"name":"delete"}
+ ]},
+ {"name":"gizmos","subcommands":[
+ {"name":"list"},
+ {"name":"get"}
+ ]},
+ {"name":"projects","subcommands":[
+ {"name":"list"},
+ {"name":"tasks","subcommands":[
+ {"name":"list"},
+ {"name":"update"}
+ ]}
+ ]},
+ {"name":"failing-resource","subcommands":[
+ {"name":"list"},
+ {"name":"get"},
+ {"name":"describe"}
+ ]},
+ {"name":"completion","subcommands":[{"name":"bash"}]}
+ ]
+}
+JSON
+ exit 0
+fi
+
+# ---------- widgets family ----------
+
+if [ "$1" = "widgets" ] && [ "$2" = "list" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+List widgets.
+
+Usage:
+ fixture-pp-cli widgets list [flags]
+
+Examples:
+ fixture-pp-cli widgets list --json
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "list" ]; then
+ if [ "${3:-}" = "--json" ]; then
+ echo '{"results":[{"id":"42"}]}'
+ exit 0
+ fi
+ echo 'widget 1'
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "get" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Get a widget.
+
+Usage:
+ fixture-pp-cli widgets get <id> [flags]
+
+Examples:
+ fixture-pp-cli widgets get 42
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "get" ]; then
+ if [ "${3:-}" = "__printing_press_invalid__" ]; then
+ echo 'not found' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ]; then
+ echo '{"id":"42"}'
+ exit 0
+ fi
+ echo "widget $3"
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "describe" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Describe a widget.
+
+Usage:
+ fixture-pp-cli widgets describe <id> [flags]
+
+Examples:
+ fixture-pp-cli widgets describe 42
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "describe" ]; then
+ if [ "${3:-}" = "__printing_press_invalid__" ]; then
+ echo 'not found' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ]; then
+ echo '{"id":"42","description":"a widget"}'
+ exit 0
+ fi
+ echo "description of widget $3"
+ exit 0
+fi
+
+# ---------- gizmos family (companion-supports-limit) ----------
+
+if [ "$1" = "gizmos" ] && [ "$2" = "list" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+List gizmos.
+
+Usage:
+ fixture-pp-cli gizmos list [flags]
+
+Examples:
+ fixture-pp-cli gizmos list --json
+
+Flags:
+ --json Output JSON
+ --limit Maximum results to return
+HELP
+ exit 0
+fi
+
+if [ "$1" = "gizmos" ] && [ "$2" = "list" ]; then
+ # Resolver appends --limit 1 because --limit is declared in Flags. Match
+ # only when both --json AND --limit are present, so a regression that
+ # stops appending --limit (bare --json) falls through to the failure
+ # branch instead of being silently accepted.
+ case "$*" in
+ *"--json"*"--limit"*|*"--limit"*"--json"*)
+ echo '{"results":[{"id":"42"}]}'
+ exit 0
+ ;;
+ esac
+ echo 'gizmo 1'
+ exit 0
+fi
+
+if [ "$1" = "gizmos" ] && [ "$2" = "get" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Get a gizmo.
+
+Usage:
+ fixture-pp-cli gizmos get <id> [flags]
+
+Examples:
+ fixture-pp-cli gizmos get 42
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "gizmos" ] && [ "$2" = "get" ]; then
+ if [ "${3:-}" = "__printing_press_invalid__" ]; then
+ echo 'not found' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ]; then
+ echo '{"id":"42"}'
+ exit 0
+ fi
+ echo "gizmo $3"
+ exit 0
+fi
+
+# ---------- projects/tasks family (chained walk) ----------
+
+if [ "$1" = "projects" ] && [ "$2" = "list" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+List projects.
+
+Usage:
+ fixture-pp-cli projects list [flags]
+
+Examples:
+ fixture-pp-cli projects list --json
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "projects" ] && [ "$2" = "list" ]; then
+ if [ "${3:-}" = "--json" ]; then
+ echo '{"results":[{"id":"P1"}]}'
+ exit 0
+ fi
+ echo 'project 1'
+ exit 0
+fi
+
+if [ "$1" = "projects" ] && [ "$2" = "tasks" ] && [ "${3:-}" = "list" ] && [ "${4:-}" = "--help" ]; then
+ cat <<'HELP'
+List tasks within a project.
+
+Usage:
+ fixture-pp-cli projects tasks list <project-id> [flags]
+
+Examples:
+ fixture-pp-cli projects tasks list P1 --json
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "projects" ] && [ "$2" = "tasks" ] && [ "${3:-}" = "list" ]; then
+ # ${4:-} is the resolved project-id (or the matrix walker's invalid-token
+ # sentinel for error_path). ${5:-} is --json when supplied. We also handle
+ # the self-companion case (4-arg "... list --json" with no project-id),
+ # which fires when the resolver's findListCompanion picks "projects tasks
+ # list" as the companion for itself; without this branch the matrix walker
+ # would silently skip the bare list probe.
+ if [ "${4:-}" = "__printing_press_invalid__" ]; then
+ echo 'invalid project' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ] || [ "${5:-}" = "--json" ]; then
+ echo '{"results":[{"id":"T7"}]}'
+ exit 0
+ fi
+ echo 'task 1'
+ exit 0
+fi
+
+if [ "$1" = "projects" ] && [ "$2" = "tasks" ] && [ "${3:-}" = "update" ] && [ "${4:-}" = "--help" ]; then
+ cat <<'HELP'
+Update a task within a project.
+
+Usage:
+ fixture-pp-cli projects tasks update <project-id> <task-id> [flags]
+
+Examples:
+ fixture-pp-cli projects tasks update P1 T7
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "projects" ] && [ "$2" = "tasks" ] && [ "${3:-}" = "update" ]; then
+ # ${4:-} is project-id (or __printing_press_invalid__ for error_path).
+ # ${5:-} is task-id (or --json for malformed error_path argv).
+ if [ "${4:-}" = "__printing_press_invalid__" ]; then
+ echo 'invalid project' >&2
+ exit 2
+ fi
+ if [ "${6:-}" = "--json" ]; then
+ echo '{"id":"T7","status":"updated"}'
+ exit 0
+ fi
+ echo 'updated'
+ exit 0
+fi
+
+# ---------- failing-resource family (negative cache) ----------
+
+if [ "$1" = "failing-resource" ] && [ "$2" = "list" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+List failing-resource items.
+
+Usage:
+ fixture-pp-cli failing-resource list [flags]
+
+Examples:
+ fixture-pp-cli failing-resource list --json
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "failing-resource" ] && [ "$2" = "list" ]; then
+ # Always fail on the actual --json call so the resolver caches a sentinel.
+ if [ "${3:-}" = "--json" ]; then
+ echo 'upstream service unavailable' >&2
+ exit 2
+ fi
+ echo 'failing-resource 1'
+ exit 0
+fi
+
+if [ "$1" = "failing-resource" ] && [ "$2" = "get" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Get a failing-resource item.
+
+Usage:
+ fixture-pp-cli failing-resource get <id> [flags]
+
+Examples:
+ fixture-pp-cli failing-resource get 42
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "failing-resource" ] && [ "$2" = "get" ]; then
+ if [ "${3:-}" = "__printing_press_invalid__" ]; then
+ echo 'not found' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ]; then
+ echo '{"id":"42"}'
+ exit 0
+ fi
+ echo "failing-resource $3"
+ exit 0
+fi
+
+if [ "$1" = "failing-resource" ] && [ "$2" = "describe" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Describe a failing-resource item.
+
+Usage:
+ fixture-pp-cli failing-resource describe <id> [flags]
+
+Examples:
+ fixture-pp-cli failing-resource describe 42
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "failing-resource" ] && [ "$2" = "describe" ]; then
+ if [ "${3:-}" = "__printing_press_invalid__" ]; then
+ echo 'not found' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ]; then
+ echo '{"id":"42","description":"a thing"}'
+ exit 0
+ fi
+ echo "description of failing-resource $3"
+ exit 0
+fi
+
+# ---------- widgets search family (U3 — search-shape error_path) ----------
+
+# widgets search: --query flag + --json flag.
+# Mode dispatch via PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE only affects the
+# error_path probe (query == __printing_press_invalid__). Walker's happy_path
+# and json_fidelity probes use a different query and always return valid JSON
+# so they don't pollute test signal.
+if [ "$1" = "widgets" ] && [ "$2" = "search" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Search widgets.
+
+Usage:
+ fixture-pp-cli widgets search <query> [flags]
+
+Examples:
+ fixture-pp-cli widgets search --query foo --json
+
+Flags:
+ --json Output JSON
+ --query Search query
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "search" ]; then
+ # Args shape: widgets search --query <q> [--json]
+ query="${4:-}"
+ if [ "$query" = "__printing_press_invalid__" ]; then
+ case "${PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE:-empty}" in
+ fallback)
+ echo '{"results":[{"id":"recent-1"},{"id":"recent-2"}]}'
+ exit 0
+ ;;
+ nonzero)
+ exit 4
+ ;;
+ invalid)
+ echo '{not-json'
+ exit 0
+ ;;
+ *)
+ echo '{"results":[]}'
+ exit 0
+ ;;
+ esac
+ fi
+ echo '{"results":[]}'
+ exit 0
+fi
+
+# widgets search-no-json: --query flag, NO --json flag. Used to verify that
+# search-shape error_path passes on exit 0 even without --json.
+if [ "$1" = "widgets" ] && [ "$2" = "search-no-json" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Search widgets without JSON support.
+
+Usage:
+ fixture-pp-cli widgets search-no-json <query> [flags]
+
+Examples:
+ fixture-pp-cli widgets search-no-json --query foo
+
+Flags:
+ --query Search query
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "search-no-json" ]; then
+ echo '0 results found.'
+ exit 0
+fi
+
+# widgets search-positional: positional <query>, no --query flag. Used to
+# verify error_path constructs the positional argv shape (no --query flag).
+if [ "$1" = "widgets" ] && [ "$2" = "search-positional" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Search widgets via a positional query.
+
+Usage:
+ fixture-pp-cli widgets search-positional <query> [flags]
+
+Examples:
+ fixture-pp-cli widgets search-positional foo --json
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "search-positional" ]; then
+ # Args shape: widgets search-positional <query> [--json]
+ echo '{"results":[]}'
+ exit 0
+fi
+
+# widgets delete: mutation-shape (no --query flag, no <query> positional;
+# accepts <id>). error_path uses non-zero-required strategy (mutating-leaf
+# deny-list overrides search-shape detection).
+if [ "$1" = "widgets" ] && [ "$2" = "delete" ] && [ "${3:-}" = "--help" ]; then
+ cat <<'HELP'
+Delete a widget.
+
+Usage:
+ fixture-pp-cli widgets delete <id> [flags]
+
+Examples:
+ fixture-pp-cli widgets delete 42
+
+Flags:
+ --json Output JSON
+HELP
+ exit 0
+fi
+
+if [ "$1" = "widgets" ] && [ "$2" = "delete" ]; then
+ if [ "${3:-}" = "__printing_press_invalid__" ]; then
+ echo 'invalid id' >&2
+ exit 2
+ fi
+ if [ "${4:-}" = "--json" ]; then
+ echo '{"id":"42","status":"deleted"}'
+ exit 0
+ fi
+ echo 'deleted'
+ exit 0
+fi
+
+echo "unexpected args: $*" >&2
+exit 99
+`
+ require.NoError(t, os.WriteFile(binPath, []byte(script), 0o755))
+ return dir, binaryName
+}
+
+// readArgvLog returns the lines from the argv log file, with empty lines
+// filtered out. Used by resolve-success and search/error_path tests to
+// assert on subprocess invocation count and content.
+func readArgvLog(t *testing.T, path string) []string {
+ t.Helper()
+ data, err := os.ReadFile(path)
+ require.NoError(t, err)
+ var lines []string
+ for line := range strings.SplitSeq(string(data), "\n") {
+ if line != "" {
+ lines = append(lines, line)
+ }
+ }
+ return lines
+}
+
+// countArgvLines returns the number of argv-log lines whose content contains
+// every substring in `must`. Tests filter on `--json` (the actual companion
+// call) to exclude `--help` invocations from companionSupportsLimit, which
+// would otherwise inflate companion-call counts.
+func countArgvLines(lines []string, must ...string) int {
+ count := 0
+outer:
+ for _, line := range lines {
+ for _, m := range must {
+ if !strings.Contains(line, m) {
+ continue outer
+ }
+ }
+ count++
+ }
+ return count
+}
+
+// findResultByCommandKind locates a single matrix-walker result by command
+// path and test kind. Used by resolve-success tests to assert on the
+// post-resolution status of a specific probe.
+func findResultByCommandKind(report *LiveDogfoodReport, command string, kind LiveDogfoodTestKind) *LiveDogfoodTestResult {
+ for i := range report.Tests {
+ if report.Tests[i].Command == command && report.Tests[i].Kind == kind {
+ return &report.Tests[i]
+ }
+ }
+ return nil
+}
+
+// setupRichFixture is the shared preamble for U2/U3 tests: skip on Windows,
+// build the rich fixture, and enable the argv-log side channel via a unique
+// per-test tempfile path. Returns the fixture dir, binary name, and the
+// argv-log path (tests that don't read the log can ignore it).
+func setupRichFixture(t *testing.T) (dir, binaryName, argvLog string) {
+ t.Helper()
+ if runtime.GOOS == "windows" {
+ t.Skip("test uses a shell script as the fake binary; skip on Windows")
+ }
+ dir, binaryName = writeLiveDogfoodRichFixture(t)
+ argvLog = filepath.Join(t.TempDir(), "argv.log")
+ t.Setenv("PRINTING_PRESS_TEST_ARGV_LOG", argvLog)
+ return
+}
+
+// runRichFixtureMatrix runs the standard full-level matrix walk against the
+// rich fixture with the same options every U2/U3 test uses. Fails the test on
+// any RunLiveDogfood error.
+func runRichFixtureMatrix(t *testing.T, dir, binaryName string) *LiveDogfoodReport {
+ t.Helper()
+ report, err := RunLiveDogfood(LiveDogfoodOptions{
+ CLIDir: dir,
+ BinaryName: binaryName,
+ Level: "full",
+ Timeout: 2 * time.Second,
+ })
+ require.NoError(t, err)
+ return report
+}
+
+func TestRunLiveDogfoodResolveSuccessSinglePositional(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // The resolver substituted the id from companion widgets list --json
+ // (which returned {"results":[{"id":"42"}]}) into widgets get,
+ // producing argv = `widgets get 42`. The probe ran and returned exit 0.
+ got := findResultByCommandKind(report, "widgets get", LiveDogfoodTestHappy)
+ require.NotNil(t, got, "expected widgets get happy_path in report")
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+
+ // Companion-leaf invariant: the resolver picked widgets list, not some
+ // other allowlisted sibling (widgets search is also in crossAPIListVerbs
+ // but sorts later alphabetically). Pin both directions: widgets list
+ // must appear AS a companion call, AND widgets search must NOT appear
+ // as one. The bare-companion shape `widgets search --json` (path +
+ // --json, nothing else) only fires when findListCompanion picks search;
+ // the walker's own probe of widgets search uses --query foo --json from
+ // Examples, which doesn't match the bare companion shape.
+ lines := readArgvLog(t, argvLog)
+ assert.GreaterOrEqual(t, countArgvLines(lines, "widgets list", "--json"), 1,
+ "expected widgets list --json to appear in argv log as the chosen companion")
+ bareSearchCompanion := 0
+ for _, line := range lines {
+ if line == "widgets search --json" {
+ bareSearchCompanion++
+ }
+ }
+ assert.Equal(t, 0, bareSearchCompanion,
+ "widgets search must NOT be picked as companion when widgets list is available; saw bare `widgets search --json` in argv log")
+ assert.GreaterOrEqual(t, countArgvLines(lines, "widgets get 42"), 1,
+ "expected widgets get 42 (post-substitution probe) to appear in argv log")
+}
+
+func TestRunLiveDogfoodResolveSuccessChainedMultiPositional(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // The chain walks projects list → projects tasks list P1, threading the
+ // resolved P1 into the second list call. The final probe is
+ // `projects tasks update P1 T7`.
+ got := findResultByCommandKind(report, "projects tasks update", LiveDogfoodTestHappy)
+ require.NotNil(t, got, "expected projects tasks update happy_path in report")
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+
+ lines := readArgvLog(t, argvLog)
+ assert.GreaterOrEqual(t, countArgvLines(lines, "projects list", "--json"), 1,
+ "expected projects list --json (depth-0 companion) in argv log")
+ assert.GreaterOrEqual(t, countArgvLines(lines, "projects tasks list", "P1", "--json"), 1,
+ "expected projects tasks list P1 --json (depth-1 companion threading P1) in argv log")
+ assert.GreaterOrEqual(t, countArgvLines(lines, "projects tasks update P1 T7"), 1,
+ "expected projects tasks update P1 T7 (post-chain probe) in argv log")
+}
+
+func TestRunLiveDogfoodResolveSuccessCacheHit(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // Both siblings successfully resolve and run their probes.
+ getResult := findResultByCommandKind(report, "widgets get", LiveDogfoodTestHappy)
+ require.NotNil(t, getResult)
+ assert.Equal(t, LiveDogfoodStatusPass, getResult.Status, getResult.Reason)
+ descResult := findResultByCommandKind(report, "widgets describe", LiveDogfoodTestHappy)
+ require.NotNil(t, descResult)
+ assert.Equal(t, LiveDogfoodStatusPass, descResult.Status, descResult.Reason)
+
+ // Cache hit: one cached id serves every widgets-family id-shape sibling.
+ // The walker probes `widgets list` itself for both happy_path and
+ // json_fidelity, both of which invoke argv `widgets list --json`
+ // (Examples already has --json, so appendJSONArg dedups and json_fidelity
+ // reuses the happy argv). The resolver invokes the companion exactly
+ // once for the first id-shape sibling probed; subsequent siblings hit
+ // the cache and add 0.
+ const walkerProbes, resolverCallsWithCacheHit = 2, 1
+ const expectedTotal = walkerProbes + resolverCallsWithCacheHit
+ lines := readArgvLog(t, argvLog)
+ companionCalls := countArgvLines(lines, "widgets list", "--json")
+ // Equality assertion (not <=) catches both directions: cache miss inflates
+ // to expectedTotal+1, walker-side dedup deflates to expectedTotal-1. Bare
+ // upper bound would silently pass on the second case.
+ assert.Equal(t, expectedTotal, companionCalls,
+ "expected exactly %d widgets list --json invocations (%d walker + %d resolver with cache hit); got %d", expectedTotal, walkerProbes, resolverCallsWithCacheHit, companionCalls)
+
+ // Both probe argvs landed in the log post-substitution.
+ assert.GreaterOrEqual(t, countArgvLines(lines, "widgets get 42"), 1)
+ assert.GreaterOrEqual(t, countArgvLines(lines, "widgets describe 42"), 1)
+}
+
+func TestRunLiveDogfoodResolveSuccessCompanionLimit(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // gizmos list declares --limit in its Flags section, so the resolver
+ // appends --limit 1 to the companion call before invoking it.
+ got := findResultByCommandKind(report, "gizmos get", LiveDogfoodTestHappy)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+
+ lines := readArgvLog(t, argvLog)
+ // The resolver's gizmos list call must include both --json and --limit 1.
+ // Order is resolver-dependent (currently --json --limit 1) so we assert
+ // substring presence rather than exact ordering.
+ limitCalls := countArgvLines(lines, "gizmos list", "--json", "--limit 1")
+ assert.GreaterOrEqual(t, limitCalls, 1,
+ "expected gizmos list call with both --json and --limit 1; got 0 such lines")
+}
+
+func TestRunLiveDogfoodResolveSuccessNegativeCacheSentinel(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // failing-resource list returns exit non-zero for --json. The walker
+ // probes commands in alphabetical order: `failing-resource describe`
+ // runs first and hits the FRESH failure (caches sentinel);
+ // `failing-resource get` runs second and hits the cached sentinel.
+ descResult := findResultByCommandKind(report, "failing-resource describe", LiveDogfoodTestHappy)
+ require.NotNil(t, descResult)
+ assert.Equal(t, LiveDogfoodStatusSkip, descResult.Status,
+ "first sibling (describe) should skip with fresh companion-failure reason")
+ assert.Contains(t, descResult.Reason, "list companion failed at depth",
+ "first sibling reason should reference the actual depth-keyed failure, not the cached sentinel")
+
+ getResult := findResultByCommandKind(report, "failing-resource get", LiveDogfoodTestHappy)
+ require.NotNil(t, getResult)
+ assert.Equal(t, LiveDogfoodStatusSkip, getResult.Status,
+ "second sibling (get) should skip via the cached negative-cache sentinel")
+ assert.Contains(t, getResult.Reason, "list companion previously failed at depth",
+ "second sibling reason should reference the cached sentinel, not re-fail the companion")
+
+ // Filter to the actual companion call (--json excludes --help). The
+ // walker probes `failing-resource list` for both happy_path and
+ // json_fidelity (both use argv `failing-resource list --json` since
+ // Examples already includes --json). The resolver invokes the companion
+ // once for the first sibling (describe), caches the sentinel, and the
+ // second sibling (get) hits the sentinel without re-invoking.
+ const walkerProbes, resolverCallsWithSentinel = 2, 1
+ const expectedTotal = walkerProbes + resolverCallsWithSentinel
+ lines := readArgvLog(t, argvLog)
+ companionCalls := countArgvLines(lines, "failing-resource list", "--json")
+ // Equality assertion (not <=) catches sentinel-bypass (4 calls) AND any
+ // future walker-side dedup that would collapse to 2 calls without going
+ // through the sentinel path.
+ assert.Equal(t, expectedTotal, companionCalls,
+ "expected exactly %d failing-resource list --json invocations (%d walker + %d resolver with sentinel); got %d", expectedTotal, walkerProbes, resolverCallsWithSentinel, companionCalls)
+}
+
+// ----- U3: search-aware error_path integration tests -----
+
+func TestRunLiveDogfoodSearchErrorPathEmptyResults(t *testing.T) {
+ // Mode unset → fixture returns exit 0 + {"results":[]} for the
+ // __printing_press_invalid__ probe.
+ dir, binaryName, _ := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ got := findResultByCommandKind(report, "widgets search", LiveDogfoodTestError)
+ require.NotNil(t, got, "expected widgets search error_path in report")
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+}
+
+func TestRunLiveDogfoodSearchErrorPathFallbackResults(t *testing.T) {
+ dir, binaryName, _ := setupRichFixture(t)
+ t.Setenv("PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE", "fallback")
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // Recency-fallback APIs return content under unmatched queries — exit 0
+ // with non-empty results is a valid "no match" signal, not a failure.
+ got := findResultByCommandKind(report, "widgets search", LiveDogfoodTestError)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+}
+
+func TestRunLiveDogfoodSearchErrorPathNoJSONSupport(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // Search-shape command without --json flag declared. Exit 0 alone is
+ // sufficient when --json wasn't supplied — no JSON validation possible.
+ got := findResultByCommandKind(report, "widgets search-no-json", LiveDogfoodTestError)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+
+ // argv-log assertion: the error_path probe ran without --json.
+ lines := readArgvLog(t, argvLog)
+ assert.GreaterOrEqual(t, countArgvLines(lines, "widgets search-no-json", "--query", "__printing_press_invalid__"), 1,
+ "expected error_path probe to use --query for the no-json search command")
+}
+
+func TestRunLiveDogfoodSearchErrorPathPositionalQuery(t *testing.T) {
+ dir, binaryName, argvLog := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ got := findResultByCommandKind(report, "widgets search-positional", LiveDogfoodTestError)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+
+ // argv-log assertion: probe used the positional argv shape, not --query
+ // (the command has <query> in Usage but no --query flag in Flags).
+ lines := readArgvLog(t, argvLog)
+ positionalCalls := countArgvLines(lines, "widgets search-positional __printing_press_invalid__", "--json")
+ assert.GreaterOrEqual(t, positionalCalls, 1,
+ "expected error_path probe to use positional <query>, not --query flag")
+ flagCalls := countArgvLines(lines, "widgets search-positional", "--query")
+ assert.Equal(t, 0, flagCalls,
+ "expected --query flag NOT to appear in error_path argv when command uses positional <query>")
+}
+
+func TestRunLiveDogfoodSearchErrorPathNonZeroExit(t *testing.T) {
+ dir, binaryName, _ := setupRichFixture(t)
+ t.Setenv("PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE", "nonzero")
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // Non-zero exit is also a valid "no match" signal for some APIs;
+ // search-shape error_path treats it as Pass, consistent with mutation.
+ got := findResultByCommandKind(report, "widgets search", LiveDogfoodTestError)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+}
+
+// TestRunLiveDogfoodSearchErrorPathInvalidJSON exercises the only fail mode
+// currently produced by the search-shape error_path code in
+// live_dogfood.go:693-711. INVARIANT: if the production code adds a new
+// search-shape Fail branch (timeout, empty stdout under --json, schema
+// mismatch), add a corresponding integration test in the same change.
+func TestRunLiveDogfoodSearchErrorPathInvalidJSON(t *testing.T) {
+ dir, binaryName, _ := setupRichFixture(t)
+ t.Setenv("PRINTING_PRESS_TEST_WIDGETS_SEARCH_MODE", "invalid")
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ got := findResultByCommandKind(report, "widgets search", LiveDogfoodTestError)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusFail, got.Status,
+ "search + invalid JSON under --json is the only search-shape error_path Fail mode")
+ assert.Contains(t, got.Reason, "invalid JSON")
+}
+
+func TestRunLiveDogfoodSearchErrorPathMutationFallthrough(t *testing.T) {
+ dir, binaryName, _ := setupRichFixture(t)
+ report := runRichFixtureMatrix(t, dir, binaryName)
+
+ // widgets delete has no --query flag and no <query> positional, so
+ // commandSupportsSearch returns false. Even if it had --query (it
+ // doesn't), the mutating-leaf deny-list (delete is in mutatingVerbs)
+ // would still suppress search-shape and route to the existing
+ // non-zero-required strategy. Fixture exit 2 → Pass.
+ got := findResultByCommandKind(report, "widgets delete", LiveDogfoodTestError)
+ require.NotNil(t, got)
+ assert.Equal(t, LiveDogfoodStatusPass, got.Status, got.Reason)
+ assert.Equal(t, 2, got.ExitCode)
+}
← e9696c9b fix(cli): preserve query params in generated MCP handlers (#
·
back to Cli Printing Press
·
fix(cli): complete bearer refresh and path handling (#584) ebfa6bf1 →