fix(skills): Phase 4.85 prompt refinements from calibration dispatch (#215)

6f8b0c7d6a05151d74131323c18d6bf8405f197b · 2026-04-13 18:29:26 -0700 · Trevin Chow
Files touched

M skills/printing-press/SKILL.md
Diff

commit 6f8b0c7d6a05151d74131323c18d6bf8405f197b
Author: Trevin Chow <trevin@trevinchow.com>
Date:   Mon Apr 13 18:29:26 2026 -0700

    fix(skills): Phase 4.85 prompt refinements from calibration dispatch (#215)
---
 skills/printing-press/SKILL.md | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/skills/printing-press/SKILL.md b/skills/printing-press/SKILL.md
index 8c7e5f06..8850456d 100644
--- a/skills/printing-press/SKILL.md
+++ b/skills/printing-press/SKILL.md
@@ -1744,13 +1744,14 @@ Use the Agent tool (general-purpose) with this prompt contract:
 
 > Review the sampled outputs from the shipped CLI at `$CLI_WORK_DIR`. You have these ground-truth sources:
 >
-> - Sampled command output: run `printing-press scorecard --dir $CLI_WORK_DIR --live-check --json` and read the `live_check.features[]` array. Each entry has the command, example invocation, actual stdout, the pass/fail reason, and a `warnings` array (populated by rule-based checks like the raw-HTML-entity detector).
+> - Sampled command output: run `printing-press scorecard --dir $CLI_WORK_DIR --live-check --json` and read the `live_check.features[]` array. Each entry has the command, example invocation, actual stdout (in `output_sample`, bounded to ~4 KiB), the pass/fail reason, and a `warnings` array (populated by rule-based checks like the raw-HTML-entity detector).
+> - **Review only `status: pass` entries.** Entries with `status: fail` either crashed, timed out, or had placeholder args (`<id>`, `<url>`) that never produced real output — their sample is empty and there's nothing for you to judge. Phase 5 dogfood handles test-coverage and exit-code concerns.
 > - `$CLI_WORK_DIR/research.json` `novel_features` (planned behavior per feature) and `novel_features_built` (verified built commands).
 > - The CLI binary at `$CLI_WORK_DIR/<cli-name>-pp-cli` — you may invoke additional commands to gather more output when a finding needs verification.
 >
 > For each of these checks, report findings under 50 words each. Only report issues a human user would notice in 5 minutes of hands-on testing — not every edge case a thorough QA pass might find:
 >
-> 1. **Output matches query intent.** For sampled novel features with a query argument, does the output contain results clearly related to the query? Watch for results that coincidentally contain the query as a substring of an unrelated term, or fallback behavior where the extractor failed and returned adjacent (not matching) content.
+> 1. **Output *semantically* matches query intent.** For sampled novel features with a query argument, judge relevance beyond what the mechanical query-token check in live-check already enforced. A feature that passed live-check's `outputMentionsQuery` test still contains *some* query token somewhere — but "buttermilk" appearing as a substring of "butter" results, or "brownies" returning a chili recipe because the extractor fell back to adjacent content, both slip past the mechanical check. Only flag when a human user would look at the top results and say "this isn't what I asked for." Skip this check when the example has no query argument.
 > 2. **No obvious format bugs.** Does the output contain raw HTML entities, mojibake (question marks or replacement chars in titles), or malformed URLs (pointing at category index pages, feed endpoints, or random-selector routes rather than canonical content permalinks)? Rule-based live-check catches numeric entities; this layer catches the broader class.
 > 3. **Aggregation commands show all requested sources.** For commands with a `--source`/`--site`/`--region` CSV flag: if the user requested N sources, does output show N, or does stderr explain the missing ones? Silent drops of failed sources are a top failure mode for fan-out commands.
 > 4. **Result ordering/ranking makes sense.** For commands that claim to rank or sort, does the top result look plausibly best given the query? Watch for broken score weights, off-by-one sort bugs, and silent fallback to recency when relevance computation fails.
← 270270ab feat(cli): machine-output-verification Wave B — live-check e · back to Cli Printing Press · feat(cli): apply Cloudflare Wrangler CLI learnings (naming c dc5a8cc4 →