← back to Cli Printing Press
feat(cli): tools-audit checks MCP descriptions in tools-manifest.json (#381)
fee6802ec499315d197c543d87be2a6b0c79a618 · 2026-04-29 02:12:59 -0700 · Trevin Chow
The tools-audit subcommand previously only audited Cobra Short fields
in internal/cli/*.go. That covered shell-out tools (Short = MCP
description for the runtime walker) but missed typed endpoint tools,
where the MCP description comes from the OpenAPI spec via the typed
registration in tools.go. Spec-derived summaries can be terse boilerplate
("Create a tag", "List domains" — 12-30 chars) that leaves an agent
guessing what fields to pass and what comes back; until now the audit
gave those a free pass.
This change extends the audit to read tools-manifest.json and emit two
new finding kinds:
- empty-mcp-description: tool description field is empty
- thin-mcp-description: under 60 chars AND fewer than 8 words
Thresholds are floor values, not assessments. The deterministic check
is a tripwire — Pass 2 of the polish playbook (agent reasoning over
every description) is where the actual quality assessment happens. A
70-char description can be perfect or terrible; the binary can't tell.
Also tightens the Cobra-side checks to honor the runtime walker's skip
rules. A command's Cobra Short is only the MCP description when the
runtime walker registers it as a shell-out tool. The walker skips:
- pp:endpoint commands (typed registration; manifest covers them)
- parent groupers (no Run/RunE; not actionable as a tool)
- framework commands (auth, doctor, version, completion, profile, etc.)
The audit now exempts these three classes from empty-short, thin-short,
and missing-read-only. On dub this drops Cobra-side noise from 54 false
positives to 3 real findings, with 51 newly-surfaced thin-mcp-description
findings on the manifest (the actual problem we'd been missing).
Updates references/tools-polish.md to document the new finding kinds,
the higher MCP-description bar (1-3 sentences naming action,
parameters, return shape, when-to-prefer), and the dual edit path
(tools-manifest.json + internal/mcp/tools.go must stay in sync).
Reframes the playbook to make explicit that the audit is a tripwire,
not the assessment — Pass 2 (agent walks every description) is where
quality gets evaluated.
Files touched
M internal/cli/tools_audit.goM skills/printing-press-polish/SKILL.mdM skills/printing-press-polish/references/tools-polish.md
Diff
commit fee6802ec499315d197c543d87be2a6b0c79a618
Author: Trevin Chow <trevin@trevinchow.com>
Date: Wed Apr 29 02:12:59 2026 -0700
feat(cli): tools-audit checks MCP descriptions in tools-manifest.json (#381)
The tools-audit subcommand previously only audited Cobra Short fields
in internal/cli/*.go. That covered shell-out tools (Short = MCP
description for the runtime walker) but missed typed endpoint tools,
where the MCP description comes from the OpenAPI spec via the typed
registration in tools.go. Spec-derived summaries can be terse boilerplate
("Create a tag", "List domains" — 12-30 chars) that leaves an agent
guessing what fields to pass and what comes back; until now the audit
gave those a free pass.
This change extends the audit to read tools-manifest.json and emit two
new finding kinds:
- empty-mcp-description: tool description field is empty
- thin-mcp-description: under 60 chars AND fewer than 8 words
Thresholds are floor values, not assessments. The deterministic check
is a tripwire — Pass 2 of the polish playbook (agent reasoning over
every description) is where the actual quality assessment happens. A
70-char description can be perfect or terrible; the binary can't tell.
Also tightens the Cobra-side checks to honor the runtime walker's skip
rules. A command's Cobra Short is only the MCP description when the
runtime walker registers it as a shell-out tool. The walker skips:
- pp:endpoint commands (typed registration; manifest covers them)
- parent groupers (no Run/RunE; not actionable as a tool)
- framework commands (auth, doctor, version, completion, profile, etc.)
The audit now exempts these three classes from empty-short, thin-short,
and missing-read-only. On dub this drops Cobra-side noise from 54 false
positives to 3 real findings, with 51 newly-surfaced thin-mcp-description
findings on the manifest (the actual problem we'd been missing).
Updates references/tools-polish.md to document the new finding kinds,
the higher MCP-description bar (1-3 sentences naming action,
parameters, return shape, when-to-prefer), and the dual edit path
(tools-manifest.json + internal/mcp/tools.go must stay in sync).
Reframes the playbook to make explicit that the audit is a tripwire,
not the assessment — Pass 2 (agent walks every description) is where
quality gets evaluated.
---
internal/cli/tools_audit.go | 231 +++++++++++++++++----
skills/printing-press-polish/SKILL.md | 23 +-
.../references/tools-polish.md | 188 +++++++++++++----
3 files changed, 355 insertions(+), 87 deletions(-)
diff --git a/internal/cli/tools_audit.go b/internal/cli/tools_audit.go
index e7493734..c925bee4 100644
--- a/internal/cli/tools_audit.go
+++ b/internal/cli/tools_audit.go
@@ -22,8 +22,39 @@ const (
statusAccepted = "accepted"
suspiciousMaxLen = 30
suspiciousMinWord = 4
+
+ // MCP tool descriptions need richer text than Cobra Shorts. Agents
+ // pick from a tool catalog without the context a human gets from
+ // --help, so spec-derived summaries like "Create a tag" or "List
+ // items" — fine for OpenAPI doc rendering — leave the agent guessing
+ // what fields to pass and what comes back. The thresholds here are
+ // floor values: 60 chars / 8 words is roughly two short clauses,
+ // enough to convey the action plus one parameter or return shape.
+ mcpDescMinLen = 60
+ mcpDescMinWords = 8
+
+ manifestFile = "tools-manifest.json"
)
+// frameworkCommands mirrors cobratree/classify.go.tmpl. The runtime
+// walker skips these names entirely — they're never registered as MCP
+// tools — so audit findings on their Cobra Short are noise.
+var frameworkCommands = map[string]bool{
+ "about": true,
+ "agent-context": true,
+ "api": true,
+ "auth": true,
+ "completion": true,
+ "doctor": true,
+ "feedback": true,
+ "help": true,
+ "profile": true,
+ "search": true,
+ "sql": true,
+ "version": true,
+ "which": true,
+}
+
// newToolsAuditCmd inspects a single printed CLI's command tree for
// MCP tool quality issues a deterministic check can catch: empty Short,
// suspiciously thin Short, and read-shaped command names that lack the
@@ -79,19 +110,29 @@ Exit 0 regardless of findings (diagnostic, not gating).`,
return cmd
}
-// ToolsAuditFinding is one mechanical issue discovered in a command
-// definition. Status and Note are ledger-only; the audit phase emits
-// findings without them (omitempty keeps --json clean for downstream
-// parsing). The agent edits the persisted ledger to flip Status to
-// "accepted" with a Note explaining why a thin-short is fine as-is.
-// On re-run the binary preserves these fields for any finding whose
-// identity key matches.
+// ToolsAuditFinding is one mechanical issue discovered in either a
+// Cobra command literal or an entry in the runtime tools manifest.
+// Status and Note are ledger-only; the audit phase emits findings
+// without them (omitempty keeps --json clean for downstream parsing).
+// The agent edits the persisted ledger to flip Status to "accepted"
+// with a Note explaining why a finding is fine as-is. On re-run the
+// binary preserves these fields for any finding whose identity key
+// matches.
+//
+// Five finding kinds, two surfaces:
+// - Cobra surface (internal/cli/*.go): "empty-short", "thin-short",
+// "missing-read-only" — these check shell-out tools the runtime
+// walker registers from Cobra metadata.
+// - Manifest surface (tools-manifest.json): "empty-mcp-description",
+// "thin-mcp-description" — these check the descriptions agents
+// actually see for typed endpoint tools, where the source is the
+// OpenAPI spec rather than the Cobra Short.
type ToolsAuditFinding struct {
- Kind string `json:"kind"` // "empty-short", "thin-short", "missing-read-only"
- Command string `json:"command"` // value of the Use field, e.g. "tiktok"
- File string `json:"file"` // path relative to cli-dir
- Line int `json:"line"` // 1-based source line of the cobra.Command literal
- Evidence string `json:"evidence"` // the offending text (Short value, etc.)
+ Kind string `json:"kind"`
+ Command string `json:"command"` // tool name (manifest) or Cobra Use head (source)
+ File string `json:"file"` // file path relative to cli-dir
+ Line int `json:"line"` // 1-based source line; 0 for manifest findings
+ Evidence string `json:"evidence"` // the offending text
Status string `json:"status,omitempty"` // "" (== pending) or "accepted"; agent writes
Note string `json:"note,omitempty"` // agent-written rationale for an accept decision
}
@@ -101,18 +142,45 @@ type ToolsAuditFinding struct {
// frameworkCommands skip set (search, sql, doctor, version) — the
// runtime walker doesn't register those as MCP tools, so a missing
// read-only annotation is meaningless noise for them.
+//
+// tail/since/report/lint were added after dub's polish-Pass-2 surfaced
+// them as commands the heuristic missed. They're consistently read-
+// shaped across domains (log tail, time-windowed listing, generated
+// report, static check). Do not add domain-specific verbs (funnel,
+// leaderboard, journey, drift) — those mean reads in some verticals
+// and writes in others; let Pass 2 catch them per CLI.
var readShapedNames = map[string]struct{}{
"list": {}, "get": {}, "show": {}, "view": {},
"find": {}, "describe": {}, "context": {}, "stats": {},
"trending": {}, "trust": {}, "health": {}, "stale": {}, "orphans": {},
- "reconcile": {}, "analytics": {},
+ "reconcile": {}, "analytics": {}, "tail": {}, "since": {},
+ "report": {}, "lint": {},
}
-// runToolsAudit reads every non-test .go file in <cliDir>/internal/cli/
-// and accumulates findings across each Command literal. Returns
-// findings sorted by file then line so the human-readable table is
-// stable and the JSON output is diff-friendly.
+// runToolsAudit walks two surfaces: the Cobra source under
+// <cliDir>/internal/cli/*.go (shell-out tools) and the runtime tools
+// manifest at <cliDir>/tools-manifest.json (typed endpoint tools).
+// Findings are sorted by file then line for stable output.
func runToolsAudit(cliDir string) ([]ToolsAuditFinding, error) {
+ cobraFindings, err := auditCobraSource(cliDir)
+ if err != nil {
+ return nil, err
+ }
+ manifestFindings := auditMCPManifest(cliDir)
+ findings := append(cobraFindings, manifestFindings...)
+ sort.Slice(findings, func(i, j int) bool {
+ if findings[i].File != findings[j].File {
+ return findings[i].File < findings[j].File
+ }
+ if findings[i].Line != findings[j].Line {
+ return findings[i].Line < findings[j].Line
+ }
+ return findings[i].Command < findings[j].Command
+ })
+ return findings, nil
+}
+
+func auditCobraSource(cliDir string) ([]ToolsAuditFinding, error) {
pkgDir := filepath.Join(cliDir, "internal", "cli")
entries, err := os.ReadDir(pkgDir)
if err != nil {
@@ -146,20 +214,75 @@ func runToolsAudit(cliDir string) ([]ToolsAuditFinding, error) {
return true
})
}
- sort.Slice(findings, func(i, j int) bool {
- if findings[i].File != findings[j].File {
- return findings[i].File < findings[j].File
- }
- return findings[i].Line < findings[j].Line
- })
return findings, nil
}
+// toolsManifest is the subset of <cli-dir>/tools-manifest.json the
+// audit reads. The full manifest carries more fields (params, method,
+// path, transport metadata) but only Name and Description matter here.
+type toolsManifest struct {
+ Tools []toolsManifestEntry `json:"tools"`
+}
+
+type toolsManifestEntry struct {
+ Name string `json:"name"`
+ Description string `json:"description"`
+}
+
+// auditMCPManifest reads tools-manifest.json and flags MCP tool
+// descriptions that fall below the agent-grade bar. The manifest is
+// the source of truth for typed endpoint tools' descriptions; for
+// shell-out tools, descriptions come from the Cobra Short and the
+// auditCommandFields path covers them. Returns nil silently if the
+// manifest is missing (older CLIs predate it) or malformed.
+func auditMCPManifest(cliDir string) []ToolsAuditFinding {
+ path := filepath.Join(cliDir, manifestFile)
+ data, err := os.ReadFile(path)
+ if err != nil {
+ return nil
+ }
+ var m toolsManifest
+ if err := json.Unmarshal(data, &m); err != nil {
+ return nil
+ }
+ var findings []ToolsAuditFinding
+ for _, t := range m.Tools {
+ if t.Name == "" {
+ continue
+ }
+ switch {
+ case t.Description == "":
+ findings = append(findings, ToolsAuditFinding{
+ Kind: "empty-mcp-description", Command: t.Name,
+ File: manifestFile, Evidence: "(empty)",
+ })
+ case thinMCPDescription(t.Description):
+ findings = append(findings, ToolsAuditFinding{
+ Kind: "thin-mcp-description", Command: t.Name,
+ File: manifestFile, Evidence: t.Description,
+ })
+ }
+ }
+ return findings
+}
+
+// thinMCPDescription flags descriptions that are both short and
+// low-word-count — the "Create a tag" / "List items" shape that's
+// fine for OpenAPI documentation and inadequate for agents. Either
+// dimension alone is acceptable: a precise 50-char one-liner with 9
+// words can be agent-grade, and a 65-char description packed with
+// jargon may still be too thin in word count. Both signals firing is
+// the suspect pattern.
+func thinMCPDescription(s string) bool {
+ return len(s) < mcpDescMinLen && len(strings.Fields(s)) < mcpDescMinWords
+}
+
type commandFields struct {
use string
short string
hasReadOnly bool
hasEndpoint bool
+ hasRunE bool // true when the literal declares Run or RunE; parent groupers omit both
}
func isCobraCommandType(expr ast.Expr) bool {
@@ -197,6 +320,8 @@ func extractCommandFields(lit *ast.CompositeLit) commandFields {
f.short = stringLit(kv.Value)
case "Annotations":
f.hasReadOnly, f.hasEndpoint = inspectAnnotations(kv.Value)
+ case "Run", "RunE":
+ f.hasRunE = true
}
}
return f
@@ -240,23 +365,40 @@ func auditCommandFields(file string, line int, f commandFields) []ToolsAuditFind
}
name := cmdName[0]
+ // The Cobra-side checks only apply to commands that actually become
+ // shell-out MCP tools at runtime. The cobratree walker skips:
+ // - pp:endpoint commands (registered as typed tools with
+ // spec-derived descriptions; the manifest audit covers those)
+ // - parent groupers (no Run/RunE means not actionable)
+ // - framework commands (auth, doctor, version, etc.) — including
+ // their entire subtree. Generated CLIs put children (e.g.
+ // `auth status`, `profile list`) in the same file as the parent,
+ // so the file basename's leading token tells us the subtree
+ // even when the child's own Use field doesn't match a framework
+ // name.
+ // For all of these, the Cobra Short isn't the MCP tool description
+ // the agent will see, so flagging it would be noise.
+ isShellOut := !f.hasEndpoint && f.hasRunE && !frameworkCommands[name] && !inFrameworkSubtree(file)
+
var out []ToolsAuditFinding
- switch {
- case f.short == "":
- out = append(out, ToolsAuditFinding{
- Kind: "empty-short", Command: name, File: file, Line: line,
- Evidence: "(empty)",
- })
- case suspiciousShort(f.short):
- out = append(out, ToolsAuditFinding{
- Kind: "thin-short", Command: name, File: file, Line: line,
- Evidence: f.short,
- })
+ if isShellOut {
+ switch {
+ case f.short == "":
+ out = append(out, ToolsAuditFinding{
+ Kind: "empty-short", Command: name, File: file, Line: line,
+ Evidence: "(empty)",
+ })
+ case suspiciousShort(f.short):
+ out = append(out, ToolsAuditFinding{
+ Kind: "thin-short", Command: name, File: file, Line: line,
+ Evidence: f.short,
+ })
+ }
}
- // pp:endpoint commands are skipped by the runtime walker (they get
- // typed-tool registration with method-derived classification), so
- // the missing-read-only check doesn't apply to them.
- if !f.hasEndpoint && !f.hasReadOnly && readShapedName(name) {
+ // missing-read-only applies only to commands that become shell-out
+ // MCP tools (typed endpoint tools get classification from the spec
+ // method; framework commands don't register at all).
+ if isShellOut && !f.hasReadOnly && readShapedName(name) {
out = append(out, ToolsAuditFinding{
Kind: "missing-read-only", Command: name, File: file, Line: line,
Evidence: "name matches read heuristic; no mcp:read-only annotation",
@@ -265,6 +407,19 @@ func auditCommandFields(file string, line int, f commandFields) []ToolsAuditFind
return out
}
+// inFrameworkSubtree returns true when the file basename's leading
+// token (split on `_`) is a framework command. Generated CLIs follow
+// the convention <parent>.go or <parent>_<child>.go, so this catches
+// both the parent and its subtree without needing to track AddCommand
+// edges across files.
+func inFrameworkSubtree(file string) bool {
+ base := strings.TrimSuffix(file, ".go")
+ if i := strings.IndexByte(base, '_'); i > 0 {
+ base = base[:i]
+ }
+ return frameworkCommands[base]
+}
+
// suspiciousShort flags Short text that's both short (under 30 chars)
// and uses fewer than 4 words. Either dimension alone is fine: a long
// 3-word phrase is OK, and a short-but-precise instruction is OK.
diff --git a/skills/printing-press-polish/SKILL.md b/skills/printing-press-polish/SKILL.md
index 1620101f..e41b88f7 100644
--- a/skills/printing-press-polish/SKILL.md
+++ b/skills/printing-press-polish/SKILL.md
@@ -116,20 +116,29 @@ fi
### Divergence check
-The internal copy at `$CLI_DIR` can drift from the public library (`mvanhorn/printing-press-library`) copy if anyone edited the public repo directly after this CLI was last published. Polishing a stale internal copy and re-publishing later would overwrite those public-only fixes.
+**Stop and run this step before Phase 1. Do not skip it. Do not proceed to diagnostics until you have completed the check and resolved any divergence.**
-Find the public library clone on the user's machine. Honor `$PRINTING_PRESS_LIBRARY_PUBLIC` if set, otherwise locate a clone however fits this platform. Validate by checking the git remote points at `mvanhorn/printing-press-library` — other directories may share the name (forks, accidental name collisions). If multiple valid clones exist, prefer the most recently modified; ask the user to disambiguate only if still unclear.
+The internal copy at `$CLI_DIR` can drift from the public library (`mvanhorn/printing-press-library`) copy if anyone edited the public repo directly after this CLI was last published. Polishing a stale internal copy and re-publishing later silently overwrites those public-only fixes — a real failure mode that shipped CLIs hit.
+
+**You must:**
+
+1. **Locate the public library clone.** Honor `$PRINTING_PRESS_LIBRARY_PUBLIC` if set; otherwise scan the user's filesystem however fits this platform. Validate every candidate by checking the git remote points at `mvanhorn/printing-press-library` — other directories may share the name (forks, accidental name collisions). If multiple valid clones exist, prefer the most recently modified; ask the user to disambiguate only if still unclear.
+2. **Locate this CLI inside the clone.** `find <clone>/library -type d -name "<api>-pp-cli"` or equivalent.
+3. **Run `diff -r <public-cli-dir> $CLI_DIR`** (excluding build artifacts, the `.printing-press-tools-polish.json` ledger, and the binary).
+4. **Surface the result** before continuing.
Outcomes:
-- **No clone found** → user doesn't have public locally; proceed silently on internal.
-- **Clone found but doesn't contain this CLI** → never published or under a different name; proceed silently on internal.
-- **Found and `diff -r` is empty** → in sync; proceed silently on internal.
-- **Found and divergent** → don't compute "which side is newer" (file mtimes lie, internal isn't a git repo). Show the user the divergent files and ask via AskUserQuestion: **sync public→internal**, or **proceed without syncing**. If the user picks sync, copy public's version of the divergent files into internal, then continue polish.
+- **No clone found** → user doesn't have public locally. State this explicitly ("public library not found locally; proceeding on internal as canonical") and continue.
+- **Clone found but doesn't contain this CLI** → never published or under a different name. State this and continue.
+- **Found and diff is empty** → in sync. State this and continue.
+- **Found and divergent** → **stop**. Do not run Phase 1 diagnostics yet. List the divergent files for the user. Ask via AskUserQuestion: **sync public→internal**, or **proceed without syncing**. If the user picks sync, copy public's version of the divergent files into internal, then continue polish on the synced internal copy.
Before showing the sync prompt, check whether internal has files modified after its `.printing-press.json` timestamp (the user has been polishing locally without publishing). If yes, hedge the prompt explicitly: syncing will overwrite their pending local work. Let them decide whether to keep their local edits or pull public's.
-After sync (or skip), the rest of polish operates on `$CLI_DIR` as canonical. The eventual `/printing-press-publish` step pushes internal back to public; no second divergence check is needed there.
+After sync (or explicit skip), the rest of polish operates on `$CLI_DIR` as canonical. The eventual `/printing-press-publish` step pushes internal back to public; no second divergence check is needed there.
+
+**The check has run only when one of the four outcomes above is explicitly stated in your response.** Silent omission counts as not having run it.
## Phase 1: Baseline diagnostics
diff --git a/skills/printing-press-polish/references/tools-polish.md b/skills/printing-press-polish/references/tools-polish.md
index 87b08ba7..da124b4d 100644
--- a/skills/printing-press-polish/references/tools-polish.md
+++ b/skills/printing-press-polish/references/tools-polish.md
@@ -1,9 +1,16 @@
# Tools Polish — MCP Tool Quality Playbook
-**Your goal:** ensure every MCP tool exposed by this CLI carries the metadata an agent needs to use it correctly. Tool descriptions and classifications are how agents discover and decide whether to call a tool — thin descriptions and missing annotations directly degrade agent UX.
+**Your goal:** ensure every MCP tool exposed by this CLI carries a description an agent can use to decide whether to call the tool and how to call it correctly. Tool descriptions are how agents discover and choose tools — thin descriptions and missing classifications directly degrade agent UX.
+
+**MCP descriptions ≠ Cobra Shorts.** They're different artifacts with different jobs:
+
+- **Cobra Short** is `--help` text. Reader is a human who can scroll, experiment, and uses surrounding context. Terse-but-precise often works ("Print version", "Check CLI health").
+- **MCP tool description** is what an agent reads while picking from a catalog of dozens of tools. The agent needs enough to decide *whether* to call this tool and *how* to call it correctly without trying it first. Usually richer than `--help` — typically 1-3 sentences naming the action, the key parameters, what comes back, and when to prefer this tool over alternatives.
This playbook has two passes. Both run on every CLI; do not skip either.
+**The audit is a tripwire, not the assessment.** The audit's length-and-word-count thresholds catch the obviously inadequate cases — the 12-char "Create a tag" entries that any sensible reading would flag. But the real question — "does this description give an agent enough to use this tool correctly?" — requires reading the description against the operation it describes, which is judgment work. A 70-char description can be perfect or terrible; the binary can't tell. **Pass 2 is where the quality assessment actually happens.** Pass 1 is the floor.
+
## Pass 1: Address mechanical findings
Run the audit:
@@ -12,48 +19,91 @@ Run the audit:
printing-press tools-audit <cli-dir> --json
```
-The audit emits findings of three kinds. Address each:
+The audit emits findings on two surfaces:
+
+- **Cobra source** (`internal/cli/*.go`) — `empty-short`, `thin-short`, `missing-read-only`. These check shell-out tools the runtime walker registers from Cobra metadata.
+- **Tools manifest** (`tools-manifest.json`) — `empty-mcp-description`, `thin-mcp-description`. These check the descriptions agents actually see for typed endpoint tools, where the source is the OpenAPI spec rather than the Cobra Short.
+
+The audit exempts `pp:endpoint`-annotated commands, parent groupers (no `RunE`), and framework-skipped commands (`auth`, `doctor`, `version`, …) from the Cobra-side checks — those don't become MCP tools the way Cobra Short would describe them.
+
+A finding here means the description is mechanically thin enough that there's no chance it's adequate. Fix it. But absence of a finding does NOT mean the description is good — the threshold is a floor, not a ceiling. That's what Pass 2 is for.
+
+#### DO-NOT-EDIT files
+
+Before fixing any finding, check the file header. Generated files carry a `// Generated by CLI Printing Press ... DO NOT EDIT.` comment. Hand-edits to those files are wiped on the next regeneration — fixing a thin Short in `bookings_create.go` (generated) only to have `mcp-sync` or a fresh generation overwrite it solves nothing.
+
+For findings in DO-NOT-EDIT files:
+
+1. **Mark the finding accepted in the ledger** with `note: "DO-NOT-EDIT generated file; needs generator-template fix"`.
+2. **Surface a retro candidate** in your final summary. The accepted-with-rationale pattern is correct here, but the rationale is a system-level ask: the generator template that emitted this file should produce better defaults. Note the pattern in the polish summary so it can be addressed in a `/printing-press-retro` pass.
+
+Files that are NOT DO-NOT-EDIT and ARE safe to edit:
+
+- Hand-written novel commands (no generator header)
+- The polish skill's own targets: README.md, SKILL.md, manifest.json
+- `internal/cli/root.go` (the generator emits the scaffold but the skill mutates it during polish)
+
+`tools-manifest.json` is a special case — it's generated, but the dual-edit path documented under `thin-mcp-description` deliberately edits both manifest and `tools.go` together. The runtime registers from `tools.go`; the manifest is a parallel artifact. If `mcp-sync` regenerates the manifest, the description edits in tools.go survive. Verify this on your CLI before relying on it; if both are regenerated atomically, this is a generator-template ask, not a polish-time fix.
### `empty-short`
-A `cobra.Command{}` with no `Short:` field. The runtime walker falls back to `"Run \`<cmd>\` through the companion CLI binary."` — a meaningless description for agents.
+A `cobra.Command{}` shell-out tool with no `Short:` field. The runtime walker falls back to `"Run \`<cmd>\` through the companion CLI binary."` — meaningless to agents.
**Fix:** write a verb-led, action-specific Short. Read the command's `RunE` body and `Flags()` block to ground the description in actual behavior. See **Description criteria** below.
### `thin-short`
-Short text under 30 characters AND fewer than 4 words. Examples from real CLIs: `"Advertiser Search"`, `"Search Ads"`, `"Subreddit Posts"`, `"Product Reviews"`.
+Cobra Short text under 30 characters AND fewer than 4 words on a shell-out tool. Examples from real CLIs: `"Advertiser Search"`, `"Search Ads"`, `"Subreddit Posts"`, `"Product Reviews"`.
**This is a suspect, not a verdict.** Some short Shorts are accurate (`"Print version"`, `"Check CLI health"`). Decide per finding:
-- If the Short is a fragment that doesn't tell an agent what action runs, what parameters it takes, or what comes back → rewrite per **Description criteria**.
-- If the Short is brief but precise → leave it; document the decision implicitly by skipping the rewrite.
+- If the Short doesn't tell an agent what action runs, what parameters it takes, or what comes back → rewrite per **Description criteria**.
+- If the Short is brief but precise → mark accepted in the ledger with a one-line rationale.
### `missing-read-only`
-Command name matches a read-shaped pattern (`list`, `get`, `search`, `show`, `view`, `find`, `describe`, `context`, `sql`, `stats`, `trending`, `trust`, `health`, `stale`, `orphans`, `reconcile`, `doctor`, `version`, `analytics`) but no `mcp:read-only` annotation, AND no `pp:endpoint` (so the runtime walker registers it as a shell-out tool).
+Shell-out tool whose name matches a read-shaped pattern (`list`, `get`, `show`, `view`, `find`, `describe`, `context`, `stats`, `trending`, `trust`, `health`, `stale`, `orphans`, `reconcile`, `analytics`) but has no `mcp:read-only` annotation.
**Fix:** add `Annotations: map[string]string{"mcp:read-only": "true"},` to the command literal. The walker then emits `readOnlyHint: true` so MCP hosts skip the per-call permission prompt.
-**Don't blindly accept.** If the command name matches the heuristic but the body actually mutates state (writes to the local store, opens a browser, sends a notification, modifies config), do NOT add the annotation. The heuristic is a starting point; the body is the truth.
+**Don't blindly accept.** If the name matches the heuristic but the body actually mutates state (writes to the local store, opens a browser, sends a notification, modifies config), do NOT add the annotation. The heuristic is a starting point; the body is the truth.
+
+### `empty-mcp-description`
+
+A typed endpoint tool whose `description` field in `tools-manifest.json` is empty.
-## Pass 2: Evaluate every command (audit-independent)
+**Fix:** write an MCP-grade description (see criteria below) and update both:
-Run a judgment pass over every user-facing command in `internal/cli/`, including ones the audit didn't flag. Mechanical detection misses two real classes:
+1. `tools-manifest.json` — the `description` field for the matching tool entry.
+2. `internal/mcp/tools.go` — the `mcplib.WithDescription("...")` call for the same tool. Keep the two in sync.
-1. **Read-only commands whose names don't match the heuristic.** Platform-named shortcuts (`tiktok`, `instagram`, `google`), one-word reads (`search`, `analytics`, `export`), commands renamed away from their `list-*`/`get-*` form (`facebook_list-post-3` → `post-transcript`). All of these are real reads that need `mcp:read-only` but won't trip the audit.
-2. **Descriptions that pass the length check but are still poor for agents.** A 50-character Short that says `"Manage your saved cookbook (list, search, tag, match)"` is category-shaped, not action-shaped — fine for human help text, weak for an agent deciding whether to call it.
+### `thin-mcp-description`
+
+A typed endpoint tool whose description is both short (<60 chars) and low-word-count (<8 words). The dominant pattern: spec-derived summaries like `"Create a tag"`, `"List domains"`, `"Update a link"` — fine for OpenAPI doc rendering, inadequate for an agent picking from a tool catalog. The agent looking at `"Create a tag"` has to guess what fields to pass, what comes back, and when to prefer this over alternatives.
+
+**Fix:** rewrite per the MCP description criteria below. Update both `tools-manifest.json` and `internal/mcp/tools.go` so the runtime and the manifest stay in sync.
+
+**Acceptance is rare here.** A 30-char description is almost never enough for a typed MCP tool. Only accept if the underlying operation is genuinely so simple that more words add no information (e.g., a synthetic ping/heartbeat tool).
+
+## Pass 2: Evaluate every command (the load-bearing pass)
+
+This is where the actual quality assessment happens. Walk every user-facing command in `internal/cli/` and every entry in `tools-manifest.json`, including ones the audit didn't flag. The mechanical check in Pass 1 misses two real classes:
+
+1. **Read-only commands whose names don't match the heuristic.** Platform-named shortcuts (`tiktok`, `instagram`, `google`), one-word reads (`search`, `analytics`, `export`), commands renamed away from their `list-*`/`get-*` form. Real reads that need `mcp:read-only` but won't trip the audit.
+2. **Descriptions that clear the length threshold but still leave an agent guessing.** A 65-character description that says `"Manage your saved cookbook (list, search, tag, match)"` is category-shaped, not action-shaped. A 70-char MCP description that names the action but no parameters or return shape is technically over the threshold and still inadequate. Length is a proxy for content; only reading the description against the operation tells you whether it earns its bytes.
For each command:
-- Read its `RunE` body. Does it call an HTTP method (`client.Get`, `resolveRead`, `c.Post`, …)? Does it write to the local store, the filesystem, or config?
+- Read its `RunE` body (or for typed endpoint tools, the underlying spec method/path). Does it call an HTTP method (`client.Get`, `resolveRead`, `c.Post`, …)? Does it write to the local store, the filesystem, or config?
- **HTTP GET only, no local writes** → mark `mcp:read-only` (even if the audit didn't flag).
- **Any local write or external mutation** → do NOT mark; the default destructive hint is correct.
- **Side effect** (browser open, notification, audio) → do NOT mark; agents should be prompted.
-- Read its `Short:` and `Long:` against the criteria below. If thin or category-shaped, rewrite.
+- Read the description (Cobra `Short` for shell-out, manifest entry for typed endpoint). Test against the criteria below — not just "is it long enough" but "does it actually answer the four questions an agent needs answered before calling this tool?" If the answer is "not really," rewrite.
## Description criteria
+### Cobra Short (shell-out tools)
+
Agent-grade Shorts share four properties:
1. **Verb-led.** Open with the action: `"Search ..."`, `"Fetch ..."`, `"List ..."`, `"Estimate ..."`, `"Save ..."`. Not `"Cookbook commands"` or `"Manage ..."`.
@@ -61,6 +111,39 @@ Agent-grade Shorts share four properties:
3. **Parameter-aware.** Name the meaningful filters/options inline so the agent knows what to pass: `"... by max cook time, recency, and dietary tags"`, not just `"Pick dinner"`.
4. **Scope-explicit.** When the command operates on a subset, say so: `"Search recipe titles across curated sites without fetching the full recipes (returns metadata only)"` — tells the agent the cost/output trade-off vs. a richer search.
+### MCP descriptions (typed endpoint tools)
+
+The same four properties, plus a higher bar on length and content. Agent looking at a tool catalog needs roughly 1-3 sentences answering:
+
+- **What action does this tool perform?** (the verb-led part)
+- **What are the key parameters?** Required ones explicitly; the most useful optional ones too. Agents call tools blind — without the description, the only signal is the parameter schema, which doesn't tell them which ones matter.
+- **What comes back?** The shape of the result. "Returns the new link's id and slug" beats "Creates a link."
+- **When to prefer this tool?** When two or three tools could plausibly handle the request, what distinguishes this one.
+
+Worked example:
+
+```
+Before: "Create a tag"
+After: "Create a new tag in the workspace. Required: name. Optional: color (hex)
+ and parentTagId for nested tag groups. Returns the tag's id and slug.
+ Tags must exist before they can be assigned to links — call this first
+ if you're tagging a new category."
+```
+
+The before is what OpenAPI auto-generators emit. The after is what an agent picking between `tags_create`, `tags_update`, `links_create` (which can also accept tags) actually needs.
+
+#### Where to mine the content from
+
+Don't write MCP descriptions from general API knowledge — that produces plausible but inaccurate prose. The CLI ships with the source material; ground every rewrite in it:
+
+1. **The OpenAPI spec** at `<cli-dir>/spec.json` (or `spec.yaml`). For each tool, find the matching operation by `path` and `method` (those are in `tools-manifest.json` next to the description). The operation object usually has:
+ - A richer `description` field, often several sentences, that the manifest's `summary` truncated. This is your primary source for the rewrite.
+ - `parameters[].description` — use these to name the meaningful parameters in your rewrite. Don't list every parameter; pick the 1-3 that drive how the agent calls the tool.
+ - `responses['200'].content[*].schema` — tells you what comes back. Translate the schema shape into a one-clause "Returns ..." rather than a field-by-field listing.
+2. **The runtime registration** at `<cli-dir>/internal/mcp/tools.go`. Find the matching `mcplib.NewTool(...)` block. The parameter list there is what the runtime actually exposes (sometimes a subset of what the spec defines). Your description should reflect that subset, not the spec's superset.
+
+If the spec's operation `description` is itself empty or as thin as the `summary`, you're upstream of an incomplete spec — flag this in the accept rationale and write the best description you can from the parameters and response schema. Don't make up parameters or behaviors the spec doesn't document.
+
### Anti-patterns to remove
- **Dev-state leakage.** `"(wip)"`, `"(planned)"`, `"(ranking integration wip)"`, `"(coming soon)"` — useless to agents, leaks dev backlog. Strip during polish.
@@ -70,9 +153,9 @@ Agent-grade Shorts share four properties:
## Worked examples
-Pre/post pairs from real polish passes. The "before" lines are what we found in the wild; the "after" lines were the rewrites that landed.
+Pre/post pairs from real polish passes.
-### recipe-goat (from PR #154 polish commit)
+### Cobra Shorts (shell-out tools)
```
Before: "Lightweight cross-site recipe search (metadata only, no fetch)"
@@ -85,41 +168,38 @@ After: "List past cooking sessions with ratings and notes, most recent first"
```
```
-Before: "View and override site trust scores (ranking integration wip)"
-After: "View and override per-site trust scores used by the cross-site ranker"
+Before: "Advertiser Search"
+After: "Search the Google Ads Transparency Center for advertisers by name or domain"
```
```
-Before: "Persist a site trust override (ranking integration wip)"
-After: "Save a local per-site trust adjustment that the ranker applies on top of the built-in scores"
+Before: "Subreddit Posts"
+After: "Fetch posts from a subreddit, with sort (hot/new/top) and timeframe filters"
```
-### scrape-creators (from PR #113 polish commit)
+### MCP descriptions (typed endpoint tools)
```
-Before: "Advertiser Search"
-After: "Search the Google Ads Transparency Center for advertisers by name or domain"
+Before: "Create a tag"
+After: "Create a new tag in the workspace. Required: name. Optional: color (hex) and parentTagId.
+ Returns the tag's id and slug. Tags must exist before they can be assigned to links."
```
```
-Before: "Search Ads"
-After: "Search the LinkedIn Ad Library by company, keyword, country, and date range"
+Before: "List all customers"
+After: "List customers in the workspace. Optional filters: email, externalId, country.
+ Returns paginated array with id, email, name, createdAt. Use customers_get-id when
+ you already have an id; use this when you need to search by attribute."
```
```
-Before: "Subreddit Posts"
-After: "Fetch posts from a subreddit, with sort (hot/new/top) and timeframe filters"
-```
-
-```
-Before: "Product Reviews"
-After: "Fetch product reviews from a TikTok Shop product page (by URL or product ID)"
+Before: "Retrieve analytics for a link, a domain, or the authenticated workspace."
+After: (already agent-grade — this is the upper bound of what the OpenAPI summary field
+ produces. No rewrite needed; mark accepted with a brief note.)
```
## Per-finding response templates
-When the audit returns a finding, follow this minimum response.
-
### Empty Short
```go
@@ -143,19 +223,40 @@ Short: "<verb> <specific scope> by <main filter>, <secondary filter>",
Annotations: map[string]string{"mcp:read-only": "true"},
```
-The audit already exempts commands carrying `pp:endpoint` (those get typed-tool registration with method-derived classification), so this finding never fires on endpoint mirrors.
+### Empty / thin MCP description
+
+Two files to update for the same tool — keep them in sync.
+
+`tools-manifest.json`:
+
+```json
+{
+ "name": "tags_create",
+ "description": "<MCP-grade description per criteria above>",
+ ...
+}
+```
+
+`internal/mcp/tools.go`:
+
+```go
+mcplib.NewTool("tags_create",
+ mcplib.WithDescription("<same MCP-grade description>"),
+ ...
+)
+```
## Ledger and resumability
-`tools-audit` writes `<cli-dir>/.printing-press-tools-polish.json` after every run. It contains the timestamp, cli-dir, and one entry per finding. The ledger serves three purposes:
+`tools-audit` writes `<cli-dir>/.printing-press-tools-polish.json` after every run. It contains the timestamp, cli-dir, and one entry per finding.
-1. **Delta computation.** On a second run within 24 hours, the audit prints `since last run: N resolved, M new` so you can see your progress without re-counting. Stale ledgers (>24h) are deleted automatically.
+1. **Delta computation.** On a second run within 24 hours, the audit prints `since last run: N resolved, M new`. Stale ledgers (>24h) are deleted automatically.
2. **Resumability.** If your context window flushes mid-polish, re-run `tools-audit <cli-dir>`. Findings you've fixed have disappeared from the new scan; findings you accepted are still recorded with status. You pick up where you left off.
-3. **Audit trail of accept decisions.** When you decide a `thin-short` is fine as-is, you mark the entry `accepted` and write a one-sentence rationale. The next run filters it out of the pending table.
+3. **Audit trail of accept decisions.** When you decide a finding is fine as-is, you mark the entry `accepted` and write a one-sentence rationale. The next run filters it out of the pending table.
### Marking a finding accepted
-When the table shows a `thin-short` whose Short is brief but precise (`"Print version"`, `"Check CLI health"`, `"Show authentication status"`), edit the ledger entry directly:
+When the table shows a finding that's actually fine, edit the ledger entry directly:
```json
{
@@ -169,13 +270,15 @@ When the table shows a `thin-short` whose Short is brief but precise (`"Print ve
}
```
-After marking, re-run `tools-audit <cli-dir>`. The pending table will exclude this finding. The accepted count in the header (`(1 accepted)`) confirms the decision persisted.
+After marking, re-run `tools-audit <cli-dir>`. The pending table excludes the accepted entry; the header shows `(N accepted)` to confirm.
+
+**Don't accept `empty-short`, `empty-mcp-description`, or `missing-read-only` findings.** They have no judgment call: empty descriptions always need text, and a read command always wants the annotation.
-**Don't accept `empty-short` or `missing-read-only` findings.** They have no judgment call to make: empty Shorts always need text, and a read command always wants the annotation. Acceptance is reserved for `thin-short`.
+**Acceptance is rare for `thin-mcp-description`.** A typed endpoint tool whose description is so brief it tripped the heuristic almost certainly leaves the agent guessing. Accept only when the operation is genuinely trivial enough that more words don't help.
### Don't manually mark findings as fixed
-If you fixed a finding via code change, do nothing in the ledger. The next audit re-scans the source; finding gone from the AST → finding gone from the table → delta line shows `1 resolved`. Never set `status: "fixed"` by hand — the tool detects this automatically and your manual marking will read as "still flagged but accepted-without-rationale."
+If you fixed a finding via code change, do nothing in the ledger. The next audit re-scans the source and the manifest; finding gone → finding gone from the table → delta line shows `1 resolved`. Never set `status: "fixed"` by hand.
## Verification checklist
@@ -184,6 +287,7 @@ After applying fixes, before declaring the polish complete:
- [ ] `go build ./...` clean (annotations don't break compilation)
- [ ] `printing-press tools-audit <cli-dir>` shows zero pending findings — every finding is either fixed (auto-removed) or explicitly accepted with a `note`
- [ ] `printing-press dogfood --dir <cli-dir>` reports `MCP Surface: PASS`
+- [ ] If `tools-manifest.json` was edited, `tools.go` was updated to match (the runtime registers from `tools.go`; the manifest documents what was registered)
- [ ] If commands were renamed or had their annotations restructured, smoke-test the binary by inspecting `--help` output for the affected commands
The ledger file persists until it ages out (24h). Once the polish PR merges and the CLI is rebuilt, the file is no longer load-bearing — the next `tools-audit` run can start fresh.
← 6727d0d8 feat(skills): add divergence check to polish skill setup (#3
·
back to Cli Printing Press
·
feat(cli): MCP description overrides + tools-audit scoring + e651d01f →