← back to Cli Printing Press
feat(cli): MCP description overrides + tools-audit scoring + polish skill nibbles (#382)
e651d01f09118dc55eb1d385d70e12492560a810 · 2026-04-29 08:13:50 -0700 · Trevin Chow
Three concerns bundled because they all hinge on the same dub/cal-com
polish observation: the audit finds real thin-mcp-description findings
on most CLIs but the polish path can't fix any of them durably (manifest
and tools.go are DO-NOT-EDIT generated artifacts), and even when polish
makes real improvements the scorer doesn't reward them.
(1) MCP description override mechanism. New internal/mcpoverrides/
package reads a hand-editable mcp-descriptions.json sidecar at the
cli-dir root and applies it to the parsed spec before generation.
mcpsync.Sync loads + applies overrides; both WriteToolsManifest and
the mcp_tools.go template see the overridden descriptions through
the same parsed spec, so the manifest and the runtime registration
stay in lockstep. The override file lives outside the generator's
emit set and survives regen. mcpsync now always regenerates the MCP
surface (was gated on !alreadyMigrated) so override edits propagate
to tools.go consistently. ToolName helper exposed for shared use
between the override loader and WriteToolsManifest.
(2) MCPDescriptionQuality scorecard dimension. Reads tools-manifest.json
via the new pipeline.ReadToolsManifest helper and counts entries
tripping pipeline.IsThinMCPDescription — the same predicate the audit
uses for thin-mcp-description findings. Stepped curve: 0% thin → 10,
<5% → 9, <15% → 7, <30% → 5, <50% → 3, else → 0. Unscored when no
manifest exists. Closes the proxy gap where polish work on MCP
description quality was invisible to the scorecard signal.
(3) Polish skill prose nibbles. printing-press-polish/SKILL.md gains
Priority 0 (auto-run mcp-sync when dogfood reports MCP-parity FAIL,
the legacy-CLI migration path), explicit env-failure classification
in the diagnostic categories table (SQLITE_BUSY, 401 mock token, qr
binary output etc. are not CLI defects), and push-higher-without-
gaming guidance after ship gates pass. The playbook (references/
tools-polish.md) now documents the override-file path and tells the
agent to write to mcp-descriptions.json + run mcp-sync rather than
edit the DO-NOT-EDIT generated files directly.
naming.OneLine split into OneLine (capped at 120 chars for compact
display) and OneLineNormalize (uncapped). MCPDescription now uses
OneLineNormalize so 363-char overrides don't get silently truncated
to 117 + "..." — the truncation defeated the entire override system
in the first end-to-end test.
Tests: mcpoverrides Apply/Load round-trips, ToolName composition,
unmatched-key surfacing, empty-overrides no-op. IsThinMCPDescription
table tests covering threshold boundaries. scoreMCPDescriptionQuality
table tests covering the full curve.
Smoke-tested end-to-end on cal-com: 363-char override applied,
manifest + tools.go both updated, audit no longer flags the tool;
removing override and re-syncing reverts both files to spec text.
Closes follow-up tasks #19 (override mechanism), #20 (scorer
integration), #21 (polish skill prose) from the dub/cal-com run
analysis.
Files touched
M internal/cli/mcp_sync.goM internal/cli/scorecard.goM internal/cli/tools_audit.goA internal/mcpoverrides/overrides.goA internal/mcpoverrides/overrides_test.goM internal/naming/naming.goM internal/pipeline/mcpsync/sync.goM internal/pipeline/scorecard.goM internal/pipeline/scorecard_tier2_test.goM internal/pipeline/toolsmanifest.goM skills/printing-press-polish/SKILL.mdM skills/printing-press-polish/references/tools-polish.mdM testdata/golden/expected/generate-golden-api/scorecard.json
Diff
commit e651d01f09118dc55eb1d385d70e12492560a810
Author: Trevin Chow <trevin@trevinchow.com>
Date: Wed Apr 29 08:13:50 2026 -0700
feat(cli): MCP description overrides + tools-audit scoring + polish skill nibbles (#382)
Three concerns bundled because they all hinge on the same dub/cal-com
polish observation: the audit finds real thin-mcp-description findings
on most CLIs but the polish path can't fix any of them durably (manifest
and tools.go are DO-NOT-EDIT generated artifacts), and even when polish
makes real improvements the scorer doesn't reward them.
(1) MCP description override mechanism. New internal/mcpoverrides/
package reads a hand-editable mcp-descriptions.json sidecar at the
cli-dir root and applies it to the parsed spec before generation.
mcpsync.Sync loads + applies overrides; both WriteToolsManifest and
the mcp_tools.go template see the overridden descriptions through
the same parsed spec, so the manifest and the runtime registration
stay in lockstep. The override file lives outside the generator's
emit set and survives regen. mcpsync now always regenerates the MCP
surface (was gated on !alreadyMigrated) so override edits propagate
to tools.go consistently. ToolName helper exposed for shared use
between the override loader and WriteToolsManifest.
(2) MCPDescriptionQuality scorecard dimension. Reads tools-manifest.json
via the new pipeline.ReadToolsManifest helper and counts entries
tripping pipeline.IsThinMCPDescription — the same predicate the audit
uses for thin-mcp-description findings. Stepped curve: 0% thin → 10,
<5% → 9, <15% → 7, <30% → 5, <50% → 3, else → 0. Unscored when no
manifest exists. Closes the proxy gap where polish work on MCP
description quality was invisible to the scorecard signal.
(3) Polish skill prose nibbles. printing-press-polish/SKILL.md gains
Priority 0 (auto-run mcp-sync when dogfood reports MCP-parity FAIL,
the legacy-CLI migration path), explicit env-failure classification
in the diagnostic categories table (SQLITE_BUSY, 401 mock token, qr
binary output etc. are not CLI defects), and push-higher-without-
gaming guidance after ship gates pass. The playbook (references/
tools-polish.md) now documents the override-file path and tells the
agent to write to mcp-descriptions.json + run mcp-sync rather than
edit the DO-NOT-EDIT generated files directly.
naming.OneLine split into OneLine (capped at 120 chars for compact
display) and OneLineNormalize (uncapped). MCPDescription now uses
OneLineNormalize so 363-char overrides don't get silently truncated
to 117 + "..." — the truncation defeated the entire override system
in the first end-to-end test.
Tests: mcpoverrides Apply/Load round-trips, ToolName composition,
unmatched-key surfacing, empty-overrides no-op. IsThinMCPDescription
table tests covering threshold boundaries. scoreMCPDescriptionQuality
table tests covering the full curve.
Smoke-tested end-to-end on cal-com: 363-char override applied,
manifest + tools.go both updated, audit no longer flags the tool;
removing override and re-syncing reverts both files to spec text.
Closes follow-up tasks #19 (override mechanism), #20 (scorer
integration), #21 (polish skill prose) from the dub/cal-com run
analysis.
---
internal/cli/mcp_sync.go | 8 +
internal/cli/scorecard.go | 1 +
internal/cli/tools_audit.go | 53 ++-----
internal/mcpoverrides/overrides.go | 119 ++++++++++++++
internal/mcpoverrides/overrides_test.go | 172 +++++++++++++++++++++
internal/naming/naming.go | 38 +++--
internal/pipeline/mcpsync/sync.go | 42 ++++-
internal/pipeline/scorecard.go | 111 ++++++++++---
internal/pipeline/scorecard_tier2_test.go | 103 +++++++++++-
internal/pipeline/toolsmanifest.go | 23 ++-
skills/printing-press-polish/SKILL.md | 32 +++-
.../references/tools-polish.md | 53 ++++---
.../expected/generate-golden-api/scorecard.json | 2 +
13 files changed, 649 insertions(+), 108 deletions(-)
diff --git a/internal/cli/mcp_sync.go b/internal/cli/mcp_sync.go
index f8e27868..5e0d4c56 100644
--- a/internal/cli/mcp_sync.go
+++ b/internal/cli/mcp_sync.go
@@ -27,6 +27,14 @@ func newMCPSyncCmd() *cobra.Command {
}
return &ExitError{Code: ExitPublishError, Err: err}
}
+ for _, name := range result.UnmatchedOverrideKeys {
+ // Surface override-file keys that didn't match any endpoint.
+ // Common causes: typo in mcp-descriptions.json, stale key
+ // after a resource/endpoint rename. Stderr-warn so the user
+ // notices but the sync still succeeds — tools-audit will
+ // flag any thin description that wasn't overridden.
+ fmt.Fprintf(cmd.ErrOrStderr(), "warning: mcp-descriptions.json key %q does not match any tool in the spec\n", name)
+ }
if result.Changed {
fmt.Fprintf(cmd.OutOrStdout(), "migrated MCP surface in %s\n", cliDir)
} else {
diff --git a/internal/cli/scorecard.go b/internal/cli/scorecard.go
index d971780c..29444c40 100644
--- a/internal/cli/scorecard.go
+++ b/internal/cli/scorecard.go
@@ -90,6 +90,7 @@ func newScorecardCmd() *cobra.Command {
fmt.Printf(" README %d/10\n", s.README)
fmt.Printf(" Doctor %d/10\n", s.Doctor)
fmt.Printf(" Agent Native %d/10\n", s.AgentNative)
+ fmt.Printf(" MCP Desc Quality %s\n", renderScore("mcp_description_quality", s.MCPDescriptionQuality, 10))
fmt.Printf(" Local Cache %d/10\n", s.LocalCache)
fmt.Printf(" Breadth %d/10\n", s.Breadth)
fmt.Printf(" Vision %d/10\n", s.Vision)
diff --git a/internal/cli/tools_audit.go b/internal/cli/tools_audit.go
index c925bee4..3f2f1b47 100644
--- a/internal/cli/tools_audit.go
+++ b/internal/cli/tools_audit.go
@@ -13,6 +13,7 @@ import (
"strings"
"time"
+ "github.com/mvanhorn/cli-printing-press/v2/internal/pipeline"
"github.com/spf13/cobra"
)
@@ -22,20 +23,12 @@ const (
statusAccepted = "accepted"
suspiciousMaxLen = 30
suspiciousMinWord = 4
-
- // MCP tool descriptions need richer text than Cobra Shorts. Agents
- // pick from a tool catalog without the context a human gets from
- // --help, so spec-derived summaries like "Create a tag" or "List
- // items" — fine for OpenAPI doc rendering — leave the agent guessing
- // what fields to pass and what comes back. The thresholds here are
- // floor values: 60 chars / 8 words is roughly two short clauses,
- // enough to convey the action plus one parameter or return shape.
- mcpDescMinLen = 60
- mcpDescMinWords = 8
-
- manifestFile = "tools-manifest.json"
)
+// MCP description thresholds (pipeline.MCPDescMinLen, MCPDescMinWords,
+// IsThinMCPDescription) live in pipeline so the scorecard applies the
+// same predicate as the audit.
+
// frameworkCommands mirrors cobratree/classify.go.tmpl. The runtime
// walker skips these names entirely — they're never registered as MCP
// tools — so audit findings on their Cobra Short are noise.
@@ -217,18 +210,6 @@ func auditCobraSource(cliDir string) ([]ToolsAuditFinding, error) {
return findings, nil
}
-// toolsManifest is the subset of <cli-dir>/tools-manifest.json the
-// audit reads. The full manifest carries more fields (params, method,
-// path, transport metadata) but only Name and Description matter here.
-type toolsManifest struct {
- Tools []toolsManifestEntry `json:"tools"`
-}
-
-type toolsManifestEntry struct {
- Name string `json:"name"`
- Description string `json:"description"`
-}
-
// auditMCPManifest reads tools-manifest.json and flags MCP tool
// descriptions that fall below the agent-grade bar. The manifest is
// the source of truth for typed endpoint tools' descriptions; for
@@ -236,15 +217,10 @@ type toolsManifestEntry struct {
// auditCommandFields path covers them. Returns nil silently if the
// manifest is missing (older CLIs predate it) or malformed.
func auditMCPManifest(cliDir string) []ToolsAuditFinding {
- path := filepath.Join(cliDir, manifestFile)
- data, err := os.ReadFile(path)
+ m, err := pipeline.ReadToolsManifest(cliDir)
if err != nil {
return nil
}
- var m toolsManifest
- if err := json.Unmarshal(data, &m); err != nil {
- return nil
- }
var findings []ToolsAuditFinding
for _, t := range m.Tools {
if t.Name == "" {
@@ -254,29 +230,18 @@ func auditMCPManifest(cliDir string) []ToolsAuditFinding {
case t.Description == "":
findings = append(findings, ToolsAuditFinding{
Kind: "empty-mcp-description", Command: t.Name,
- File: manifestFile, Evidence: "(empty)",
+ File: pipeline.ToolsManifestFilename, Evidence: "(empty)",
})
- case thinMCPDescription(t.Description):
+ case pipeline.IsThinMCPDescription(t.Description):
findings = append(findings, ToolsAuditFinding{
Kind: "thin-mcp-description", Command: t.Name,
- File: manifestFile, Evidence: t.Description,
+ File: pipeline.ToolsManifestFilename, Evidence: t.Description,
})
}
}
return findings
}
-// thinMCPDescription flags descriptions that are both short and
-// low-word-count — the "Create a tag" / "List items" shape that's
-// fine for OpenAPI documentation and inadequate for agents. Either
-// dimension alone is acceptable: a precise 50-char one-liner with 9
-// words can be agent-grade, and a 65-char description packed with
-// jargon may still be too thin in word count. Both signals firing is
-// the suspect pattern.
-func thinMCPDescription(s string) bool {
- return len(s) < mcpDescMinLen && len(strings.Fields(s)) < mcpDescMinWords
-}
-
type commandFields struct {
use string
short string
diff --git a/internal/mcpoverrides/overrides.go b/internal/mcpoverrides/overrides.go
new file mode 100644
index 00000000..0f543416
--- /dev/null
+++ b/internal/mcpoverrides/overrides.go
@@ -0,0 +1,119 @@
+// Package mcpoverrides reads the per-CLI MCP description override file
+// (mcp-descriptions.json at the cli-dir root) and applies it to a parsed
+// spec before generation. The override file is the sanctioned path for
+// hand-authored MCP tool descriptions on typed endpoint tools: direct
+// edits to internal/mcp/tools.go and tools-manifest.json are wiped on
+// the next regen because both files carry the generator's DO-NOT-EDIT
+// header. The override file is hand-editable and survives regeneration
+// because it lives outside the generator's emit set; mcp-sync calls
+// Apply on the parsed spec so both the manifest writer and the
+// mcp_tools.go template see the overridden descriptions.
+package mcpoverrides
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io/fs"
+ "os"
+ "path/filepath"
+
+ "github.com/mvanhorn/cli-printing-press/v2/internal/naming"
+ "github.com/mvanhorn/cli-printing-press/v2/internal/spec"
+)
+
+// Filename is the per-CLI override file name. Lives at the cli-dir
+// root alongside spec.yaml / spec.json / .printing-press.json.
+const Filename = "mcp-descriptions.json"
+
+// Overrides holds the user-authored description overrides keyed by
+// MCP tool name. Tool name format matches tools-manifest.json's name
+// field: snake_case resource name + "_" + snake_case endpoint name,
+// optionally with a sub-resource segment in the middle (e.g.,
+// "tags_create", "bookings_attendees_booking-add").
+type Overrides struct {
+ Descriptions map[string]string `json:"descriptions"`
+}
+
+// Load reads <cliDir>/mcp-descriptions.json. Returns an empty
+// Overrides (not an error) when the file is absent — most CLIs don't
+// have one and that's the expected steady state. Malformed JSON
+// returns a wrapped error so the caller can decide whether to abort.
+func Load(cliDir string) (Overrides, error) {
+ path := filepath.Join(cliDir, Filename)
+ data, err := os.ReadFile(path)
+ if err != nil {
+ if errors.Is(err, fs.ErrNotExist) {
+ return Overrides{}, nil
+ }
+ return Overrides{}, fmt.Errorf("reading %s: %w", path, err)
+ }
+ var o Overrides
+ if err := json.Unmarshal(data, &o); err != nil {
+ return Overrides{}, fmt.Errorf("parsing %s: %w", path, err)
+ }
+ return o, nil
+}
+
+// Apply mutates parsed.Resources in place: for each endpoint whose
+// computed tool name matches an override key, replace the endpoint's
+// Description with the override text. Both the manifest writer and
+// the mcp_tools.go template read from the same parsed spec, so this
+// single in-place patch flows through to both surfaces.
+//
+// Returns the override keys that did not match any endpoint. A typo in
+// the override file (`tags_creat` instead of `tags_create`, or a stale
+// key from before a spec rename) would otherwise silently no-op; the
+// caller should surface unmatched keys so the user can debug them.
+//
+// Tool name computation matches WriteToolsManifest's iteration:
+// snake(resource) + "_" + snake(endpoint) for top-level endpoints,
+// snake(resource) + "_" + snake(sub) + "_" + snake(endpoint) for
+// sub-resource endpoints.
+func (o Overrides) Apply(parsed *spec.APISpec) []string {
+ if len(o.Descriptions) == 0 || parsed == nil {
+ return nil
+ }
+ matched := make(map[string]bool, len(o.Descriptions))
+ for rName, resource := range parsed.Resources {
+ for eName, endpoint := range resource.Endpoints {
+ name := ToolName(rName, "", eName)
+ if override, ok := o.Descriptions[name]; ok {
+ endpoint.Description = override
+ resource.Endpoints[eName] = endpoint
+ matched[name] = true
+ }
+ }
+ for subName, sub := range resource.SubResources {
+ for eName, endpoint := range sub.Endpoints {
+ name := ToolName(rName, subName, eName)
+ if override, ok := o.Descriptions[name]; ok {
+ endpoint.Description = override
+ sub.Endpoints[eName] = endpoint
+ matched[name] = true
+ }
+ }
+ resource.SubResources[subName] = sub
+ }
+ parsed.Resources[rName] = resource
+ }
+ var unmatched []string
+ for name := range o.Descriptions {
+ if !matched[name] {
+ unmatched = append(unmatched, name)
+ }
+ }
+ return unmatched
+}
+
+// ToolName composes the snake_case MCP tool name from a resource +
+// optional sub-resource + endpoint name. Mirrors the iteration in
+// pipeline.WriteToolsManifest and the mcp_tools.go template's NewTool
+// call. Exported so the override loader and the manifest writer share
+// one definition; if the naming scheme ever changes, both follow.
+func ToolName(resource, sub, endpoint string) string {
+ if sub == "" {
+ return naming.Snake(resource) + "_" + naming.Snake(endpoint)
+ }
+ return naming.Snake(resource) + "_" + naming.Snake(sub) + "_" + naming.Snake(endpoint)
+}
diff --git a/internal/mcpoverrides/overrides_test.go b/internal/mcpoverrides/overrides_test.go
new file mode 100644
index 00000000..b4dd5f0d
--- /dev/null
+++ b/internal/mcpoverrides/overrides_test.go
@@ -0,0 +1,172 @@
+package mcpoverrides
+
+import (
+ "os"
+ "path/filepath"
+ "sort"
+ "testing"
+
+ "github.com/mvanhorn/cli-printing-press/v2/internal/spec"
+)
+
+func TestLoad_FileAbsentReturnsEmpty(t *testing.T) {
+ dir := t.TempDir()
+ o, err := Load(dir)
+ if err != nil {
+ t.Fatalf("expected nil error when file absent, got %v", err)
+ }
+ if len(o.Descriptions) != 0 {
+ t.Fatalf("expected empty Descriptions, got %v", o.Descriptions)
+ }
+}
+
+func TestLoad_MalformedJSONReturnsError(t *testing.T) {
+ dir := t.TempDir()
+ if err := os.WriteFile(filepath.Join(dir, Filename), []byte("{not json"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := Load(dir); err == nil {
+ t.Fatal("expected error for malformed JSON")
+ }
+}
+
+func TestLoad_RoundtripsDescriptions(t *testing.T) {
+ dir := t.TempDir()
+ want := `{"descriptions":{"tags_create":"Create a new tag","tags_update":"Update existing tag"}}`
+ if err := os.WriteFile(filepath.Join(dir, Filename), []byte(want), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ o, err := Load(dir)
+ if err != nil {
+ t.Fatalf("unexpected error: %v", err)
+ }
+ if got := o.Descriptions["tags_create"]; got != "Create a new tag" {
+ t.Errorf("tags_create = %q, want %q", got, "Create a new tag")
+ }
+ if got := o.Descriptions["tags_update"]; got != "Update existing tag" {
+ t.Errorf("tags_update = %q, want %q", got, "Update existing tag")
+ }
+}
+
+func TestApply_TopLevelEndpoint(t *testing.T) {
+ parsed := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "tags": {
+ Endpoints: map[string]spec.Endpoint{
+ "create": {Description: "Create a tag"},
+ },
+ },
+ },
+ }
+ o := Overrides{Descriptions: map[string]string{
+ "tags_create": "Create a new tag with required name and optional color.",
+ }}
+ unmatched := o.Apply(parsed)
+ if len(unmatched) != 0 {
+ t.Errorf("expected zero unmatched, got %v", unmatched)
+ }
+ got := parsed.Resources["tags"].Endpoints["create"].Description
+ want := "Create a new tag with required name and optional color."
+ if got != want {
+ t.Errorf("description = %q, want %q", got, want)
+ }
+}
+
+func TestApply_SubResourceEndpoint(t *testing.T) {
+ parsed := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "bookings": {
+ SubResources: map[string]spec.Resource{
+ "attendees": {
+ Endpoints: map[string]spec.Endpoint{
+ "add": {Description: "Add an attendee"},
+ },
+ },
+ },
+ },
+ },
+ }
+ o := Overrides{Descriptions: map[string]string{
+ "bookings_attendees_add": "Add an attendee to a booking by email or userId.",
+ }}
+ unmatched := o.Apply(parsed)
+ if len(unmatched) != 0 {
+ t.Errorf("expected zero unmatched, got %v", unmatched)
+ }
+ got := parsed.Resources["bookings"].SubResources["attendees"].Endpoints["add"].Description
+ want := "Add an attendee to a booking by email or userId."
+ if got != want {
+ t.Errorf("description = %q, want %q", got, want)
+ }
+}
+
+func TestApply_UnmatchedKeysReturned(t *testing.T) {
+ parsed := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "tags": {
+ Endpoints: map[string]spec.Endpoint{
+ "create": {Description: "Create a tag"},
+ },
+ },
+ },
+ }
+ o := Overrides{Descriptions: map[string]string{
+ "tags_create": "OK",
+ "tags_destroy": "TYPO — does not exist",
+ "links_lint": "STALE — endpoint removed",
+ }}
+ unmatched := o.Apply(parsed)
+ sort.Strings(unmatched)
+ want := []string{"links_lint", "tags_destroy"}
+ if len(unmatched) != len(want) {
+ t.Fatalf("unmatched len = %d (%v), want %v", len(unmatched), unmatched, want)
+ }
+ for i := range want {
+ if unmatched[i] != want[i] {
+ t.Errorf("unmatched[%d] = %q, want %q", i, unmatched[i], want[i])
+ }
+ }
+}
+
+func TestApply_EmptyOverridesIsNoOp(t *testing.T) {
+ parsed := &spec.APISpec{
+ Resources: map[string]spec.Resource{
+ "tags": {
+ Endpoints: map[string]spec.Endpoint{
+ "create": {Description: "Create a tag"},
+ },
+ },
+ },
+ }
+ o := Overrides{}
+ unmatched := o.Apply(parsed)
+ if unmatched != nil {
+ t.Errorf("expected nil unmatched for empty overrides, got %v", unmatched)
+ }
+ if got := parsed.Resources["tags"].Endpoints["create"].Description; got != "Create a tag" {
+ t.Errorf("description mutated unexpectedly to %q", got)
+ }
+}
+
+func TestApply_NilParsedIsNoOp(t *testing.T) {
+ o := Overrides{Descriptions: map[string]string{"x": "y"}}
+ if got := o.Apply(nil); got != nil {
+ t.Errorf("expected nil unmatched for nil parsed, got %v", got)
+ }
+}
+
+func TestToolName(t *testing.T) {
+ tests := []struct {
+ resource, sub, endpoint, want string
+ }{
+ {"tags", "", "create", "tags_create"},
+ {"bookings", "attendees", "add", "bookings_attendees_add"},
+ {"event-types", "", "list", "event-types_list"},
+ }
+ for _, tt := range tests {
+ got := ToolName(tt.resource, tt.sub, tt.endpoint)
+ if got != tt.want {
+ t.Errorf("ToolName(%q, %q, %q) = %q, want %q", tt.resource, tt.sub, tt.endpoint, got, tt.want)
+ }
+ }
+}
diff --git a/internal/naming/naming.go b/internal/naming/naming.go
index eb548548..c5d270c9 100644
--- a/internal/naming/naming.go
+++ b/internal/naming/naming.go
@@ -154,17 +154,13 @@ func EnvVarPlaceholder(envVar string) string {
}
// OneLine normalizes generated descriptions for compact template and manifest
-// output.
+// output, then truncates anything longer than 120 chars. Use for callers that
+// need a compact single-line form (param descriptions in `--help`, terminal
+// summaries). For MCP tool descriptions where richer content is intentional —
+// including hand-authored mcp-descriptions.json overrides — use OneLineNormalize
+// instead, which does the same normalization without the length cap.
func OneLine(s string) string {
- s = strings.ReplaceAll(s, "\r\n", " ")
- s = strings.ReplaceAll(s, "\n", " ")
- s = strings.ReplaceAll(s, "\r", " ")
- s = strings.ReplaceAll(s, `"`, `'`)
- s = strings.ReplaceAll(s, "\\", "")
- for strings.Contains(s, " ") {
- s = strings.ReplaceAll(s, " ", " ")
- }
- s = strings.TrimSpace(s)
+ s = OneLineNormalize(s)
if len(s) > 120 {
cut := s[:117]
if idx := strings.LastIndex(cut, " "); idx > 60 {
@@ -176,6 +172,22 @@ func OneLine(s string) string {
return s
}
+// OneLineNormalize collapses whitespace, newlines, and quotes into a
+// single-line safe form without imposing a length cap. Use for content
+// that's already curated for length (MCP tool descriptions, agent-authored
+// overrides) where truncating would defeat the purpose.
+func OneLineNormalize(s string) string {
+ s = strings.ReplaceAll(s, "\r\n", " ")
+ s = strings.ReplaceAll(s, "\n", " ")
+ s = strings.ReplaceAll(s, "\r", " ")
+ s = strings.ReplaceAll(s, `"`, `'`)
+ s = strings.ReplaceAll(s, "\\", "")
+ for strings.Contains(s, " ") {
+ s = strings.ReplaceAll(s, " ", " ")
+ }
+ return strings.TrimSpace(s)
+}
+
// MCPDescription builds an MCP tool description with optional minority-side
// auth annotation. It annotates only when an API has a mix of public and
// auth-required tools, and only the minority side gets annotated.
@@ -200,7 +212,11 @@ func MCPDescription(desc string, noAuth bool, authType string, publicCount, tota
}
}
- return OneLine(desc)
+ // MCP descriptions intentionally allow rich content (1-3 sentences naming
+ // action, params, return shape, when to prefer). Length is curated by the
+ // agent or by the spec; we do single-line normalization but not the
+ // 120-char cap that OneLine imposes for compact display.
+ return OneLineNormalize(desc)
}
func DogfoodBinary(name string) string {
diff --git a/internal/pipeline/mcpsync/sync.go b/internal/pipeline/mcpsync/sync.go
index fe8baf8d..c72e232e 100644
--- a/internal/pipeline/mcpsync/sync.go
+++ b/internal/pipeline/mcpsync/sync.go
@@ -12,6 +12,7 @@ import (
"github.com/mvanhorn/cli-printing-press/v2/internal/generator"
"github.com/mvanhorn/cli-printing-press/v2/internal/graphql"
+ "github.com/mvanhorn/cli-printing-press/v2/internal/mcpoverrides"
"github.com/mvanhorn/cli-printing-press/v2/internal/openapi"
"github.com/mvanhorn/cli-printing-press/v2/internal/pipeline"
"github.com/mvanhorn/cli-printing-press/v2/internal/spec"
@@ -33,6 +34,12 @@ var endpointAnnotationLine = regexp.MustCompile(`(?m)^\s*Annotations: map\[strin
type Result struct {
Changed bool
Detail string
+ // UnmatchedOverrideKeys lists keys from mcp-descriptions.json that
+ // did not correspond to any endpoint in the spec — typos, stale
+ // keys after a rename, or overrides for endpoints removed from the
+ // spec. The library returns these so the CLI layer can surface them;
+ // Sync itself does not print, keeping output formatting at the edge.
+ UnmatchedOverrideKeys []string
}
type Options struct {
@@ -86,11 +93,26 @@ func Sync(cliDir string, opts Options) (Result, error) {
parsed.DisplayName = existing
}
}
+ // Apply hand-authored MCP description overrides before generating
+ // any surface that consumes endpoint.Description. Both the manifest
+ // writer and the mcp_tools.go template read from this parsed spec,
+ // so a single in-place patch flows through to both. The override
+ // file is the sanctioned path for replacing thin spec-derived
+ // descriptions; direct edits to internal/mcp/tools.go and
+ // tools-manifest.json are wiped on regen because both files carry
+ // the generator's DO-NOT-EDIT header.
+ overrides, err := mcpoverrides.Load(cliDir)
+ if err != nil {
+ return Result{}, fmt.Errorf("loading mcp description overrides: %w", err)
+ }
+ unmatched := overrides.Apply(parsed)
modulePath, err := readModulePath(cliDir)
if err != nil {
return Result{}, err
}
features := loadNovelFeatures(cliDir)
+ // Migration-only steps run when the surface is on the legacy
+ // template. Already-migrated CLIs skip these.
if !alreadyMigrated {
if err := ensureRootCmdExport(cliDir); err != nil {
return Result{}, err
@@ -109,12 +131,15 @@ func Sync(cliDir string, opts Options) (Result, error) {
if err := removeStaleMCPHandlersFile(cliDir, opts.Force); err != nil {
return Result{}, err
}
- gen := generator.New(parsed, cliDir)
- gen.NovelFeatures = features
- gen.ModulePath = modulePath
- if err := gen.GenerateMCPSurface(); err != nil {
- return Result{}, fmt.Errorf("rendering MCP surface: %w", err)
- }
+ }
+ // Surface regen runs every sync — overrides applied above must reach
+ // tools.go, and WriteToolsManifest below rewrites the manifest
+ // unconditionally, so keeping tools.go in lockstep avoids drift.
+ gen := generator.New(parsed, cliDir)
+ gen.NovelFeatures = features
+ gen.ModulePath = modulePath
+ if err := gen.GenerateMCPSurface(); err != nil {
+ return Result{}, fmt.Errorf("rendering MCP surface: %w", err)
}
if err := pipeline.WriteToolsManifest(cliDir, parsed); err != nil {
return Result{}, fmt.Errorf("regenerating tools-manifest.json: %w", err)
@@ -137,10 +162,11 @@ func Sync(cliDir string, opts Options) (Result, error) {
if err := pipeline.WriteMCPBManifest(cliDir); err != nil {
return Result{}, fmt.Errorf("regenerating manifest.json: %w", err)
}
+ detail := "migrated MCP surface to runtime Cobra-tree mirror"
if alreadyMigrated {
- return Result{Changed: true, Detail: "refreshed manifest.json + tools-manifest.json from current spec/.printing-press.json"}, nil
+ detail = "refreshed MCP surface, manifest.json, and tools-manifest.json from current spec / .printing-press.json / mcp-descriptions.json"
}
- return Result{Changed: true, Detail: "migrated MCP surface to runtime Cobra-tree mirror"}, nil
+ return Result{Changed: true, Detail: detail, UnmatchedOverrideKeys: unmatched}, nil
}
func loadArchivedSpec(cliDir string) (*spec.APISpec, error) {
diff --git a/internal/pipeline/scorecard.go b/internal/pipeline/scorecard.go
index 10b86f3a..f159a675 100644
--- a/internal/pipeline/scorecard.go
+++ b/internal/pipeline/scorecard.go
@@ -43,25 +43,26 @@ type Scorecard struct {
// SteinerScore breaks down the Steinberger bar into 11 dimensions, each 0-10.
type SteinerScore struct {
- OutputModes int `json:"output_modes"` // 0-10
- Auth int `json:"auth"` // 0-10
- ErrorHandling int `json:"error_handling"` // 0-10
- TerminalUX int `json:"terminal_ux"` // 0-10
- README int `json:"readme"` // 0-10
- Doctor int `json:"doctor"` // 0-10
- AgentNative int `json:"agent_native"` // 0-10
- MCPQuality int `json:"mcp_quality"` // 0-10
- MCPTokenEff int `json:"mcp_token_efficiency"` // 0-10; unscored when no MCP surface
- MCPRemoteTransport int `json:"mcp_remote_transport"` // 0-10; unscored when no MCP surface. Rewards remote-capable servers per Anthropic's 2026-04 MCP guidance.
- MCPToolDesign int `json:"mcp_tool_design"` // 0-10; unscored when no MCP surface or endpoint count below toolDesignMinEndpoints. Rewards intent-grouped tools vs. endpoint mirrors.
- MCPSurfaceStrategy int `json:"mcp_surface_strategy"` // 0-10; unscored unless the endpoint surface exceeds surfaceStrategyLargeThreshold or code-orchestration is explicitly used. Penalizes endpoint-mirror at scale.
- LocalCache int `json:"local_cache"` // 0-10
- CacheFreshness int `json:"cache_freshness"` // 0-10; unscored when the CLI has no local store
- Breadth int `json:"breadth"` // 0-10: how many commands (penalizes empty CLIs)
- Vision int `json:"vision"` // 0-10
- Workflows int `json:"workflows"` // 0-10
- Insight int `json:"insight"` // 0-10
- AgentWorkflow int `json:"agent_workflow_readiness"` // 0-10: HeyGen-derived - async jobs, profiles, deliver, feedback
+ OutputModes int `json:"output_modes"` // 0-10
+ Auth int `json:"auth"` // 0-10
+ ErrorHandling int `json:"error_handling"` // 0-10
+ TerminalUX int `json:"terminal_ux"` // 0-10
+ README int `json:"readme"` // 0-10
+ Doctor int `json:"doctor"` // 0-10
+ AgentNative int `json:"agent_native"` // 0-10
+ MCPQuality int `json:"mcp_quality"` // 0-10
+ MCPDescriptionQuality int `json:"mcp_description_quality"` // 0-10; unscored when no tools-manifest.json. Penalizes thin per-tool descriptions (the same threshold as `printing-press tools-audit` thin-mcp-description).
+ MCPTokenEff int `json:"mcp_token_efficiency"` // 0-10; unscored when no MCP surface
+ MCPRemoteTransport int `json:"mcp_remote_transport"` // 0-10; unscored when no MCP surface. Rewards remote-capable servers per Anthropic's 2026-04 MCP guidance.
+ MCPToolDesign int `json:"mcp_tool_design"` // 0-10; unscored when no MCP surface or endpoint count below toolDesignMinEndpoints. Rewards intent-grouped tools vs. endpoint mirrors.
+ MCPSurfaceStrategy int `json:"mcp_surface_strategy"` // 0-10; unscored unless the endpoint surface exceeds surfaceStrategyLargeThreshold or code-orchestration is explicitly used. Penalizes endpoint-mirror at scale.
+ LocalCache int `json:"local_cache"` // 0-10
+ CacheFreshness int `json:"cache_freshness"` // 0-10; unscored when the CLI has no local store
+ Breadth int `json:"breadth"` // 0-10: how many commands (penalizes empty CLIs)
+ Vision int `json:"vision"` // 0-10
+ Workflows int `json:"workflows"` // 0-10
+ Insight int `json:"insight"` // 0-10
+ AgentWorkflow int `json:"agent_workflow_readiness"` // 0-10: HeyGen-derived - async jobs, profiles, deliver, feedback
// Tier 2: Domain Correctness (semantic checks)
PathValidity int `json:"path_validity"` // 0-10
AuthProtocol int `json:"auth_protocol"` // 0-10
@@ -118,6 +119,8 @@ func scoreInfrastructureDimensions(sc *Scorecard, outputDir string) {
sc.Steinberger.Doctor = scoreDoctor(outputDir)
sc.Steinberger.AgentNative = scoreAgentNative(outputDir)
sc.Steinberger.MCPQuality = scoreMCPQuality(outputDir)
+ mcpDescScore, mcpDescScored := scoreMCPDescriptionQuality(outputDir)
+ recordOptionalScore(sc, &sc.Steinberger.MCPDescriptionQuality, "mcp_description_quality", mcpDescScore, mcpDescScored)
mcpTokenScore, mcpTokenScored := scoreMCPTokenEfficiency(outputDir)
recordOptionalScore(sc, &sc.Steinberger.MCPTokenEff, "mcp_token_efficiency", mcpTokenScore, mcpTokenScored)
remoteScore, remoteScored := scoreMCPRemoteTransport(outputDir)
@@ -673,6 +676,71 @@ func scoreMCPQuality(dir string) int {
return score
}
+// MCPDescMinLen and MCPDescMinWords are the agent-grade-description
+// thresholds used by both `printing-press tools-audit` and the
+// scorecard's mcp_description_quality dimension. Defined here so a
+// single edit keeps both surfaces in lockstep — internal/cli imports
+// these for its thin-mcp-description check.
+const (
+ MCPDescMinLen = 60
+ MCPDescMinWords = 8
+)
+
+// IsThinMCPDescription reports whether a description trips the
+// agent-grade floor: empty or both shorter than MCPDescMinLen AND
+// fewer words than MCPDescMinWords. Shared between the audit (cli
+// package) and the scorer so both apply identical semantics.
+func IsThinMCPDescription(desc string) bool {
+ d := strings.TrimSpace(desc)
+ if d == "" {
+ return true
+ }
+ return len(d) < MCPDescMinLen && len(strings.Fields(d)) < MCPDescMinWords
+}
+
+// scoreMCPDescriptionQuality measures how many of a CLI's typed MCP
+// tools carry agent-grade descriptions vs. terse spec-derived
+// summaries. Reads tools-manifest.json (the source of truth for typed
+// endpoint tools' descriptions at runtime) and counts entries whose
+// description trips IsThinMCPDescription — the same predicate
+// `printing-press tools-audit` uses for thin-mcp-description findings.
+//
+// Unscored when no manifest exists (legacy CLIs predating the
+// manifest schema) or when the manifest has no tools.
+//
+// Score curve favors low thin-percentage steeply; the goal is to
+// reward CLIs that have done the work to provide rich descriptions
+// rather than to give partial credit to ones that haven't. CLIs whose
+// descriptions are 50%+ thin score 0 — there's no signal to credit
+// when most of the surface is unusable to an agent.
+func scoreMCPDescriptionQuality(dir string) (score int, scored bool) {
+ m, err := ReadToolsManifest(dir)
+ if err != nil || len(m.Tools) == 0 {
+ return 0, false
+ }
+ thin := 0
+ for _, t := range m.Tools {
+ if IsThinMCPDescription(t.Description) {
+ thin++
+ }
+ }
+ pct := float64(thin) / float64(len(m.Tools))
+ switch {
+ case pct == 0:
+ return 10, true
+ case pct < 0.05:
+ return 9, true
+ case pct < 0.15:
+ return 7, true
+ case pct < 0.30:
+ return 5, true
+ case pct < 0.50:
+ return 3, true
+ default:
+ return 0, true
+ }
+}
+
func scoreLocalCache(dir string) int {
clientContent := readFileContent(filepath.Join(dir, "internal", "client", "client.go"))
score := 0
@@ -860,6 +928,7 @@ func recomputeScorecardTotals(sc *Scorecard) {
sc.Steinberger.Doctor,
sc.Steinberger.AgentNative,
sc.Steinberger.MCPQuality,
+ sc.Steinberger.MCPDescriptionQuality,
sc.Steinberger.MCPTokenEff,
sc.Steinberger.MCPRemoteTransport,
sc.Steinberger.MCPToolDesign,
@@ -873,7 +942,7 @@ func recomputeScorecardTotals(sc *Scorecard) {
sc.Steinberger.AgentWorkflow,
)
- tier1Max := scorecardTierMax(sc, 190, "mcp_token_efficiency", "cache_freshness", "mcp_remote_transport", "mcp_tool_design", "mcp_surface_strategy")
+ tier1Max := scorecardTierMax(sc, 200, "mcp_description_quality", "mcp_token_efficiency", "cache_freshness", "mcp_remote_transport", "mcp_tool_design", "mcp_surface_strategy")
tier1Normalized := 0
if tier1Max > 0 {
tier1Normalized = (tier1Raw * 50) / tier1Max
@@ -2305,6 +2374,7 @@ func buildGapReport(s SteinerScore, unscored []string) []string {
{"doctor", s.Doctor},
{"agent_native", s.AgentNative},
{"mcp_quality", s.MCPQuality},
+ {"mcp_description_quality", s.MCPDescriptionQuality},
{"mcp_token_efficiency", s.MCPTokenEff},
{"local_cache", s.LocalCache},
{"breadth", s.Breadth},
@@ -2435,6 +2505,7 @@ func writeScorecardMD(sc *Scorecard, pipelineDir string) error {
{"Doctor", "doctor", s.Doctor},
{"Agent Native", "agent_native", s.AgentNative},
{"MCP Quality", "mcp_quality", s.MCPQuality},
+ {"MCP Description Quality", "mcp_description_quality", s.MCPDescriptionQuality},
{"MCP Token Efficiency", "mcp_token_efficiency", s.MCPTokenEff},
{"Local Cache", "local_cache", s.LocalCache},
{"Breadth", "breadth", s.Breadth},
diff --git a/internal/pipeline/scorecard_tier2_test.go b/internal/pipeline/scorecard_tier2_test.go
index 75ae69db..194a5186 100644
--- a/internal/pipeline/scorecard_tier2_test.go
+++ b/internal/pipeline/scorecard_tier2_test.go
@@ -10,6 +10,105 @@ import (
"github.com/stretchr/testify/assert"
)
+func TestIsThinMCPDescription(t *testing.T) {
+ tests := []struct {
+ name string
+ desc string
+ want bool
+ }{
+ {"empty is thin", "", true},
+ {"whitespace is thin", " ", true},
+ {"short and few words", "Get a tag", true}, // 9 chars / 3 words → both trip
+ {"both signals trip on short low-word string", "verylongidentifier verylongidentifier", true}, // 37 chars / 2 words → both below thresholds
+ {"long enough chars passes even if few words", "verylongidentifier verylongidentifier verylongidentifier verylongidentifier", false}, // 73 chars / 4 words → length passes
+ {"enough words passes even if short", "Create a new tag in the user workspace", false}, // 38 chars / 8 words → words passes
+ {"genuinely rich passes", "Create a new tag in the workspace. Required: name. Returns id and slug.", false},
+ }
+ for _, tt := range tests {
+ t.Run(tt.name, func(t *testing.T) {
+ if got := IsThinMCPDescription(tt.desc); got != tt.want {
+ t.Errorf("IsThinMCPDescription(%q) = %v, want %v", tt.desc, got, tt.want)
+ }
+ })
+ }
+}
+
+func TestScoreMCPDescriptionQuality(t *testing.T) {
+ mk := func(t *testing.T, descs []string) string {
+ t.Helper()
+ dir := t.TempDir()
+ tools := make([]map[string]any, 0, len(descs))
+ for i, d := range descs {
+ tools = append(tools, map[string]any{
+ "name": "tool_" + string(rune('a'+i)),
+ "description": d,
+ })
+ }
+ manifest := map[string]any{"tools": tools}
+ data, err := json.Marshal(manifest)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(dir, "tools-manifest.json"), data, 0o644); err != nil {
+ t.Fatal(err)
+ }
+ return dir
+ }
+
+ t.Run("missing manifest is unscored", func(t *testing.T) {
+ dir := t.TempDir()
+ score, scored := scoreMCPDescriptionQuality(dir)
+ if scored {
+ t.Errorf("expected unscored, got scored=%d", score)
+ }
+ })
+
+ t.Run("empty tools list is unscored", func(t *testing.T) {
+ dir := t.TempDir()
+ if err := os.WriteFile(filepath.Join(dir, "tools-manifest.json"), []byte(`{"tools":[]}`), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ score, scored := scoreMCPDescriptionQuality(dir)
+ if scored {
+ t.Errorf("expected unscored, got scored=%d", score)
+ }
+ })
+
+ rich := "Create a new tag in the workspace. Required: name. Returns id and slug."
+ thin := "Create a tag"
+
+ cases := []struct {
+ name string
+ descs []string
+ want int
+ }{
+ {"all rich -> 10", []string{rich, rich, rich, rich, rich}, 10},
+ {"4% thin -> 9", appendN([]string{thin}, rich, 24), 9},
+ {"10% thin -> 7", appendN([]string{thin}, rich, 9), 7},
+ {"25% thin -> 5", []string{rich, rich, rich, thin}, 5},
+ {"40% thin -> 3", []string{rich, rich, rich, thin, thin}, 3},
+ {"100% thin -> 0", []string{thin, thin, thin, thin, thin}, 0},
+ }
+ for _, c := range cases {
+ t.Run(c.name, func(t *testing.T) {
+ dir := mk(t, c.descs)
+ score, scored := scoreMCPDescriptionQuality(dir)
+ if !scored || score != c.want {
+ t.Errorf("score=%d scored=%v, want %d/true", score, scored, c.want)
+ }
+ })
+ }
+}
+
+func appendN(prefix []string, val string, n int) []string {
+ out := make([]string, 0, len(prefix)+n)
+ out = append(out, prefix...)
+ for range n {
+ out = append(out, val)
+ }
+ return out
+}
+
func TestScoreDeadCode(t *testing.T) {
t.Run("penalizes dead flags and helper functions", func(t *testing.T) {
dir := t.TempDir()
@@ -514,7 +613,7 @@ func runLinks() string {
pipelineDir := t.TempDir()
sc, err := RunScorecard(dir, pipelineDir, "", nil)
assert.NoError(t, err)
- assert.ElementsMatch(t, []string{"mcp_token_efficiency", "mcp_remote_transport", "mcp_tool_design", "mcp_surface_strategy", "cache_freshness", "path_validity", "auth_protocol", "live_api_verification"}, sc.UnscoredDimensions)
+ assert.ElementsMatch(t, []string{"mcp_description_quality", "mcp_token_efficiency", "mcp_remote_transport", "mcp_tool_design", "mcp_surface_strategy", "cache_freshness", "path_validity", "auth_protocol", "live_api_verification"}, sc.UnscoredDimensions)
assert.NotContains(t, sc.GapReport, "path_validity scored 0/10 - needs improvement")
assert.NotContains(t, sc.GapReport, "auth_protocol scored 0/10 - needs improvement")
})
@@ -905,7 +1004,7 @@ func runLinks() string {
body := string(data)
assert.True(t, strings.Contains(body, `"path_validity":0`))
assert.True(t, strings.Contains(body, `"auth_protocol":0`))
- assert.True(t, strings.Contains(body, `"unscored_dimensions":["mcp_token_efficiency","mcp_remote_transport","mcp_tool_design","mcp_surface_strategy","cache_freshness","path_validity","auth_protocol","live_api_verification"]`))
+ assert.True(t, strings.Contains(body, `"unscored_dimensions":["mcp_description_quality","mcp_token_efficiency","mcp_remote_transport","mcp_tool_design","mcp_surface_strategy","cache_freshness","path_validity","auth_protocol","live_api_verification"]`))
})
}
diff --git a/internal/pipeline/toolsmanifest.go b/internal/pipeline/toolsmanifest.go
index 767b67cd..1040b2f9 100644
--- a/internal/pipeline/toolsmanifest.go
+++ b/internal/pipeline/toolsmanifest.go
@@ -10,6 +10,7 @@ import (
"sort"
"strings"
+ "github.com/mvanhorn/cli-printing-press/v2/internal/mcpoverrides"
"github.com/mvanhorn/cli-printing-press/v2/internal/naming"
"github.com/mvanhorn/cli-printing-press/v2/internal/spec"
)
@@ -76,6 +77,24 @@ type ManifestHeader struct {
Value string `json:"value"`
}
+// ReadToolsManifest decodes <dir>/tools-manifest.json into a
+// ToolsManifest. Returns the wrapped error from os.ReadFile when the
+// file is missing — callers that treat absence as "no manifest"
+// should check errors.Is(err, fs.ErrNotExist) at the call site.
+// Shared between WriteToolsManifest's downstream consumers (audit,
+// scorer) so the on-disk schema has a single decode site.
+func ReadToolsManifest(dir string) (*ToolsManifest, error) {
+ data, err := os.ReadFile(filepath.Join(dir, ToolsManifestFilename))
+ if err != nil {
+ return nil, err
+ }
+ var m ToolsManifest
+ if err := json.Unmarshal(data, &m); err != nil {
+ return nil, fmt.Errorf("parsing %s: %w", ToolsManifestFilename, err)
+ }
+ return &m, nil
+}
+
// WriteToolsManifest generates a tools-manifest.json from a parsed API spec.
// It iterates Resources/SubResources/Endpoints in sorted key order (matching
// the MCP template's RegisterTools pattern) and writes deterministic JSON.
@@ -131,7 +150,7 @@ func WriteToolsManifest(dir string, parsed *spec.APISpec) error {
if cookieOrComposed && !endpoint.NoAuth {
continue
}
- toolName := naming.Snake(rName) + "_" + naming.Snake(eName)
+ toolName := mcpoverrides.ToolName(rName, "", eName)
desc := naming.MCPDescription(endpoint.Description, endpoint.NoAuth, parsed.Auth.Type, public, total)
tool := buildManifestTool(toolName, desc, endpoint)
manifest.Tools = append(manifest.Tools, tool)
@@ -147,7 +166,7 @@ func WriteToolsManifest(dir string, parsed *spec.APISpec) error {
if cookieOrComposed && !endpoint.NoAuth {
continue
}
- toolName := naming.Snake(rName) + "_" + naming.Snake(subName) + "_" + naming.Snake(eName)
+ toolName := mcpoverrides.ToolName(rName, subName, eName)
desc := naming.MCPDescription(endpoint.Description, endpoint.NoAuth, parsed.Auth.Type, public, total)
tool := buildManifestTool(toolName, desc, endpoint)
manifest.Tools = append(manifest.Tools, tool)
diff --git a/skills/printing-press-polish/SKILL.md b/skills/printing-press-polish/SKILL.md
index e41b88f7..6bc1cc3d 100644
--- a/skills/printing-press-polish/SKILL.md
+++ b/skills/printing-press-polish/SKILL.md
@@ -179,7 +179,14 @@ Parse findings into categories:
| Go vet issues | go vet | Any output |
| Output entity warnings | scorecard JSON | `live_check.features[].warnings` — raw HTML entities in human output |
| Output plausibility | Phase 4.85 | Findings from the agentic output review |
-| MCP tool quality | tools-audit | Empty Short, thin Short, missing read-only annotations |
+| MCP tool quality | tools-audit | Empty Short, thin Short, missing read-only annotations, thin MCP descriptions |
+
+**Environmental failures vs. CLI defects.** Some Phase 1 outputs surface failures that aren't real CLI bugs and should not block ship:
+
+- `scorecard --live-check` reporting `SQLITE_BUSY`, network timeouts, `401` from a mock or expired token, or HTTP errors that depend on the test workspace's permissions/state — these are test-environment issues, not CLI defects.
+- `verify` mock-harness flakes on commands with binary output (e.g., `qr` returning a PNG that the substring matcher can't validate) or commands with optional positional args where dry-run output legitimately doesn't contain the verify probe string.
+
+Classify these as environmental in `skipped_findings` with the specific reason; do not spend Phase 2 cycles trying to "fix" them. The polish skill's ship logic already excludes live-check failures from gating, but the agent should still annotate them so reviewers can see they were considered and dismissed deliberately.
### Phase 4.85 — Agentic output review (Wave B)
@@ -206,6 +213,18 @@ Fix in priority order. After each priority level, update the lock heartbeat:
printing-press lock update --cli "$CLI_NAME" --phase polish 2>/dev/null
```
+### Priority 0: MCP surface migration (legacy CLIs)
+
+If Phase 1's `dogfood` reported `MCP Surface: FAIL` with a parity mismatch, the CLI was generated before the runtime cobratree walker existed and is still on the static `internal/mcp/tools.go` surface. The fix is mechanical:
+
+```bash
+printing-press mcp-sync "$CLI_DIR"
+```
+
+That migrates the MCP surface to the runtime walker, regenerates `tools-manifest.json` and `internal/mcp/tools.go`, and applies any `mcp-descriptions.json` overrides. Re-run `dogfood` after; the parity gate flips to PASS. This is a known migration path for every CLI generated before the cobratree landed; running it on a CLI already on the runtime walker is a no-op refresh.
+
+Skip this priority on CLIs where dogfood's MCP gate is already passing.
+
### Priority 1: Verify failures
For each command that fails verify dry-run or exec:
@@ -387,6 +406,17 @@ Compute the ship recommendation:
- **`ship-with-gaps`**: verify >= 65%, scorecard >= 65, non-critical gaps remain, **AND** the SKILL/workflow gates above hold. Reserved for the rare case where a refactor or external-dependency blocker prevents a clean fix; the gap must be documented in the remaining issues.
- **`hold`**: verify < 65% or scorecard < 65 or critical failures, **OR** verify-skill has unresolved findings, **OR** workflow-verify reports `workflow-fail` and the workflow is the CLI's primary value.
+### Push higher without gaming
+
+The ship gates are a floor, not a ceiling. After they pass, look at scorecard dimensions still below max and ask whether each gap is real or structural:
+
+1. **Find the underlying deficit, not the score.** The scorecard is a proxy for quality, not the goal itself. A README scoring 8/10 might be missing a Cookbook section or have outdated commands — that's a real, fixable gap. A `mcp_surface_strategy` scoring 2/10 on a 200-endpoint API might be flagging that the surface is mostly endpoint mirrors — also potentially fixable.
+2. **If there's a real, agent-grade improvement available, make it.** Better description, missing flag doc, weak README section, an example that doesn't reflect actual usage. The CLI gets better and the score follows.
+3. **If the deficit is structural, document and accept.** Some dimensions assume capabilities the CLI's domain doesn't have (a read-only API scored against write-workflow dimensions, a CLI with no auth scored on auth dimensions, a small API penalized on `surface_strategy` thresholds calibrated for large APIs). Note the reason in `skipped_findings` and move on.
+4. **Never add scaffolding to satisfy the scorer.** Fake commands, fake tests, fake flags, or boilerplate prose written purely to nudge a number — those degrade the CLI to satisfy the proxy. The scorer is imperfect by design (the "scoring may be imperfect" caveat in AGENTS.md applies). Trust the underlying judgment, not the number.
+
+Rule of thumb: if your fix would still be valuable if the scorecard didn't exist, do it. If the only motivation is "to push the score," don't.
+
## Display delta and emit result block
Display the delta to the user, then emit the structured `---POLISH-RESULT---` block. The block lets calling skills (e.g., main printing-press SKILL.md Phase 5.5) parse the recommendation and scores reliably; the human-readable table above is for the user.
diff --git a/skills/printing-press-polish/references/tools-polish.md b/skills/printing-press-polish/references/tools-polish.md
index da124b4d..96b21d28 100644
--- a/skills/printing-press-polish/references/tools-polish.md
+++ b/skills/printing-press-polish/references/tools-polish.md
@@ -42,8 +42,9 @@ Files that are NOT DO-NOT-EDIT and ARE safe to edit:
- Hand-written novel commands (no generator header)
- The polish skill's own targets: README.md, SKILL.md, manifest.json
- `internal/cli/root.go` (the generator emits the scaffold but the skill mutates it during polish)
+- `mcp-descriptions.json` — the sanctioned override file for MCP tool descriptions (see `thin-mcp-description` below)
-`tools-manifest.json` is a special case — it's generated, but the dual-edit path documented under `thin-mcp-description` deliberately edits both manifest and `tools.go` together. The runtime registers from `tools.go`; the manifest is a parallel artifact. If `mcp-sync` regenerates the manifest, the description edits in tools.go survive. Verify this on your CLI before relying on it; if both are regenerated atomically, this is a generator-template ask, not a polish-time fix.
+`tools-manifest.json` and `internal/mcp/tools.go` are both DO-NOT-EDIT generated files. Don't edit them directly for description fixes — the override file is the path that survives regen.
### `empty-short`
@@ -72,19 +73,35 @@ Shell-out tool whose name matches a read-shaped pattern (`list`, `get`, `show`,
A typed endpoint tool whose `description` field in `tools-manifest.json` is empty.
-**Fix:** write an MCP-grade description (see criteria below) and update both:
-
-1. `tools-manifest.json` — the `description` field for the matching tool entry.
-2. `internal/mcp/tools.go` — the `mcplib.WithDescription("...")` call for the same tool. Keep the two in sync.
+**Fix:** write an MCP-grade description (see criteria below) to the override file (see "Override path" below). Do NOT edit `tools-manifest.json` or `internal/mcp/tools.go` directly — both are generated and edits get wiped on the next `mcp-sync`.
### `thin-mcp-description`
A typed endpoint tool whose description is both short (<60 chars) and low-word-count (<8 words). The dominant pattern: spec-derived summaries like `"Create a tag"`, `"List domains"`, `"Update a link"` — fine for OpenAPI doc rendering, inadequate for an agent picking from a tool catalog. The agent looking at `"Create a tag"` has to guess what fields to pass, what comes back, and when to prefer this over alternatives.
-**Fix:** rewrite per the MCP description criteria below. Update both `tools-manifest.json` and `internal/mcp/tools.go` so the runtime and the manifest stay in sync.
+**Fix:** rewrite per the MCP description criteria below and write the new text to the override file (see "Override path" below).
**Acceptance is rare here.** A 30-char description is almost never enough for a typed MCP tool. Only accept if the underlying operation is genuinely so simple that more words add no information (e.g., a synthetic ping/heartbeat tool).
+### Override path: `mcp-descriptions.json`
+
+Both `tools-manifest.json` and `internal/mcp/tools.go` are generated artifacts (each carries the generator's "DO NOT EDIT" header) and get rewritten on every `mcp-sync` run. The sanctioned path for replacing thin spec-derived descriptions is the `mcp-descriptions.json` sidecar at the cli-dir root. Format:
+
+```json
+{
+ "descriptions": {
+ "tags_create": "Create a new tag in the workspace. Required: name. Optional: color (hex) and parentTagId. Returns the tag's id and slug. Tags must exist before they can be assigned to links.",
+ "tags_update": "..."
+ }
+}
+```
+
+Keys are the MCP tool names exactly as they appear in `tools-manifest.json`'s `name` field (snake_case, e.g. `tags_create`, `bookings_attendees_booking-add`). Values are the new descriptions written per the MCP criteria below.
+
+After writing one or more overrides, **run `printing-press mcp-sync <cli-dir>`** to regenerate the manifest and runtime registration with the override applied. Then re-run the audit; the finding disappears because the manifest now reflects the override.
+
+The override file is hand-editable, persists across regenerations (it lives outside the generator's emit set), and survives `mcp-sync` runs. It's the only place an agent should write MCP descriptions that have to last.
+
## Pass 2: Evaluate every command (the load-bearing pass)
This is where the actual quality assessment happens. Walk every user-facing command in `internal/cli/` and every entry in `tools-manifest.json`, including ones the audit didn't flag. The mechanical check in Pass 1 misses two real classes:
@@ -225,27 +242,23 @@ Annotations: map[string]string{"mcp:read-only": "true"},
### Empty / thin MCP description
-Two files to update for the same tool — keep them in sync.
-
-`tools-manifest.json`:
+Write to `<cli-dir>/mcp-descriptions.json`, then run `mcp-sync` to apply.
```json
{
- "name": "tags_create",
- "description": "<MCP-grade description per criteria above>",
- ...
+ "descriptions": {
+ "tags_create": "<MCP-grade description per criteria above>",
+ "tags_update": "<another override>"
+ }
}
```
-`internal/mcp/tools.go`:
-
-```go
-mcplib.NewTool("tags_create",
- mcplib.WithDescription("<same MCP-grade description>"),
- ...
-)
+```bash
+printing-press mcp-sync <cli-dir>
```
+The sync regenerates `tools-manifest.json` and `internal/mcp/tools.go` with the overrides applied. Both generated files now carry the richer text; both are wiped and re-emitted from the override file on every regen, so edits persist. Re-run `tools-audit` to confirm the finding is gone.
+
## Ledger and resumability
`tools-audit` writes `<cli-dir>/.printing-press-tools-polish.json` after every run. It contains the timestamp, cli-dir, and one entry per finding.
@@ -287,7 +300,7 @@ After applying fixes, before declaring the polish complete:
- [ ] `go build ./...` clean (annotations don't break compilation)
- [ ] `printing-press tools-audit <cli-dir>` shows zero pending findings — every finding is either fixed (auto-removed) or explicitly accepted with a `note`
- [ ] `printing-press dogfood --dir <cli-dir>` reports `MCP Surface: PASS`
-- [ ] If `tools-manifest.json` was edited, `tools.go` was updated to match (the runtime registers from `tools.go`; the manifest documents what was registered)
+- [ ] If `mcp-descriptions.json` was edited, `printing-press mcp-sync <cli-dir>` was run to apply the overrides into `tools-manifest.json` and `internal/mcp/tools.go`. Re-run `tools-audit` after the sync to confirm thin-mcp-description findings disappeared.
- [ ] If commands were renamed or had their annotations restructured, smoke-test the binary by inspecting `--help` output for the affected commands
The ledger file persists until it ages out (24h). Once the polish PR merges and the CLI is rebuilt, the file is no longer load-bearing — the next `tools-audit` run can start fresh.
diff --git a/testdata/golden/expected/generate-golden-api/scorecard.json b/testdata/golden/expected/generate-golden-api/scorecard.json
index 5ef058d9..888d6c04 100644
--- a/testdata/golden/expected/generate-golden-api/scorecard.json
+++ b/testdata/golden/expected/generate-golden-api/scorecard.json
@@ -20,6 +20,7 @@
"insight": 8,
"live_api_verification": 0,
"local_cache": 10,
+ "mcp_description_quality": 0,
"mcp_quality": 10,
"mcp_remote_transport": 5,
"mcp_surface_strategy": 0,
@@ -37,6 +38,7 @@
"workflows": 10
},
"unscored_dimensions": [
+ "mcp_description_quality",
"mcp_tool_design",
"mcp_surface_strategy",
"path_validity",
← fee6802e feat(cli): tools-audit checks MCP descriptions in tools-mani
·
back to Cli Printing Press
·
feat(cli,skills): polish ledger per-item enforcement + AGENT 0320269d →