← back to Cli Printing Press

fix(cli): support $-prefixed pagination params for Socrata-style APIs (#1204)

3819b67f8118617725671467a7a0f6ee712c4304 · 2026-05-13 14:47:22 -0700 · CB

Adds an optional `url_name` field to spec.Param that, when set, overrides
the URL query-key while leaving the CLI flag, Go identifier, and JSON body
emission unchanged. Specs targeting Socrata-backed APIs (NYC OpenData and
~70 other major US municipal portals) can now declare:

    params:
      - name: limit
        url_name: "$limit"
        type: integer

and the generated handler emits params["$limit"] in the URL while keeping
--limit as the cobra flag.

Previously generated CLIs against Socrata returned HTTP 400
"Unrecognized arguments [limit]" because the wire-side key was the plain
Name. Specs without url_name are unaffected — verified by regenerating
bexar-cad-pp-cli and confirming params["f"], params["outFields"], etc.
remain plain.

Changes
- internal/spec/spec.go: add URLName field + Param.WireName() method
- internal/generator/flag_collision.go: add paramWireName helper, mirroring
  paramIdent. Updated paramIdent comment to clarify URL emission goes
  through paramWireName, not Name directly.
- internal/generator/generator.go: register paramWireName template helper
- internal/generator/templates/command_endpoint.go.tmpl: 6 emission sites
  switched from {{.Name}} to {{paramWireName .}}
- internal/generator/templates/command_promoted.go.tmpl: 8 emission sites
  switched from {{.Name}} to {{paramWireName .}}
- internal/generator/url_name_test.go: new file. Three tests cover
  WireName() unit behavior, $-prefixed URL emission via url_name, and
  the regression guard for plain specs.

Notes
- Path substitution, JSON body keys, and MCP tool bindings still use Name.
  url_name affects URL query keys only.
- Pagination's limit_param/cursor_param already accepted user-supplied
  strings (e.g. "$limit"), so the pagination loop alone was working;
  this fix closes the remaining gap for non-pagination SoQL params like
  $where, $select, $order.
- Secondary Socrata gap (id_field extraction needs $select=:id,*
  auto-injection) is NOT addressed here. Filed as follow-up.

Live-tested
- nyc-liens-pp-cli regen with url_name on limit/where/select returns
  valid Brooklyn water-only lien records from
  https://data.cityofnewyork.us/resource/9rz4-mjek.json against borough=3
  and water_debt_only=YES.
- bexar-cad-pp-cli regen unchanged — ArcGIS plain params preserved.

Signed-off-by: youdidwhat-towho <chris@lovephoenixhomes.com>

Files touched

• M internal/generator/flag_collision.go
• M internal/generator/generator.go
• M internal/generator/templates/command_endpoint.go.tmpl
• M internal/generator/templates/command_promoted.go.tmpl
• A internal/generator/url_name_test.go
• M internal/spec/spec.go

Diff

commit 3819b67f8118617725671467a7a0f6ee712c4304
Author: CB <chris@lovephoenixhomes.com>
Date:   Wed May 13 14:47:22 2026 -0700

    fix(cli): support $-prefixed pagination params for Socrata-style APIs (#1204)
    
    Adds an optional `url_name` field to spec.Param that, when set, overrides
    the URL query-key while leaving the CLI flag, Go identifier, and JSON body
    emission unchanged. Specs targeting Socrata-backed APIs (NYC OpenData and
    ~70 other major US municipal portals) can now declare:
    
        params:
          - name: limit
            url_name: "$limit"
            type: integer
    
    and the generated handler emits params["$limit"] in the URL while keeping
    --limit as the cobra flag.
    
    Previously generated CLIs against Socrata returned HTTP 400
    "Unrecognized arguments [limit]" because the wire-side key was the plain
    Name. Specs without url_name are unaffected — verified by regenerating
    bexar-cad-pp-cli and confirming params["f"], params["outFields"], etc.
    remain plain.
    
    Changes
    - internal/spec/spec.go: add URLName field + Param.WireName() method
    - internal/generator/flag_collision.go: add paramWireName helper, mirroring
      paramIdent. Updated paramIdent comment to clarify URL emission goes
      through paramWireName, not Name directly.
    - internal/generator/generator.go: register paramWireName template helper
    - internal/generator/templates/command_endpoint.go.tmpl: 6 emission sites
      switched from {{.Name}} to {{paramWireName .}}
    - internal/generator/templates/command_promoted.go.tmpl: 8 emission sites
      switched from {{.Name}} to {{paramWireName .}}
    - internal/generator/url_name_test.go: new file. Three tests cover
      WireName() unit behavior, $-prefixed URL emission via url_name, and
      the regression guard for plain specs.
    
    Notes
    - Path substitution, JSON body keys, and MCP tool bindings still use Name.
      url_name affects URL query keys only.
    - Pagination's limit_param/cursor_param already accepted user-supplied
      strings (e.g. "$limit"), so the pagination loop alone was working;
      this fix closes the remaining gap for non-pagination SoQL params like
      $where, $select, $order.
    - Secondary Socrata gap (id_field extraction needs $select=:id,*
      auto-injection) is NOT addressed here. Filed as follow-up.
    
    Live-tested
    - nyc-liens-pp-cli regen with url_name on limit/where/select returns
      valid Brooklyn water-only lien records from
      https://data.cityofnewyork.us/resource/9rz4-mjek.json against borough=3
      and water_debt_only=YES.
    - bexar-cad-pp-cli regen unchanged — ArcGIS plain params preserved.
    
    Signed-off-by: youdidwhat-towho <chris@lovephoenixhomes.com>
---
 internal/generator/flag_collision.go               |  12 +-
 internal/generator/generator.go                    |   1 +
 .../generator/templates/command_endpoint.go.tmpl   |  12 +-
 .../generator/templates/command_promoted.go.tmpl   |  16 +--
 internal/generator/url_name_test.go                | 128 +++++++++++++++++++++
 internal/spec/spec.go                              |  13 +++
 6 files changed, 166 insertions(+), 16 deletions(-)

diff --git a/internal/generator/flag_collision.go b/internal/generator/flag_collision.go
index b13abee2..3bf89ca6 100644
--- a/internal/generator/flag_collision.go
+++ b/internal/generator/flag_collision.go
@@ -304,8 +304,8 @@ func validatePublicFlagEntry(resKey, epName string, entry publicFlagEntry, reser
 // identifiers (via camel) or cobra flag names (via flagName). It is
 // IdentName when populated by the dedup pass and Name otherwise. The
 // resulting string must never be used for wire-side serialization;
-// callers writing URL params, JSON keys, or path substitutions read
-// Name directly.
+// callers writing URL params read paramWireName, while JSON keys, or
+// path substitutions read Name directly.
 func paramIdent(p spec.Param) string {
 	if p.IdentName != "" {
 		return p.IdentName
@@ -313,6 +313,14 @@ func paramIdent(p spec.Param) string {
 	return p.Name
 }
 
+// paramWireName returns the URL query-key for this param. URLName overrides
+// when set (e.g., "$limit" for Socrata APIs); otherwise Name. Used by
+// generator templates for URL emission only — not for JSON body keys or
+// path substitution.
+func paramWireName(p spec.Param) string {
+	return p.WireName()
+}
+
 func publicFlagName(p spec.Param) string {
 	if p.FlagName != "" {
 		return p.FlagName
diff --git a/internal/generator/generator.go b/internal/generator/generator.go
index ddc1ebf4..d91dd090 100644
--- a/internal/generator/generator.go
+++ b/internal/generator/generator.go
@@ -248,6 +248,7 @@ func New(s *spec.APISpec, outputDir string) *Generator {
 		"mcpParamDesc":           g.mcpParamDescription,
 		"flagName":               flagName,
 		"paramIdent":             paramIdent,
+		"paramWireName":          paramWireName,
 		"typeFieldIdent":         typeFieldIdent,
 		"safeTypeName":           safeTypeName,
 		"hasNonScalarType": func(types map[string]spec.TypeDef) bool {
diff --git a/internal/generator/templates/command_endpoint.go.tmpl b/internal/generator/templates/command_endpoint.go.tmpl
index fa671a9f..65375db7 100644
--- a/internal/generator/templates/command_endpoint.go.tmpl
+++ b/internal/generator/templates/command_endpoint.go.tmpl
@@ -212,10 +212,10 @@ func new{{camel .FuncPrefix}}{{camel .EndpointName}}Cmd(flags *rootFlags) *cobra
 			data, prov, err := resolvePaginatedRead(cmd.Context(), c, flags, "{{lower .ResourceName}}", path, map[string]string{
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-				"{{.Name}}": args[{{positionalIndex $.Endpoint .Name}}],
+				"{{paramWireName .}}": args[{{positionalIndex $.Endpoint .Name}}],
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
-				"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
+				"{{paramWireName .}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
 {{- end}}
 {{- end}}
 			}, {{if .Endpoint.HeaderOverrides}}headerOverrides{{else}}nil{{end}}, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
@@ -223,10 +223,10 @@ func new{{camel .FuncPrefix}}{{camel .EndpointName}}Cmd(flags *rootFlags) *cobra
 			data, err := paginatedGet(c, path, map[string]string{
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-				"{{.Name}}": args[{{positionalIndex $.Endpoint .Name}}],
+				"{{paramWireName .}}": args[{{positionalIndex $.Endpoint .Name}}],
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
-				"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
+				"{{paramWireName .}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
 {{- end}}
 {{- end}}
 			}, {{if .Endpoint.HeaderOverrides}}headerOverrides{{else}}nil{{end}}, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
@@ -235,11 +235,11 @@ func new{{camel .FuncPrefix}}{{camel .EndpointName}}Cmd(flags *rootFlags) *cobra
 			params := map[string]string{}
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-			params["{{.Name}}"] = args[{{positionalIndex $.Endpoint .Name}}]
+			params["{{paramWireName .}}"] = args[{{positionalIndex $.Endpoint .Name}}]
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
 			if flag{{camel (paramIdent .)}} != {{zeroValForParam .Name .Type}} {
-				params["{{.Name}}"] = fmt.Sprintf("%v", flag{{camel (paramIdent .)}})
+				params["{{paramWireName .}}"] = fmt.Sprintf("%v", flag{{camel (paramIdent .)}})
 			}
 {{- end}}
 {{- end}}
diff --git a/internal/generator/templates/command_promoted.go.tmpl b/internal/generator/templates/command_promoted.go.tmpl
index 82e432d6..e7811916 100644
--- a/internal/generator/templates/command_promoted.go.tmpl
+++ b/internal/generator/templates/command_promoted.go.tmpl
@@ -121,11 +121,11 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
 			htmlRequestParams := map[string]string{}
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-			htmlRequestParams["{{.Name}}"] = args[{{positionalIndex $.Endpoint .Name}}]
+			htmlRequestParams["{{paramWireName .}}"] = args[{{positionalIndex $.Endpoint .Name}}]
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
 			if flag{{camel (paramIdent .)}} != {{zeroValForParam .Name .Type}} {
-				htmlRequestParams["{{.Name}}"] = fmt.Sprintf("%v", flag{{camel (paramIdent .)}})
+				htmlRequestParams["{{paramWireName .}}"] = fmt.Sprintf("%v", flag{{camel (paramIdent .)}})
 			}
 {{- end}}
 {{- end}}
@@ -136,10 +136,10 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
 			data, prov, err := resolvePaginatedRead(cmd.Context(), c, flags, "{{lower .ResourceName}}", path, map[string]string{
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-				"{{.Name}}": args[{{positionalIndex $.Endpoint .Name}}],
+				"{{paramWireName .}}": args[{{positionalIndex $.Endpoint .Name}}],
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
-				"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
+				"{{paramWireName .}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
 {{- end}}
 {{- end}}
 			}, nil, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
@@ -147,10 +147,10 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
 			data, err := paginatedGet(c, path, map[string]string{
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-				"{{.Name}}": args[{{positionalIndex $.Endpoint .Name}}],
+				"{{paramWireName .}}": args[{{positionalIndex $.Endpoint .Name}}],
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
-				"{{.Name}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
+				"{{paramWireName .}}": fmt.Sprintf("%v", flag{{camel (paramIdent .)}}),
 {{- end}}
 {{- end}}
 			}, nil, flagAll, "{{.Endpoint.Pagination.CursorParam}}", "{{.Endpoint.Pagination.NextCursorPath}}", "{{.Endpoint.Pagination.HasMoreField}}")
@@ -160,11 +160,11 @@ func new{{camel .PromotedName}}PromotedCmd(flags *rootFlags) *cobra.Command {
 			params := map[string]string{}
 {{- range .Endpoint.Params}}
 {{- if and .Positional (not (pathContainsParam $.Endpoint.Path .Name))}}
-			params["{{.Name}}"] = args[{{positionalIndex $.Endpoint .Name}}]
+			params["{{paramWireName .}}"] = args[{{positionalIndex $.Endpoint .Name}}]
 {{- end}}
 {{- if and (not .Positional) (not .PathParam)}}
 			if flag{{camel (paramIdent .)}} != {{zeroValForParam .Name .Type}} {
-				params["{{.Name}}"] = fmt.Sprintf("%v", flag{{camel (paramIdent .)}})
+				params["{{paramWireName .}}"] = fmt.Sprintf("%v", flag{{camel (paramIdent .)}})
 			}
 {{- end}}
 {{- end}}
diff --git a/internal/generator/url_name_test.go b/internal/generator/url_name_test.go
new file mode 100644
index 00000000..5328aafb
--- /dev/null
+++ b/internal/generator/url_name_test.go
@@ -0,0 +1,128 @@
+package generator
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/mvanhorn/cli-printing-press/v4/internal/spec"
+	"github.com/stretchr/testify/require"
+)
+
+// TestParamURLNameOverridesWireKey covers Socrata-style APIs where the URL
+// query key needs a literal "$" prefix ($limit, $offset, $where) while the
+// user-facing CLI flag stays clean (--limit, --offset, --where). The fix adds
+// an optional url_name field on Param that, when set, overrides Name as the
+// wire-side URL key without touching the CLI flag derivation.
+func TestParamURLNameOverridesWireKey(t *testing.T) {
+	t.Parallel()
+
+	apiSpec := minimalSpec("socrata-url-name")
+	apiSpec.Resources["records"] = spec.Resource{
+		Description: "Records",
+		Endpoints: map[string]spec.Endpoint{
+			"query": {
+				Method:      "GET",
+				Path:        "/records",
+				Description: "Query records",
+				Params: []spec.Param{
+					{Name: "limit", URLName: "$limit", Type: "integer", Description: "Max rows"},
+					{Name: "offset", URLName: "$offset", Type: "integer", Description: "Offset"},
+					{Name: "where", URLName: "$where", Type: "string", Description: "SoQL WHERE"},
+					{Name: "borough", Type: "integer", Description: "Plain (no override)"},
+				},
+			},
+		},
+	}
+
+	outputDir := filepath.Join(t.TempDir(), "socrata-url-name-pp-cli")
+	require.NoError(t, New(apiSpec, outputDir).Generate())
+
+	content := readGeneratedHandler(t, outputDir, "records")
+
+	// Wire-side URL keys must be the $-prefixed override
+	require.Contains(t, content, `params["$limit"]`, "URLName $limit must appear in the params map")
+	require.Contains(t, content, `params["$offset"]`, "URLName $offset must appear in the params map")
+	require.Contains(t, content, `params["$where"]`, "URLName $where must appear in the params map")
+
+	// Params without url_name must keep plain Name
+	require.Contains(t, content, `params["borough"]`, "Param without URLName must emit plain Name as URL key")
+
+	// CLI flag identifiers must stay plain (no $ in Go identifiers, no $ on cobra flag names)
+	require.Contains(t, content, "flagLimit", "Go identifier flagLimit must remain plain")
+	require.Contains(t, content, `"limit"`, "cobra flag --limit must remain plain")
+	require.NotContains(t, content, "flag$Limit", "no $ should leak into Go identifiers")
+	require.NotContains(t, content, "flag\\$Limit", "no escaped $ should leak into Go identifiers either")
+
+	// The Name field must NOT appear as a URL key when URLName is set (regression guard)
+	if strings.Contains(content, `params["limit"]`) {
+		t.Errorf("when URLName is $limit, params[\"limit\"] must not also be emitted as a URL key")
+	}
+}
+
+// TestParamWithoutURLNameUnchanged guards against regression: existing specs
+// without url_name must continue to emit Name as the URL key.
+func TestParamWithoutURLNameUnchanged(t *testing.T) {
+	t.Parallel()
+
+	apiSpec := minimalSpec("plain-param")
+	apiSpec.Resources["records"] = spec.Resource{
+		Description: "Records",
+		Endpoints: map[string]spec.Endpoint{
+			"query": {
+				Method: "GET", Path: "/records", Description: "Query records",
+				Params: []spec.Param{
+					{Name: "limit", Type: "integer", Description: "Max rows"},
+					{Name: "owner", Type: "string", Description: "Owner filter"},
+				},
+			},
+		},
+	}
+
+	outputDir := filepath.Join(t.TempDir(), "plain-param-pp-cli")
+	require.NoError(t, New(apiSpec, outputDir).Generate())
+
+	content := readGeneratedHandler(t, outputDir, "records")
+
+	require.Contains(t, content, `params["limit"]`, "plain Name limit must emit as URL key when URLName unset")
+	require.Contains(t, content, `params["owner"]`, "plain Name owner must emit as URL key when URLName unset")
+}
+
+// readGeneratedHandler returns the contents of the generated CLI handler for a
+// resource. The generator may emit it as either `<resource>.go` (multi-endpoint
+// resource) or `promoted_<resource>.go` (single-endpoint promoted pattern), so
+// try both.
+func readGeneratedHandler(t *testing.T, outputDir, resource string) string {
+	t.Helper()
+	candidates := []string{
+		filepath.Join(outputDir, "internal", "cli", resource+".go"),
+		filepath.Join(outputDir, "internal", "cli", "promoted_"+resource+".go"),
+	}
+	for _, p := range candidates {
+		if src, err := os.ReadFile(p); err == nil {
+			return string(src)
+		}
+	}
+	t.Fatalf("no generated handler found for resource %q (tried %v)", resource, candidates)
+	return ""
+}
+
+// TestParamWireNameUnit exercises the spec.Param.WireName() method directly.
+func TestParamWireNameUnit(t *testing.T) {
+	t.Parallel()
+	cases := []struct {
+		name, n, urlName, want string
+	}{
+		{"name only", "limit", "", "limit"},
+		{"url_name overrides", "limit", "$limit", "$limit"},
+		{"url_name empty falls back to Name", "where", "", "where"},
+		{"url_name with special chars", "complex", "$query.where", "$query.where"},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			p := spec.Param{Name: c.n, URLName: c.urlName}
+			require.Equal(t, c.want, p.WireName())
+		})
+	}
+}
diff --git a/internal/spec/spec.go b/internal/spec/spec.go
index 3b0c5296..264d140d 100644
--- a/internal/spec/spec.go
+++ b/internal/spec/spec.go
@@ -1184,6 +1184,7 @@ func (h *HTMLExtract) EffectiveScriptSelector() string {
 type Param struct {
 	Name        string   `yaml:"name" json:"name"`
 	FlagName    string   `yaml:"flag_name,omitempty" json:"flag_name,omitempty"`
+	URLName     string   `yaml:"url_name,omitempty" json:"url_name,omitempty"` // optional override for URL query-key emission (e.g., "$limit" for Socrata while keeping --limit flag)
 	Aliases     []string `yaml:"aliases,omitempty" json:"aliases,omitempty"`
 	Type        string   `yaml:"type" json:"type"`
 	Required    bool     `yaml:"required" json:"required"`
@@ -1209,6 +1210,18 @@ type Param struct {
 	FlagNameSet bool `yaml:"-" json:"-"`
 }
 
+// WireName returns the URL query-key name for this param when emitted in a
+// generated HTTP request. URLName takes precedence when set (e.g., "$limit" for
+// Socrata-style APIs that require the literal "$" prefix on pagination + SoQL
+// params); otherwise Name is used. The CLI flag name is independent (derived
+// from FlagName or paramIdent), so this only affects what shows up in the URL.
+func (p Param) WireName() string {
+	if p.URLName != "" {
+		return p.URLName
+	}
+	return p.Name
+}
+
 func (p Param) PublicInputName() string {
 	if p.FlagName != "" {
 		return p.FlagName

← 900b335c feat(cli): emit fetchFull<X> companion for GET-embedded page  ·  back to Cli Printing Press  ·  fix(cli): propagate --timeout to surf transport ResponseHead 69cefbe9 →