← back to Cli Printing Press
feat(cli): cost-based throttling primitives for GraphQL CLIs (#434)
de3a4e7c07e1182ebb8b1e921f50ad5e7cb453d9 · 2026-05-01 03:03:41 -0500 · Cathryn Lavery
* feat(cli): add ThrottlingConfig spec field + HasCostThrottling helper
Plumbing for PR-3's cost-based throttling primitives. When
Throttling.Enabled is set, the generator will emit throttle.go and
related conditional blocks; when unset (default), printed CLIs
regenerate byte-identical to today.
The shape is intentionally minimal — Enabled is a single bool — so
spec authors don't decide retry counts before knowing they want
throttling at all. Future fields (max retries, custom extensions
path for non-Shopify shapes) belong on this struct.
* feat(cli): emit cost-based throttle primitives in throttle.go
Adds throttle.go.tmpl producing internal/client/throttle.go in any
printed CLI whose spec sets throttling.enabled. The file declares:
- ThrottleStatus: bucket snapshot reported by the API
- ThrottleState: mutex-guarded current status, seeded flag for
pre-first-response queries
- WaitForBudget(ctx, cost): blocks until projected bucket can
admit the request; returns immediately when !seeded so the
first call learns its budget from the response
- computeWait(now, cost): pure projection math, split out so
tests can verify forward-projection without sleeping
- Update(status): callable from the GraphQL client after every
response that carries extensions.cost
- HandleThrottleError(resp, body, attempt): detects HTTP 429s
(with Retry-After) and GraphQL extensions code "THROTTLED";
caps retries at MaxThrottleRetries with exponential backoff
plus jitter
- ThrottleMode constants: strict (default, block), lenient
(warn but proceed), off (bypass)
The math fixes V2's seeded-init bug already (returns nil before
the first observation rather than blocking on a zero-status
bucket). Distinct from cliutil.AdaptiveLimiter — that's reactive
RPS pacing from 429 observation, this projects forward from the
bucket the API reports. They compose: limiter caps RPS, throttle
gates on remaining query cost.
The generator gates emission on g.Spec.HasCostThrottling() so
existing GraphQL CLIs (Linear) and REST CLIs regenerate
byte-identical when throttling is unset.
* feat(cli): wire --throttle-mode flag through generated client + graphql retry
Three-template wiring for the cost-throttling surface, all gated on
.HasCostThrottling so byte-compat is preserved when throttling is
off.
client.go.tmpl: when enabled, Client gains Throttle (*ThrottleState,
auto-instantiated in New()) and ThrottleMode (defaults to
ThrottleModeStrict). The fields exist on the struct so any novel
command can call c.Throttle.WaitForBudget directly without a
type-assertion dance.
graphql_client.go.tmpl: when enabled, Query routes through
queryWithThrottle, which (1) calls c.awaitBudget before the Post
(strict blocks via WaitForBudget; lenient warns and proceeds; off
bypasses), (2) parses extensions.cost.throttleStatus from every
response — Shopify's shape today; defensive (no-op on missing) so
other GraphQL APIs are safe to opt in, and (3) retries on
"THROTTLED" extension codes via HandleThrottleError. HTTP 429s are
still handled by Client.do()'s existing AdaptiveLimiter retry —
this path stacks on top without double-handling.
root.go.tmpl: registers --throttle-mode strict|lenient|off
(default strict; documented in the flag's help text), validates
the value in PersistentPreRunE, and propagates it to Client via
newClient(). Strict default keeps first-time runs correct; agents
and scripted callers can opt to lenient/off.
* test(cli): cover cost-based throttling primitives + byte-compat
Two integration tests in cost_throttling_test.go.
TestGenerateNoCostThrottlingByteCompat — guards the byte-compat
promise. Generates a Linear-shape GraphQL CLI with
Throttling.Enabled unset and asserts throttle.go is absent;
client.go has no Throttle/ThrottleMode references; graphql.go
keeps its byte-identical Query path; root.go registers no
--throttle-mode flag. Linear is the canonical existing GraphQL
CLI; loops/stytch/clerk cover REST through TestGenerateProjectsCompile.
TestGenerateCostThrottlingPrimitives — generates a fixture spec
with Throttling.Enabled, compiles the generated tree, and runs an
injected behavioral test that exercises every PR-3 acceptance
path:
(a) WaitForBudget returns immediately when !seeded
(b) computeWait projects forward (status 100/1000 with
restoreRate 50, requested 800 after 5s elapsed → projected
350, deficit 450, sleep 9s)
(c) HandleThrottleError detects HTTP 429 with Retry-After
(d) HandleThrottleError detects GraphQL "THROTTLED" extension
code body
(e) HandleThrottleError caps at MaxThrottleRetries
(f) Concurrent Update/computeWait/Status calls are race-free
under -race
(g) Off-mode awaitBudget short-circuits even when bucket is
drained
(h) updateThrottleFromBody parses Shopify's
extensions.cost.throttleStatus shape
(i) updateThrottleFromBody no-ops when extensions.cost is
missing — the defensive guard that lets any GraphQL spec
opt in safely
Mirrors the inject-and-go-test pattern PR-2 used in
TestGenerateEndpointTemplateVarsRuntimeSubstitution. The -race
pass on the concurrent test exercises the mutex shape directly,
which is the explicit V3 fix for V2's sync.Map design.
* fix(cli): reserve throttle budget on Wait + drop unused Extensions field
Two findings from a Codex review pass on PR-3.
1. WaitForBudget now debits requested cost from the local bucket
under the same mutex that computes the wait. Without the debit,
concurrent goroutines all read the same projected balance and
race through, defeating the throttle for fanout/paginated GraphQL
workflows. The math is split across two helpers: computeWait
stays pure (used by lenient mode and tests), reserveBudget
computes-and-debits inside one critical section (used by the
blocking WaitForBudget path). Reservation drift is bounded —
the next Update() resyncs from the API's authoritative status,
so divergence collapses to one in-flight request per goroutine.
Better to over-throttle locally than double-spend a shared
budget.
2. GraphQLResponse loses its unused Extensions field. The cost
parser reads extensions.cost from the raw body via a separate
typed struct (throttleExtensions), so the field on
GraphQLResponse was dead code that drifted byte-compat for
existing GraphQL CLIs (Linear) regardless of throttling opt-in.
Dropping it restores the byte-identical contract.
Adds TestThrottleReserveBudgetDebitsConcurrentCallers as the
behavioral receipt: two reserveBudget(50) calls on a 100-bucket
must each admit with no wait, but the third call must wait for the
bucket to refill. This exercises the reservation path end-to-end.
* refactor(throttle): named-adapter shape + CI lint fix
Two changes:
1. **CI fix.** internal/spec/spec.go:108 had json:"throttling,omitempty"
on a struct value; modernize lint correctly flagged that omitempty has
no effect on Go struct values for JSON encoding. Drop ,omitempty from
the json tag to match the convention used by Cache, Share, and MCP in
the same struct (yaml:"x,omitempty" json:"x").
2. **Named-adapter shape.** The PR's "generic primitives for cost-throttled
GraphQL CLIs" framing is mostly already true — the bucket math, retry,
modes, and --throttle-mode flag in throttle.go.tmpl are all API-agnostic.
The only Shopify-specific code is ~30 lines in graphql_client.go.tmpl
(the throttleExtensions struct + updateThrottleFromBody parser).
Make that shape-specific code switchable via a new Shape field on
ThrottlingConfig:
throttling:
enabled: true
shape: shopify
- Add ThrottleShape typed string with ShapeShopify constant
- ThrottlingConfig.Validate rejects Enabled=true without a recognized
Shape (wired into APISpec.Validate alongside the cache/share/MCP checks)
- Wrap the Shopify-shape parser block in graphql_client.go.tmpl on
{{- if eq (printf "%s" .Throttling.Shape) "shopify" -}} so adding a
second shape is a new gated block, not a refactor of the existing one
- Update existing test fixture and add TestThrottlingValidate covering
enabled-without-shape, unknown-shape, and the off case
The contract for adding GitHub or Datadog support becomes mechanical:
add a constant, extend Validate to accept it, add the parser block.
No core code changes. Goldens still pass byte-identical.
* refactor(spec): polish from review — free fn, pinning test, doc trim
Three small follow-ups to the named-adapter commit:
1. Convert ThrottlingConfig.Validate method to validateThrottling free
function for symmetry with validateMCP / validateExtraCommands. The
rest of the package favors free functions for spec validation.
2. Add TestThrottleShapeShopifyValue pinning the constant's wire value
to "shopify". Go templates can't import the constant, so the parser
gate in graphql_client.go.tmpl uses the literal "shopify". A silent
rename of the Go constant would strand the template gate; this test
surfaces the mismatch as a test failure instead.
3. Trim ThrottlingConfig's doc comment to point at ThrottleShape's doc
for valid values rather than restating them. Removes the duplicated
"Today only ThrottleShapeShopify is valid" sentence and the parallel
"no refactor needed" framing — both now live with the type.
---------
Co-authored-by: Trevin Chow <trevin@trevinchow.com>
Files touched
A internal/generator/cost_throttling_test.goM internal/generator/generator.goM internal/generator/templates/client.go.tmplM internal/generator/templates/graphql_client.go.tmplM internal/generator/templates/root.go.tmplA internal/generator/templates/throttle.go.tmplM internal/spec/spec.goM internal/spec/spec_test.go
Diff
commit de3a4e7c07e1182ebb8b1e921f50ad5e7cb453d9
Author: Cathryn Lavery <50469282+cathrynlavery@users.noreply.github.com>
Date: Fri May 1 03:03:41 2026 -0500
feat(cli): cost-based throttling primitives for GraphQL CLIs (#434)
* feat(cli): add ThrottlingConfig spec field + HasCostThrottling helper
Plumbing for PR-3's cost-based throttling primitives. When
Throttling.Enabled is set, the generator will emit throttle.go and
related conditional blocks; when unset (default), printed CLIs
regenerate byte-identical to today.
The shape is intentionally minimal — Enabled is a single bool — so
spec authors don't decide retry counts before knowing they want
throttling at all. Future fields (max retries, custom extensions
path for non-Shopify shapes) belong on this struct.
* feat(cli): emit cost-based throttle primitives in throttle.go
Adds throttle.go.tmpl producing internal/client/throttle.go in any
printed CLI whose spec sets throttling.enabled. The file declares:
- ThrottleStatus: bucket snapshot reported by the API
- ThrottleState: mutex-guarded current status, seeded flag for
pre-first-response queries
- WaitForBudget(ctx, cost): blocks until projected bucket can
admit the request; returns immediately when !seeded so the
first call learns its budget from the response
- computeWait(now, cost): pure projection math, split out so
tests can verify forward-projection without sleeping
- Update(status): callable from the GraphQL client after every
response that carries extensions.cost
- HandleThrottleError(resp, body, attempt): detects HTTP 429s
(with Retry-After) and GraphQL extensions code "THROTTLED";
caps retries at MaxThrottleRetries with exponential backoff
plus jitter
- ThrottleMode constants: strict (default, block), lenient
(warn but proceed), off (bypass)
The math fixes V2's seeded-init bug already (returns nil before
the first observation rather than blocking on a zero-status
bucket). Distinct from cliutil.AdaptiveLimiter — that's reactive
RPS pacing from 429 observation, this projects forward from the
bucket the API reports. They compose: limiter caps RPS, throttle
gates on remaining query cost.
The generator gates emission on g.Spec.HasCostThrottling() so
existing GraphQL CLIs (Linear) and REST CLIs regenerate
byte-identical when throttling is unset.
* feat(cli): wire --throttle-mode flag through generated client + graphql retry
Three-template wiring for the cost-throttling surface, all gated on
.HasCostThrottling so byte-compat is preserved when throttling is
off.
client.go.tmpl: when enabled, Client gains Throttle (*ThrottleState,
auto-instantiated in New()) and ThrottleMode (defaults to
ThrottleModeStrict). The fields exist on the struct so any novel
command can call c.Throttle.WaitForBudget directly without a
type-assertion dance.
graphql_client.go.tmpl: when enabled, Query routes through
queryWithThrottle, which (1) calls c.awaitBudget before the Post
(strict blocks via WaitForBudget; lenient warns and proceeds; off
bypasses), (2) parses extensions.cost.throttleStatus from every
response — Shopify's shape today; defensive (no-op on missing) so
other GraphQL APIs are safe to opt in, and (3) retries on
"THROTTLED" extension codes via HandleThrottleError. HTTP 429s are
still handled by Client.do()'s existing AdaptiveLimiter retry —
this path stacks on top without double-handling.
root.go.tmpl: registers --throttle-mode strict|lenient|off
(default strict; documented in the flag's help text), validates
the value in PersistentPreRunE, and propagates it to Client via
newClient(). Strict default keeps first-time runs correct; agents
and scripted callers can opt to lenient/off.
* test(cli): cover cost-based throttling primitives + byte-compat
Two integration tests in cost_throttling_test.go.
TestGenerateNoCostThrottlingByteCompat — guards the byte-compat
promise. Generates a Linear-shape GraphQL CLI with
Throttling.Enabled unset and asserts throttle.go is absent;
client.go has no Throttle/ThrottleMode references; graphql.go
keeps its byte-identical Query path; root.go registers no
--throttle-mode flag. Linear is the canonical existing GraphQL
CLI; loops/stytch/clerk cover REST through TestGenerateProjectsCompile.
TestGenerateCostThrottlingPrimitives — generates a fixture spec
with Throttling.Enabled, compiles the generated tree, and runs an
injected behavioral test that exercises every PR-3 acceptance
path:
(a) WaitForBudget returns immediately when !seeded
(b) computeWait projects forward (status 100/1000 with
restoreRate 50, requested 800 after 5s elapsed → projected
350, deficit 450, sleep 9s)
(c) HandleThrottleError detects HTTP 429 with Retry-After
(d) HandleThrottleError detects GraphQL "THROTTLED" extension
code body
(e) HandleThrottleError caps at MaxThrottleRetries
(f) Concurrent Update/computeWait/Status calls are race-free
under -race
(g) Off-mode awaitBudget short-circuits even when bucket is
drained
(h) updateThrottleFromBody parses Shopify's
extensions.cost.throttleStatus shape
(i) updateThrottleFromBody no-ops when extensions.cost is
missing — the defensive guard that lets any GraphQL spec
opt in safely
Mirrors the inject-and-go-test pattern PR-2 used in
TestGenerateEndpointTemplateVarsRuntimeSubstitution. The -race
pass on the concurrent test exercises the mutex shape directly,
which is the explicit V3 fix for V2's sync.Map design.
* fix(cli): reserve throttle budget on Wait + drop unused Extensions field
Two findings from a Codex review pass on PR-3.
1. WaitForBudget now debits requested cost from the local bucket
under the same mutex that computes the wait. Without the debit,
concurrent goroutines all read the same projected balance and
race through, defeating the throttle for fanout/paginated GraphQL
workflows. The math is split across two helpers: computeWait
stays pure (used by lenient mode and tests), reserveBudget
computes-and-debits inside one critical section (used by the
blocking WaitForBudget path). Reservation drift is bounded —
the next Update() resyncs from the API's authoritative status,
so divergence collapses to one in-flight request per goroutine.
Better to over-throttle locally than double-spend a shared
budget.
2. GraphQLResponse loses its unused Extensions field. The cost
parser reads extensions.cost from the raw body via a separate
typed struct (throttleExtensions), so the field on
GraphQLResponse was dead code that drifted byte-compat for
existing GraphQL CLIs (Linear) regardless of throttling opt-in.
Dropping it restores the byte-identical contract.
Adds TestThrottleReserveBudgetDebitsConcurrentCallers as the
behavioral receipt: two reserveBudget(50) calls on a 100-bucket
must each admit with no wait, but the third call must wait for the
bucket to refill. This exercises the reservation path end-to-end.
* refactor(throttle): named-adapter shape + CI lint fix
Two changes:
1. **CI fix.** internal/spec/spec.go:108 had json:"throttling,omitempty"
on a struct value; modernize lint correctly flagged that omitempty has
no effect on Go struct values for JSON encoding. Drop ,omitempty from
the json tag to match the convention used by Cache, Share, and MCP in
the same struct (yaml:"x,omitempty" json:"x").
2. **Named-adapter shape.** The PR's "generic primitives for cost-throttled
GraphQL CLIs" framing is mostly already true — the bucket math, retry,
modes, and --throttle-mode flag in throttle.go.tmpl are all API-agnostic.
The only Shopify-specific code is ~30 lines in graphql_client.go.tmpl
(the throttleExtensions struct + updateThrottleFromBody parser).
Make that shape-specific code switchable via a new Shape field on
ThrottlingConfig:
throttling:
enabled: true
shape: shopify
- Add ThrottleShape typed string with ShapeShopify constant
- ThrottlingConfig.Validate rejects Enabled=true without a recognized
Shape (wired into APISpec.Validate alongside the cache/share/MCP checks)
- Wrap the Shopify-shape parser block in graphql_client.go.tmpl on
{{- if eq (printf "%s" .Throttling.Shape) "shopify" -}} so adding a
second shape is a new gated block, not a refactor of the existing one
- Update existing test fixture and add TestThrottlingValidate covering
enabled-without-shape, unknown-shape, and the off case
The contract for adding GitHub or Datadog support becomes mechanical:
add a constant, extend Validate to accept it, add the parser block.
No core code changes. Goldens still pass byte-identical.
* refactor(spec): polish from review — free fn, pinning test, doc trim
Three small follow-ups to the named-adapter commit:
1. Convert ThrottlingConfig.Validate method to validateThrottling free
function for symmetry with validateMCP / validateExtraCommands. The
rest of the package favors free functions for spec validation.
2. Add TestThrottleShapeShopifyValue pinning the constant's wire value
to "shopify". Go templates can't import the constant, so the parser
gate in graphql_client.go.tmpl uses the literal "shopify". A silent
rename of the Go constant would strand the template gate; this test
surfaces the mismatch as a test failure instead.
3. Trim ThrottlingConfig's doc comment to point at ThrottleShape's doc
for valid values rather than restating them. Removes the duplicated
"Today only ThrottleShapeShopify is valid" sentence and the parallel
"no refactor needed" framing — both now live with the type.
---------
Co-authored-by: Trevin Chow <trevin@trevinchow.com>
---
internal/generator/cost_throttling_test.go | 391 +++++++++++++++++++++
internal/generator/generator.go | 15 +
internal/generator/templates/client.go.tmpl | 23 ++
.../generator/templates/graphql_client.go.tmpl | 176 ++++++++++
internal/generator/templates/root.go.tmpl | 25 ++
internal/generator/templates/throttle.go.tmpl | 281 +++++++++++++++
internal/spec/spec.go | 62 ++++
internal/spec/spec_test.go | 57 +++
8 files changed, 1030 insertions(+)
diff --git a/internal/generator/cost_throttling_test.go b/internal/generator/cost_throttling_test.go
new file mode 100644
index 00000000..dc7159a7
--- /dev/null
+++ b/internal/generator/cost_throttling_test.go
@@ -0,0 +1,391 @@
+package generator
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+
+ "github.com/mvanhorn/cli-printing-press/v3/internal/graphql"
+ "github.com/mvanhorn/cli-printing-press/v3/internal/naming"
+ "github.com/mvanhorn/cli-printing-press/v3/internal/spec"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+)
+
+// TestGenerateNoCostThrottlingByteCompat guards PR-3's byte-compat promise:
+// a spec that doesn't opt into Throttling.Enabled must regenerate without
+// throttle.go, without the --throttle-mode flag, and without any
+// Throttle/ThrottleMode references in client.go or graphql.go. Linear is
+// the canonical existing GraphQL CLI; loops/stytch/clerk cover the REST
+// path through TestGenerateProjectsCompile.
+func TestGenerateNoCostThrottlingByteCompat(t *testing.T) {
+ t.Parallel()
+
+ gqlSpec, err := graphql.ParseSDL(filepath.Join("..", "..", "testdata", "graphql", "test.graphql"))
+ require.NoError(t, err)
+ require.False(t, gqlSpec.HasCostThrottling(),
+ "GraphQL fixture must keep cost throttling off for the byte-compat case")
+
+ outputDir := filepath.Join(t.TempDir(), naming.CLI(gqlSpec.Name))
+ gen := New(gqlSpec, outputDir)
+ require.NoError(t, gen.Generate())
+
+ _, err = os.Stat(filepath.Join(outputDir, "internal", "client", "throttle.go"))
+ assert.True(t, os.IsNotExist(err),
+ "throttle.go must NOT be emitted when Throttling.Enabled is unset; got err=%v", err)
+
+ clientGoBytes, err := os.ReadFile(filepath.Join(outputDir, "internal", "client", "client.go"))
+ require.NoError(t, err)
+ clientGo := string(clientGoBytes)
+ assert.NotContains(t, clientGo, "Throttle",
+ "client.go must not reference Throttle when throttling is off")
+ assert.NotContains(t, clientGo, "ThrottleMode",
+ "client.go must not reference ThrottleMode when throttling is off")
+
+ graphqlGoBytes, err := os.ReadFile(filepath.Join(outputDir, "internal", "client", "graphql.go"))
+ require.NoError(t, err)
+ graphqlGo := string(graphqlGoBytes)
+ assert.NotContains(t, graphqlGo, "queryWithThrottle",
+ "graphql.go must keep its byte-identical Query path when throttling is off")
+ assert.NotContains(t, graphqlGo, "updateThrottleFromBody",
+ "graphql.go must not emit the cost parser when throttling is off")
+ // Negative on the bare-Throttle identifier as well; the throttle-aware
+ // helpers all start with that prefix, so any reference would be a
+ // regression on the byte-compat promise.
+ assert.NotContains(t, graphqlGo, "c.Throttle",
+ "graphql.go must not reference c.Throttle when throttling is off")
+
+ rootGoBytes, err := os.ReadFile(filepath.Join(outputDir, "internal", "cli", "root.go"))
+ require.NoError(t, err)
+ rootGo := string(rootGoBytes)
+ assert.NotContains(t, rootGo, "throttle-mode",
+ "root.go must not register --throttle-mode when throttling is off")
+ assert.NotContains(t, rootGo, "throttleMode",
+ "rootFlags must not carry throttleMode when throttling is off")
+}
+
+// TestGenerateCostThrottlingPrimitives covers PR-3's contract end-to-end:
+// when a GraphQL spec opts into Throttling.Enabled, the generator emits
+// throttle.go, the --throttle-mode flag, and the Client throttle field.
+// The test compiles the generated tree and runs an injected behavioral
+// test that exercises every required path — !seeded fast path, project-
+// forward bucket math, 429 + THROTTLED detection, max-retry cap, and
+// concurrent update under -race. Mirrors the inject-and-go-test pattern
+// PR-2 used in TestGenerateEndpointTemplateVarsRuntimeSubstitution.
+func TestGenerateCostThrottlingPrimitives(t *testing.T) {
+ t.Parallel()
+
+ apiSpec := &spec.APISpec{
+ Name: "shoppy",
+ Description: "Cost-throttled GraphQL fixture",
+ Version: "1.0.0",
+ BaseURL: "https://api.shoppy.example",
+ GraphQLEndpointPath: "/admin/api/graphql.json",
+ Throttling: spec.ThrottlingConfig{Enabled: true, Shape: spec.ThrottleShapeShopify},
+ Auth: spec.AuthConfig{
+ Type: "api_key",
+ Header: "X-API-Token",
+ EnvVars: []string{"SHOPPY_API_TOKEN"},
+ },
+ Config: spec.ConfigSpec{
+ Format: "toml",
+ Path: "~/.config/shoppy-pp-cli/config.toml",
+ },
+ Resources: map[string]spec.Resource{
+ "orders": {
+ Description: "Orders",
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/graphql",
+ Description: "List orders",
+ ResponsePath: "data.orders.nodes",
+ Pagination: &spec.Pagination{
+ Type: "cursor",
+ LimitParam: "first",
+ CursorParam: "after",
+ NextCursorPath: "data.orders.pageInfo.endCursor",
+ HasMoreField: "data.orders.pageInfo.hasNextPage",
+ },
+ Response: spec.ResponseDef{Type: "array", Item: "Order"},
+ },
+ },
+ },
+ },
+ Types: map[string]spec.TypeDef{
+ "Order": {Fields: []spec.TypeField{
+ {Name: "id", Type: "string"},
+ {Name: "name", Type: "string"},
+ }},
+ },
+ }
+
+ outputDir := filepath.Join(t.TempDir(), naming.CLI(apiSpec.Name))
+ gen := New(apiSpec, outputDir)
+ require.NoError(t, gen.Generate())
+
+ // throttle.go must exist; this is where ThrottleState lives.
+ throttleGoPath := filepath.Join(outputDir, "internal", "client", "throttle.go")
+ throttleBytes, err := os.ReadFile(throttleGoPath)
+ require.NoError(t, err, "throttle.go should be generated when Throttling.Enabled is set")
+ throttle := string(throttleBytes)
+ assert.Contains(t, throttle, "type ThrottleState struct",
+ "throttle.go must declare ThrottleState")
+ assert.Contains(t, throttle, "WaitForBudget",
+ "throttle.go must declare WaitForBudget")
+ assert.Contains(t, throttle, "HandleThrottleError",
+ "throttle.go must declare HandleThrottleError")
+
+ // client.go must carry the Throttle field and instantiate it.
+ clientGoBytes, err := os.ReadFile(filepath.Join(outputDir, "internal", "client", "client.go"))
+ require.NoError(t, err)
+ clientGo := string(clientGoBytes)
+ assert.Contains(t, clientGo, "Throttle *ThrottleState",
+ "client.go must declare the Throttle field")
+ assert.Contains(t, clientGo, "ThrottleMode: ThrottleModeStrict",
+ "client.go must default ThrottleMode to strict in New()")
+
+ // graphql.go must wire updateThrottleFromBody and queryWithThrottle.
+ graphqlGoBytes, err := os.ReadFile(filepath.Join(outputDir, "internal", "client", "graphql.go"))
+ require.NoError(t, err)
+ graphqlGo := string(graphqlGoBytes)
+ assert.Contains(t, graphqlGo, "queryWithThrottle",
+ "graphql.go must use queryWithThrottle when throttling is enabled")
+ assert.Contains(t, graphqlGo, "updateThrottleFromBody",
+ "graphql.go must parse extensions.cost via updateThrottleFromBody")
+ assert.Contains(t, graphqlGo, "throttleStatus",
+ "graphql.go cost parser must read throttleStatus")
+
+ // root.go must register --throttle-mode and validate its value.
+ rootGoBytes, err := os.ReadFile(filepath.Join(outputDir, "internal", "cli", "root.go"))
+ require.NoError(t, err)
+ rootGo := string(rootGoBytes)
+ assert.Contains(t, rootGo, "--throttle-mode",
+ "root.go must register the --throttle-mode flag (mentioned in --help)")
+ assert.Contains(t, rootGo, `"strict", "lenient", "off"`,
+ "root.go must validate --throttle-mode against the enum")
+ assert.Contains(t, rootGo, "c.ThrottleMode = f.throttleMode",
+ "root.go must propagate --throttle-mode to the client")
+
+ // Compile before injecting tests so a syntax error in throttle.go.tmpl
+ // surfaces here, not as a confusing test runner failure.
+ runGoCommand(t, outputDir, "mod", "tidy")
+ runGoCommand(t, outputDir, "build", "./...")
+
+ // Behavioral tests injected into the generated client package. Verify
+ // every acceptance path from PR-3 against real generated code:
+ // (a) WaitForBudget returns immediately when !seeded
+ // (b) computeWait projects forward (status 100/1000, restoreRate 50,
+ // requested 800 after 5s elapsed → projected 350, deficit 450,
+ // sleep 9s)
+ // (c) HandleThrottleError detects 429 with Retry-After
+ // (d) HandleThrottleError detects GraphQL extensions code "THROTTLED"
+ // (e) HandleThrottleError caps at MaxThrottleRetries
+ // (f) Concurrent Update calls are race-free (run under -race below)
+ const behaviorTest = `package client
+
+import (
+ "context"
+ "net/http"
+ "sync"
+ "testing"
+ "time"
+)
+
+func TestThrottleWaitForBudgetUnseededReturnsImmediately(t *testing.T) {
+ state := NewThrottleState()
+ start := time.Now()
+ ctx, cancel := context.WithTimeout(context.Background(), 250*time.Millisecond)
+ defer cancel()
+ if err := state.WaitForBudget(ctx, 100); err != nil {
+ t.Fatalf("unexpected error from unseeded WaitForBudget: %v", err)
+ }
+ if elapsed := time.Since(start); elapsed > 50*time.Millisecond {
+ t.Fatalf("unseeded WaitForBudget should return immediately; took %s", elapsed)
+ }
+}
+
+func TestThrottleComputeWaitProjectsForward(t *testing.T) {
+ state := NewThrottleState()
+ state.Update(ThrottleStatus{
+ MaximumAvailable: 1000,
+ CurrentlyAvailable: 100,
+ RestoreRate: 50,
+ })
+ // Force the LastUpdated stamp 5s into the past so the projection has
+ // observable elapsed time without sleeping the test.
+ state.mu.Lock()
+ state.status.LastUpdated = time.Now().Add(-5 * time.Second)
+ state.mu.Unlock()
+
+ got := state.computeWait(time.Now(), 800)
+ const want = 9 * time.Second
+ const tolerance = 100 * time.Millisecond
+ if got < want-tolerance || got > want+tolerance {
+ t.Fatalf("computeWait: got %s, want ~%s (projected 100 + 5*50 = 350; deficit 450; 450/50 = 9s)", got, want)
+ }
+}
+
+func TestThrottleComputeWaitZeroWhenProjectedExceedsRequested(t *testing.T) {
+ state := NewThrottleState()
+ state.Update(ThrottleStatus{
+ MaximumAvailable: 1000,
+ CurrentlyAvailable: 900,
+ RestoreRate: 50,
+ })
+ if got := state.computeWait(time.Now(), 100); got != 0 {
+ t.Fatalf("expected 0 wait when projected balance covers cost; got %s", got)
+ }
+}
+
+func TestThrottleHandleErrorDetects429WithRetryAfter(t *testing.T) {
+ state := NewThrottleState()
+ resp := &http.Response{
+ StatusCode: http.StatusTooManyRequests,
+ Header: http.Header{},
+ }
+ resp.Header.Set("Retry-After", "3")
+
+ retry, wait := state.HandleThrottleError(resp, nil, 0)
+ if !retry {
+ t.Fatal("HandleThrottleError must request retry on 429")
+ }
+ if wait < 3*time.Second {
+ t.Fatalf("HandleThrottleError must honor Retry-After (3s); got %s", wait)
+ }
+}
+
+func TestThrottleHandleErrorDetectsGraphQLThrottledExtension(t *testing.T) {
+ state := NewThrottleState()
+ body := []byte(` + "`" + `{"errors":[{"message":"Throttled","extensions":{"code":"THROTTLED"}}]}` + "`" + `)
+ retry, wait := state.HandleThrottleError(nil, body, 0)
+ if !retry {
+ t.Fatal("HandleThrottleError must request retry on THROTTLED extension code")
+ }
+ if wait <= 0 {
+ t.Fatalf("HandleThrottleError must return a positive wait for THROTTLED; got %s", wait)
+ }
+}
+
+func TestThrottleHandleErrorCapsAtMaxRetries(t *testing.T) {
+ state := NewThrottleState()
+ resp := &http.Response{StatusCode: http.StatusTooManyRequests, Header: http.Header{}}
+ if retry, _ := state.HandleThrottleError(resp, nil, MaxThrottleRetries); retry {
+ t.Fatalf("HandleThrottleError must stop retrying at attempt=MaxThrottleRetries (%d)", MaxThrottleRetries)
+ }
+}
+
+func TestThrottleReserveBudgetDebitsConcurrentCallers(t *testing.T) {
+ // Codex P2 from PR-3 review: WaitForBudget without reservation lets
+ // every concurrent caller read the same projected balance and race
+ // through. reserveBudget is the fix — under one mutex, project AND
+ // debit. Two callers asking for 50 against a 100/1000 bucket
+ // (restoreRate 50) must each be admitted with no wait, but the
+ // third caller arriving immediately after must wait because the
+ // local bucket is now drained.
+ state := NewThrottleState()
+ state.Update(ThrottleStatus{
+ MaximumAvailable: 1000,
+ CurrentlyAvailable: 100,
+ RestoreRate: 50,
+ })
+ now := time.Now()
+ if got := state.reserveBudget(now, 50); got != 0 {
+ t.Fatalf("first reserveBudget(50) on 100-bucket must not wait; got %s", got)
+ }
+ if got := state.reserveBudget(now, 50); got != 0 {
+ t.Fatalf("second reserveBudget(50) on the now-50 bucket must not wait; got %s", got)
+ }
+ // The third concurrent call lands on a debited-to-zero bucket. With
+ // negligible elapsed time, projected ≈ 0, deficit = 50, sleep = 1s.
+ got := state.reserveBudget(now, 50)
+ const want = time.Second
+ if got < want-50*time.Millisecond || got > want+50*time.Millisecond {
+ t.Fatalf("third reserveBudget after two debits must wait ~1s for the bucket to refill; got %s", got)
+ }
+}
+
+func TestThrottleConcurrentUpdateNoRace(t *testing.T) {
+ // This test is meaningful when run under -race; the bare exec also
+ // verifies the mutex shape is correct (no deadlock).
+ state := NewThrottleState()
+ var wg sync.WaitGroup
+ for i := 0; i < 50; i++ {
+ wg.Add(1)
+ go func(i int) {
+ defer wg.Done()
+ state.Update(ThrottleStatus{
+ MaximumAvailable: 1000,
+ CurrentlyAvailable: float64(i),
+ RestoreRate: 50,
+ })
+ _ = state.computeWait(time.Now(), 100)
+ _, _ = state.Status()
+ }(i)
+ }
+ wg.Wait()
+}
+
+func TestThrottleModeOffSkipsAwaitBudget(t *testing.T) {
+ // awaitBudget is the integration glue used by queryWithThrottle.
+ // Off-mode must short-circuit even when the bucket is exhausted, so
+ // the call returns immediately regardless of computeWait's verdict.
+ c := &Client{Throttle: NewThrottleState(), ThrottleMode: ThrottleModeOff}
+ c.Throttle.Update(ThrottleStatus{
+ MaximumAvailable: 1000,
+ CurrentlyAvailable: 0,
+ RestoreRate: 1,
+ })
+ c.Throttle.mu.Lock()
+ c.Throttle.status.LastUpdated = time.Now()
+ c.Throttle.mu.Unlock()
+
+ start := time.Now()
+ if err := c.awaitBudget(context.Background(), 999); err != nil {
+ t.Fatalf("off-mode awaitBudget should never error: %v", err)
+ }
+ if elapsed := time.Since(start); elapsed > 50*time.Millisecond {
+ t.Fatalf("off-mode awaitBudget should not block; took %s", elapsed)
+ }
+}
+
+func TestUpdateThrottleFromBodyParsesShopifyShape(t *testing.T) {
+ c := &Client{Throttle: NewThrottleState()}
+ body := []byte(` + "`" + `{"data":{},"extensions":{"cost":{"requestedQueryCost":10,"actualQueryCost":7,"throttleStatus":{"maximumAvailable":1000,"currentlyAvailable":993,"restoreRate":50}}}}` + "`" + `)
+ if !c.updateThrottleFromBody(body) {
+ t.Fatal("updateThrottleFromBody must consume Shopify's extensions.cost.throttleStatus shape")
+ }
+ status, seeded := c.Throttle.Status()
+ if !seeded {
+ t.Fatal("Throttle must be seeded after updateThrottleFromBody")
+ }
+ if status.MaximumAvailable != 1000 || status.CurrentlyAvailable != 993 || status.RestoreRate != 50 {
+ t.Fatalf("unexpected status %+v", status)
+ }
+}
+
+func TestUpdateThrottleFromBodyNoOpsWithoutCost(t *testing.T) {
+ // APIs that don't return extensions.cost (Linear, GitHub via the
+ // non-cost rateLimit node, vanilla GraphQL servers) must be a silent
+ // no-op. This is the defensive guard that lets ANY GraphQL spec opt
+ // in without breaking when extensions.cost isn't in the response.
+ c := &Client{Throttle: NewThrottleState()}
+ if c.updateThrottleFromBody([]byte(` + "`" + `{"data":{"viewer":{"id":"x"}}}` + "`" + `)) {
+ t.Fatal("updateThrottleFromBody must no-op when extensions.cost is missing")
+ }
+ if _, seeded := c.Throttle.Status(); seeded {
+ t.Fatal("Throttle must remain unseeded when extensions.cost is missing")
+ }
+}
+`
+ testPath := filepath.Join(outputDir, "internal", "client", "throttle_behavior_test.go")
+ require.NoError(t, os.WriteFile(testPath, []byte(behaviorTest), 0o644))
+
+ // First plain test pass to verify all paths.
+ runGoCommand(t, outputDir, "test", "./internal/client", "-run", "TestThrottle|TestUpdateThrottle")
+ // Then a -race pass focused on the concurrent test to prove the mutex
+ // is correct under the race detector. -race is opt-in elsewhere in
+ // this suite (PR-2 didn't run it); the mutex is the explicit V3 fix
+ // from V2's sync.Map shape, so we exercise it directly.
+ runGoCommand(t, outputDir, "test", "./internal/client", "-run", "TestThrottleConcurrentUpdateNoRace", "-race")
+}
diff --git a/internal/generator/generator.go b/internal/generator/generator.go
index e28e89b3..24966d34 100644
--- a/internal/generator/generator.go
+++ b/internal/generator/generator.go
@@ -1321,6 +1321,21 @@ func (g *Generator) renderOptionalSupportFiles() error {
}
}
+ // Specs that opt into cost-based throttling (Throttling.Enabled) get
+ // the ThrottleState primitives — bucket projection, cost-extension
+ // parser, retry helper. Gated by spec opt-in so existing GraphQL
+ // CLIs (Linear) and REST CLIs regenerate byte-identically. Lives
+ // under internal/client/ alongside graphql.go because the parser is
+ // graphql-shaped today; if a REST API ever exposes cost-bucket data
+ // in response headers this file would extend to read those, but the
+ // surface (ThrottleState, WaitForBudget, HandleThrottleError) is
+ // transport-agnostic.
+ if g.Spec.HasCostThrottling() {
+ if err := g.renderTemplate("throttle.go.tmpl", filepath.Join("internal", "client", "throttle.go"), g.Spec); err != nil {
+ return fmt.Errorf("rendering throttle helper: %w", err)
+ }
+ }
+
// Specs that declare per-tenant URL placeholders (e.g. Shopify's
// "{shop}" / "{version}") get a buildURL helper that resolves the
// {var} markers against env-backed Config.TemplateVars at request
diff --git a/internal/generator/templates/client.go.tmpl b/internal/generator/templates/client.go.tmpl
index e71d5a03..562f2e2f 100644
--- a/internal/generator/templates/client.go.tmpl
+++ b/internal/generator/templates/client.go.tmpl
@@ -39,6 +39,21 @@ type Client struct {
{{- if eq .Auth.Type "session_handshake"}}
Session *SessionManager
{{- end}}
+{{- if .HasCostThrottling}}
+ // Throttle tracks calculated-cost rate-limit budgets reported by the
+ // API in extensions.cost. Initialized in New() so callers don't need
+ // to nil-check; the !seeded guard inside ThrottleState handles the
+ // pre-first-response case. Distinct from `limiter` (cliutil's
+ // AdaptiveLimiter), which paces request count reactively from 429
+ // observation; this one is forward-projecting from the bucket the API
+ // itself reports. They compose: limiter caps RPS, Throttle gates on
+ // remaining query cost.
+ Throttle *ThrottleState
+ // ThrottleMode is one of ThrottleModeStrict (default; block on
+ // WaitForBudget), ThrottleModeLenient (warn but proceed), or
+ // ThrottleModeOff (bypass). Wired from the --throttle-mode root flag.
+ ThrottleMode string
+{{- end}}
}
{{if eq .ClientPattern "proxy-envelope"}}
@@ -125,6 +140,10 @@ func New(cfg *config.Config, timeout time.Duration, rateLimit float64) *Client {
cacheDir: cacheDir,
limiter: cliutil.NewAdaptiveLimiter(rateLimit),
Session: sess,
+{{- if .HasCostThrottling}}
+ Throttle: NewThrottleState(),
+ ThrottleMode: ThrottleModeStrict,
+{{- end}}
}
{{- else}}
httpClient := newHTTPClient(timeout, nil)
@@ -134,6 +153,10 @@ func New(cfg *config.Config, timeout time.Duration, rateLimit float64) *Client {
HTTPClient: httpClient,
cacheDir: cacheDir,
limiter: cliutil.NewAdaptiveLimiter(rateLimit),
+{{- if .HasCostThrottling}}
+ Throttle: NewThrottleState(),
+ ThrottleMode: ThrottleModeStrict,
+{{- end}}
}
{{- end}}
}
diff --git a/internal/generator/templates/graphql_client.go.tmpl b/internal/generator/templates/graphql_client.go.tmpl
index 22154063..b4ff6bea 100644
--- a/internal/generator/templates/graphql_client.go.tmpl
+++ b/internal/generator/templates/graphql_client.go.tmpl
@@ -4,9 +4,18 @@
package client
import (
+{{- if .HasCostThrottling}}
+ "context"
+{{- end}}
"encoding/json"
"fmt"
+{{- if .HasCostThrottling}}
+ "os"
+{{- end}}
"strings"
+{{- if .HasCostThrottling}}
+ "time"
+{{- end}}
)
// GraphQLRequest is a standard GraphQL POST body.
@@ -59,9 +68,106 @@ func isGraphQLAccessDenialCode(code string) bool {
// "/admin/api/{version}/graphql.json") work without editing the client.
const graphqlEndpointPath = "{{.GraphQLEndpointPath}}"
+{{- if .HasCostThrottling}}
+
+// defaultThrottleQueryCost is the cost we ask WaitForBudget to ensure
+// before issuing a GraphQL query. The real cost is whatever the API
+// computes from the query AST and reports back in extensions.cost; we
+// don't compute it locally. Asking for 1.0 means "ensure the bucket can
+// admit at least the cheapest possible query" — over-cost queries still
+// trigger a THROTTLED retry, but the project-forward math at least pauses
+// us when we've drained the bucket entirely.
+const defaultThrottleQueryCost = 1.0
+
+// updateThrottleFromBody is the per-shape parser the GraphQL client uses
+// to convert a raw response body into a ThrottleStatus update. Each shape
+// (selected via spec Throttling.Shape) provides its own implementation in
+// a gated block below. Spec validation rejects Enabled=true without a
+// recognized Shape, so callers can rely on this method always existing
+// when HasCostThrottling is true.
+{{- if eq (printf "%s" .Throttling.Shape) "shopify"}}
+
+// throttleExtensions is the parser shape for Shopify's
+// extensions.cost.throttleStatus payload. Defensive — when extensions.cost
+// is missing or malformed, Unmarshal silently leaves the zero value and
+// updateThrottleFromBody no-ops rather than seeding bad data.
+type throttleExtensions struct {
+ Cost struct {
+ ThrottleStatus struct {
+ MaximumAvailable float64 `json:"maximumAvailable"`
+ CurrentlyAvailable float64 `json:"currentlyAvailable"`
+ RestoreRate float64 `json:"restoreRate"`
+ } `json:"throttleStatus"`
+ } `json:"cost"`
+}
+
+func (c *Client) updateThrottleFromBody(body []byte) bool {
+ if c == nil || c.Throttle == nil || len(body) == 0 {
+ return false
+ }
+ var env struct {
+ Extensions throttleExtensions `json:"extensions"`
+ }
+ if err := json.Unmarshal(body, &env); err != nil {
+ return false
+ }
+ ts := env.Extensions.Cost.ThrottleStatus
+ if ts.MaximumAvailable == 0 && ts.CurrentlyAvailable == 0 && ts.RestoreRate == 0 {
+ return false
+ }
+ c.Throttle.Update(ThrottleStatus{
+ MaximumAvailable: ts.MaximumAvailable,
+ CurrentlyAvailable: ts.CurrentlyAvailable,
+ RestoreRate: ts.RestoreRate,
+ })
+ return true
+}
+{{- end}}
+
+// throttleModeOrDefault returns the configured ThrottleMode, falling back
+// to ThrottleModeStrict so the field's zero value still produces blocking
+// behavior. Off / lenient must be opted into explicitly.
+func (c *Client) throttleModeOrDefault() string {
+ switch c.ThrottleMode {
+ case ThrottleModeOff, ThrottleModeLenient, ThrottleModeStrict:
+ return c.ThrottleMode
+ default:
+ return ThrottleModeStrict
+ }
+}
+
+// awaitBudget consults the throttle mode and either blocks on
+// WaitForBudget (strict), warns to stderr without blocking (lenient), or
+// returns immediately (off / no throttle state). The graphql Query path
+// calls this before every Post; novel commands can call it directly when
+// composing multi-query workflows.
+func (c *Client) awaitBudget(ctx context.Context, cost float64) error {
+ if c == nil || c.Throttle == nil {
+ return nil
+ }
+ mode := c.throttleModeOrDefault()
+ if mode == ThrottleModeOff {
+ return nil
+ }
+ if mode == ThrottleModeLenient {
+ // Lenient: don't block, but surface the projected wait so users
+ // running scripts can spot a saturated bucket without staring at
+ // stalled CLI output.
+ if wait := c.Throttle.computeWait(time.Now(), cost); wait > 0 {
+ fmt.Fprintf(os.Stderr, "throttle: bucket low, would wait %s in strict mode\n", wait.Round(time.Millisecond))
+ }
+ return nil
+ }
+ return c.Throttle.WaitForBudget(ctx, cost)
+}
+{{- end}}
+
// Query executes a GraphQL query via POST against graphqlEndpointPath and
// returns the raw data payload.
func (c *Client) Query(query string, variables map[string]any) (json.RawMessage, error) {
+{{- if .HasCostThrottling}}
+ return c.queryWithThrottle(context.Background(), query, variables)
+{{- else}}
req := GraphQLRequest{Query: query, Variables: variables}
raw, _, err := c.Post(graphqlEndpointPath, req)
if err != nil {
@@ -91,7 +197,77 @@ func (c *Client) Query(query string, variables map[string]any) (json.RawMessage,
return nil, fmt.Errorf("graphql: %s", strings.Join(msgs, "; "))
}
return resp.Data, nil
+{{- end}}
+}
+
+{{- if .HasCostThrottling}}
+
+// queryWithThrottle is the throttle-aware Query path. Wraps the bare Post
+// call with WaitForBudget (mode-gated), parses extensions.cost from every
+// response (including error responses, since Shopify reports the bucket
+// even when the query was rejected), and retries on THROTTLED extension
+// codes via HandleThrottleError. HTTP 429s are still handled by the
+// Client.do() layer's existing AdaptiveLimiter retry — this path stacks
+// on top of that without double-handling.
+func (c *Client) queryWithThrottle(ctx context.Context, query string, variables map[string]any) (json.RawMessage, error) {
+ req := GraphQLRequest{Query: query, Variables: variables}
+ mode := c.throttleModeOrDefault()
+ for attempt := 0; ; attempt++ {
+ if err := c.awaitBudget(ctx, defaultThrottleQueryCost); err != nil {
+ return nil, err
+ }
+ raw, _, err := c.Post(graphqlEndpointPath, req)
+ if err != nil {
+ return nil, err
+ }
+ // Update throttle state from extensions.cost regardless of whether
+ // the response carried errors — Shopify includes the bucket even
+ // when reporting a THROTTLED error, and skipping the update would
+ // strand WaitForBudget on stale data after a throttle rejection.
+ c.updateThrottleFromBody(raw)
+
+ var resp GraphQLResponse
+ if err := json.Unmarshal(raw, &resp); err != nil {
+ return nil, fmt.Errorf("decoding graphql response: %w", err)
+ }
+ if len(resp.Errors) == 0 {
+ return resp.Data, nil
+ }
+
+ // Off-mode skips the THROTTLED retry path entirely so users who
+ // opt out of throttling don't pay an unexpected sleep. The error
+ // surfaces just like any other GraphQL error in that case.
+ if mode != ThrottleModeOff {
+ retry, wait := c.Throttle.HandleThrottleError(nil, raw, attempt)
+ if retry {
+ select {
+ case <-ctx.Done():
+ return nil, ctx.Err()
+ case <-time.After(wait):
+ }
+ continue
+ }
+ }
+
+ msgs := make([]string, len(resp.Errors))
+ codes := make([]string, 0, len(resp.Errors))
+ allDenial := true
+ for i, e := range resp.Errors {
+ msgs[i] = e.Message
+ if e.Extensions.Code != "" {
+ codes = append(codes, e.Extensions.Code)
+ }
+ if !isGraphQLAccessDenialCode(e.Extensions.Code) {
+ allDenial = false
+ }
+ }
+ if allDenial && len(codes) > 0 {
+ return nil, &GraphQLAccessDeniedError{Codes: codes, Messages: msgs}
+ }
+ return nil, fmt.Errorf("graphql: %s", strings.Join(msgs, "; "))
+ }
}
+{{- end}}
// Mutate executes a GraphQL mutation via POST against graphqlEndpointPath and
// returns the raw data payload. Identical transport to Query; separate method
diff --git a/internal/generator/templates/root.go.tmpl b/internal/generator/templates/root.go.tmpl
index 77000193..43601a49 100644
--- a/internal/generator/templates/root.go.tmpl
+++ b/internal/generator/templates/root.go.tmpl
@@ -39,6 +39,9 @@ type rootFlags struct {
rateLimit float64
dataSource string
freshnessMeta any
+{{- if .HasCostThrottling}}
+ throttleMode string
+{{- end}}
// deliverBuf captures command output when --deliver is set to a
// non-stdout sink. Flushed to the sink after Execute returns.
@@ -155,6 +158,15 @@ Run '{{.Name}}-pp-cli doctor' to verify auth and connectivity.{{backtick}},
{{- else}}
rootCmd.PersistentFlags().Float64Var(&flags.rateLimit, "rate-limit", 0, "Max requests per second (0 to disable)")
{{- end}}
+{{- if .HasCostThrottling}}
+ // --throttle-mode controls how the cost-based throttle reacts to a low
+ // budget. strict (default) blocks on WaitForBudget; lenient warns but
+ // proceeds (good for scripts where a momentary over-spend is preferable
+ // to an unexpected pause); off bypasses the throttle entirely. Default
+ // strict so first-time users see correct behavior; document the
+ // tradeoff loudly in --help so scripted callers know to opt out.
+ rootCmd.PersistentFlags().StringVar(&flags.throttleMode, "throttle-mode", "strict", "Cost-based throttle mode: strict (block when bucket is low), lenient (warn but proceed), off (bypass)")
+{{- end}}
rootCmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
if flags.deliverSpec != "" {
@@ -207,6 +219,14 @@ Run '{{.Name}}-pp-cli doctor' to verify auth and connectivity.{{backtick}},
default:
return fmt.Errorf("invalid --data-source value %q: must be auto, live, or local", flags.dataSource)
}
+{{- if .HasCostThrottling}}
+ switch flags.throttleMode {
+ case "strict", "lenient", "off":
+ // valid
+ default:
+ return fmt.Errorf("invalid --throttle-mode value %q: must be strict, lenient, or off", flags.throttleMode)
+ }
+{{- end}}
{{- if .Cache.Enabled}}
// Auto-refresh stale local caches before serving read commands.
// Looks up the current command path in readCommandResources and
@@ -293,6 +313,11 @@ func (f *rootFlags) newClient() (*client.Client, error) {
c := client.New(cfg, f.timeout, f.rateLimit)
c.DryRun = f.dryRun
c.NoCache = f.noCache
+{{- if .HasCostThrottling}}
+ if f.throttleMode != "" {
+ c.ThrottleMode = f.throttleMode
+ }
+{{- end}}
return c, nil
}
diff --git a/internal/generator/templates/throttle.go.tmpl b/internal/generator/templates/throttle.go.tmpl
new file mode 100644
index 00000000..ea154fe1
--- /dev/null
+++ b/internal/generator/templates/throttle.go.tmpl
@@ -0,0 +1,281 @@
+// Copyright {{currentYear}} {{.Owner}}. Licensed under Apache-2.0. See LICENSE.
+// Generated by CLI Printing Press (https://github.com/mvanhorn/cli-printing-press). DO NOT EDIT.
+
+package client
+
+import (
+ "context"
+ "encoding/json"
+ "math"
+ "math/rand"
+ "net/http"
+ "strconv"
+ "sync"
+ "time"
+)
+
+// Throttle modes selected by the --throttle-mode root flag. Strict is the
+// default: WaitForBudget actually blocks the caller until the projected
+// bucket rises above the requested cost. Lenient drops the wait but still
+// updates state from observed responses (useful for scripted runs where a
+// momentary over-spend is preferable to an unexpected multi-second pause).
+// Off bypasses the entire surface — no waits, no state updates, no retries
+// from HandleThrottleError — and is intended as an escape hatch when an
+// upstream API misreports its bucket and the throttle would otherwise
+// stall the CLI.
+const (
+ ThrottleModeStrict = "strict"
+ ThrottleModeLenient = "lenient"
+ ThrottleModeOff = "off"
+)
+
+// MaxThrottleRetries caps HandleThrottleError's retry decision. The fifth
+// attempt fails through to the caller — at that point the bucket math is
+// either wrong or the API is sustaining a deeper outage than client-side
+// retries can paper over.
+const MaxThrottleRetries = 5
+
+// ThrottleStatus mirrors the cost bucket the API reports back in response
+// extensions. Today the parser populates it from Shopify's
+// extensions.cost.throttleStatus shape; APIs that expose the same data
+// under a different path are a follow-up (the parser no-ops on absence,
+// so unrelated GraphQL APIs are safe to opt in).
+type ThrottleStatus struct {
+ MaximumAvailable float64
+ CurrentlyAvailable float64
+ RestoreRate float64
+ LastUpdated time.Time
+}
+
+// ThrottleState tracks the most recent ThrottleStatus reported by the API
+// and projects the bucket forward by elapsed wall-clock time. Safe for
+// concurrent use — the GraphQL client may issue multiple paginated queries
+// in flight, and Update / WaitForBudget can race during fanout.
+type ThrottleState struct {
+ mu sync.Mutex
+ status ThrottleStatus
+ seeded bool // first call has no prior status; WaitForBudget returns immediately until the API reports back
+}
+
+// NewThrottleState returns a fresh, unseeded throttle state. The first
+// WaitForBudget call before any Update returns immediately so the CLI can
+// make its first request and learn its budget from the response.
+func NewThrottleState() *ThrottleState {
+ return &ThrottleState{}
+}
+
+// WaitForBudget blocks until the projected bucket can admit
+// nextRequestedCost, then debits the cost from the local bucket so
+// concurrent callers see the post-debit balance instead of all reading
+// the same projected value and racing through. Returns immediately when
+// the state has not been seeded by a prior response (no bucket data to
+// project), when the projected balance already covers the cost (debited
+// in place), or when the API reports a non-positive restore rate (no way
+// to wait it out — fail forward and let HandleThrottleError handle the
+// inevitable 429).
+//
+// Reservation drift: the local bucket may briefly diverge from the API's
+// real bucket between the debit and the next response. The next Update()
+// resyncs from the authoritative status, so drift is bounded to one
+// in-flight request per goroutine. Better to over-throttle locally than
+// double-spend a shared budget.
+func (t *ThrottleState) WaitForBudget(ctx context.Context, nextRequestedCost float64) error {
+ sleep := t.reserveBudget(time.Now(), nextRequestedCost)
+ if sleep <= 0 {
+ return nil
+ }
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ case <-time.After(sleep):
+ return nil
+ }
+}
+
+// computeWait returns the time WaitForBudget would block, given the
+// observed status. Pure projection — does not mutate the bucket. Lenient
+// mode uses this to surface the projected wait without reserving budget;
+// tests use it to verify the math without sleeping. Reservation lives in
+// reserveBudget so the two concerns stay distinct.
+func (t *ThrottleState) computeWait(now time.Time, requested float64) time.Duration {
+ t.mu.Lock()
+ defer t.mu.Unlock()
+ return t.computeWaitLocked(now, requested)
+}
+
+// computeWaitLocked is the lock-held variant of computeWait. Internal
+// helper so reserveBudget can compute and mutate inside one critical
+// section.
+func (t *ThrottleState) computeWaitLocked(now time.Time, requested float64) time.Duration {
+ if !t.seeded {
+ return 0
+ }
+ s := t.status
+ elapsed := now.Sub(s.LastUpdated).Seconds()
+ if elapsed < 0 {
+ elapsed = 0
+ }
+ projected := math.Min(s.MaximumAvailable, s.CurrentlyAvailable+elapsed*s.RestoreRate)
+ if projected >= requested || s.RestoreRate <= 0 {
+ return 0
+ }
+ deficit := requested - projected
+ return time.Duration((deficit / s.RestoreRate) * float64(time.Second))
+}
+
+// reserveBudget atomically computes the wait duration AND debits the
+// requested cost from the local bucket so concurrent callers don't all
+// read the same projected balance and race through. Returns the duration
+// the caller must sleep before issuing the request; 0 means proceed
+// immediately (the cost has been debited).
+//
+// No-wait path: project the bucket forward, deduct the request, stamp
+// LastUpdated to now. The next caller projects from the debited state.
+//
+// Wait path: assume the caller actually waits, so by the time it wakes
+// the bucket will have refilled to exactly `requested` and we debit it
+// to zero. LastUpdated advances to the wake time so the next concurrent
+// caller's projection starts from the cleared bucket.
+//
+// !seeded short-circuits to 0 with no debit so the first request can
+// learn its budget from the response. Same for non-positive restore
+// rate — there's no way to wait it out, so admit and rely on
+// HandleThrottleError for the inevitable 429.
+func (t *ThrottleState) reserveBudget(now time.Time, requested float64) time.Duration {
+ t.mu.Lock()
+ defer t.mu.Unlock()
+ if !t.seeded {
+ return 0
+ }
+ if t.status.RestoreRate <= 0 {
+ return 0
+ }
+ elapsed := now.Sub(t.status.LastUpdated).Seconds()
+ if elapsed < 0 {
+ elapsed = 0
+ }
+ projected := math.Min(t.status.MaximumAvailable, t.status.CurrentlyAvailable+elapsed*t.status.RestoreRate)
+ if projected >= requested {
+ t.status.CurrentlyAvailable = math.Max(0, projected-requested)
+ t.status.LastUpdated = now
+ return 0
+ }
+ deficit := requested - projected
+ sleep := time.Duration((deficit / t.status.RestoreRate) * float64(time.Second))
+ t.status.CurrentlyAvailable = 0
+ t.status.LastUpdated = now.Add(sleep)
+ return sleep
+}
+
+// Update is called by the GraphQL client after every response that carries
+// a parseable throttle status. LastUpdated is stamped here (callers don't
+// need to set it) so the projection in computeWait always uses the time
+// the CLI observed the snapshot.
+func (t *ThrottleState) Update(s ThrottleStatus) {
+ t.mu.Lock()
+ defer t.mu.Unlock()
+ s.LastUpdated = time.Now()
+ t.status = s
+ t.seeded = true
+}
+
+// Status returns a snapshot of the current throttle status. Useful for
+// observability surfaces (doctor, debug logs) that want to render the
+// last-seen budget without racing on the internal mutex.
+func (t *ThrottleState) Status() (ThrottleStatus, bool) {
+ t.mu.Lock()
+ defer t.mu.Unlock()
+ return t.status, t.seeded
+}
+
+// HandleThrottleError detects throttle signals on a completed HTTP exchange
+// and returns whether the caller should retry, plus the suggested wait
+// before the next attempt. Caps retries at MaxThrottleRetries; the caller
+// is expected to track its own attempt counter and pass it in.
+//
+// Two signals trigger a retry:
+// - HTTP 429 (any GraphQL or REST shape — the status is the authoritative
+// "back off"). When a Retry-After header is present, that wait wins;
+// otherwise we fall back to exponential backoff with jitter.
+// - GraphQL response body containing extensions.code == "THROTTLED" on
+// any error entry. Shopify and several other GraphQL-cost APIs return
+// 200 with a THROTTLED error rather than 429 when the cost exceeds the
+// bucket; handling only 429 would miss them.
+//
+// resp may be nil when only the body is available (e.g., the caller already
+// closed the response). body may be nil when only the status is interesting.
+// Either signal alone is enough.
+func (t *ThrottleState) HandleThrottleError(resp *http.Response, body []byte, attempt int) (retry bool, wait time.Duration) {
+ if attempt >= MaxThrottleRetries {
+ return false, 0
+ }
+ if resp != nil && resp.StatusCode == http.StatusTooManyRequests {
+ w := parseRetryAfter(resp)
+ if w == 0 {
+ w = throttleBackoff(attempt)
+ }
+ return true, w + throttleJitter()
+ }
+ if len(body) > 0 {
+ var ge struct {
+ Errors []struct {
+ Extensions struct {
+ Code string `json:"code"`
+ } `json:"extensions"`
+ } `json:"errors"`
+ }
+ if err := json.Unmarshal(body, &ge); err == nil {
+ for _, e := range ge.Errors {
+ if e.Extensions.Code == "THROTTLED" {
+ return true, throttleBackoff(attempt) + throttleJitter()
+ }
+ }
+ }
+ }
+ return false, 0
+}
+
+// parseRetryAfter reads RFC 7231 Retry-After in either delta-seconds or
+// HTTP-date form. Returns 0 when the header is missing, malformed, or
+// expressed as a date in the past — callers are expected to fall through
+// to exponential backoff in that case.
+func parseRetryAfter(resp *http.Response) time.Duration {
+ if resp == nil {
+ return 0
+ }
+ h := resp.Header.Get("Retry-After")
+ if h == "" {
+ return 0
+ }
+ if secs, err := strconv.Atoi(h); err == nil {
+ if secs <= 0 {
+ return 0
+ }
+ return time.Duration(secs) * time.Second
+ }
+ if when, err := http.ParseTime(h); err == nil {
+ d := time.Until(when)
+ if d <= 0 {
+ return 0
+ }
+ return d
+ }
+ return 0
+}
+
+// throttleBackoff is the deterministic part of the retry wait: 1s, 2s, 4s,
+// 8s, 16s for attempts 0..4. Renamed from "backoff" to avoid colliding with
+// any helper a hand-authored novel command might define in the same package.
+func throttleBackoff(attempt int) time.Duration {
+ if attempt < 0 {
+ attempt = 0
+ }
+ return time.Duration(math.Pow(2, float64(attempt))) * time.Second
+}
+
+// throttleJitter returns a [0, 500ms) randomized addend on top of the
+// deterministic backoff. Prevents a fleet of CLIs from synchronizing on
+// the same retry tick after a shared 429.
+func throttleJitter() time.Duration {
+ return time.Duration(rand.Intn(500)) * time.Millisecond
+}
diff --git a/internal/spec/spec.go b/internal/spec/spec.go
index e6d4c9e1..2afb9b6c 100644
--- a/internal/spec/spec.go
+++ b/internal/spec/spec.go
@@ -105,6 +105,65 @@ type APISpec struct {
Cache CacheConfig `yaml:"cache,omitempty" json:"cache"` // cache freshness + auto-refresh config; when enabled, generated read commands auto-refresh stale local data before serving
Share ShareConfig `yaml:"share,omitempty" json:"share"` // git-backed snapshot sharing config; when enabled, emits a `share` subcommand that publishes/subscribes to a git repo
MCP MCPConfig `yaml:"mcp,omitempty" json:"mcp"` // MCP server generation config; when unset, the emitted MCP binary is stdio-only (today's default). Opting into http adds a --transport/--addr flag surface so the same binary can serve cloud-hosted agents.
+ Throttling ThrottlingConfig `yaml:"throttling,omitempty" json:"throttling"` // cost-based throttling config; when Enabled with a recognized Shape, the generator emits a ThrottleState (generic harness) plus a per-Shape parser that reads the API's cost bucket. Only the "shopify" Shape ships in v1.
+}
+
+// ThrottleShape names the API-specific cost-bucket parser the generator
+// wires into the GraphQL client. The generic harness (bucket math, retry,
+// --throttle-mode flag) is shape-agnostic; only the parser that reads the
+// API's response into a ThrottleStatus differs per shape, because every
+// API surfaces its calculated cost in a different place. Adding a new
+// shape means: (1) add a constant here, (2) extend validateThrottling to
+// accept it, (3) add the parser block to graphql_client.go.tmpl gated on
+// `eq .Throttling.Shape "<name>"`. No core code changes.
+type ThrottleShape string
+
+const (
+ // ThrottleShapeShopify reads `extensions.cost.throttleStatus.{maximumAvailable,
+ // currentlyAvailable,restoreRate}` from each GraphQL response. This is the
+ // only shape supported in v1; GitHub's queryable `rateLimit` field and
+ // Datadog's header-based cost limits will need their own shapes (and the
+ // GitHub case will need a query-rewrite layer, since rateLimit is a schema
+ // field rather than a response extension).
+ ThrottleShapeShopify ThrottleShape = "shopify"
+)
+
+// ThrottlingConfig opts a printed CLI into the cost-based throttling
+// primitives. Enabled turns the surface on (--throttle-mode flag,
+// ThrottleState, budget projection, retry helper); default off so existing
+// CLIs regenerate byte-identically when this field is unset. Shape selects
+// the per-API parser and is required when Enabled is true; see ThrottleShape
+// for the valid values and how to add new ones.
+//
+// Authors opt in by writing `throttling: { enabled: true, shape: shopify }`.
+type ThrottlingConfig struct {
+ Enabled bool `yaml:"enabled,omitempty" json:"enabled,omitempty"`
+ Shape ThrottleShape `yaml:"shape,omitempty" json:"shape,omitempty"`
+}
+
+// validateThrottling ensures Shape is set and recognized when Enabled is
+// true. Off case returns nil so unrelated specs aren't penalized for never
+// opting in.
+func validateThrottling(c ThrottlingConfig) error {
+ if !c.Enabled {
+ return nil
+ }
+ switch c.Shape {
+ case ThrottleShapeShopify:
+ return nil
+ case "":
+ return fmt.Errorf("throttling.shape is required when throttling.enabled is true (valid: %q)", ThrottleShapeShopify)
+ default:
+ return fmt.Errorf("throttling.shape %q is not recognized (valid: %q)", c.Shape, ThrottleShapeShopify)
+ }
+}
+
+// HasCostThrottling reports whether the spec opts into cost-based throttling
+// primitives. Used by the generator to gate emission of throttle.go and the
+// related conditional blocks in client.go / graphql_client.go / root.go.
+// Specs without this flag regenerate byte-identical to the pre-PR-3 output.
+func (s *APISpec) HasCostThrottling() bool {
+ return s != nil && s.Throttling.Enabled
}
// ExtraCommand declares a hand-written cobra command so the SKILL.md
@@ -1017,6 +1076,9 @@ func (s *APISpec) Validate() error {
if err := validateMCP(s.MCP, s.Resources); err != nil {
return err
}
+ if err := validateThrottling(s.Throttling); err != nil {
+ return err
+ }
if s.ClientPattern == "proxy-envelope" && s.HasResourceBaseURLOverride() {
return fmt.Errorf("resource base_url overrides are incompatible with client_pattern=proxy-envelope; the proxy POSTs every request to the spec-level BaseURL, so per-resource overrides would be silently ignored")
}
diff --git a/internal/spec/spec_test.go b/internal/spec/spec_test.go
index d0d04362..137fabf9 100644
--- a/internal/spec/spec_test.go
+++ b/internal/spec/spec_test.go
@@ -87,6 +87,63 @@ func TestValidation(t *testing.T) {
}
}
+// TestThrottleShapeShopifyValue pins the wire value of ThrottleShapeShopify
+// to "shopify" because the graphql_client.go.tmpl template gates its
+// Shopify parser block on the literal string "shopify" (Go templates can't
+// import the constant). A silent rename of the Go constant would leave the
+// template gate stranded; this test surfaces the mismatch immediately.
+func TestThrottleShapeShopifyValue(t *testing.T) {
+ assert.Equal(t, "shopify", string(ThrottleShapeShopify),
+ "changing this value requires updating graphql_client.go.tmpl's gate to match")
+}
+
+// TestThrottlingValidate guards the named-adapter contract: enabling
+// throttling without a Shape (or with an unrecognized one) must fail at
+// spec-load time, not silently emit Shopify-shape parser code for an API
+// that isn't Shopify. The off case stays unconditional — specs that don't
+// use throttling never have to think about Shape.
+func TestThrottlingValidate(t *testing.T) {
+ tests := []struct {
+ name string
+ cfg ThrottlingConfig
+ wantErr string
+ }{
+ {
+ name: "off is always valid",
+ cfg: ThrottlingConfig{},
+ },
+ {
+ name: "off with stray shape is still valid",
+ cfg: ThrottlingConfig{Shape: ThrottleShapeShopify},
+ },
+ {
+ name: "shopify shape is valid when enabled",
+ cfg: ThrottlingConfig{Enabled: true, Shape: ThrottleShapeShopify},
+ },
+ {
+ name: "enabled without shape is rejected",
+ cfg: ThrottlingConfig{Enabled: true},
+ wantErr: "throttling.shape is required",
+ },
+ {
+ name: "unknown shape is rejected",
+ cfg: ThrottlingConfig{Enabled: true, Shape: "github-graphql"},
+ wantErr: "not recognized",
+ },
+ }
+ for _, tt := range tests {
+ t.Run(tt.name, func(t *testing.T) {
+ err := validateThrottling(tt.cfg)
+ if tt.wantErr == "" {
+ require.NoError(t, err)
+ return
+ }
+ require.Error(t, err)
+ assert.Contains(t, err.Error(), tt.wantErr)
+ })
+ }
+}
+
func TestVersionPassedThrough(t *testing.T) {
base := func(v string) APISpec {
return APISpec{
← 2c59d1c6 refactor(cli): extract body-map block to a shared helper (#4
·
back to Cli Printing Press
·
chore(main): release 3.1.0 (#419) 052187a7 →