← back to Cli Printing Press
feat(cli): add --dates sync flag and wrapper-object list detection (#154)
49148b437ddc577786549c413fe62bced135b43f · 2026-04-09 13:50:39 -0700 · Trevin Chow
* fix(skills): require retro findings to strip API-specific details from machine fixes
The retro skill's Phase 3 "durable fix" question lacked guidance on separating
general machine improvements from printed-CLI-specific implementation details.
This caused the ESPN retro to propose --sport/--league flags and monthly chunking
as machine template changes, when those are ESPN-specific behaviors that belong
in Phase 4 printed-CLI work.
Adds explicit instruction with a concrete before/after example showing the
anti-pattern: hardcoded param names, date formats, and chunking strategies
leaking from one API's retro into a machine recommendation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): add DateRangeParam detection and wrapper-object list endpoint heuristic
Profiler now detects date-range query params (dates, date_range, dateRange)
and exposes them as DateRangeParam on PaginationProfile. Also improves
isListEndpoint to recognize GET endpoints returning wrapper objects with
known array keys (data, results, items, events, entries, records, nodes).
These changes enable the sync template to emit --dates flags and detect
more syncable resources across APIs that use wrapper response patterns.
* feat(cli): emit --dates flag in sync template when DateRangeParam detected
The sync.go template now conditionally emits a --dates flag when the
profiler detects a date-range query parameter (dates, date_range,
dateRange). The flag passes the value directly to the API as the
detected param name — no chunking or format assumptions.
Gated entirely on {{if .Pagination.DateRangeParam}} so APIs without
date-range params get no extra flags. Verified: ESPN gets --dates,
Stytch does not.
* test(cli): add profiler tests for DateRangeParam and wrapper-object detection
Four new test cases:
- TestProfileDateRangeParam: plural 'dates' param detected
- TestProfileDateRangeParam_SingularDateNotMatched: singular 'date' excluded
- TestProfileWrapperObjectDetection: wrapper object with 'events' field is syncable
- TestProfileWrapperObjectDetection_NoFalsePositive: plain object not syncable
* fix(cli): remove unused resourceLimitExplicit var blocking lint
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Files touched
M internal/generator/templates/sync.go.tmplM internal/openapi/parser.goM internal/profiler/profiler.goM internal/profiler/profiler_test.goM skills/printing-press-retro/SKILL.md
Diff
commit 49148b437ddc577786549c413fe62bced135b43f
Author: Trevin Chow <trevin@trevinchow.com>
Date: Thu Apr 9 13:50:39 2026 -0700
feat(cli): add --dates sync flag and wrapper-object list detection (#154)
* fix(skills): require retro findings to strip API-specific details from machine fixes
The retro skill's Phase 3 "durable fix" question lacked guidance on separating
general machine improvements from printed-CLI-specific implementation details.
This caused the ESPN retro to propose --sport/--league flags and monthly chunking
as machine template changes, when those are ESPN-specific behaviors that belong
in Phase 4 printed-CLI work.
Adds explicit instruction with a concrete before/after example showing the
anti-pattern: hardcoded param names, date formats, and chunking strategies
leaking from one API's retro into a machine recommendation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(cli): add DateRangeParam detection and wrapper-object list endpoint heuristic
Profiler now detects date-range query params (dates, date_range, dateRange)
and exposes them as DateRangeParam on PaginationProfile. Also improves
isListEndpoint to recognize GET endpoints returning wrapper objects with
known array keys (data, results, items, events, entries, records, nodes).
These changes enable the sync template to emit --dates flags and detect
more syncable resources across APIs that use wrapper response patterns.
* feat(cli): emit --dates flag in sync template when DateRangeParam detected
The sync.go template now conditionally emits a --dates flag when the
profiler detects a date-range query parameter (dates, date_range,
dateRange). The flag passes the value directly to the API as the
detected param name — no chunking or format assumptions.
Gated entirely on {{if .Pagination.DateRangeParam}} so APIs without
date-range params get no extra flags. Verified: ESPN gets --dates,
Stytch does not.
* test(cli): add profiler tests for DateRangeParam and wrapper-object detection
Four new test cases:
- TestProfileDateRangeParam: plural 'dates' param detected
- TestProfileDateRangeParam_SingularDateNotMatched: singular 'date' excluded
- TestProfileWrapperObjectDetection: wrapper object with 'events' field is syncable
- TestProfileWrapperObjectDetection_NoFalsePositive: plain object not syncable
* fix(cli): remove unused resourceLimitExplicit var blocking lint
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---
internal/generator/templates/sync.go.tmpl | 35 ++++++++-
internal/openapi/parser.go | 2 -
internal/profiler/profiler.go | 55 +++++++++++++-
internal/profiler/profiler_test.go | 117 ++++++++++++++++++++++++++++++
skills/printing-press-retro/SKILL.md | 15 ++++
5 files changed, 217 insertions(+), 7 deletions(-)
diff --git a/internal/generator/templates/sync.go.tmpl b/internal/generator/templates/sync.go.tmpl
index e62ac85e..35d3d397 100644
--- a/internal/generator/templates/sync.go.tmpl
+++ b/internal/generator/templates/sync.go.tmpl
@@ -32,6 +32,9 @@ func newSyncCmd(flags *rootFlags) *cobra.Command {
var since string
var concurrency int
var dbPath string
+{{- if .Pagination.DateRangeParam}}
+ var dates string
+{{- end}}
cmd := &cobra.Command{
Use: "sync",
@@ -52,7 +55,11 @@ Once synced, use the 'search' command for instant full-text search.`,
{{.Name}}-pp-cli sync --since 7d
# Parallel sync with 8 workers
- {{.Name}}-pp-cli sync --concurrency 8`,
+ {{.Name}}-pp-cli sync --concurrency 8
+{{- if .Pagination.DateRangeParam}}
+
+ # Sync historical data for a date range
+ {{.Name}}-pp-cli sync --dates 20250101-20250131{{end}}`,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := flags.newClient()
if err != nil {
@@ -92,6 +99,11 @@ Once synced, use the 'search' command for instant full-text search.`,
sinceTS = ts.Format(time.RFC3339)
}
+{{- if .Pagination.DateRangeParam}}
+ // If --dates is provided, set it as extra sync context
+ syncDates := dates
+{{- end}}
+
// Worker pool: produce resources, N workers consume
if concurrency < 1 {
concurrency = 4
@@ -107,7 +119,7 @@ Once synced, use the 'search' command for instant full-text search.`,
go func() {
defer wg.Done()
for resource := range work {
- res := syncResource(c, db, resource, sinceTS, full)
+ res := syncResource(c, db, resource, sinceTS, full{{if .Pagination.DateRangeParam}}, syncDates{{end}})
results <- res
}
}()
@@ -157,6 +169,9 @@ Once synced, use the 'search' command for instant full-text search.`,
cmd.Flags().StringVar(&since, "since", "", "Incremental sync duration (e.g. 7d, 24h, 1w, 30m)")
cmd.Flags().IntVar(&concurrency, "concurrency", 4, "Number of parallel sync workers")
cmd.Flags().StringVar(&dbPath, "db", "", "Database path (default: ~/.local/share/{{.Name}}-pp-cli/data.db)")
+{{- if .Pagination.DateRangeParam}}
+ cmd.Flags().StringVar(&dates, "dates", "", "Date or date range to sync (passed as {{.Pagination.DateRangeParam}} query parameter)")
+{{- end}}
return cmd
}
@@ -166,7 +181,7 @@ Once synced, use the 'search' command for instant full-text search.`,
func syncResource(c interface {
Get(string, map[string]string) (json.RawMessage, error)
RateLimit() float64
-}, db *store.Store, resource, sinceTS string, full bool) syncResult {
+}, db *store.Store, resource, sinceTS string, full bool{{if .Pagination.DateRangeParam}}, dates string{{end}}) syncResult {
started := time.Now()
if !humanFriendly {
@@ -208,6 +223,13 @@ func syncResource(c interface {
if effectiveSince != "" {
params[sinceParam] = effectiveSince
}
+{{- if .Pagination.DateRangeParam}}
+
+ // Set date range filter (pass-through to API)
+ if dates != "" {
+ params[determineDateRangeParam()] = dates
+ }
+{{- end}}
data, err := c.Get(path, params)
if err != nil {
@@ -305,6 +327,13 @@ func determinePaginationDefaults() paginationDefaults {
func determineSinceParam() string {
return "{{if .Pagination.SinceParam}}{{.Pagination.SinceParam}}{{else}}since{{end}}"
}
+{{- if .Pagination.DateRangeParam}}
+
+// determineDateRangeParam returns the query parameter name for date-range sync filtering.
+func determineDateRangeParam() string {
+ return "{{.Pagination.DateRangeParam}}"
+}
+{{- end}}
// extractPageItems attempts to extract an array of items and pagination cursor from a response.
// It tries multiple strategies:
diff --git a/internal/openapi/parser.go b/internal/openapi/parser.go
index 6c0ed334..66aa9743 100644
--- a/internal/openapi/parser.go
+++ b/internal/openapi/parser.go
@@ -22,7 +22,6 @@ var (
maxResources = 500
maxEndpointsPerResource = 50
endpointLimitExplicit = false // true when user set --max-endpoints-per-resource
- resourceLimitExplicit = false // true when user set --max-resources
)
// SetMaxResources overrides the default resource limit. When not called,
@@ -30,7 +29,6 @@ var (
func SetMaxResources(n int) {
if n > 0 {
maxResources = n
- resourceLimitExplicit = true
}
}
diff --git a/internal/profiler/profiler.go b/internal/profiler/profiler.go
index ba5b2819..0f318a7c 100644
--- a/internal/profiler/profiler.go
+++ b/internal/profiler/profiler.go
@@ -40,6 +40,7 @@ type PaginationProfile struct {
CursorParam string `json:"cursor_param"` // most common cursor param name (after, cursor, page_token, offset)
PageSizeParam string `json:"page_size_param"` // most common page size param (limit, per_page, page_size, first)
SinceParam string `json:"since_param"` // temporal filter param (since, updated_after, modified_since)
+ DateRangeParam string `json:"date_range_param"` // date-range filter param (dates, date_range, dateRange)
ItemsKey string `json:"items_key"` // response array key (data, results, items, or "" for root array)
DefaultPageSize int `json:"default_page_size"` // detected or default 100
}
@@ -117,6 +118,7 @@ func Profile(s *spec.APISpec) *APIProfile {
cursorParams := make(map[string]int)
pageSizeParams := make(map[string]int)
sinceParams := make(map[string]int)
+ dateRangeParams := make(map[string]int)
responsePaths := make(map[string]int)
var walk func(name string, r spec.Resource)
@@ -234,7 +236,7 @@ func Profile(s *spec.APISpec) *APIProfile {
p.HasChronological = true
}
- if isListEndpoint(endpointName, endpoint) {
+ if isListEndpoint(endpointName, endpoint, s.Types) {
listCapableGETs++
listResources[resourceName] = struct{}{}
@@ -291,6 +293,9 @@ func Profile(s *spec.APISpec) *APIProfile {
if strings.Contains(name, "since") || strings.Contains(name, "updated_after") || strings.Contains(name, "modified_since") || strings.Contains(name, "updated_at") {
sinceParams[param.Name]++
}
+ if name == "dates" || name == "date_range" || name == "daterange" {
+ dateRangeParams[param.Name]++
+ }
}
if len(endpoint.Body) > 10 {
@@ -362,6 +367,7 @@ func Profile(s *spec.APISpec) *APIProfile {
CursorParam: mostCommon(cursorParams, "after"),
PageSizeParam: mostCommon(pageSizeParams, "limit"),
SinceParam: mostCommon(sinceParams, ""),
+ DateRangeParam: mostCommon(dateRangeParams, ""),
ItemsKey: mostCommon(responsePaths, ""),
DefaultPageSize: 100,
}
@@ -572,7 +578,7 @@ func hasRequiredScopeParams(endpoint spec.Endpoint) bool {
return false
}
-func isListEndpoint(name string, endpoint spec.Endpoint) bool {
+func isListEndpoint(name string, endpoint spec.Endpoint, types map[string]spec.TypeDef) bool {
if strings.ToUpper(endpoint.Method) != "GET" {
return false
}
@@ -583,10 +589,55 @@ func isListEndpoint(name string, endpoint spec.Endpoint) bool {
return true
}
+ // Check for wrapper-object responses: the endpoint returns type "object"
+ // and the referenced type has a field matching a known wrapper key. These
+ // are list endpoints that wrap their arrays (e.g., {events: [...]}).
+ // The key list matches extractPageItems in sync.go.tmpl plus "events".
+ if endpoint.Response.Type == "object" && endpoint.Response.Item != "" {
+ if hasWrapperArrayField(endpoint.Response.Item, types) {
+ return true
+ }
+ }
+
name = strings.ToLower(name)
return containsAny(name, []string{"list", "all"})
}
+// wrapperArrayKeys are response object field names that indicate the object
+// wraps a list of items. Kept in sync with extractPageItems in sync.go.tmpl.
+var wrapperArrayKeys = map[string]bool{
+ "data": true,
+ "results": true,
+ "items": true,
+ "events": true,
+ "entries": true,
+ "records": true,
+ "nodes": true,
+}
+
+// hasWrapperArrayField checks whether a named type in the spec's types map
+// has any field whose name matches a known wrapper key, or whether the type
+// name itself suggests a list wrapper (contains "Response", "List", "Result",
+// or "Collection"). The type-name heuristic is a fallback for specs where the
+// types map is empty or incomplete.
+func hasWrapperArrayField(typeName string, types map[string]spec.TypeDef) bool {
+ if typeDef, ok := types[typeName]; ok {
+ for _, field := range typeDef.Fields {
+ if wrapperArrayKeys[strings.ToLower(field.Name)] {
+ return true
+ }
+ }
+ }
+
+ // Fallback: if the type name itself suggests a list wrapper, treat it
+ // as a wrapper even when the types map lacks field definitions.
+ nameUpper := strings.ToUpper(typeName)
+ return strings.Contains(nameUpper, "RESPONSE") ||
+ strings.Contains(nameUpper, "LIST") ||
+ strings.Contains(nameUpper, "RESULT") ||
+ strings.Contains(nameUpper, "COLLECTION")
+}
+
// findEntityTypeEnum returns the first required enum query param on a list endpoint
// that looks like an entity type selector. Heuristics:
// 1. Param is required with 2+ enum values
diff --git a/internal/profiler/profiler_test.go b/internal/profiler/profiler_test.go
index bbcc878a..2a00f0e2 100644
--- a/internal/profiler/profiler_test.go
+++ b/internal/profiler/profiler_test.go
@@ -461,3 +461,120 @@ func discordSpec() *spec.APISpec {
},
}
}
+
+func TestProfileDateRangeParam(t *testing.T) {
+ s := &spec.APISpec{
+ Name: "sportsdata",
+ Resources: map[string]spec.Resource{
+ "scoreboard": {
+ Endpoints: map[string]spec.Endpoint{
+ "get": {
+ Method: "GET",
+ Path: "/scoreboard",
+ Params: []spec.Param{
+ {Name: "dates", Type: "string", Description: "Date range YYYYMMDD-YYYYMMDD"},
+ {Name: "limit", Type: "int", Default: 100},
+ },
+ Response: spec.ResponseDef{Type: "object", Item: "ScoreboardResponse"},
+ },
+ },
+ },
+ },
+ Types: map[string]spec.TypeDef{
+ "ScoreboardResponse": {
+ Fields: []spec.TypeField{
+ {Name: "events", Type: "string"},
+ {Name: "leagues", Type: "string"},
+ },
+ },
+ },
+ }
+
+ profile := Profile(s)
+ assert.Equal(t, "dates", profile.Pagination.DateRangeParam)
+}
+
+func TestProfileDateRangeParam_SingularDateNotMatched(t *testing.T) {
+ s := &spec.APISpec{
+ Name: "calendar",
+ Resources: map[string]spec.Resource{
+ "events": {
+ Endpoints: map[string]spec.Endpoint{
+ "list": {
+ Method: "GET",
+ Path: "/events",
+ Params: []spec.Param{{Name: "date", Type: "string"}, {Name: "limit", Type: "int"}},
+ Response: spec.ResponseDef{Type: "array"},
+ },
+ },
+ },
+ },
+ }
+
+ profile := Profile(s)
+ assert.Empty(t, profile.Pagination.DateRangeParam, "singular 'date' must not match DateRangeParam")
+}
+
+func TestProfileWrapperObjectDetection(t *testing.T) {
+ s := &spec.APISpec{
+ Name: "sportsdata",
+ Resources: map[string]spec.Resource{
+ "scoreboard": {
+ Endpoints: map[string]spec.Endpoint{
+ "get": {
+ Method: "GET",
+ Path: "/scoreboard",
+ Response: spec.ResponseDef{Type: "object", Item: "ScoreboardResponse"},
+ },
+ },
+ },
+ },
+ Types: map[string]spec.TypeDef{
+ "ScoreboardResponse": {
+ Fields: []spec.TypeField{
+ {Name: "events", Type: "string"},
+ {Name: "leagues", Type: "string"},
+ },
+ },
+ },
+ }
+
+ profile := Profile(s)
+ syncNames := make([]string, len(profile.SyncableResources))
+ for i, sr := range profile.SyncableResources {
+ syncNames[i] = sr.Name
+ }
+ assert.Contains(t, syncNames, "scoreboard", "wrapper-object endpoint with 'events' field should be syncable")
+}
+
+func TestProfileWrapperObjectDetection_NoFalsePositive(t *testing.T) {
+ s := &spec.APISpec{
+ Name: "config",
+ Resources: map[string]spec.Resource{
+ "settings": {
+ Endpoints: map[string]spec.Endpoint{
+ "get": {
+ Method: "GET",
+ Path: "/settings",
+ Response: spec.ResponseDef{Type: "object", Item: "Settings"},
+ },
+ },
+ },
+ },
+ Types: map[string]spec.TypeDef{
+ "Settings": {
+ Fields: []spec.TypeField{
+ {Name: "theme", Type: "string"},
+ {Name: "language", Type: "string"},
+ },
+ },
+ },
+ }
+
+ profile := Profile(s)
+ syncNames := make([]string, len(profile.SyncableResources))
+ for i, sr := range profile.SyncableResources {
+ syncNames[i] = sr.Name
+ }
+ assert.NotContains(t, syncNames, "settings", "non-wrapper object should not be syncable")
+}
diff --git a/skills/printing-press-retro/SKILL.md b/skills/printing-press-retro/SKILL.md
index da5e8519..a40de325 100644
--- a/skills/printing-press-retro/SKILL.md
+++ b/skills/printing-press-retro/SKILL.md
@@ -309,6 +309,21 @@ propose the cheapest mitigation.
**7. What is the durable fix?** Prefer: template fix > binary post-processing > skill instruction.
+**Strip API-specific details from the proposed fix.** The durable fix must work across
+APIs, not just the one that surfaced the finding. If the fix includes hardcoded param
+names (e.g., `--sport`, `--league`), date formats (e.g., `YYYYMMDD`), chunking
+strategies (e.g., monthly), or domain-specific logic, those are printed-CLI details
+leaking into the machine recommendation. The machine fix should be parameterized —
+driven by what the profiler detects in the spec, not by what one API happens to need.
+
+Example of the anti-pattern:
+- Finding: "ESPN sync needs `--dates` for historical data"
+- Bad fix: "Add `--dates` with `YYYYMMDD-YYYYMMDD` format, `--sport`/`--league` flags, and monthly chunking to the sync template"
+- Good fix: "When the profiler detects a date-range query param, emit a `--dates` flag that passes the value through to the API"
+
+The bad fix bakes ESPN's date format, scope params, and chunking strategy into the
+machine. The good fix lets the profiler drive behavior from the spec.
+
## Phase 4: Prioritize
Sort findings into two buckets:
← af88767c chore(main): release 1.2.1 (#151)
·
back to Cli Printing Press
·
fix(skills): correct publish package flag name and staging w 776d433c →