feat(skills): add API reachability gate before generation (#91)

bed2f97c3afb99c6a9b89ad2c7582fdf1f4191b1 · 2026-03-30 23:32:50 -0700 · Matt Van Horn

After the Redfin failure (85/100 scorecard, 0% functional - every call
returns 403), add Phase 1.9 that tests one real API call before spending
tokens on generation. HARD STOP if the API blocks programmatic access.

Also adds issue scanning to Phase 1 research - check the top wrapper
library's GitHub issues for "403", "blocked", "broken" signals. The
Redfin wrapper had 6+ issues about this going back over a year.

If the sniff gate already failed with bot detection, the reachability
gate becomes an automatic HARD STOP.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Files touched

A docs/plans/2026-03-31-002-feat-api-reachability-gate-plan.md
M skills/printing-press/SKILL.md

Diff

commit bed2f97c3afb99c6a9b89ad2c7582fdf1f4191b1
Author: Matt Van Horn <mvanhorn@users.noreply.github.com>
Date:   Mon Mar 30 23:32:50 2026 -0700

    feat(skills): add API reachability gate before generation (#91)
    
    After the Redfin failure (85/100 scorecard, 0% functional - every call
    returns 403), add Phase 1.9 that tests one real API call before spending
    tokens on generation. HARD STOP if the API blocks programmatic access.
    
    Also adds issue scanning to Phase 1 research - check the top wrapper
    library's GitHub issues for "403", "blocked", "broken" signals. The
    Redfin wrapper had 6+ issues about this going back over a year.
    
    If the sniff gate already failed with bot detection, the reachability
    gate becomes an automatic HARD STOP.
    
    Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---
 ...26-03-31-002-feat-api-reachability-gate-plan.md | 103 +++++++++++++++++++++
 skills/printing-press/SKILL.md                     |  57 ++++++++++++
 2 files changed, 160 insertions(+)

diff --git a/docs/plans/2026-03-31-002-feat-api-reachability-gate-plan.md b/docs/plans/2026-03-31-002-feat-api-reachability-gate-plan.md
new file mode 100644
index 00000000..f868fa70
--- /dev/null
+++ b/docs/plans/2026-03-31-002-feat-api-reachability-gate-plan.md
@@ -0,0 +1,103 @@
+---
+title: "feat: API reachability gate before generation"
+type: feat
+status: active
+date: 2026-03-31
+---
+
+# feat: API reachability gate before generation
+
+## Overview
+
+Add a mandatory "can we actually reach this API?" check between the absorb gate (Phase 1.5) and generation (Phase 2). One real HTTP call to one real endpoint. If it fails, STOP and tell the user before burning tokens on 10,000 lines of dead code.
+
+## Problem Frame
+
+The Redfin CLI scored 85/100 on the scorecard and 96% on verify. Every API call returns HTTP 403. The CLI is structurally perfect and functionally zero.
+
+The signals were everywhere:
+- Sniff gate failed with bot detection (Redfin blocked automated browsers)
+- The primary wrapper library (reteps/redfin) had 6+ open issues about 403 errors going back over a year
+- Issue #22 was literally titled "Revive broken library: fix 403 errors"
+
+None of this stopped the press from generating, building, polishing, and publishing a CLI that cannot make a single successful API call. The scorecard tests structure, not function. There is no gate that tests "does this API actually respond?"
+
+## Requirements Trace
+
+- R1. Before Phase 2 (generate), test one real API call against the resolved spec's base URL
+- R2. If the call fails (403, 401, timeout, DNS error, connection refused), STOP and present the user with options - do not silently proceed
+- R3. During Phase 1 research, check GitHub issues on the primary wrapper library for "403", "blocked", "broken", "deprecated" signals
+- R4. If the sniff gate failed due to bot detection, treat that as a strong signal that the API itself may block programmatic access - escalate the reachability check to a hard gate, not a warning
+
+## Scope Boundaries
+
+- Only modifying `skills/printing-press/SKILL.md`
+- Not changing the scorecard or verify tools (those test code quality, which is fine)
+- Not adding a new Go binary command (this is a skill-level check using curl/fetch)
+- Not blocking APIs that require auth keys the user hasn't provided yet (that's the existing API key gate)
+
+## Key Technical Decisions
+
+- **Placement: between Phase 1.5 and Phase 2**: After the absorb manifest is approved but before generation starts. This is the last cheap exit point. Once Phase 2 runs, you're committed to tokens.
+- **One call, not a suite**: Pick the simplest GET endpoint from the spec (or the base URL with a health/version path). One 200 OK is enough to prove reachability. One 403/timeout is enough to flag the problem.
+- **Hard gate for sniff-failed APIs**: If the sniff gate already failed with bot detection/403, the reachability check becomes a hard STOP, not a warning. The evidence is already there.
+- **Research-phase issue scanning**: During Phase 1, when fetching competitor repos, also check the Issues tab for 403/blocked/deprecated signals. This catches the Redfin pattern where the wrapper library's own issues were screaming that the API is dead.
+
+## Implementation Units
+
+- [ ] **Unit 1: Add API reachability gate (Phase 1.9) to SKILL.md**
+
+  **Goal:** Before generation, test one real API call. If it fails, present options to the user.
+
+  **Requirements:** R1, R2, R4
+
+  **Files:**
+  - Modify: `skills/printing-press/SKILL.md`
+
+  **Approach:**
+  - Add a new "Phase 1.9: API Reachability Gate" section between Phase 1.5 (absorb gate) and Phase 2 (generate)
+  - The gate runs a single curl/WebFetch against the API's base URL or simplest GET endpoint
+  - For APIs with a resolved spec: pick the first GET endpoint with no required params, or fall back to the base URL
+  - For sniffed/docs-based APIs: use the base URL directly
+  - Success (HTTP 2xx or 3xx): proceed silently
+  - Auth error (401/403 without a key): skip silently if the API key gate already noted "no key provided" - the user knows
+  - Auth error (403 WITH bot detection signals like HTML error pages, "Are You a Robot", Cloudflare challenge): HARD STOP
+  - If the sniff gate previously failed with bot detection: HARD STOP regardless of this call's result
+  - On HARD STOP, present via AskUserQuestion:
+    > "Warning: `<API>` appears to block programmatic access. [details of what failed]. Building a CLI that can't reach the API is a waste of time. What do you want to do?"
+    > 1. **Try anyway** - proceed knowing the CLI may not work against the live API
+    > 2. **Pick a different API** - start over with an API that has public access
+    > 3. **Done** - stop here
+  - Add a MANDATORY checkpoint marker so the LLM can't skip it
+
+  **Verification:**
+  - Phase 1.9 section exists between Phase 1.5 and Phase 2
+  - Running against an API that returns 403 triggers the HARD STOP with user options
+  - Running against a working API proceeds silently
+
+- [ ] **Unit 2: Add issue scanning to Phase 1 research**
+
+  **Goal:** During research, check competitor repo issues for "API is broken/blocked" signals.
+
+  **Requirements:** R3
+
+  **Dependencies:** None (independent of Unit 1)
+
+  **Files:**
+  - Modify: `skills/printing-press/SKILL.md` (Phase 1 research checklist)
+
+  **Approach:**
+  - In the Phase 1 research checklist, after "Find the top 1-2 competitors", add:
+    "Check GitHub issues on the top wrapper library for '403', 'blocked', 'broken', 'deprecated', 'rate limit'. If multiple issues report the API is inaccessible, flag this in the research brief as a reachability risk."
+  - This feeds into Phase 1.9 - if research already found 403 signals, the reachability gate should be stricter
+
+  **Verification:**
+  - Phase 1 research checklist includes issue scanning instruction
+  - Research brief has a place for reachability risk signals
+
+## Sources & References
+
+- Redfin failure report: `docs/retros/2026-03-30-redfin-failure-report.md`
+- reteps/redfin issues #7, #15, #19, #20, #21, #22 - all 403 errors
+- Sniff gate: `skills/printing-press/SKILL.md` Phase 1.7
+- Absorb gate: `skills/printing-press/SKILL.md` Phase 1.5
diff --git a/skills/printing-press/SKILL.md b/skills/printing-press/SKILL.md
index 860a3c38..a6baed93 100644
--- a/skills/printing-press/SKILL.md
+++ b/skills/printing-press/SKILL.md
@@ -353,6 +353,7 @@ The brief must answer:
 Research checklist:
 - Find the spec or docs source
 - Find the top 1-2 competitors
+- **Check GitHub issues on the top wrapper/SDK repo for "403", "blocked", "broken", "deprecated", "rate limit".** If multiple issues report the API is inaccessible or broken, flag this in the research brief as a reachability risk. This is critical for unofficial/reverse-engineered APIs.
 - Find official and popular SDK wrappers on npm (`site:npmjs.com`) and PyPI (`site:pypi.org`)
 - Find 2-3 concrete user pain points
 - Identify the highest-gravity entities
@@ -380,6 +381,9 @@ Suggested shape:
 - Users:
 - Data profile:
 
+## Reachability Risk
+- [None / Low / High] [evidence: e.g., "6 open issues on reteps/redfin about 403 errors since 2025"]
+
 ## Top Workflows
 1. ...
 
@@ -1011,6 +1015,59 @@ WAIT for approval. Do NOT generate until approved.
 
 ---
 
+## Phase 1.9: API Reachability Gate
+
+**MANDATORY. Do NOT skip this phase. Do NOT proceed to Phase 2 without running this check.**
+
+Before spending tokens on generation, verify the API actually responds to programmatic requests. One real HTTP call. If it fails, STOP.
+
+### The Check
+
+Pick the simplest GET endpoint from the resolved spec (no required params, no auth if possible). If no such endpoint exists, use the spec's base URL. Run one HTTP request:
+
+```bash
+curl -s -o /dev/null -w "%{http_code}" -m 10 "<base_url>/<simplest_get_path>" 2>/dev/null
+```
+
+Or use `WebFetch` if curl is unavailable. The goal is one real response code.
+
+### Decision Matrix
+
+| Result | Sniff gate failed? | Research found 403 issues? | Action |
+|--------|-------------------|---------------------------|--------|
+| 2xx/3xx | Any | Any | **PASS** - proceed to Phase 2 |
+| 401 (no key provided) | No | No | **PASS** - expected when API needs auth and user declined key gate |
+| 403 with HTML/bot detection | Any | Any | **HARD STOP** |
+| 403 | Yes (bot detection) | Any | **HARD STOP** |
+| 403 | No | Yes (issues found) | **HARD STOP** |
+| 403 | No | No | **WARN** - ask user |
+| Timeout/DNS/connection refused | Any | Any | **WARN** - ask user |
+
+### On HARD STOP
+
+Present via `AskUserQuestion`:
+
+> "WARNING: `<API>` appears to block programmatic access. [what failed: e.g., 'HTTP 403 with HTML error page', 'sniff gate failed with bot detection', 'reteps/redfin has 6+ issues about 403 errors']. Building a CLI against an unreachable API wastes time and tokens."
+>
+> 1. **Try anyway** - proceed knowing the CLI may not work against the live API
+> 2. **Pick a different API** - start over
+> 3. **Done** - stop here
+
+### On WARN
+
+Present via `AskUserQuestion`:
+
+> "The API returned [error]. This might be temporary, or it might mean programmatic access is blocked. Want to proceed?"
+>
+> 1. **Yes - proceed** - generate the CLI anyway
+> 2. **No - stop** - pick a different API or provide a spec manually
+
+### On PASS
+
+Proceed silently to Phase 2.
+
+---
+
 ## Phase 2: Generate
 
 Use the resolved spec source and generate immediately.

← 03487972 feat(skills): add /printing-press-polish standalone skill (# · back to Cli Printing Press · fix(cli): actionable auth errors with env var names, key URL 151ec979 →