[object Object]

← back to Ventura Corridor

feat(news): summarizer pre-flight + 4-hour poll launchd

4b74df3e07644bc7b2b5ebb202fe3d521fb705c2 · 2026-05-07 20:26:34 -0700 · SteveStudio2

Two changes that let the news summarizer drain naturally when Mac1
frees up, instead of burning a 62-min backoff per item or sleeping
through the gap:

1. Pre-flight probe in summarize_news.ts
   - Sends a tiny "reply OK" call before opening the work queue.
   - If Mac1 returns 503/429 immediately (i.e. the 2000-feature
     generate_features batch is hammering it), exit 0 cleanly with a
     "DEFERRED" log line.
   - Saves 60+ minutes of pointless backoff per failed run.

2. Poll launchd com.steve.ventura-corridor-news-summarize-poll
   - StartInterval 14400 (every 4 hours).
   - Pre-flight gates work; when Mac1 is busy this fires + immediately
     exits, costing ~/bin/bash + ~50ms.
   - The first run after magazine-gen winds down will catch the gap
     and drain all 76 unsummarized items.

Existing 5:30am daily summarize launchd unchanged — this poll is
additive coverage during the magazine-gen marathon (~ 15 more hours
to go at current 40s/feature rate).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files touched

Diff

commit 4b74df3e07644bc7b2b5ebb202fe3d521fb705c2
Author: SteveStudio2 <stevestudio2@SteveStacStudio.lan>
Date:   Thu May 7 20:26:34 2026 -0700

    feat(news): summarizer pre-flight + 4-hour poll launchd
    
    Two changes that let the news summarizer drain naturally when Mac1
    frees up, instead of burning a 62-min backoff per item or sleeping
    through the gap:
    
    1. Pre-flight probe in summarize_news.ts
       - Sends a tiny "reply OK" call before opening the work queue.
       - If Mac1 returns 503/429 immediately (i.e. the 2000-feature
         generate_features batch is hammering it), exit 0 cleanly with a
         "DEFERRED" log line.
       - Saves 60+ minutes of pointless backoff per failed run.
    
    2. Poll launchd com.steve.ventura-corridor-news-summarize-poll
       - StartInterval 14400 (every 4 hours).
       - Pre-flight gates work; when Mac1 is busy this fires + immediately
         exits, costing ~/bin/bash + ~50ms.
       - The first run after magazine-gen winds down will catch the gap
         and drain all 76 unsummarized items.
    
    Existing 5:30am daily summarize launchd unchanged — this poll is
    additive coverage during the magazine-gen marathon (~ 15 more hours
    to go at current 40s/feature rate).
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 ...teve.ventura-corridor-news-summarize-poll.plist | 33 ++++++++++++++++++++
 src/jobs/summarize_news.ts                         | 35 ++++++++++++++++++++++
 2 files changed, 68 insertions(+)

diff --git a/launchd/com.steve.ventura-corridor-news-summarize-poll.plist b/launchd/com.steve.ventura-corridor-news-summarize-poll.plist
new file mode 100644
index 0000000..690c229
--- /dev/null
+++ b/launchd/com.steve.ventura-corridor-news-summarize-poll.plist
@@ -0,0 +1,33 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+  <key>Label</key>
+  <string>com.steve.ventura-corridor-news-summarize-poll</string>
+  <key>ProgramArguments</key>
+  <array>
+    <string>/bin/bash</string>
+    <string>-lc</string>
+    <string>cd /Users/stevestudio2/Projects/ventura-corridor &amp;&amp; /usr/local/bin/npm run news:summarize -- --limit=200</string>
+  </array>
+  <key>StartInterval</key>
+  <integer>14400</integer>
+  <key>StandardOutPath</key>
+  <string>/Users/stevestudio2/Projects/ventura-corridor/logs/news-summarize-poll.out.log</string>
+  <key>StandardErrorPath</key>
+  <string>/Users/stevestudio2/Projects/ventura-corridor/logs/news-summarize-poll.err.log</string>
+  <key>EnvironmentVariables</key>
+  <dict>
+    <key>PATH</key>
+    <string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin</string>
+    <key>PGHOST</key>
+    <string>/tmp</string>
+    <key>OLLAMA_URL</key>
+    <string>http://100.94.103.98:11434</string>
+    <key>OLLAMA_MODEL</key>
+    <string>qwen3:14b</string>
+  </dict>
+  <key>RunAtLoad</key>
+  <false/>
+</dict>
+</plist>
diff --git a/src/jobs/summarize_news.ts b/src/jobs/summarize_news.ts
index 427cf8c..077d94f 100644
--- a/src/jobs/summarize_news.ts
+++ b/src/jobs/summarize_news.ts
@@ -70,7 +70,42 @@ async function callOllama(prompt: string, attempt = 0): Promise<string> {
   return text;
 }
 
+// Pre-flight: probe Ollama with a tiny call. If it returns 503 "server busy"
+// immediately (no backoff), Mac1 is locked by another job (typically the
+// 2000-feature generate_features run). Exit 0 cleanly so the launchd job
+// reschedules instead of burning the patient backoff budget on every item.
+async function preflight(): Promise<{ ok: boolean; reason?: string }> {
+  try {
+    const res = await fetch(`${OLLAMA}/api/chat`, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({
+        model: MODEL, stream: false,
+        messages: [{ role: 'user', content: 'reply with OK' }],
+        options: { num_predict: 4, num_ctx: 256 }
+      }),
+      signal: AbortSignal.timeout(15000)
+    });
+    if (res.status === 503 || res.status === 429) {
+      const txt = (await res.text()).slice(0, 120);
+      return { ok: false, reason: `mac1_busy_${res.status}: ${txt}` };
+    }
+    if (!res.ok) return { ok: false, reason: `mac1_${res.status}` };
+    return { ok: true };
+  } catch (e: any) {
+    return { ok: false, reason: `mac1_unreachable: ${e?.message || e}` };
+  }
+}
+
 async function main() {
+  const pf = await preflight();
+  if (!pf.ok) {
+    console.log(`[summarize_news] DEFERRED — ${pf.reason}`);
+    console.log(`[summarize_news] Will retry on the next scheduled launchd tick (5:30am daily) or when Steve runs manually.`);
+    await pool.end();
+    process.exit(0);
+  }
+
   const r = await query<Row>(`
     SELECT n.id, n.business_id, n.title, n.body,
            b.name AS business_name, b.category AS business_category

← 0f24227 feat(news): /api/news/leaderboard + Top publishers section o  ·  back to Ventura Corridor  ·  feat(news): /api/news/sources + source-domain bar chart on / 888783b →