← back to Ventura Corridor
feat(news): /api/news/by-day + 90-day calendar heatmap on /coverage.html
f9ee539e8d38c7e97d004eca62c08f8322fa9652 · 2026-05-07 22:17:10 -0700 · SteveStudio2
GitHub-style daily scrape heatmap so editors can see at a glance whether
the nightly 03:15 cron is landing fresh content vs drifting silent.
Routes:
GET /api/news/by-day 90-day generate_series LEFT JOIN news_items
by fetched_at::date in America/Los_Angeles tz.
10-min cache.
UI:
/coverage.html new "News scrape calendar" section above the
existing News heatmap. 14px gold-tinted squares,
5 brightness levels keyed to max-day count.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Files touched
M public/coverage.htmlM src/server/index.ts
Diff
commit f9ee539e8d38c7e97d004eca62c08f8322fa9652
Author: SteveStudio2 <stevestudio2@SteveStacStudio.lan>
Date: Thu May 7 22:17:10 2026 -0700
feat(news): /api/news/by-day + 90-day calendar heatmap on /coverage.html
GitHub-style daily scrape heatmap so editors can see at a glance whether
the nightly 03:15 cron is landing fresh content vs drifting silent.
Routes:
GET /api/news/by-day 90-day generate_series LEFT JOIN news_items
by fetched_at::date in America/Los_Angeles tz.
10-min cache.
UI:
/coverage.html new "News scrape calendar" section above the
existing News heatmap. 14px gold-tinted squares,
5 brightness levels keyed to max-day count.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
public/coverage.html | 25 +++++++++++++++++++++++++
src/server/index.ts | 27 +++++++++++++++++++++++++++
2 files changed, 52 insertions(+)
diff --git a/public/coverage.html b/public/coverage.html
index 9ebcc3e..777a006 100644
--- a/public/coverage.html
+++ b/public/coverage.html
@@ -84,6 +84,12 @@ main{padding:32px;max-width:1200px;margin:0 auto}
<div id="city"></div>
</div>
+<div class="bar-section">
+ <h3>News scrape calendar · last 90 days</h3>
+ <p style="font-family:'Cormorant Garamond',serif;font-style:italic;font-size:13px;color:var(--ink-mute);margin:0 0 14px">Daily count of articles landed by the news scraper. Quiet stretches mean the nightly 03:15 scrape didn't find new posts; bright days mean fresh content.</p>
+ <div id="news-cal" style="display:flex;gap:3px;flex-wrap:wrap;align-items:center"></div>
+</div>
+
<div class="bar-section">
<h3>News heatmap · which categories are publishing</h3>
<p style="font-family:'Cormorant Garamond',serif;font-style:italic;font-size:13px;color:var(--ink-mute);margin:0 0 14px">Articles scraped from each business's own website, bucketed by OSM category. Top buckets currently producing the most blog/news/press content.</p>
@@ -148,6 +154,25 @@ async function load() {
renderBars(d.by_vertical, document.getElementById('vert'));
renderBars(d.by_city, document.getElementById('city'));
+ // News scrape calendar — 90-day heatmap above the heatmap section
+ try {
+ const cal = await fetch('/api/news/by-day').then(r => r.json());
+ const days = cal.days || [];
+ if (days.length) {
+ const max = Math.max(1, ...days.map(d => Number(d.n)));
+ document.getElementById('news-cal').innerHTML = days.map(d => {
+ const n = Number(d.n);
+ const lvl = n === 0 ? 0 : Math.min(4, Math.ceil((n / max) * 4));
+ const colors = ['rgba(255,255,255,0.04)', 'rgba(184,153,104,0.22)', 'rgba(184,153,104,0.45)', 'rgba(184,153,104,0.7)', 'rgba(212,182,131,1)'];
+ return `<div style="width:14px;height:14px;background:${colors[lvl]};border:1px solid var(--rule)" title="${d.day}: ${n} article${n===1?'':'s'}"></div>`;
+ }).join('');
+ } else {
+ document.getElementById('news-cal').innerHTML = '<p style="color:var(--ink-mute);font-style:italic;font-family:Cormorant Garamond,serif">No scrape data yet.</p>';
+ }
+ } catch(e) {
+ document.getElementById('news-cal').innerHTML = '<p style="color:#c47b7b">calendar unavailable</p>';
+ }
+
// News heatmap — pulled separately so coverage.html still renders if news data is empty.
try {
const nv = await fetch('/api/news/by-vertical').then(r => r.json());
diff --git a/src/server/index.ts b/src/server/index.ts
index 71e21e5..53cdf3e 100644
--- a/src/server/index.ts
+++ b/src/server/index.ts
@@ -5126,6 +5126,33 @@ app.get('/api/news/archive', async (_req, res) => {
}
});
+// 90-day daily news-fetch heatmap. /coverage.html renders it as a GitHub-
+// style grid so editors see "is the scraper actually landing fresh stuff?"
+app.get('/api/news/by-day', async (_req, res) => {
+ try {
+ const r = await query(`
+ WITH days AS (
+ SELECT generate_series(
+ (now() AT TIME ZONE 'America/Los_Angeles')::date - interval '89 days',
+ (now() AT TIME ZONE 'America/Los_Angeles')::date,
+ '1 day'::interval
+ )::date AS d
+ )
+ SELECT days.d AS day,
+ COALESCE(COUNT(n.id), 0)::int AS n
+ FROM days
+ LEFT JOIN news_items n
+ ON (n.fetched_at AT TIME ZONE 'America/Los_Angeles')::date = days.d
+ GROUP BY days.d
+ ORDER BY days.d ASC
+ `);
+ res.set('Cache-Control', 'public, max-age=600');
+ res.json({ days: r.rows });
+ } catch (e: any) {
+ res.status(500).json({ error: e.message });
+ }
+});
+
// Source-domain breakdown — what platforms are corridor businesses publishing
// on? wordpress.com, squarespace, custom, etc. Useful editorial signal:
// "are these mostly self-hosted blogs vs aggregator-pages?"
← 4035b3c feat(news): news-recency badge on /pitches.html name column
·
back to Ventura Corridor
·
feat(coverage): idea-loop feed bridge on /coverage.html dab6585 →