feat(seo): include news pages in /sitemap.xml

902783291e080d07a43bf228f8e38d82a8182901 · 2026-05-07 17:06:52 -0700 · SteveStudio2

The news pipeline shipped 5 new URL surfaces this session
(/news.html, /this-week.html, /news/feed.xml-discovery via /news.html,
/business/:id/news, /coverage.html news heatmap). Add them to the
sitemap so search engines find them.

Static pages added:
  /news.html        hourly  pri 0.9
  /this-week.html   daily   pri 0.8
  /coverage.html    daily   pri 0.5

Dynamic pages added:
  /business/:id/news for every biz with at least one news_item
  Cap 1000, ordered by most-recently-fetched, lastmod = MAX(fetched_at)

Real numbers: 46 /business/:id/news entries flowing on first build.
xmllint validates clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files touched

M src/server/index.ts

Diff

commit 902783291e080d07a43bf228f8e38d82a8182901
Author: SteveStudio2 <stevestudio2@SteveStacStudio.lan>
Date:   Thu May 7 17:06:52 2026 -0700

    feat(seo): include news pages in /sitemap.xml
    
    The news pipeline shipped 5 new URL surfaces this session
    (/news.html, /this-week.html, /news/feed.xml-discovery via /news.html,
    /business/:id/news, /coverage.html news heatmap). Add them to the
    sitemap so search engines find them.
    
    Static pages added:
      /news.html        hourly  pri 0.9
      /this-week.html   daily   pri 0.8
      /coverage.html    daily   pri 0.5
    
    Dynamic pages added:
      /business/:id/news for every biz with at least one news_item
      Cap 1000, ordered by most-recently-fetched, lastmod = MAX(fetched_at)
    
    Real numbers: 46 /business/:id/news entries flowing on first build.
    xmllint validates clean.
    
    Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 src/server/index.ts | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/src/server/index.ts b/src/server/index.ts
index c205699..995fb77 100644
--- a/src/server/index.ts
+++ b/src/server/index.ts
@@ -3058,7 +3058,21 @@ app.get('/sitemap.xml', async (_req, res) => {
       `<url><loc>${baseUrl}/scoreboard.html</loc><changefreq>daily</changefreq><priority>0.6</priority></url>`,
       `<url><loc>${baseUrl}/words</loc><changefreq>weekly</changefreq><priority>0.5</priority></url>`,
       `<url><loc>${baseUrl}/rate-card.html</loc><changefreq>monthly</changefreq><priority>0.7</priority></url>`,
+      `<url><loc>${baseUrl}/news.html</loc><changefreq>hourly</changefreq><priority>0.9</priority></url>`,
+      `<url><loc>${baseUrl}/this-week.html</loc><changefreq>daily</changefreq><priority>0.8</priority></url>`,
+      `<url><loc>${baseUrl}/coverage.html</loc><changefreq>daily</changefreq><priority>0.5</priority></url>`,
     ];
+    // One <url> per business with at least one scraped article — these
+    // /business/:id/news pages are real content surfaces, not just stubs,
+    // and are worth crawling. Cap at 1000 so the sitemap stays sane.
+    const newsBiz = await query(`
+      SELECT DISTINCT business_id, MAX(fetched_at) AS last_fetched
+        FROM news_items GROUP BY business_id
+       ORDER BY MAX(fetched_at) DESC LIMIT 1000
+    `);
+    for (const nb of newsBiz.rows) {
+      urls.push(`<url><loc>${baseUrl}/business/${nb.business_id}/news</loc><lastmod>${fmt(nb.last_fetched)}</lastmod><changefreq>weekly</changefreq><priority>0.5</priority></url>`);
+    }
     for (const c of cats.rows) {
       urls.push(`<url><loc>${baseUrl}/issue/${c.category_tag}</loc><changefreq>weekly</changefreq><priority>0.8</priority></url>`);
     }

← 33b09e9 feat(news): "Latest from their site" section on /magazine/:i · back to Ventura Corridor · fix(news): hardenings caught by codex-2way round 1 audit 840b1a4 →