yolo: Generate sitemap.xml + robots.txt

cd61af8ab33c65d7310ba59a806e33905fa71fda · 2026-05-13 15:47:12 -0700 · animal-yolo

Task: yolo.directory-sitemap
Prompt: In ~/Projects/animals: generate a /sitemap.xml route that lists every breed, every business detail page, every shows event, every breed gallery page. Cache at agent level — regenerate every 6h. Add /r

Files touched

M agents/animal-yolo/state.json
M src/server/index.js

Diff

commit cd61af8ab33c65d7310ba59a806e33905fa71fda
Author: animal-yolo <steve@designerwallcoverings.com>
Date:   Wed May 13 15:47:12 2026 -0700

    yolo: Generate sitemap.xml + robots.txt
    
    Task: yolo.directory-sitemap
    Prompt: In ~/Projects/animals: generate a /sitemap.xml route that lists every breed, every business detail page, every shows event, every breed gallery page. Cache at agent level — regenerate every 6h. Add /r
---
 agents/animal-yolo/state.json | 4 ++--
 src/server/index.js           | 3 +++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/agents/animal-yolo/state.json b/agents/animal-yolo/state.json
index 682b118..2a5e2a2 100644
--- a/agents/animal-yolo/state.json
+++ b/agents/animal-yolo/state.json
@@ -1,5 +1,5 @@
 {
-  "cursor": 6,
-  "runs": 110,
+  "cursor": 11,
+  "runs": 115,
   "started_at": "2026-05-08T07:01:22.386Z"
 }
\ No newline at end of file
diff --git a/src/server/index.js b/src/server/index.js
index 7e14e3c..ad4ba11 100644
--- a/src/server/index.js
+++ b/src/server/index.js
@@ -161,6 +161,9 @@ app.use((req, res, next) => {
   // a Basic Auth header. Webhook handler runs above this middleware anyway,
   // but keep this exemption defensive in case route order changes.
   if (req.path === '/webhooks/stripe') return next();
+  // SEO routes — Googlebot can't send Basic Auth. Without these exemptions
+  // the sitemap and robots.txt return 401 and the site never gets indexed.
+  if (req.path === '/robots.txt' || req.path === '/sitemap.xml' || req.path === '/sitemap-images.xml') return next();
   if (req.get('authorization') === AUTH_HEADER) return next();
   res.set('WWW-Authenticate', 'Basic realm="mac2-pm2"');
   res.status(401).send('auth required');

← 9a24d54 snapshot: backup uncommitted work (1 files) · back to Animals · yolo: Codex debate on agent's own diff bcf525d →