harden all-dw cache write: atomic tmp+rename, refresh disk-cache sig

2bc0f06432e590c72350274ab377158c771be4c1 · 2026-09-22 13:58:08 -0700 · Steve Abrams

Write the ~hundreds-of-MB all-catalog.json via a sibling .tmp + rename() so an
interrupted write can never leave a truncated cache the feed-down fallback would
choke on. Also stamp _diskCacheSig from the just-written file so the first
feed-down tick after a refresh skips a redundant reparse (TK-11962 guard).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0142i1ci5jdMrmc8DNr6RjUS

Files touched

M server.js

Diff

commit 2bc0f06432e590c72350274ab377158c771be4c1
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Tue Sep 22 13:58:08 2026 -0700

    harden all-dw cache write: atomic tmp+rename, refresh disk-cache sig
    
    Write the ~hundreds-of-MB all-catalog.json via a sibling .tmp + rename() so an
    interrupted write can never leave a truncated cache the feed-down fallback would
    choke on. Also stamp _diskCacheSig from the just-written file so the first
    feed-down tick after a refresh skips a redundant reparse (TK-11962 guard).
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0142i1ci5jdMrmc8DNr6RjUS
---
 server.js | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/server.js b/server.js
index 7569885..0ebc5d3 100644
--- a/server.js
+++ b/server.js
@@ -2772,7 +2772,22 @@ async function buildCatalog() {
     const feed = await fetchAllDwCatalog();
     if (feed && Array.isArray(feed.rows) && feed.rows.length) {
       CATALOG = feed.rows.map(feedRowToItem);
-      try { fs.writeFileSync(CATALOG_CACHE, JSON.stringify(feed)); } catch (e) { /* cache best-effort */ }
+      try {
+        // Atomic write: this cache is a multi-hundred-MB JSON; an in-place writeFileSync that is
+        // interrupted (OOM/restart/disk-full mid-write) leaves a TRUNCATED file that the feed-down
+        // fallback (branch 2 below) would then JSON.parse and fail on every 15-min tick. Write to a
+        // sibling .tmp on the same filesystem, then rename() — rename is atomic, so a reader ever
+        // only sees the complete old file or the complete new one, never a torn one.
+        const tmp = CATALOG_CACHE + '.tmp';
+        fs.writeFileSync(tmp, JSON.stringify(feed));
+        fs.renameSync(tmp, CATALOG_CACHE);
+        // Record the signature of the file we just wrote so a later feed-down tick recognizes that
+        // CATALOG already reflects this exact cache and skips a redundant ~hundreds-of-MB reparse
+        // (the TK-11962 guard keys on size+mtime). Without this, the first feed-down tick after every
+        // refresh needlessly re-reads+re-parses the file it just produced — costly with a flapping feed.
+        const st = fs.statSync(CATALOG_CACHE);
+        _diskCacheSig = st.size + ':' + st.mtimeMs;
+      } catch (e) { /* cache best-effort */ }
       console.log(`catalog indexed: ${CATALOG.length} products from all-dw feed`);
       return rebuildIndex();
     }

← 77ae6ab harden(TK-11962): catalog-cache guard survives corrupt/unrea · back to Dw Photo Capture · gitignore 5x/out/ screen-capture test artifacts 0d34d1b →