backfill-rooms-cron: indent=2 + byte-size guard (95%) to stop daily designs.json field-shrink clobber

1b9059d3d8c62a8dd58c97d4ea60d8d3bd1508d4 · 2026-05-30 08:23:58 -0700 · Steve Abrams

2026-05-30 06:30 UTC incident: inline-Python wrote compact json.dump that
shrunk pretty-printed designs.json 25.2MB -> 20.3MB (~4min /api/designs=0
blackout). The existing count-only catastrophe guard ('len(d) < 1000')
missed it (rows preserved, fields not). Now serializes with indent=2 to
match refresh_designs_snapshot.py's canonical format, AND refuses writes
where new bytes < 95% of existing (same proven guard pattern at lines
209-216 of refresh_designs_snapshot.py). Override WALLCO_FORCE_BACKFILL=1.

DTD verdict 2/2 panelists (qwen errored): option B over A/C/D.

Files touched

M scripts/backfill-rooms-cron.sh

Diff

commit 1b9059d3d8c62a8dd58c97d4ea60d8d3bd1508d4
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Sat May 30 08:23:58 2026 -0700

    backfill-rooms-cron: indent=2 + byte-size guard (95%) to stop daily designs.json field-shrink clobber
    
    2026-05-30 06:30 UTC incident: inline-Python wrote compact json.dump that
    shrunk pretty-printed designs.json 25.2MB -> 20.3MB (~4min /api/designs=0
    blackout). The existing count-only catastrophe guard ('len(d) < 1000')
    missed it (rows preserved, fields not). Now serializes with indent=2 to
    match refresh_designs_snapshot.py's canonical format, AND refuses writes
    where new bytes < 95% of existing (same proven guard pattern at lines
    209-216 of refresh_designs_snapshot.py). Override WALLCO_FORCE_BACKFILL=1.
    
    DTD verdict 2/2 panelists (qwen errored): option B over A/C/D.
---
 scripts/backfill-rooms-cron.sh | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/scripts/backfill-rooms-cron.sh b/scripts/backfill-rooms-cron.sh
index 017c3e5..a422835 100755
--- a/scripts/backfill-rooms-cron.sh
+++ b/scripts/backfill-rooms-cron.sh
@@ -37,15 +37,28 @@ if t:
   if len(d) < 1000:
     print(f'ABORT: refusing to write designs.json with only {len(d)} rows (catastrophe guard)')
   else:
+    # Serialize with indent=2 to match the canonical pretty-printed format produced
+    # by refresh_designs_snapshot.py. Compact json.dump(d,f,separators=(',',':')) was
+    # writing a ~20% smaller file on every run (e.g. 2026-05-30 06:30 clobber dropped
+    # designs.json from 25.2MB → 20.3MB while preserving row count, blackout ~4min).
+    new_blob = json.dumps(d, indent=2)
+    # Byte-size guard (mirrors refresh_designs_snapshot.py lines 209-216). The COUNT
+    # guard above misses field-shrink clobbers. Refuse a materially smaller FILE
+    # unless override flag WALLCO_FORCE_BACKFILL=1 is set.
+    if os.path.exists(DJ) and os.environ.get('WALLCO_FORCE_BACKFILL') != '1':
+      old_bytes = os.path.getsize(DJ)
+      if old_bytes > 1000000 and len(new_blob) < old_bytes * 0.95:
+        print(f'ABORT: new designs.json {len(new_blob)}B < 95% of existing {old_bytes}B (field-shrink clobber guard). Keeping existing. Override WALLCO_FORCE_BACKFILL=1.')
+        raise SystemExit(0)
     shutil.copy(DJ, DJ+'.bak.'+str(int(time.time())))
     # ATOMIC write: stream to a temp file then os.replace() (atomic rename) so the
     # server NEVER reads a half-written/0-byte file. open(DJ,'w') truncated the live
     # 25MB file to zero and took seconds to refill — any read in that window saw 0
     # designs → the recurring ~4-min catalog blackout. (fix 2026-05-30)
     tmp=DJ+'.tmp'
-    with open(tmp,'w') as f: json.dump(d, f, separators=(',',':'))
+    with open(tmp,'w') as f: f.write(new_blob)
     os.replace(tmp, DJ)
-    print(f'patched {t} rows in designs.json (atomic)')
+    print(f'patched {t} rows in designs.json (atomic, pretty)')
 else:
   print('no new room_mockups to patch')
 PY

← a37f6a3 _evidence: snapshot bubbe RCE routes (hooks + upload-bubbe) · back to Wallco Ai · fix(ui): admin-toolbar no longer overlaps fixed header 5f9d9db →