← back to Scarlet Riverboat Masquerade
riverboat: Phase 1 archive.org reliability — cap year-count fetches at 5 concurrent, polite retry/backoff/breaker, localStorage count cache (kills the false "0" tiles)
ab812885b03239843c01a26767d5569717fbf57d · 2026-09-02 11:55:24 -0700 · Steve Abrams
buildMap fired all 31 per-year count queries at archive.org at once; archive
drops ~40% of that burst as steady-state, so ~40% of tiles rendered a false "0"
(a failed fetch became n:-1 then cached/painted as 0). Phase 1:
- mrPool: cap the count-fetch burst at 5 concurrent (was unbounded Promise.all).
- mrFetchCount: max 2 retries, jittered backoff, 429 backs off HARDER than a
timeout; r.ok/status===429 checked BEFORE r.json() so a 429 is never parsed as
success; batch-shared circuit breaker opens after 6 consecutive failures and
short-circuits the rest instead of hammering a struggling nonprofit. Count
timeout tightened to 6s (counts return in ~0.18s) to bound time-to-error.
- mrCacheLoad/Save: per-source+year localStorage cache; NEVER persists a
failed/zero/negative count (a cached 0 is a poisoned read), current calendar
year always re-fetches, ~30-day TTL. buildMap paints known-good cached counts
instantly then refreshes live via the capped pool; a live failure NEVER
overwrites a good cached value with 0.
- failed tiles show a non-destructive "—" error state, never a false 0.
Tests (global playwright via PW env, no npm deps):
- tests/mock-reliability.cjs: route-interception mock proving concurrency cap
(saturates at 5, never exceeds), no false "0" under a 40% failure mix, bounded
retries + circuit-breaker halt (no retry storm), and 429 backing off harder
than a timeout. 14/14.
- tests/live-happy.cjs: real archive.org, no regression — 31 GD tiles populate
(1994=460), D&C source loads, no false 0, no page errors. 7/7.
Audio/playback, source selector, and UI markup unchanged. No Relisten (Phase 2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k4CBzSCAmou3w8SYxQJLb
Files touched
M index.htmlA tests/live-happy.cjsA tests/mock-reliability.cjs
Diff
commit ab812885b03239843c01a26767d5569717fbf57d
Author: Steve Abrams <steve@designerwallcoverings.com>
Date: Wed Sep 2 11:55:24 2026 -0700
riverboat: Phase 1 archive.org reliability — cap year-count fetches at 5 concurrent, polite retry/backoff/breaker, localStorage count cache (kills the false "0" tiles)
buildMap fired all 31 per-year count queries at archive.org at once; archive
drops ~40% of that burst as steady-state, so ~40% of tiles rendered a false "0"
(a failed fetch became n:-1 then cached/painted as 0). Phase 1:
- mrPool: cap the count-fetch burst at 5 concurrent (was unbounded Promise.all).
- mrFetchCount: max 2 retries, jittered backoff, 429 backs off HARDER than a
timeout; r.ok/status===429 checked BEFORE r.json() so a 429 is never parsed as
success; batch-shared circuit breaker opens after 6 consecutive failures and
short-circuits the rest instead of hammering a struggling nonprofit. Count
timeout tightened to 6s (counts return in ~0.18s) to bound time-to-error.
- mrCacheLoad/Save: per-source+year localStorage cache; NEVER persists a
failed/zero/negative count (a cached 0 is a poisoned read), current calendar
year always re-fetches, ~30-day TTL. buildMap paints known-good cached counts
instantly then refreshes live via the capped pool; a live failure NEVER
overwrites a good cached value with 0.
- failed tiles show a non-destructive "—" error state, never a false 0.
Tests (global playwright via PW env, no npm deps):
- tests/mock-reliability.cjs: route-interception mock proving concurrency cap
(saturates at 5, never exceeds), no false "0" under a 40% failure mix, bounded
retries + circuit-breaker halt (no retry storm), and 429 backing off harder
than a timeout. 14/14.
- tests/live-happy.cjs: real archive.org, no regression — 31 GD tiles populate
(1994=460), D&C source loads, no false 0, no page errors. 7/7.
Audio/playback, source selector, and UI markup unchanged. No Relisten (Phase 2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k4CBzSCAmou3w8SYxQJLb
---
index.html | 138 +++++++++++++++++++++++++++++++----
tests/live-happy.cjs | 82 +++++++++++++++++++++
tests/mock-reliability.cjs | 178 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 383 insertions(+), 15 deletions(-)
diff --git a/index.html b/index.html
index ac0394c..768f6b6 100644
--- a/index.html
+++ b/index.html
@@ -175,6 +175,8 @@
.mr-yr .yc{font-size:8.5px;color:var(--ink-dim);letter-spacing:.02em;line-height:1}
.mr-yr.on .yc{color:#6a2c12}
.mr-yr.loading{animation:mrPulse 1s ease-in-out infinite}
+ .mr-yr.err{opacity:.55;border-style:dashed}
+ .mr-yr.err .yc{color:var(--ink-dim)}
@keyframes mrPulse{0%,100%{opacity:.4}50%{opacity:.8}}
.mr-map-err{font-size:11px;color:#e2a08a;padding:6px 4px;font-style:italic}
.mr-results{max-height:150px;overflow:auto;margin-bottom:8px;display:flex;flex-direction:column;gap:2px}
@@ -1540,6 +1542,79 @@ function fetchT(url, ms=9000, extSignal){
return fetch(url, {mode:'cors', signal: ctl?ctl.signal:undefined})
.finally(()=>{ if(t) clearTimeout(t); });
}
+
+// --- Phase 1 reliability: capped pool + polite retry/backoff/breaker + count cache ---
+// archive.org serves ONE count query fast but drops ~40% of a 31-way concurrent burst,
+// so we throttle, retry timeouts/429s (429 backs off harder), and stop hammering a
+// struggling nonprofit once failures pile up.
+const MR_POOL_LIMIT=5, MR_RETRY_MAX=2, MR_BREAKER_TRIP=6, MR_COUNT_TIMEOUT=6000;
+function mrDelay(ms){ return new Promise(r=>setTimeout(r,ms)); }
+function mrJitter(base){ return base+Math.floor(Math.random()*base); }
+// Run fn over items with at most `limit` in flight at once. fn must not throw.
+function mrPool(items, limit, fn){
+ const out=new Array(items.length); let next=0;
+ const worker=async()=>{ while(next<items.length){ const i=next++; out[i]=await fn(items[i], i); } };
+ const n=Math.min(Math.max(1,limit), items.length||1);
+ return Promise.all(Array.from({length:n}, worker)).then(()=>out);
+}
+// Fetch one advancedsearch count with bounded retries. r.ok / status===429 are checked
+// BEFORE r.json() so a 429 or HTTP error is never parsed as a success. `breaker` is a
+// batch-shared {fails,open}: after MR_BREAKER_TRIP consecutive failures it opens and
+// remaining calls short-circuit instead of piling on.
+async function mrFetchCount(url, sig, breaker){
+ for(let attempt=0; ; attempt++){
+ if(breaker.open) throw new Error('breaker-open');
+ if(sig&&sig.aborted) throw new Error('aborted');
+ let outcome, value=null, err=null;
+ try{
+ const r=await fetchT(url, MR_COUNT_TIMEOUT, sig); // counts return in ~0.18s — a tight timeout bounds time-to-error under a hang
+ if(r.status===429){ outcome='retry-429'; err=new Error('http-429'); }
+ else if(!r.ok){ outcome='fail'; err=new Error('http-'+r.status); }
+ else{
+ const j=await r.json();
+ const n=(j&&j.response&&j.response.numFound);
+ if(typeof n==='number'){ outcome='ok'; value=n; }
+ else { outcome='fail'; err=new Error('malformed'); }
+ }
+ }catch(e){
+ if(sig&&sig.aborted) throw e;
+ outcome='retry-timeout'; err=e; // network drop / abort-timeout
+ }
+ if(outcome==='ok'){ breaker.fails=0; return value; }
+ const canRetry = attempt<MR_RETRY_MAX && (outcome==='retry-timeout'||outcome==='retry-429');
+ if(canRetry){
+ await mrDelay(mrJitter(outcome==='retry-429' ? 1500*(attempt+1) : 400*(attempt+1)));
+ continue; // 429 backs off HARDER than a timeout
+ }
+ breaker.fails++;
+ if(breaker.fails>=MR_BREAKER_TRIP) breaker.open=true;
+ throw err||new Error('fail');
+ }
+}
+// Per-(source|year) count cache in localStorage. HARD RULE: never persist a
+// failed/zero/negative count (every real GD/DC year is nonzero, so a cached 0 is a
+// poisoned read). Historical years live ~30 days; the current calendar year always
+// re-fetches (its count keeps growing).
+const MR_CACHE_KEY='srm.counts.v1', MR_CACHE_TTL=30*24*60*60*1000;
+function mrCacheLoad(src,y){
+ try{
+ const store=JSON.parse(localStorage.getItem(MR_CACHE_KEY)||'{}');
+ const rec=store[ck(src,y)];
+ if(!rec || !(rec.n>0)) return null; // missing / poisoned 0/neg → not cached
+ if(Number(y)===new Date().getFullYear()) return null;
+ if(Date.now()-rec.t>MR_CACHE_TTL) return null; // stale
+ return rec.n;
+ }catch(e){ return null; }
+}
+function mrCacheSave(src,y,n){
+ if(!(n>0)) return; // NEVER persist a failed/zero/negative count
+ if(Number(y)===new Date().getFullYear()) return;
+ try{
+ const store=JSON.parse(localStorage.getItem(MR_CACHE_KEY)||'{}');
+ store[ck(src,y)]={n:n, t:Date.now()};
+ localStorage.setItem(MR_CACHE_KEY, JSON.stringify(store));
+ }catch(e){}
+}
// Browse the WHOLE Grateful Dead collection — empty = most-popular shows; text and/or a year narrow it.
async function searchArchive(q){
const res=$('mrResults'); if(!res) return;
@@ -1705,9 +1780,34 @@ function setYear(y){
searchArchive(searchEl?searchEl.value:'');
}
+// paint a resolved count onto its tile; leave it non-destructive if we lack a good value
+function mrPaintCount(src,y,n){
+ if(src!==window.MR_SRC) return;
+ const el=mapEl.querySelector('[data-y="'+y+'"]'); if(!el) return;
+ el.classList.remove('loading','err');
+ el.querySelector('.yc').textContent=n.toLocaleString();
+ el.title=y+' — '+n.toLocaleString()+' shows';
+}
+// a tile whose live fetch failed with NO good (cached) value: show an error mark, never a false 0
+function mrMarkFailed(src,y){
+ if(src!==window.MR_SRC) return;
+ const el=mapEl.querySelector('[data-y="'+y+'"]'); if(!el) return;
+ if(counts[ck(src,y)]!=null) return; // already have a good value → keep it
+ el.classList.remove('loading'); el.classList.add('err');
+ el.querySelector('.yc').textContent='—';
+ el.title=y+' — count unavailable · tap to browse this year';
+}
+// heat-color every tile from the known counts (sqrt to tame the '77/'89 spikes)
+function mrApplyHeat(src){
+ if(src!==window.MR_SRC) return;
+ const tiles=[...mapEl.querySelectorAll('.mr-yr')];
+ const max=Math.max(1,...tiles.map(el=>{const n=counts[ck(src,el.dataset.y)];return n>0?Math.sqrt(n):0;}));
+ tiles.forEach(el=>{ const n=counts[ck(src,el.dataset.y)]; if(n>0) el.style.setProperty('--heat',(Math.sqrt(n)/max).toFixed(3)); });
+}
+
let buildCtl=null; // cancels the previous batch of year-count fetches on a source switch
function buildMap(){
- const src=window.MR_SOURCES[window.MR_SRC], mySrc=window.MR_SRC;
+ const src=window.MR_SOURCES[window.MR_SRC], mySrc=window.MR_SRC, srcId=src.id;
if(buildCtl) buildCtl.abort(); // kill in-flight fetches from the prior build
buildCtl=(typeof AbortController!=='undefined')?new AbortController():null;
const sig=buildCtl?buildCtl.signal:undefined;
@@ -1721,20 +1821,28 @@ function buildMap(){
b.addEventListener('mouseenter',()=>{ const c=counts[ck(window.MR_SRC,y)]; if(c!=null) mapLabel.textContent=`${y} · ${c.toLocaleString()} shows`; });
mapEl.appendChild(b);
});
- // per-year show counts → heat-color the tiles (sqrt to tame the '77/'89 spikes)
- Promise.all(YEARS.map(y=>
- fetchT('https://archive.org/advancedsearch.php?q='+encodeURIComponent('collection:'+src.id+' AND year:'+y)+'&rows=0&output=json', 9000, sig)
- .then(r=>r.json()).then(j=>({y,n:(j.response&&j.response.numFound)||0})).catch(()=>({y,n:-1}))
- )).then(rows=>{
- if(mySrc!==window.MR_SRC) return; // source switched mid-flight
- if(!rows.some(r=>r.n>=0)){ mapEl.innerHTML='<div class="mr-map-err">Couldn’t load the tour history — archive.org may be unreachable. Search still works.</div>'; return; }
- const max=Math.max(1,...rows.filter(r=>r.n>0).map(r=>Math.sqrt(r.n)));
- rows.forEach(({y,n})=>{ const el=mapEl.querySelector('[data-y="'+y+'"]'); if(!el) return;
- el.classList.remove('loading'); const nn=Math.max(0,n); counts[ck(mySrc,y)]=nn;
- el.style.setProperty('--heat',(Math.sqrt(nn)/max).toFixed(3));
- el.querySelector('.yc').textContent=nn.toLocaleString();
- el.title=y+' — '+nn.toLocaleString()+' shows';
- });
+
+ // 1) paint known-good CACHED counts immediately — kills the "…" ellipsis for repeat visitors
+ YEARS.forEach(y=>{ const c=mrCacheLoad(mySrc,y); if(c!=null){ counts[ck(mySrc,y)]=c; mrPaintCount(mySrc,y,c); } });
+ mrApplyHeat(mySrc);
+
+ // 2) refresh live via a capped, retrying, circuit-broken pool (max 5 concurrent)
+ const breaker={fails:0, open:false};
+ mrPool(YEARS, MR_POOL_LIMIT, y=>
+ mrFetchCount('https://archive.org/advancedsearch.php?q='+encodeURIComponent('collection:'+srcId+' AND year:'+y)+'&rows=0&output=json', sig, breaker)
+ .then(n=>{
+ if(mySrc!==window.MR_SRC) return {y,ok:false};
+ const nn=Math.max(0,n);
+ if(nn>0){ counts[ck(mySrc,y)]=nn; mrCacheSave(mySrc,y,nn); mrPaintCount(mySrc,y,nn); mrApplyHeat(mySrc); }
+ else { mrMarkFailed(mySrc,y); } // genuine 0 for a real year — don't cache, don't fake it
+ return {y,ok:true};
+ })
+ .catch(()=>{ if(mySrc===window.MR_SRC) mrMarkFailed(mySrc,y); return {y,ok:false}; }) // live failure NEVER overwrites a good cached value
+ ).then(()=>{
+ if(mySrc!==window.MR_SRC) return; // source switched mid-flight
+ mrApplyHeat(mySrc);
+ const anyKnown=YEARS.some(y=>counts[ck(mySrc,y)]!=null);
+ if(!anyKnown){ mapEl.innerHTML='<div class="mr-map-err">Couldn’t load the tour history — archive.org may be unreachable. Search still works.</div>'; return; }
const cur=yearEl.value, c=cur?counts[ck(mySrc,cur)]:null;
if(cur&&c!=null) mapLabel.textContent=`${cur} · ${c.toLocaleString()} shows`;
});
diff --git a/tests/live-happy.cjs b/tests/live-happy.cjs
new file mode 100644
index 0000000..1203de4
--- /dev/null
+++ b/tests/live-happy.cjs
@@ -0,0 +1,82 @@
+// Phase 1 happy-path — REAL archive.org (no route interception), local edited index.html.
+// Confirms no regression: 31 GD tiles populate with real counts (1994 ~= 460), D&C source loads.
+// Run: PW=/Users/macstudio3/.npm-global/lib/node_modules/playwright node tests/live-happy.cjs
+const { chromium } = require(process.env.PW);
+const http = require('http');
+const fs = require('fs');
+const path = require('path');
+
+const ROOT = path.resolve(__dirname, '..');
+const MIME = { '.html':'text/html', '.js':'text/javascript', '.css':'text/css', '.json':'application/json', '.png':'image/png', '.jpg':'image/jpeg', '.svg':'image/svg+xml', '.ico':'image/x-icon' };
+function serve() {
+ return new Promise(res => {
+ const srv = http.createServer((req, r) => {
+ let p = decodeURIComponent(req.url.split('?')[0]);
+ if (p === '/' || p === '') p = '/index.html';
+ const f = path.join(ROOT, p);
+ if (!f.startsWith(ROOT) || !fs.existsSync(f) || fs.statSync(f).isDirectory()) { r.writeHead(404); return r.end('nf'); }
+ r.writeHead(200, { 'Content-Type': MIME[path.extname(f)] || 'application/octet-stream' });
+ fs.createReadStream(f).pipe(r);
+ });
+ srv.listen(0, '127.0.0.1', () => res({ srv, port: srv.address().port }));
+ });
+}
+
+(async () => {
+ const { srv, port } = await serve();
+ const base = `http://127.0.0.1:${port}/index.html`;
+ const browser = await chromium.launch({ channel: 'chrome', args: ['--use-gl=swiftshader'] });
+ const page = await browser.newPage();
+ const errs = [];
+ page.on('pageerror', e => errs.push(String(e.message || e)));
+ const fails = [];
+ const pass = (c, m) => { console.log((c ? 'PASS' : 'FAIL') + ' — ' + m); if (!c) fails.push(m); };
+
+ try {
+ await page.goto(base, { waitUntil: 'load', timeout: 60000 });
+ await page.waitForFunction(() => document.querySelectorAll('.mr-yr').length > 0, { timeout: 30000 });
+ const tiles = await page.$$eval('.mr-yr', els => els.length);
+ pass(tiles === 31, `GD source renders 31 year tiles (got ${tiles})`);
+
+ // wait for real counts to land (allow generous time — capped pool + real network)
+ await page.waitForFunction(() => {
+ const el = document.querySelector('.mr-yr[data-y="1994"] .yc');
+ const t = el ? (el.textContent || '').trim() : '';
+ return /^[0-9][0-9,]*$/.test(t) && parseInt(t.replace(/,/g, ''), 10) > 0;
+ }, { timeout: 60000 });
+
+ const y1994 = await page.$eval('.mr-yr[data-y="1994"] .yc', e => (e.textContent || '').trim());
+ const n1994 = parseInt(y1994.replace(/,/g, ''), 10);
+ pass(n1994 >= 400 && n1994 <= 520, `1994 shows a real count ~= 460 (got ${y1994})`);
+
+ // count how many of the 31 tiles populated with a real number
+ const populated = await page.$$eval('.mr-yr .yc', els => els.filter(e => /^[0-9][0-9,]*$/.test((e.textContent||'').trim()) && +(e.textContent.replace(/,/g,'')) > 0).length);
+ pass(populated >= 28, `most GD tiles populated with real counts (${populated}/31)`);
+ const zero = await page.$$eval('.mr-yr .yc', els => els.some(e => (e.textContent || '').trim() === '0'));
+ pass(!zero, `no GD tile shows a false "0"`);
+
+ // cache written for historical years
+ const cacheN = await page.evaluate(() => Object.keys(JSON.parse(localStorage.getItem('srm.counts.v1') || '{}')).length);
+ pass(cacheN >= 20, `historical counts cached to localStorage (${cacheN} entries)`);
+
+ // switch to Dead & Company source
+ await page.evaluate(() => { const b = document.querySelector('#mrSrc button[data-src="dc"]'); if (b) b.click(); });
+ await page.waitForFunction(() => {
+ const tiles = document.querySelectorAll('.mr-yr');
+ return tiles.length >= 8 && [...tiles].some(t => /^[0-9][0-9,]*$/.test((t.querySelector('.yc')?.textContent||'').trim()));
+ }, { timeout: 60000 });
+ const dcTiles = await page.$$eval('.mr-yr', els => els.length);
+ pass(dcTiles >= 8 && dcTiles <= 9, `D&C source loads (2015-2023 → ${dcTiles} tiles)`);
+
+ pass(errs.length === 0, `no page errors: ${errs.join(' | ') || 'clean'}`);
+ } catch (e) {
+ fails.push('EXCEPTION: ' + (e && e.stack || e));
+ console.log('EXCEPTION', e);
+ } finally {
+ await browser.close();
+ srv.close();
+ }
+
+ console.log('\n' + (fails.length ? `RESULT: ${fails.length} FAILURE(S)` : 'RESULT: LIVE HAPPY-PATH PASS'));
+ process.exit(fails.length ? 1 : 0);
+})();
diff --git a/tests/mock-reliability.cjs b/tests/mock-reliability.cjs
new file mode 100644
index 0000000..a426b42
--- /dev/null
+++ b/tests/mock-reliability.cjs
@@ -0,0 +1,178 @@
+// Phase 1 reliability test — archive.org MOCKED via route interception (independent of real archive.org).
+// Run: PW=/Users/macstudio3/.npm-global/lib/node_modules/playwright node tests/mock-reliability.cjs
+const { chromium } = require(process.env.PW);
+const http = require('http');
+const fs = require('fs');
+const path = require('path');
+
+const ROOT = path.resolve(__dirname, '..');
+const MIME = { '.html':'text/html', '.js':'text/javascript', '.css':'text/css', '.json':'application/json', '.png':'image/png', '.jpg':'image/jpeg', '.svg':'image/svg+xml', '.ico':'image/x-icon' };
+
+function serve() {
+ return new Promise(res => {
+ const srv = http.createServer((req, r) => {
+ let p = decodeURIComponent(req.url.split('?')[0]);
+ if (p === '/' || p === '') p = '/index.html';
+ const f = path.join(ROOT, p);
+ if (!f.startsWith(ROOT) || !fs.existsSync(f) || fs.statSync(f).isDirectory()) { r.writeHead(404); return r.end('nf'); }
+ r.writeHead(200, { 'Content-Type': MIME[path.extname(f)] || 'application/octet-stream' });
+ fs.createReadStream(f).pipe(r);
+ });
+ srv.listen(0, '127.0.0.1', () => res({ srv, port: srv.address().port }));
+ });
+}
+
+const CORS = { 'access-control-allow-origin': '*', 'content-type': 'application/json' };
+const okBody = (year) => JSON.stringify({ response: { numFound: (year - 1960) * 30, docs: [] } });
+const searchBody = JSON.stringify({ response: { numFound: 12345, docs: [{ identifier: 'gd1994', title: 'Grateful Dead', date: '1994-01-01', venue: 'Test', coverage: 'X', downloads: 9 }] } });
+
+function yearOf(url) { const m = /year%3A(\d{4})|year:(\d{4})/.exec(decodeURIComponent(url)); return m ? +(m[1] || m[2]) : null; }
+const isCount = (url) => /rows=0/.test(url);
+
+// HOLD each count request open ~120ms so concurrent requests actually pile up — otherwise
+// instant fulfillment means nothing ever overlaps and the cap assertion is meaningless.
+const HOLD = 120;
+
+// Instrumented route handler factory. mode: 'partial' | 'sustained'
+function makeHandler(state, mode) {
+ return async (route) => {
+ const url = route.request().url();
+ if (!/archive\.org\/advancedsearch/.test(url)) return route.continue();
+ if (!isCount(url)) { // search query (rows=40) — not part of the throttled burst
+ return route.fulfill({ status: 200, headers: CORS, body: searchBody });
+ }
+ const year = yearOf(url);
+ state.totalCount++;
+ state.attempts[year] = (state.attempts[year] || 0) + 1;
+ (state.times[year] = state.times[year] || []).push(Date.now()); // for backoff-timing assertions
+ state.active++;
+ if (state.active > state.maxActive) state.maxActive = state.active;
+ await new Promise(r => setTimeout(r, HOLD)); // hold the slot so the pool visibly saturates
+ const done = (fn) => { state.active--; return fn(); };
+
+ if (mode === 'sustained') { // everything fails → circuit breaker must halt the storm
+ return done(() => route.abort('failed'));
+ }
+ // partial (~40% failure mix): index buckets across the GD 1965-1995 range
+ const idx = year - 1965;
+ const bucket = ((idx % 5) + 5) % 5;
+ const attempt = state.attempts[year];
+ if (bucket === 3) { // permanent failure — half timeout, half malformed JSON
+ if (year % 2 === 0) return done(() => route.abort('failed'));
+ return done(() => route.fulfill({ status: 200, headers: { 'access-control-allow-origin': '*', 'content-type': 'application/json' }, body: 'THIS-IS-NOT-JSON' }));
+ }
+ if (bucket === 4) { // 429 on first attempt, success after — tests distinct 429 handling + backoff
+ state.saw429.add(year);
+ if (attempt === 1) return done(() => route.fulfill({ status: 429, headers: CORS, body: '{"error":"rate limited"}' }));
+ return done(() => route.fulfill({ status: 200, headers: CORS, body: okBody(year) }));
+ }
+ return done(() => route.fulfill({ status: 200, headers: CORS, body: okBody(year) }));
+ };
+}
+
+async function settleTiles(page) {
+ // wait until no tile is still "loading" (all resolved to a number or an err mark), OR the map error appears
+ await page.waitForFunction(() => {
+ if (document.querySelector('.mr-map-err')) return true;
+ const tiles = document.querySelectorAll('.mr-yr');
+ if (!tiles.length) return false;
+ return ![...tiles].some(t => t.classList.contains('loading'));
+ }, { timeout: 45000 });
+}
+
+(async () => {
+ const { srv, port } = await serve();
+ const base = `http://127.0.0.1:${port}/index.html`;
+ const browser = await chromium.launch({ channel: 'chrome', args: ['--use-gl=swiftshader'] });
+ const fails = [];
+ const pass = (c, m) => { console.log((c ? 'PASS' : 'FAIL') + ' — ' + m); if (!c) fails.push(m); };
+
+ try {
+ // ---------- Scenario A: partial 40% failure mix ----------
+ {
+ const ctx = await browser.newContext();
+ const page = await ctx.newPage();
+ const errs = [];
+ page.on('pageerror', e => errs.push(String(e.message || e)));
+ const state = { active: 0, maxActive: 0, totalCount: 0, attempts: {}, times: {}, saw429: new Set() };
+ await page.route('**/archive.org/**', makeHandler(state, 'partial'));
+ await page.goto(base, { waitUntil: 'load', timeout: 60000 });
+ await page.waitForFunction(() => document.querySelectorAll('.mr-yr').length > 0, { timeout: 30000 });
+ await settleTiles(page);
+ await page.waitForTimeout(500); // let any trailing paint flush
+
+ // (a) concurrency cap — must saturate the pool (>=4 in flight) yet NEVER exceed 5.
+ // The old unbounded Promise.all would drive this to 31; the cap holds it at 5.
+ pass(state.maxActive <= 5 && state.maxActive >= 4, `(a) count-query concurrency saturated at ${state.maxActive} (cap 5, never exceeded; unbounded would be 31)`);
+
+ // (b) no tile shows "0"; failed tiles show err/—; failed years never cached
+ const ycTexts = await page.$$eval('.mr-yr .yc', els => els.map(e => (e.textContent || '').trim()));
+ pass(!ycTexts.includes('0'), `(b) no year tile displays "0" (values: ${[...new Set(ycTexts)].slice(0,8).join(',')}…)`);
+ const errYears = await page.$$eval('.mr-yr.err', els => els.map(e => e.dataset.y));
+ pass(errYears.length > 0 && (await page.$$eval('.mr-yr.err .yc', els => els.every(e => (e.textContent||'').trim() === '—'))),
+ `(b) ${errYears.length} failed tiles render "—" error state, not a false 0`);
+ const cache = await page.evaluate(() => JSON.parse(localStorage.getItem('srm.counts.v1') || '{}'));
+ const cachedFailed = errYears.filter(y => cache['gd:' + y] !== undefined);
+ pass(cachedFailed.length === 0, `(b) failed years never persisted to cache (leaked: ${cachedFailed.join(',') || 'none'})`);
+ const cacheVals = Object.values(cache).map(v => v.n);
+ pass(cacheVals.length > 0 && cacheVals.every(n => n > 0), `(b) cache holds only positive counts (${cacheVals.length} entries)`);
+
+ // (c) bounded retries per request
+ const maxAttempts = Math.max(0, ...Object.values(state.attempts));
+ pass(maxAttempts <= 3, `(c) per-request attempts <= 3 (1 + 2 retries); observed max ${maxAttempts}`);
+
+ // (d) 429 handled distinctly — a 429-then-200 year shows its REAL number (not parsed as success, not a 0/—)
+ const y429 = [...state.saw429][0];
+ const t429 = y429 == null ? null : await page.$eval(`.mr-yr[data-y="${y429}"] .yc`, e => (e.textContent || '').trim());
+ const expect429 = y429 == null ? null : ((y429 - 1960) * 30).toLocaleString();
+ pass(y429 != null && t429 === expect429 && state.attempts[y429] >= 2,
+ `(d) 429 year ${y429}: backed off + retried (attempts ${state.attempts[y429]}) → shows real ${t429} (expected ${expect429}), not parsed as JSON success`);
+
+ // (d) prove the 429 backoff is HARDER than a timeout backoff (not just "it retried").
+ // Gap between consecutive handler entries for a year = HOLD + that year's inter-attempt backoff.
+ const gap = (y) => { const t = state.times[y]; return t && t.length >= 2 ? t[1] - t[0] : null; };
+ const g429 = gap(y429);
+ // a timeout-class retry year: bucket 3 AND even (route.abort) — it retries on the network-drop path
+ const toYear = Object.keys(state.times).map(Number).find(y => ((y - 1965) % 5 + 5) % 5 === 3 && y % 2 === 0 && (state.times[y] || []).length >= 2);
+ const gTO = toYear != null ? gap(toYear) : null;
+ pass(g429 != null && gTO != null && g429 > gTO,
+ `(d) 429 backoff ${g429}ms > timeout backoff ${gTO}ms (year ${toYear}) — 429 backs off HARDER, distinct code path`);
+
+ pass(errs.length === 0, `(no page errors) ${errs.join(' | ') || 'clean'}`);
+ await ctx.close();
+ }
+
+ // ---------- Scenario B: sustained failure → circuit breaker halts the storm ----------
+ {
+ const ctx = await browser.newContext();
+ const page = await ctx.newPage();
+ const state = { active: 0, maxActive: 0, totalCount: 0, attempts: {}, times: {}, saw429: new Set() };
+ await page.route('**/archive.org/**', makeHandler(state, 'sustained'));
+ await page.goto(base, { waitUntil: 'load', timeout: 60000 });
+ await page.waitForFunction(() => document.querySelector('.mr-map-err') || [...document.querySelectorAll('.mr-yr')].every(t => !t.classList.contains('loading')), { timeout: 45000 });
+ await page.waitForTimeout(800); // ensure no late retry storm
+
+ pass(state.maxActive <= 5, `(a) sustained: max concurrent ${state.maxActive} <= 5`);
+ // circuit breaker: total requests must be far below unthrottled worst case (31 * 3 = 93) and years short-circuited
+ const totalYears = 31;
+ const yearsRequested = Object.keys(state.attempts).length;
+ pass(state.totalCount <= 30, `(c) circuit-breaker bounds total requests to ${state.totalCount} (<=30; unthrottled worst case ~93)`);
+ pass(yearsRequested < totalYears, `(c) circuit-breaker short-circuited ${totalYears - yearsRequested} years (only ${yearsRequested} hit the network)`);
+ const maxAttempts = Math.max(0, ...Object.values(state.attempts));
+ pass(maxAttempts <= 3, `(c) sustained: per-request attempts <= 3; observed ${maxAttempts}`);
+ const hasErr = await page.$('.mr-map-err');
+ const zeroTile = await page.$$eval('.mr-yr .yc', els => els.some(e => (e.textContent || '').trim() === '0'));
+ pass(!!hasErr && !zeroTile, `(b) sustained failure → graceful map error, no tile shows "0"`);
+ await ctx.close();
+ }
+ } catch (e) {
+ fails.push('EXCEPTION: ' + (e && e.stack || e));
+ console.log('EXCEPTION', e);
+ } finally {
+ await browser.close();
+ srv.close();
+ }
+
+ console.log('\n' + (fails.length ? `RESULT: ${fails.length} FAILURE(S)` : 'RESULT: ALL MOCK GUARDRAILS PASS'));
+ process.exit(fails.length ? 1 : 0);
+})();
← 9899b13 contrarian FIX FIRST: per-build AbortController cancels stal
·
back to Scarlet Riverboat Masquerade
·
snapshot before restart: preserve in-flight work a5a2c5a →