{"slug":"paul-conrad-archive","total":36,"limit":100,"offset":0,"since":null,"commits":[{"hash":"167743e","date":"2026-09-25 11:51:56 -0700","author":"Steve Abrams","subject":"Robots re-audit + web-image hotlink allowlist (TK-12199)","body":"- Re-checked all 17,795 cartoon_sources record/image/thumbnail URLs against fresh robots.txt (26 hosts) with RobotRules.\n latimes photogallery is ALLOWED for our UA: 'Disallow: /*photogallery' is only in the Googlebot-News group, not '*'.\n 0 source rows deleted. Tests pin the real latimes robots.txt (fixture) and prove Http.get refuses the gallery\n before any I/O if the '*' group ever disallows it.\n- imagehost: allowlist ca-times.brightspotcdn.com, library.syracuse.edu, www.original-political-cartoon.com,\n pophistorydig.com, www.truthdig.com (49/49 image URLs verified 200 image/* by HEAD, no bytes kept). Page URLs on\n those hosts still return null. image.invaluable.com excluded (robots.txt 403 -> treated disallowed). CSP follows.\n- Re-ran dedupe, exports, report --gaps; pytest 153 passed. Viewer: 67 records with display_image.\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01SW5KHgwrh2Rr9TKHM6ndqM"},{"hash":"1c16566","date":"2026-09-25 11:26:08 -0700","author":"Steve Abrams","subject":"Google-style robots matcher (wildcards, $, longest-match, product-token groups) + tests; web_images crawler: Syracuse exhibit, dealer, Invaluable, Pop History Dig, Truthdig (32 records w/ image URLs); probes: Stanford Daily API, Truthdig wp-json","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01SW5KHgwrh2Rr9TKHM6ndqM"},{"hash":"80ae388","date":"2026-09-25 11:24:00 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-25T11:23:40 (1 data files) — data/conrad.db","body":""},{"hash":"7735e27","date":"2026-09-25 11:08:54 -0700","author":"Steve Abrams","subject":"IA FTS: Editor & Publisher syndicate-ad pages, IA access-restricted check before recording page images, FTS 400 retry; probes: LBJ/JFK eMuseum 403 (no UA swap), Billy Ireland DC 504, Chronicling America 23 hits/0 cartoons; DPLA creator query","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01SW5KHgwrh2Rr9TKHM6ndqM"},{"hash":"92c289d","date":"2026-09-25 11:06:51 -0700","author":"Steve Abrams","subject":"docs: 5 unsent draft research-access letters (CDNC, Colorado CHNC, Portal to Texas History, Utah Digital Newspapers, LA Times permissions)","body":"Draft only, never sent. Google News Archive skipped (no contact channel); Billy Ireland already covered by osu-request-DRAFT.md. TK-12199\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01SW5KHgwrh2Rr9TKHM6ndqM"},{"hash":"f2625ec","date":"2026-09-25 10:55:30 -0700","author":"Steve Abrams","subject":"IA full-text classifier: strict credit-line + prose-reprint rules, stated-title extraction, award/roster/music negatives","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01SW5KHgwrh2Rr9TKHM6ndqM"},{"hash":"851fc79","date":"2026-09-25 10:53:07 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-25T10:52:57 (1 data files) — data/conrad.db","body":""},{"hash":"25daa26","date":"2026-09-25 10:52:07 -0700","author":"Steve Abrams","subject":"LA Times: sitemap discovery + 2010 obituary gallery parser (26 cartoons w/ image URLs) + quote-verified letters citation; DPLA NPG Time covers; base.py gzip truncation fix; IA full-text + newspaper-probe crawlers; tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01SW5KHgwrh2Rr9TKHM6ndqM"},{"hash":"020434a","date":"2026-09-25 00:34:12 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-25T00:34:03 (1 data files) — data/conrad.db","body":""},{"hash":"95d0fc0","date":"2026-09-24 23:32:50 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-24T23:32:41 (1 data files) — data/sources.json","body":""},{"hash":"56bb0e9","date":"2026-09-24 23:12:03 -0700","author":"Steve Abrams","subject":"Rights note: reflect hotlinked institutional images (no copy stored) instead of 'no image is reproduced' (68 rows)","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"fb7c43e","date":"2026-09-24 23:11:07 -0700","author":"Steve Abrams","subject":"Reserve card image height before lazy load so filtered grids don't collapse","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01W5NGeZrf7GXNQrfMRCTnRc"},{"hash":"63f07b5","date":"2026-09-24 23:08:21 -0700","author":"Steve Abrams","subject":"Show institution-published cartoon images by hotlink only (allowlist + CSP), All cartoons view + Has image filter; replace blanket no-img test with allowlist tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \nClaude-Session: https://claude.ai/code/session_01W5NGeZrf7GXNQrfMRCTnRc"},{"hash":"ddb5417","date":"2026-09-24 19:18:44 -0700","author":"Steve Abrams","subject":"Draft (unsent) data-request letters to Syracuse, Wichita State, Ohio State Billy Ireland","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"0f34d5c","date":"2026-09-24 18:58:20 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-24T18:58:12 (1 data files) — data/conrad.db","body":""},{"hash":"ee3f521","date":"2026-09-24 18:27:43 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-24T18:27:35 (1 data files) — data/conrad.db","body":""},{"hash":"e1a4fb4","date":"2026-09-24 18:16:48 -0700","author":"Steve Abrams","subject":"NCR citation: use live natcath.org original (200, no robots) instead of Wayback copy — Wayback-only-for-retired-origins policy now holds 29/29 (Cody c3)","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"f12cde5","date":"2026-09-24 18:11:45 -0700","author":"Steve Abrams","subject":"Fix PEP701 nested-quote f-strings (broke Kamatera py3.10); add py3.10 py_compile test over all sources (proven red on the bug)","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"e4fc551","date":"2026-09-24 18:10:34 -0700","author":"Steve Abrams","subject":"Copyright rail layer 2: refuse image/PDF responses by Content-Type or magic bytes before body read; negative test proven red without guard","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"5ed3bca","date":"2026-09-24 18:08:47 -0700","author":"Steve","subject":"Cycle 3 data refresh: dedupe/exports/REPORT after live 10-sample re-verification (10/10) + non-PBS re-check","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"e088856","date":"2026-09-24 18:07:28 -0700","author":"Steve","subject":"Cycle 3: quote-verified secondary-citation crawler (PBS gallery, LA Times obit, LOC bulletin, Wikipedia, NCR, Canyon News) + Commons/Wikidata/Pulitzer/Nixon/NYT/WaPo probes; Http refuses image URLs; REPORT 'How to get more'; tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"f0b0302","date":"2026-09-24 17:48:14 -0700","author":"Steve Abrams","subject":"Daily Iowan: text-layer probe (sidecars 404, dir 403, search=Google CSE robots-disallowed, IDL 403) -> no permitted text layer, 0 records; full 1945-1950 issue list export (1,845 issues) for manual review; REPORT/sources updated; tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"5d9802d","date":"2026-09-24 17:41:46 -0700","author":"Steve Abrams","subject":"Viewer search: q now matches repository, collection_name, publication (denormalised cartoon_index.search_text, migration + auto-rebuild); tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"b6988b4","date":"2026-09-24 17:36:29 -0700","author":"Steve Abrams","subject":"LOC deep enumeration run: +1 group record (2010634752), 2 lots reclassified folder, unprocessed single prints stay items; exports + REPORT regenerated with acquisition_method counts","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"0fbcd4e","date":"2026-09-24 17:26:36 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-24T17:26:27 (1 data files) — data/conrad.db","body":""},{"hash":"84113e0","date":"2026-09-24 17:04:54 -0700","author":"Steve Abrams","subject":"LOC deep enumeration: robots audit per path, more format endpoints, notes-based Conrad attribution, group-component check, bounded checkpointed neighbour-LCCN probing; tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"e9c5e1c","date":"2026-09-24 17:03:01 -0700","author":"Steve Abrams","subject":"Dedupe: generic/[bracketed] title matches need identifier or finer-than-year date agreement; different LOC pks never auto-link; negative test","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"88b34d2","date":"2026-09-24 17:01:41 -0700","author":"Steve Abrams","subject":"Add acquisition_method provenance field: idempotent migration + backfill, API/search flags, exports, detail-page reader-bypass badge, REPORT counts, tests","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"9144a7e","date":"2026-09-24 16:48:09 -0700","author":"Steve Abrams","subject":"Ignore SQLite WAL/SHM sidecar files","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"0441366","date":"2026-09-24 16:47:59 -0700","author":"Steve Abrams","subject":"Search index, cross-year duplicate review, README, Huntington request draft, py3.10 requirements, data + REPORT","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"00dd3c6","date":"2026-09-24 16:40:51 -0700","author":"Steve Abrams","subject":"Dedupe, exports, REPORT, FastAPI viewer (link-out only, basic auth, noindex), tests incl. negative no-image test","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"a9bba5a","date":"2026-09-24 16:28:05 -0700","author":"Steve Abrams","subject":"Live crawlers: LOC item JSON, OAC, Syracuse/OSU/Wichita probes, paywalled registrations, Daily Iowan inventory, IA books, DPLA/SI discovery","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"bc3f93c","date":"2026-09-24 16:24:37 -0700","author":"auto-commit-fleet","subject":"auto-data-snapshot: 2026-09-24T16:24:12 (3 data files) — data/conrad.db-shm data/conrad.db-wal pyproject.toml","body":""},{"hash":"8ff2c42","date":"2026-09-24 16:16:35 -0700","author":"Steve Abrams","subject":"Live Huntington EAD crawler: 8628 CON slots verified, box inventory, charset fix","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"380110b","date":"2026-09-24 16:13:53 -0700","author":"Steve Abrams","subject":"Core schema, models, normalizers, polite HTTP base and seed importer","body":"Co-Authored-By: Claude Opus 5.5 (1M context) "},{"hash":"e3065ee","date":"2026-09-24 16:07:15 -0700","author":"Steve Abrams","subject":"initial scaffold","body":"Co-Authored-By: Claude Opus 5.5 (1M context) \n"}]}