elements bake — recover 4 truncated Round-1 PNGs via targeted LOAD_TRUNCATED_IMAGES fallback

cf8fd5da68368e0ce2409e73f7b2ab49853a69a4 · 2026-05-29 17:09:44 -0700 · Steve Abrams

Patched elements_extract_prod.py bake_one() with a fallback path: on
PIL decode failure, retry once with ImageFile.LOAD_TRUNCATED_IMAGES=True
and tag result with truncated_ok=True for audit. Strict-mode remains
default so genuinely corrupt new bytes still fail loudly.

4 prod-PG rows (5108, 5287, 5346, 5354 — all replicate-generator) had
PNGs missing the trailing CRC/IEND chunk but intact 1024x1024 RGB pixel
data. Bake recovered all 4 (motif 66-200KB, ref 58-173KB).

Final prod elements bake ratio: 1210/1212 = 99.83% baked. The 2
remaining (10161, 10162) are recolor variants whose source PNGs are
missing on disk — logged to data/elements-unbaked-notes.md, left
unbaked per 'only clean designs welcome' rule.

DTD verdict B (2/2, Qwen errored), Codex+Claude agreed: targeted
fallback over global flag — preserves corruption signal for new
outputs while salvaging known-good Round-1 bytes.

Files touched

A data/elements-unbaked-notes.md
M scripts/elements_extract_prod.py

Diff

commit cf8fd5da68368e0ce2409e73f7b2ab49853a69a4
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Fri May 29 17:09:44 2026 -0700

    elements bake — recover 4 truncated Round-1 PNGs via targeted LOAD_TRUNCATED_IMAGES fallback
    
    Patched elements_extract_prod.py bake_one() with a fallback path: on
    PIL decode failure, retry once with ImageFile.LOAD_TRUNCATED_IMAGES=True
    and tag result with truncated_ok=True for audit. Strict-mode remains
    default so genuinely corrupt new bytes still fail loudly.
    
    4 prod-PG rows (5108, 5287, 5346, 5354 — all replicate-generator) had
    PNGs missing the trailing CRC/IEND chunk but intact 1024x1024 RGB pixel
    data. Bake recovered all 4 (motif 66-200KB, ref 58-173KB).
    
    Final prod elements bake ratio: 1210/1212 = 99.83% baked. The 2
    remaining (10161, 10162) are recolor variants whose source PNGs are
    missing on disk — logged to data/elements-unbaked-notes.md, left
    unbaked per 'only clean designs welcome' rule.
    
    DTD verdict B (2/2, Qwen errored), Codex+Claude agreed: targeted
    fallback over global flag — preserves corruption signal for new
    outputs while salvaging known-good Round-1 bytes.
---
 data/elements-unbaked-notes.md   | 34 ++++++++++++++++++++++++++++++++++
 scripts/elements_extract_prod.py | 17 ++++++++++++++++-
 2 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/data/elements-unbaked-notes.md b/data/elements-unbaked-notes.md
new file mode 100644
index 0000000..8956301
--- /dev/null
+++ b/data/elements-unbaked-notes.md
@@ -0,0 +1,34 @@
+# Elements Bake — Unbaked IDs Log
+
+Last updated: 2026-05-29 (ELEMENTS-77-TAIL thread)
+
+## Final state
+- Total candidates (prod PG): 1212
+- Baked: 1210
+- Unbaked: 2 (0.17%)
+
+## Unbaked — root cause categorized
+
+### missing_png (2)
+Source PNG not present on prod disk. local_path points at a
+`/root/public-projects/wallco-ai/data/generated/recolor_*` file that no
+longer exists. These are derivative recolor variants of design 10159
+(bottega-veneta + valentino colorways) — the parent root is intact and
+baked; these two variants were likely produced in a recolor batch whose
+PNGs were never rsynced or were cleaned up.
+
+- 10161 — recolor_1779703868396_p10159_bottega-veneta_1965166434.png
+- 10162 — recolor_1779703878294_p10159_valentino_349926439.png
+
+Action: leave unbaked. is_published is NOT modified per the project
+"only clean designs welcome — once flagged out, stays out" rule, but
+the rows surface no defect; the bake just can't run without the source
+file. If the recolor batch is ever re-emitted, the bake will pick them
+up on the next `--all` run.
+
+## Round-1 tail fix — 4 truncated PNGs recovered (2026-05-29)
+IDs 5108, 5287, 5346, 5354 were truncated (missing trailing CRC/IEND
+chunk) but had intact 1024x1024 RGB pixel data. Patched extract.py
+with a targeted fallback (DTD verdict B) — strict-mode by default, on
+decode-fail retry with `ImageFile.LOAD_TRUNCATED_IMAGES = True` and tag
+the result with `truncated_ok=True`. All 4 baked cleanly.
diff --git a/scripts/elements_extract_prod.py b/scripts/elements_extract_prod.py
index 31783c0..cd19058 100644
--- a/scripts/elements_extract_prod.py
+++ b/scripts/elements_extract_prod.py
@@ -140,10 +140,24 @@ def bake_one(job):
     ref_path   = OUT_DIR / f'{design_id}_ref512.png'
 
     OUT_DIR.mkdir(parents=True, exist_ok=True)
+    truncated_ok = False
     try:
         img = Image.open(src).convert('RGB')
     except Exception as e:
-        return {'id': design_id, 'ok': False, 'error': f'image_decode_fail: {e}'}
+        # Targeted fallback (DTD verdict B, 2026-05-29): some Round-1 PNGs
+        # are missing the trailing CRC/IEND chunk but have intact pixel data.
+        # Retry with LOAD_TRUNCATED_IMAGES, tag result for audit.
+        try:
+            from PIL import ImageFile as _ImageFile
+            _prev = _ImageFile.LOAD_TRUNCATED_IMAGES
+            _ImageFile.LOAD_TRUNCATED_IMAGES = True
+            try:
+                img = Image.open(src).convert('RGB')
+                truncated_ok = True
+            finally:
+                _ImageFile.LOAD_TRUNCATED_IMAGES = _prev
+        except Exception:
+            return {'id': design_id, 'ok': False, 'error': f'image_decode_fail: {e}'}
     W, H = img.size
     if min(W, H) < 256:
         return {'id': design_id, 'ok': False, 'error': f'image_too_small {W}x{H}'}
@@ -189,6 +203,7 @@ def bake_one(job):
         'dense_pattern': dense_pattern,
         'motif_bytes': motif_path.stat().st_size,
         'ref_bytes': ref_path.stat().st_size,
+        'truncated_ok': truncated_ok,
     }

← 1d9bacf fix(generator_tick): guard finishRun against non-numeric run · back to Wallco Ai · Publish 19 clean aztec-kilim designs live (ids 54417-54436, c334676 →