TK-11471: make test_weight_guard.py hermetic — it was green on 09-11 and red on 09-12 with no code change

2b5408bffdb3533ac9d4d3e05b1857c1ca7bce17 · 2026-09-12 06:40:32 -0700 · Steve Abrams

The 4 TestMigratePublishGate cases drive the REAL process() apply path, which calls
require_no_prior_creation() — and that reads the SHARED, LIVE, append-only
executed-reversible ledger. The fixture SKU GRS-27580 is a real SKU, so once the live
migration recorded creation receipts for it the guard began raising

  HISTORY_HOLD: GRS-27580 already has 3 recorded product(s) ...

These tests passed on 2026-09-11 and errored on 2026-09-12 with ZERO code change between
them, purely because production state moved underneath them.

That is the same defect class this file exists to prevent, one level up: a check whose
verdict is decided by something it does not actually measure. It is harmful in BOTH
directions — it can go red on healthy code, and once it is known to be flaky a real red
gets waved off as "just that one".

Fix is test-local; NO production change, so no new seam a scheduled job could trip
(require_no_prior_creation's own docstring is explicit that ledger_path is a function
argument only, never an env var or CLI flag, for exactly that reason). The harness points
m.LEDGER at an empty temp file, cleaned up via addCleanup.

The guard is NOT stubbed out — it still runs for real against a controlled input.
Verified after the fix: seeding that temp ledger with a correctly-shaped
create_grs_draft receipt still raises HISTORY_HOLD, so a genuine duplicate-identity
regression is still caught; only the shared live file is swapped for a deterministic one.
The guard also keeps its own dedicated coverage in scripts/tests/test_receipt_guard.py
(22/22 passing).

Also checked, because "3 recorded products" could have meant a real customer-facing
duplicate: GRS-27580 has exactly ONE live product on the store
(gid://shopify/Product/7953330307123, ACTIVE). The other two receipts are historical /
archived. No duplicate defect.

  before: Ran 14 tests — FAILED (errors=4)
  after:  Ran 14 tests — OK

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Files touched

M scripts/lib/test_weight_guard.py

Diff

commit 2b5408bffdb3533ac9d4d3e05b1857c1ca7bce17
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Sat Sep 12 06:40:32 2026 -0700

    TK-11471: make test_weight_guard.py hermetic — it was green on 09-11 and red on 09-12 with no code change
    
    The 4 TestMigratePublishGate cases drive the REAL process() apply path, which calls
    require_no_prior_creation() — and that reads the SHARED, LIVE, append-only
    executed-reversible ledger. The fixture SKU GRS-27580 is a real SKU, so once the live
    migration recorded creation receipts for it the guard began raising
    
      HISTORY_HOLD: GRS-27580 already has 3 recorded product(s) ...
    
    These tests passed on 2026-09-11 and errored on 2026-09-12 with ZERO code change between
    them, purely because production state moved underneath them.
    
    That is the same defect class this file exists to prevent, one level up: a check whose
    verdict is decided by something it does not actually measure. It is harmful in BOTH
    directions — it can go red on healthy code, and once it is known to be flaky a real red
    gets waved off as "just that one".
    
    Fix is test-local; NO production change, so no new seam a scheduled job could trip
    (require_no_prior_creation's own docstring is explicit that ledger_path is a function
    argument only, never an env var or CLI flag, for exactly that reason). The harness points
    m.LEDGER at an empty temp file, cleaned up via addCleanup.
    
    The guard is NOT stubbed out — it still runs for real against a controlled input.
    Verified after the fix: seeding that temp ledger with a correctly-shaped
    create_grs_draft receipt still raises HISTORY_HOLD, so a genuine duplicate-identity
    regression is still caught; only the shared live file is swapped for a deterministic one.
    The guard also keeps its own dedicated coverage in scripts/tests/test_receipt_guard.py
    (22/22 passing).
    
    Also checked, because "3 recorded products" could have meant a real customer-facing
    duplicate: GRS-27580 has exactly ONE live product on the store
    (gid://shopify/Product/7953330307123, ACTIVE). The other two receipts are historical /
    archived. No duplicate defect.
    
      before: Ran 14 tests — FAILED (errors=4)
      after:  Ran 14 tests — OK
    
    Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
 scripts/lib/test_weight_guard.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/scripts/lib/test_weight_guard.py b/scripts/lib/test_weight_guard.py
index 243b5a8..941bfed 100644
--- a/scripts/lib/test_weight_guard.py
+++ b/scripts/lib/test_weight_guard.py
@@ -13,6 +13,7 @@ result=PUBLISHED_NO_TWIN, 37 ensure=create + 3 ensure=update).
 """
 import importlib.util
 import os
+import tempfile
 import sys
 import unittest
 
@@ -191,6 +192,27 @@ class TestMigratePublishGate(unittest.TestCase):
         m.verify_image = lambda pid, url, **kw: (True, "READY", ["READY"])
         m.publish_active = fake_publish
         m.archive_dwpw = lambda twin, grs: state.__setitem__("archived", True)
+        # HERMETICITY (fixed 2026-09-12): require_no_prior_creation() reads the SHARED, LIVE,
+        # append-only executed-reversible ledger. The fixture SKU GRS-27580 is a REAL sku, so once
+        # the live migration recorded creation receipts for it the guard began raising HISTORY_HOLD
+        # and these 4 tests errored — they passed on 2026-09-11 and failed on 2026-09-12 with ZERO
+        # code change between, purely because production state moved underneath them.
+        #
+        # That is the same defect class this whole test file exists to prevent, one level up: a
+        # check whose verdict is decided by something it does not actually measure. A test that can
+        # silently flip on unrelated live activity is worthless as a signal in BOTH directions —
+        # it can go red on healthy code, and it can be dismissed as "just that flaky one" when it
+        # goes red for a real reason.
+        #
+        # Point the guard at an EMPTY temp ledger. The guard itself is NOT stubbed out — it still
+        # runs for real against a controlled input, so a genuine duplicate-identity regression is
+        # still caught; only the shared live file is swapped for a deterministic one.
+        # (Verified while fixing: GRS-27580 has exactly ONE live product, so the 3 ledger receipts
+        # were historical/archived, not a real customer-facing duplicate.)
+        tmp = tempfile.NamedTemporaryFile("w", suffix=".jsonl", delete=False)
+        tmp.close()
+        self.addCleanup(os.unlink, tmp.name)
+        m.LEDGER = tmp.name
         return m, state
 
     def test_zero_weight_product_is_HEALED_then_PUBLISHED(self):

← b7632d3 chore: lint, refactor, v0.1.14 (session close) · back to Designerwallcoverings · chore: v0.1.15 (TK-11471 session close — weight go-live guar 5b0e813 →