[object Object]

← back to Wallco Ai

tests: mask catalog content in isolate-visual VR so tests stop drifting

38b72fe04fabe3739468f44e9f837d74bbd4d71a · 2026-05-12 14:27:37 -0700 · SteveStudio2

Previously VR-1/2/3 failed every time a new design landed in the
Newest sort, because the saved PNG baked in 4 specific design tiles
that the assertion required to match pixel-for-pixel.

Now Playwright masks five selectors in both expected and actual:
.card-img, .card-room, .chips, .iso-swatch, .iso-card-img — solid
#FF00FF blocks in both screenshots so the diff zeros out for those
regions. The chrome (header, nav, search bar, modal frame, sliders,
buttons, footer) is still compared at maxDiffPixelRatio 0.02-0.03.

Verified stable: two back-to-back runs both 3/3 pass without
--update-snapshots in between. This is what the VR tests should
have been doing from the start.

Files touched

Diff

commit 38b72fe04fabe3739468f44e9f837d74bbd4d71a
Author: SteveStudio2 <stevestudio2@SteveStacStudio.lan>
Date:   Tue May 12 14:27:37 2026 -0700

    tests: mask catalog content in isolate-visual VR so tests stop drifting
    
    Previously VR-1/2/3 failed every time a new design landed in the
    Newest sort, because the saved PNG baked in 4 specific design tiles
    that the assertion required to match pixel-for-pixel.
    
    Now Playwright masks five selectors in both expected and actual:
    .card-img, .card-room, .chips, .iso-swatch, .iso-card-img — solid
    #FF00FF blocks in both screenshots so the diff zeros out for those
    regions. The chrome (header, nav, search bar, modal frame, sliders,
    buttons, footer) is still compared at maxDiffPixelRatio 0.02-0.03.
    
    Verified stable: two back-to-back runs both 3/3 pass without
    --update-snapshots in between. This is what the VR tests should
    have been doing from the start.
---
 tests/integration/isolate-visual.spec.js           |  31 +++++++++++++++++----
 ...s-review-grid-desktop-local-chromium-darwin.png | Bin 1024328 -> 74614 bytes
 ...-open-palette-desktop-local-chromium-darwin.png | Bin 725586 -> 77360 bytes
 .../3-isolate-mobile-local-chromium-darwin.png     | Bin 149280 -> 49974 bytes
 4 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/tests/integration/isolate-visual.spec.js b/tests/integration/isolate-visual.spec.js
index df3d580..c3f6044 100644
--- a/tests/integration/isolate-visual.spec.js
+++ b/tests/integration/isolate-visual.spec.js
@@ -4,13 +4,34 @@
 // review-card image-wrap aspect ratio off). Drift becomes a test fail next time
 // instead of a visual ask from Steve.
 //
-// First-time run creates the baseline PNGs. Subsequent runs diff against them.
-// Re-baseline intentionally: `npx playwright test --update-snapshots`.
+// Catalog-churn resilience: the design tiles themselves (the actual pattern
+// images and color swatches inside the cards) are MASKED in both the
+// expected and actual screenshots — Playwright paints them with a solid
+// pink #FF00FF in both images so they always match. This way the test
+// asserts chrome and layout (header, search bar, card frame, modal
+// position, sliders, buttons) without breaking every time a new design
+// lands in the "Newest" sort. Re-baseline intentionally:
+//   npx playwright test integration/isolate-visual --update-snapshots
 
 const { test, expect } = require('@playwright/test');
 
 const BASE = 'http://127.0.0.1:9792';
 
+// Elements whose content drifts with catalog state — masked in every test.
+// .card-img = design tile background (the pattern image)
+// .card-room = the mini room-mockup overlay
+// .chips = LLM-generated chip pills
+// .iso-swatch = the extracted-palette color squares (drift with which design is picked)
+// .iso-card-img = the isolate-modal's preview thumbnail
+const CONTENT_MASKS = [
+  '.card-img', '.card-room', '.chips',
+  '.iso-swatch', '.iso-card-img',
+];
+
+function masks(page) {
+  return CONTENT_MASKS.map(sel => page.locator(sel));
+}
+
 test.describe('isolate modal visual regression', () => {
   test.beforeEach(async ({ page }) => {
     // Dismiss the age-prompt overlay (intercepts clicks in /designs review mode)
@@ -23,9 +44,9 @@ test.describe('isolate modal visual regression', () => {
     await page.waitForSelector('.design-card.review-card', { timeout: 15_000 });
     // Wait for chips to lazy-load so baseline isn't flaky
     await page.waitForTimeout(2000);
-    // Mask the first 3 cards only (rest scroll out of frame) — clip to header + first row
     await expect(page).toHaveScreenshot('1-designs-review-grid-desktop.png', {
       clip: { x: 0, y: 0, width: 1440, height: 900 },
+      mask: masks(page),
       maxDiffPixelRatio: 0.02,
       fullPage: false,
     });
@@ -38,11 +59,10 @@ test.describe('isolate modal visual regression', () => {
     const firstCard = page.locator('.design-card.review-card').first();
     await firstCard.hover();
     await page.keyboard.press('i');
-    // Wait for the isolate panel to render
     await page.waitForTimeout(3000);
-    // The isolate modal may be at the top of the page or a side panel — capture the viewport
     await expect(page).toHaveScreenshot('2-isolate-open-palette-desktop.png', {
       clip: { x: 0, y: 0, width: 1440, height: 900 },
+      mask: masks(page),
       maxDiffPixelRatio: 0.02,
       fullPage: false,
     });
@@ -58,6 +78,7 @@ test.describe('isolate modal visual regression', () => {
     await page.waitForTimeout(3000);
     await expect(page).toHaveScreenshot('3-isolate-mobile.png', {
       clip: { x: 0, y: 0, width: 390, height: 844 },
+      mask: masks(page),
       maxDiffPixelRatio: 0.03,
       fullPage: false,
     });
diff --git a/tests/integration/isolate-visual.spec.js-snapshots/1-designs-review-grid-desktop-local-chromium-darwin.png b/tests/integration/isolate-visual.spec.js-snapshots/1-designs-review-grid-desktop-local-chromium-darwin.png
index 7b51817..8c5433e 100644
Binary files a/tests/integration/isolate-visual.spec.js-snapshots/1-designs-review-grid-desktop-local-chromium-darwin.png and b/tests/integration/isolate-visual.spec.js-snapshots/1-designs-review-grid-desktop-local-chromium-darwin.png differ
diff --git a/tests/integration/isolate-visual.spec.js-snapshots/2-isolate-open-palette-desktop-local-chromium-darwin.png b/tests/integration/isolate-visual.spec.js-snapshots/2-isolate-open-palette-desktop-local-chromium-darwin.png
index 4956870..12e6214 100644
Binary files a/tests/integration/isolate-visual.spec.js-snapshots/2-isolate-open-palette-desktop-local-chromium-darwin.png and b/tests/integration/isolate-visual.spec.js-snapshots/2-isolate-open-palette-desktop-local-chromium-darwin.png differ
diff --git a/tests/integration/isolate-visual.spec.js-snapshots/3-isolate-mobile-local-chromium-darwin.png b/tests/integration/isolate-visual.spec.js-snapshots/3-isolate-mobile-local-chromium-darwin.png
index 3f0b2ef..1611a77 100644
Binary files a/tests/integration/isolate-visual.spec.js-snapshots/3-isolate-mobile-local-chromium-darwin.png and b/tests/integration/isolate-visual.spec.js-snapshots/3-isolate-mobile-local-chromium-darwin.png differ

← 1e5cbb8 tests: re-baseline isolate-visual snapshots after full-suite  ·  back to Wallco Ai  ·  wallco.ai · audit log Phase 1 — capture all user + admin act 48baf5a →