← back to Designer Wallcoverings

CLAUDE.md

974 lines

- all vendors need this fix so we are always savinf data #memory
- **CRITICAL DATABASE**: ALWAYS use `dw_unified` database - NEVER create new databases! Connection string lives in `.env` (DATABASE_URL); use `dw_admin@127.0.0.1:5432/dw_unified`. NEVER commit credentials to docs.

## Shopify Plus — 100K+ Product Push Strategy

### Why Plus (vs Standard)
| Capability | Standard | Plus |
|---|---|---|
| REST req/s | 2/s (40 bucket) | 20/s (400 bucket) — **10x** |
| GraphQL pts/s | 100 pts/s | 1,000 pts/s — **10x** |
| Daily variant limit | 1,000/day (>50K variants) | **REMOVED** |
| Bulk mutations concurrent | 1 | 5 (API 2026-01+) |
| Shopify Flow | No | Yes |
| Launchpad (scheduled publishes) | No | Yes |
| B2B / wholesale | No | Yes |
| Price | — | $2,300-$2,500/mo |

### Migration: Same-store upgrade, ~24 hours, no data migration. SEO preserved.

### Bulk Operation Pipeline (100K products)
1. **Generate JSONL** — one JSON object per product line, max 100MB per file (~30K-50K products)
2. **stagedUploadsCreate** — reserve upload space (resource: BULK_MUTATION_VARIABLES, mime: text/jsonl)
3. **Upload JSONL** — POST multipart form to staged URL
4. **bulkOperationRunMutation** — submit async (NOT rate-limited)
5. **Poll or webhook** — BULK_OPERATION_FINISH webhook fires on completion
6. **Results** — JSONL download available 7 days

### Time Estimates for 100K Products on Plus
| Method | Time |
|---|---|
| REST API (20 req/s) | 5-10 hours |
| GraphQL single mutations | 2-5 hours |
| GraphQL bulk ops (1 concurrent) | 30 min - 3 hours |
| GraphQL bulk ops (5 parallel, 2026-01) | **10 min - 1 hour** |

### Image Strategy: Use external CDN URLs in JSONL `images: [{src: "https://..."}]`. Shopify fetches async. Do NOT pre-stage 100K images individually.

### Constraints
- JSONL max: 100MB per file
- Max operation runtime: 24 hours
- Max images per product: 250
- Single query cap: 1,000 points (all plans)
- Results available: 7 days

## CRITICAL: Discontinued Products → ARCHIVE (Never Draft, Never Delete) #memory

**When a vendor discontinues a product, ARCHIVE it on Shopify. Never draft, never delete.**

### Rules
1. **Discontinued = ARCHIVE** — Set `status: "archived"` via Shopify Admin API
2. **Never use draft** for discontinued products — draft is for unpublished new products
3. **Never delete** discontinued products — archived products preserve SEO and order history
4. **Detection**: If vendor product URL returns 301/404/redirect to category page → product is discontinued
5. **Update both**: Set `status: 'archived'` in PostgreSQL catalog table AND Shopify

### API Call
```javascript
// Archive a discontinued product on Shopify
PUT /admin/api/2024-01/products/{product_id}.json
{ "product": { "id": product_id, "status": "archived" } }
```

## CRITICAL: NEVER Use "Unknown" as a Pattern Name #memory

**🚫 "UNKNOWN" MUST NEVER APPEAR IN A SHOPIFY PRODUCT TITLE — EVER!**

### Rules
1. **If no pattern name is found from scraping** — do NOT set pattern to "Unknown"
2. **If pattern name cannot be determined** — use the MFR SKU as the pattern name instead (e.g. `T10958 Blue | Thibaut`)
3. **If MFR SKU is also missing** — use only the color name (e.g. `Blue | Thibaut`)
4. **NEVER publish a product with "Unknown" in the title** — block it from the schedule
5. **Always scrape vendor website FIRST** and save pattern name to PostgreSQL BEFORE setting Shopify title
6. **Tag products for review** if pattern name cannot be found — add `Needs-Pattern-Name` tag

### Fallback Order for Pattern Name
1. Vendor website scrape (thibautdesign.com, koroseal.com, etc.)
2. Spreadsheet data (TAK, Type II Vinyl, etc.)
3. Metafields from Shopify (global.Pattern, custom.name_of_pattern)
4. MFR SKU as pattern name (last resort before "Unknown")
5. **NEVER "Unknown"** — skip the pattern entirely if nothing found

### Validation Check (MANDATORY before Shopify title update)
```javascript
function validateTitle(title) {
  if (!title || title.toLowerCase().includes('unknown')) {
    throw new Error('BLOCKED: Title contains "Unknown" - fix pattern name first');
  }
  return title;
}
```

## CRITICAL: Sample Variant Required for ALL Products #memory

**🏷️ EVERY PRODUCT MUST HAVE A "SAMPLE" VARIANT - NO EXCEPTIONS!**

### Sample Variant Requirements
| Field | Value | Notes |
|-------|-------|-------|
| **Title** | `Sample` | Always "Sample" |
| **SKU** | `{DW_SKU}-Sample` | DW SKU + "-Sample" suffix |
| **Price** | `$4.25` | Standard sample price for most vendors |
| **Inventory** | `null` | No inventory tracking |

### DW SKU Assignment Rules
1. **Assign at FIRST IMPORT ONLY** - Never change once assigned
2. **Format**: `DW{XX}-{NUMBER}` — DW + 2 random letters + dash + sequential number
3. **NEVER reference vendor name** in prefix — too easy to identify vendor
4. **NEVER use numbers** in prefix — letters only
5. **FORBIDDEN letters** (hard to hear on phone): **B, D, S, F, M, N, V**
6. **ALLOWED letters**: A, C, E, G, H, I, J, K, L, O, P, Q, R, T, U, W, X, Y, Z (19 chars)
7. Each vendor gets a **unique** 2-letter code from allowed set
8. **VCC auto-generates compliant prefixes** via `generateCompliantPrefix()` on first SKU assignment
9. **Sequential numbering** - Each vendor's range starts 5000 ahead of the last
10. **Store in PostgreSQL** - Save to vendor catalog table `dw_sku` column
11. **Legacy prefixes** (DWSC, DWTT, etc.) — leave existing, only apply new rules going forward

### Product Variant Structure (MANDATORY)
```javascript
variants: [
  {
    title: 'Sample',
    sku: `${dwSku}-Sample`,  // e.g., "DWK-040251-Sample"
    price: '4.25',
    inventory_management: null
  }
]
```

### Implementation in Shopify Product Creation
```javascript
// Generate DW SKU FIRST
const dwSku = await generateDwSku(vendorPrefix, vendorName, mfrSku);

// Create product with Sample variant
const productData = {
  product: {
    title: productTitle,
    variants: [{
      title: 'Sample',
      sku: `${dwSku}-Sample`,
      price: '4.25',
      inventory_management: null
    }],
    // ... other fields
  }
};
```

### Why This Matters
- Customers need to order samples before full rolls
- DW SKU provides unique identifier across all vendors
- Sample pricing is standardized at $4.25
- SKU format enables easy tracking and reordering

---

## CRITICAL: Never Activate SKU Without Specs + Description + ALL Images #memory

**🚫 NEVER set a Shopify product to `status: "active"` unless ALL of these hold**
(extended 2026-06-20 from the old width+image rule to a fuller gate):
1. **SPECS** — `global.width` (or `custom.width`) set + non-empty (HARD-required),
   PLUS the core set (length, repeat, content/material, unit_of_measure) **where
   the vendor provides them** (don't block on a spec the vendor genuinely lacks).
2. **DESCRIPTION** — a non-empty `body_html`/description that is NOT a placeholder,
   NOT "Unknown"/"Page Not Found"/404/error, and NOT bare legal/disclaimer text.
3. **IMAGES** — ≥1 product image AND **all** of the vendor's available images for
   that SKU attached (`vendor_catalog.all_images` / `image_url`; full-page-scrape).
4. **GUARDS** — no banned word "Wallpaper" in title, no "Unknown" in title, sample
   variant present (`{DW_SKU}-Sample`).

**Single enforced validator** (use this, don't reinvent):
`shopify/scripts/lib/validate-before-activate.js` → `validateBeforeActivate(product)`.
Wired into BOTH activation chokepoints — `cadence/cadence-import.js` (create-time)
and `cadence/activate-gated.js` (DRAFT→ACTIVE promote). On fail → keep DRAFT +
tag `Needs-Specs` / `Needs-Description` / `Needs-Image`.

Legacy summary (still true, subset of the above):
1. **A width metafield** — `global.width` or `custom.width` must be set (not null/empty)
2. **At least one product image** — `product.images.length >= 1`

### Rules
1. **Products missing width OR image remain as `draft`** — do not activate them
2. **Check BEFORE any status change** — validate both fields before calling Shopify API
3. **Bulk imports**: Skip activation for products that fail this check; tag them `Needs-Width` or `Needs-Image`
4. **Width sources** (in priority order): vendor website scrape, spreadsheet data, metafield sync
5. **Image sources**: vendor CDN, uploaded images, room settings

### Validation Code (MANDATORY before activation)
```javascript
function canActivateProduct(product) {
  const hasImage = product.images && product.images.length > 0;
  const hasWidth = product.metafields?.some(m =>
    (m.namespace === 'global' || m.namespace === 'custom') &&
    m.key === 'width' && m.value && m.value.trim() !== ''
  );
  if (!hasImage) console.warn(`BLOCKED: ${product.title} has no images`);
  if (!hasWidth) console.warn(`BLOCKED: ${product.title} has no width metafield`);
  return hasImage && hasWidth;
}
```

### Why This Matters
- Products without width confuse trade buyers who need to calculate coverage
- Products without images get zero engagement and hurt SEO
- Better to keep as draft until complete than show incomplete listings

---

## CRITICAL: SKU Registry for New Product Imports #memory

**🔢 BEFORE CREATING ANY NEW PRODUCT IN SHOPIFY - CHECK FOR DUPLICATES!**

### Import the SKU Registry Module
```javascript
const { checkDuplicate, registerSku, updateShopifyInfo } = require('./lib/sku-registry');
// Located: /root/Projects/Designer-Wallcoverings/shopify/scripts/lib/sku-registry.js
```

### Mandatory Workflow for New Products
```javascript
async function importNewProduct(vendorPrefix, vendorName, mfrSku, productData) {
  // Step 1: CHECK for duplicate
  const dup = await checkDuplicate(vendorPrefix, mfrSku);
  if (dup.exists) {
    console.log(`⚠️ SKIP: ${mfrSku} already exists as ${dup.existingSku}`);
    return null;
  }

  // Step 2: REGISTER before Shopify creation
  const dwSku = await registerSku(vendorPrefix, vendorName, mfrSku);
  if (!dwSku) return null;

  // Step 3: CREATE in Shopify with DW SKU
  const shopifyProduct = await createShopifyProduct({ ...productData, sku: dwSku });

  // Step 4: UPDATE registry with Shopify info
  await updateShopifyInfo(dwSku, shopifyProduct.id, shopifyProduct.handle);

  return { dwSku, shopifyId: shopifyProduct.id };
}
```

### Vendor Prefix Quick Reference
| Prefix | Vendor | Prefix | Vendor |
|--------|--------|--------|--------|
| DWK- | Koroseal | DWMR- | Maya Romanoff |
| DWC-Arte- | Arte | DWCC- | Novasuede |
| DWTT- | Thibaut | DWKK- | Kravet |
| DWRR- | Ralph Lauren | DWSC- | Schumacher |
| DWJS- | York | DWPJ- | Phillip Jeffries |
| DWWC- | Default/Generic | | |

**Database**: `dw_unified` → Table: `dw_sku_registry`

---

## CRITICAL: AI Analysis MUST Be Saved (Analyze + Accept) #memory

**🎨 AI ANALYSIS REQUIRES TWO STEPS - ANALYZE THEN ACCEPT!**

The `/api/analyze-image` endpoint only RETURNS analysis - it does NOT save to database!
You MUST call `/api/accept-analysis` to actually SAVE the results.

### API Endpoints (Arte Dashboard - Port 9955)

| Endpoint | Purpose | Saves to DB? |
|----------|---------|--------------|
| `POST /api/analyze-image` | Run Gemini AI analysis on image | ❌ NO - Returns only |
| `POST /api/accept-analysis` | Save analysis to PostgreSQL | ✅ YES - Saves data |
| `POST /api/scrape-arte` | Fetch MFR data from Arte website | ❌ NO - Returns only |

### Complete AI Analysis Workflow

```bash
AUTH="Basic $(printf '%s' \"$ADMIN_USER:$ADMIN_PASS\" | base64)"  # creds via env — see /secrets
API="http://localhost:9955"

# Step 1: AI Analyze (returns but does NOT save)
AI_RESULT=$(curl -s -X POST "$API/api/analyze-image" \
  -H "Authorization: $AUTH" \
  -H "Content-Type: application/json" \
  -d '{"imageUrl":"IMAGE_URL","title":"Pattern Color"}')

# Step 2: Extract FULL analysis object
ANALYSIS=$(echo "$AI_RESULT" | jq '.analysis')

# Step 3: SAVE to database - aiAnalysis MUST be nested object!
curl -s -X POST "$API/api/accept-analysis" \
  -H "Authorization: $AUTH" \
  -H "Content-Type: application/json" \
  -d "{\"arteSku\":\"$SKU\",\"aiAnalysis\":$ANALYSIS}"
```

**⚠️ CRITICAL: The `aiAnalysis` parameter MUST be a nested object containing:**
- `colors` (array)
- `tags` (array)
- `description` (string)
- `backgroundColor` (string)
- `styles` (array)
- `patterns` (array)

### Database Columns Updated by Accept

| Column | Type | Description |
|--------|------|-------------|
| `ai_colors` | text[] | Array of detected colors |
| `ai_tags` | text[] | Interior design tags |
| `ai_description` | text | 2-sentence commercial description |
| `ai_background_color` | text | Dominant background color |
| `ai_styles` | text[] | Style periods (Contemporary, Art Deco, etc.) |
| `ai_patterns` | text[] | Pattern types (Geometric, Floral, etc.) |

### Gemini 3.0 Prompt (Interior Design Tagger)

```javascript
const GEMINI_API_KEY = process.env.GEMINI_API_KEY  /* 2026-05-05 SCRUB: was hardcoded; rotate via /secrets */;

// NEVER put example values in prompt - AI will return them!
// BAD: "backgroundColor": "Navy"
// GOOD: "backgroundColor": "<actual color you see>"
```

### Scheduling Products (Calendar API - Port 9958)

```bash
CAL="http://localhost:9958"

curl -s -X POST "$CAL/api/schedule" \
  -H "Authorization: $AUTH" \
  -H "Content-Type: application/json" \
  -d '{
    "productId": "SKU",
    "productTitle": "Pattern Color",
    "vendorChannelId": 1,
    "scheduledDate": "2026-02-04T12:00:00",
    "imageUrl": "IMAGE_URL",
    "mfrSku": "SKU"
  }'
```

### UI Card Turns Purple When Scheduled

The card at http://45.61.58.125:9660/ automatically turns purple when scheduled:
```javascript
background: scheduled ? 'linear-gradient(135deg, #7c3aed 0%, #6d28d9 100%)' : 'white'
```

## CRITICAL: PostgreSQL BEFORE Shopify #memory

**🔴 ALWAYS UPDATE POSTGRESQL FIRST, THEN SHOPIFY!**

When updating ANY product data:
1. **FIRST**: Scrape vendor website for ALL data (colors, specs, images, SKUs)
2. **SECOND**: Save ALL data to PostgreSQL (vendor_catalog or arte_catalog table)
3. **THIRD**: Only THEN update Shopify from PostgreSQL data

**Why PostgreSQL First?**
- PostgreSQL is the source of truth for ALL vendor data
- Prevents data loss if Shopify API fails
- Enables bulk operations and rollback
- Allows verification before pushing to Shopify
- Manufacturer color names MUST come from scraped data, not AI

**Never skip PostgreSQL!** Even for single product updates.

## CRITICAL: BANNED WORD "WALLPAPER" - NEVER USE! #memory

**🚨 The word "WALLPAPER" is BANNED from ALL product titles, descriptions, and AI output.**

| ❌ NEVER Use | ✅ ALWAYS Use |
|-------------|--------------|
| Wallpaper | Wallcovering |
| Wallpapers | Wallcoverings |
| Walls Wallpaper | Wallcoverings |

**Applies to ALL vendors:**
- DW Bespoke Studios (all variants)
- Koroseal
- Maya Romanoff
- Thibaut
- Arte
- All other architectural vendors

**Post-processing cleanup (MANDATORY in all updaters):**
⚠️ MUST be **brand-aware** — 17 legitimate brand names END in "Wallpaper" (Ralph Lauren,
Missoni, Grasscloth, Scalamandre, Schumacher, etc.) and a blanket `\bWallpaper\b` swap
CORRUPTS them (e.g. "Ralph Lauren Wallpaper" → "Ralph Lauren Wallcovering"). It must also
skip the derived words `wallpapered`/`wallpapering`. Use this version (TK-00134):
```javascript
const cleanWallpaper = (str) => {
  if (!str) return str;
  // Preserve the 17 brands ending in "Wallpaper"; skip wallpapered/wallpapering; case-preserving.
  const WALLPAPER_BRANDS = ['Apartment','Boråstapeter','China Seas','DW Exclusive','Edge','Fentucci','Grasscloth','Laura Ashley','Malibu','MC Escher','Missoni','Nicolette Mayer','PS Removable','Ralph Lauren','Roberto Cavalli','Scalamandre','Schumacher'];
  const brandRe = new RegExp('(' + WALLPAPER_BRANDS.map(b => b.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')).join('|') + ')\\s+Wallpaper', 'gi');
  const SENT = '', hold = [];
  let out = String(str).replace(brandRe, m => { hold.push(m); return SENT + (hold.length - 1) + SENT; });
  out = out.replace(/\bWallpaper(s?)\b(?!ed|ing)/gi, (m, plural) => {
    const base = plural ? 'wallcoverings' : 'wallcovering';
    if (m === m.toUpperCase()) return base.toUpperCase();
    if (m[0] === m[0].toUpperCase()) return base[0].toUpperCase() + base.slice(1);
    return base;
  });
  hold.forEach((h, i) => { out = out.split(SENT + i + SENT).join(h); });
  return out;
};
```

---

## CRITICAL: Product Title Formatting Rules #memory

**🏷️ ALL PRODUCT TITLES MUST BE IN TITLE CASE**

### Title Format
```
Pattern Name Real Color Name | Brand Name
```

### DW Bespoke Vendors - INCLUDE AI STYLE IN TITLE #memory
**🎨 For DW Bespoke vendors ONLY, include Gemini AI-detected style in the title:**

```
Design Name [AI Style] | Architectural Wallcoverings
```

**DW Bespoke Vendors:**
- DW Bespoke Studios (all variants)
- DW Bespoke Studio To Go
- DW Bespoke Murals
- Retro DW Bespoke Studios To Go

**Examples:**
- `1917 Deco Kitchen Art Deco | Architectural Wallcoverings` (AI: Art Deco)
- `1970's Funky Geometrics Retro | Architectural Wallcoverings` (AI: Retro)
- `Tropical Palm Pop Art | Architectural Wallcoverings` (AI: Pop Art)
- `Vintage Floral Mid-Century Modern | Architectural Wallcoverings` (AI: Mid-Century Modern)

**Implementation:**
```javascript
// For DW Bespoke vendors ONLY:
const aiStyles = geminiAnalysis.styles; // e.g., ["Art Deco", "Traditional"]
const bespokeTitle = `${designName} ${aiStyles[0]} | Architectural Wallcoverings`;
```

### Title Case Rules
- **Capitalize** first letter of every word
- **Exceptions**: articles (a, an, the), prepositions (in, on, at, for), conjunctions (and, but, or) - unless first word
- Use `toTitleCase()` function on all title components

### Real Color Names (NOT AI Colors)
- **Maya Romanoff**: Use creative names like "Let's Be Platinum", "Walking in the Woods", "Clear Midnight Sky"
- **Koroseal**: Use spreadsheet colors like "Gold", "Copper", "Pearl Silver"
- **AI colors (Blue, Gray, etc.)** → TAGS ONLY, never in titles

### Title Examples
| ✅ Correct | ❌ Wrong |
|-----------|---------|
| Ajiro Burst Of Happiness Let's Be Platinum \| Maya Romanoff | ajiro burst of happiness let's be platinum \| maya romanoff |
| Opulent Leaf Gold \| Architectural Wallcoverings | OPULENT LEAF GOLD \| ARCHITECTURAL WALLCOVERINGS |
| Palm Beach Blue Green \| Thibaut | Palm beach blue green \| thibaut |

### Metafields for Real Color Names
```javascript
// Save REAL color name to custom metafields:
metafields.push({ namespace: 'custom', key: 'color', value: realColorName, type: 'single_line_text_field' });
metafields.push({ namespace: 'custom', key: 'real_color_name', value: realColorName, type: 'single_line_text_field' });
metafields.push({ namespace: 'dwc', key: 'color', value: realColorName, type: 'single_line_text_field' });
metafields.push({ namespace: 'global', key: 'Color-Way', value: realColorName, type: 'single_line_text_field' });
```

### SEO Title Format (Commercial/Architect Focus)
```
Shop [Pattern Name] in [Real Color Name] | [Brand]
```
Example: "Shop Ajiro Burst Of Happiness in Clear Midnight Sky | Maya Romanoff"

---

## CRITICAL: Git Commits Every Session #memory
**At the END of every session, ALWAYS:**
1. Commit all changes to git with descriptive message
2. Update `/root/.claude/SESSION-LEARNINGS.md` with session summary
3. Include key learnings, fixes applied, and pending tasks

## CRITICAL: Always Update Manufacturer SKU in Metafields #memory

**🏷️ MANDATORY FOR ALL SHOPIFY PRODUCT UPDATES:**
- **ALWAYS** set `custom.manufacturer_sku` metafield with the MFR SKU
- **ALWAYS** set `dwc.manufacturer_sku` metafield as backup
- Extract MFR SKU from product tags (patterns like `ABH-1109`, `TAK-CA01-01`, `AL21-15`)
- This is the PRIMARY identifier for vendor product lookups

```javascript
// ALWAYS include these metafields in every product update:
metafields.push({ namespace: 'custom', key: 'manufacturer_sku', value: mfrSku, type: 'single_line_text_field' });
metafields.push({ namespace: 'dwc', key: 'manufacturer_sku', value: mfrSku, type: 'single_line_text_field' });
```

**Why Critical:**
- MFR SKU is needed for website scraping (Koroseal, Maya Romanoff, etc.)
- Links Shopify products to vendor catalogs
- Required for spec lookups and updates
- Used by multiple automation scripts

---

## Gemini for Image Analysis
Gemini API keys and endpoints are in the global `/root/.claude/CLAUDE.md`. Use `gemini-3.5-flash` for all image analysis.

## CRITICAL: Koroseal Product Spreadsheets #memory

**🚨 USE BOTH SPREADSHEETS - NEVER GUESS COLORS!**

### 1. TAK Specialty Products (PRIORITY FOR TAK-* SKUs)
- **URL**: https://docs.google.com/spreadsheets/d/1EjqtPSo9hqTai_UHW-iLVzaglOTHUdQxDmbjz3vnU8A/edit?usp=sharing
- **CSV**: `https://docs.google.com/spreadsheets/d/1EjqtPSo9hqTai_UHW-iLVzaglOTHUdQxDmbjz3vnU8A/gviz/tq?tqx=out:csv`
- **Contains**: TAK-AA01 through TAK-GA01 specialty products (foils, cork, raffia, etc.)
- **Key Columns**:
  - `Manufacturer Number` = TAK-AA04-02
  - `V-NAME` = Pattern name (Opulent Leaf)
  - `color` = **ACTUAL COLOR** (Gold, Copper, Pearl Silver, Taupe Grey)
  - `Content` = Material type (Foil Leaf, Cork, Raffia)

### 2. Type 2 Vinyl Products
- **URL**: https://docs.google.com/spreadsheets/d/1FYG7TGUUiim_RL25YJx5R9tBcgL3ilFlLy3dQISVSMo/edit?usp=sharing
- **CSV**: `https://docs.google.com/spreadsheets/d/1FYG7TGUUiim_RL25YJx5R9tBcgL3ilFlLy3dQISVSMo/gviz/tq?tqx=out:csv`
- **Contains**: SE81, SG13, AL21, etc. Type 2 Vinyl products

### Color Priority Chain (MANDATORY ORDER)
1. **TAK Specialty Spreadsheet** (for TAK-* SKUs) - ALWAYS FIRST!
2. **Type 2 Vinyl Spreadsheet** (for other SKUs)
3. Scraped colorName from Koroseal website
4. AI analysis (ONLY if no spreadsheet data exists)

**⚠️ NEVER use AI/Gemini for colors when spreadsheet has the data!**

### Type II Vinyl Scraping - Prefix Search Fallback #memory
If full SKU search fails (e.g., SG13-02 not found), try prefix search:
- `SG13-02` → search `SG13` at https://koroseal.com/search-results?searchtext=SG13
- `AL21-15` → search `AL21`
- Pattern: `/^[A-Z]{1,2}\d{2,3}-\d+$/i`

**NEVER SKIP products** - if Koroseal page not found, still update using spreadsheet data!

### Example Color Mappings (from TAK spreadsheet)
| MFR SKU | Pattern | Color | Content |
|---------|---------|-------|---------|
| TAK-AA04-01 | Opulent Leaf | Gold | Foil Leaf |
| TAK-AA04-02 | Opulent Leaf | Copper | Foil Leaf |
| TAK-AA08-01 | Languid Leaf | Pearl Silver | Foil Leaf |
| TAK-AA08-02 | Languid Leaf | Gold | Foil Leaf |

## Brewster / York Vendor Login Credentials #memory

**🔐 Trade Portal Access for Brewster & York Wallcoverings**

| Field | Value |
|-------|-------|
| **Login URL** | https://www.yorkwall.com/Login/ |
| **Username** | see `BREWSTER_PORTAL_USER` (secrets-manager) |
| **Password** | see `BREWSTER_PORTAL_PASS` (secrets-manager) |
| **Covers** | Both Brewster and York brands (same parent company) |

### Usage
- Use for scraping product data, specs, images, pricing from trade portal
- York and Brewster share the same login system at yorkwall.com
- Required for accessing full product catalog, room settings, and spec sheets
- Vendor codes in crossref: `BRE` (Brewster), `yor`/`YOR` (York)

---

## Site Status & Port Assignments

### Production Sites (Fixed for Cloudflare - DO NOT CHANGE PORTS)
- **flockedwallpaper.com** - Port 3200 (PM2: flockedwallpaper)
- **glassbeadedwallpaper.com** - Port 9888 (PM2: glassbeadedwallpaper)  
- **grassclothwallcoverings.com** - Ports 8080/9080 (PM2: grassclothwallcoverings-8080/9080)
- **grassclothwallpaper.com** - Port 7123 (PM2: grassclothwallpaper) - Fixed 12/08 after crash
- **novasuede.com** - Port 9878 (PM2: novasuede-website)
- **bubbe.ai** - Port 3011 (PM2: bubbe)

### Steve's Monitoring Reports
- Runs at 6am, 12pm, 6pm daily via cron
- Sends to Slack #claude-to-steve channel
- Monitors all 6 sites above
- Script: /root/Projects/Designer-Wallcoverings/DW-Websites/steve-hourly-report.sh

## CRITICAL: Notify Steve on Process Completion #memory

**🔔 ALWAYS send Slack notification to Steve when ANY process completes!**

When a script, scrape, update, or any automated process finishes:
1. Send summary to Slack #claude-to-steve channel
2. Include: process name, products processed, success/fail count, duration

```bash
# Slack webhook for #claude-to-steve
SLACK_WEBHOOK="${SLACK_WEBHOOK_URL}"

# Send completion notification
curl -X POST -H 'Content-type: application/json' \
  --data '{"text":"✅ *Process Complete*\n• Name: Brand McKenzie Scrape\n• Processed: 115 products\n• Saved to PostgreSQL: 115\n• Duration: 25 min"}' \
  $SLACK_WEBHOOK
```

**Always notify on:**
- Vendor scrapes/recrawls
- AI analysis batches
- Shopify bulk updates
- Database migrations
- Any long-running process
- always look for the .md scraper and ts file for every vendor
## Shopify Store Configuration
- **Store**: designer-laboratory-sandbox
- **Admin URL**: https://admin.shopify.com/store/designer-laboratory-sandbox/
- **API Domain**: designer-laboratory-sandbox.myshopify.com
- **Access Token**: see `SHOPIFY_ADMIN_TOKEN` in secrets-manager/.env — never hardcode
- **API Version**: 2024-10
- **Note**: This token lacks `read_customers`/`write_customers` scope. Need Full Access API token from Shopify Admin for customer operations.
- **GraphQL Endpoint**: `https://designer-laboratory-sandbox.myshopify.com/admin/api/2024-01/graphql.json`

## Shopify Metafield Manager - Global Namespace Fields #memory

**CRITICAL: Use `global` namespace for these product metafields (type: single_line_text_field unless noted)**

### Core Specs (ALWAYS UPDATE FOR WALLPAPER)
| Key | Description | Example |
|-----|-------------|---------|
| `width` | Roll width | "27 Inches" |
| `length` | Roll length | "27 Feet" |
| `repeat` | Pattern repeat | "25.25 Inches" |
| `v_prods_quantity_order_min` | Minimum order qty | "2" |
| `v_prods_quantity_order_units` | Qty increment/step | "2" |
| `unit_of_measure` | Pricing unit | "Priced Per Single Roll" |
| `lead_time` | Lead time | "3-5 Business Days" |
| `packaged` | Packaging info | "Double Roll" |

### Product Info
```
title_tag, title, description_tag, color, Brand, SubBrand
Book_Name, Book-Name, Book_Number, Book-Number, BOOK_NUMBER
Collection, Country, Country-of-Origin, COUNTRY_OF_ORIGIN
Style, Style1, Style2, DESIGN_STYLE, Type, Type1, Type2
Color-Way, Color-of-Pattern, Color-1, Color-2, Color-3, Color_1
```

### Specifications
```
Contents, Content, Construction, Finish, FINISH, Finishing
Weight, Durability, Cleaning, Cleaning-Code, Clean-Code, CARECODE
Usage, Use, FABRIC_USAGE, Direction, MATCH, MATCH_TYPE
ADDITIONAL_CARE_INSTRUCTIONS_E
```

### Pattern Repeats
```
Vert-Rpt, Vert-Repeat, PATTERN_REPEAT_VERTICAL
Horz-Rpt, Horz-Repeat, PATTERN_REPEAT_HORIZONTAL
VERTICAL_HALFDROP, HORIZONTAL_HALFDROP
```

### Commercial/Contract
```
Class-Description, FLAMMABILITY, Wyzenbeek-#, Martindale-#
SURFACE_ABRASION_COUNT, SURFACE_ABRASION_TEST
CONTRACT, CA-TB117, UFAC, Prop-65
```

### Dimensions & Coverage
```
Panel-Height, Design-Height, Panel-Count
COVERAGE_AREA_SQ_FT, COVERAGE_UNIT, YARDS_PER_DR
Wallcover-Length(YD), EMBROIDERED_WIDTH, WIDTH_EXTENDABLE, AVAILABLE_SIZES
```

### Fabric-Specific
```
FABRIC_TYPE, TRIM_TYPE, WALLCOVERING_TYPE, WEIGHTED_HEM
BLACKOUT, DIMOUT, RAILROADED, REVERSIBLE
WASHABLE, HI_TEMP_WASHABLE, ECO_FRIENDLY
```

### Strength Testing
```
SEAMSLIPPAGEWARP, SEAMSLIPPAGEFILLING
BREAKINGSTRENGTHWARP, BREAKINGSTRENGTHFILLING
TEARSTRENGTHWARP, TEARSTRENGTHFILLING
```

### Internal/Trade
```
discount_trade, WHLS-Price, js_internal_number, vendor_name_internal
mfr-pattern-number, Exclusive, SoldPerMeta, Sold-Per, Minimum
```

### Inventory & Display
```
Display-Status, Inventory-Available, Image-exists
Image-File-Name---LO-RES, seohidden (Integer)
```

### Marketing/SEO
```
Bullet-Point-1, Bullet-Point-2, Bullet-Point-3
```

### Shipping
```
Ship-From, Lead-Time-in-Days, Memo-Sample-Available, DOMESTIC
```

### Variant-Level
```
SKU, Variant-Sku, Title, Color
```

### Special Types
```
color_swatches (Custom Object)
seohidden (Integer)
```

### DWC Namespace (Custom Fields)
| Key | Description |
|-----|-------------|
| `dwc.order_unit` | Order unit (text field) |
| `dwc.manufacturer_sku` | Manufacturer SKU |
| `dwc.pattern_name` | Pattern name |
| `dwc.color` | Color |
| `dwc.background_color` | Background color |
| `dwc.brand` | Brand |
| `dwc.real_vendor` | Real vendor name |
| `dwc.ai_generated_colors` | AI colors (list) |
| `dwc.ai_generated_tags` | AI tags (list) |

**CRITICAL - ALWAYS USE GRAPHQL API** #memory
- REST API pagination through 145K+ products is too slow (600+ pages)
- GraphQL `shop { productVendors }` returns all vendors in ONE call
- GraphQL `products(query: "vendor:X")` for filtering
- Example vendor query:
```graphql
{ shop { productVendors(first: 250) { edges { node } } } }
```

**CRITICAL - NEVER USE**: `designerwallcoverings.myshopify.com` - This domain does NOT work for API calls!
**ALWAYS USE**: `designer-laboratory-sandbox.myshopify.com` - This is the correct API domain for the live store.

## Architectural Wallcoverings Update Agent
- **URL**: http://45.61.58.125:9897
- **Auth**: admin / see `BASIC_AUTH` env var (secrets-manager)
- **PM2**: dw-architectural-update
- **Directory**: /root/Projects/Designer-Wallcoverings/DW-Agents/dw-agents/
- **Progress File**: /root/Projects/Designer-Wallcoverings/DW-Agents/dw-agents/ARCHITECTURAL-UPDATE-PROGRESS.md
- **Title Format**: `Pattern Name Color (Real-MFR#) | Architectural Wallcoverings`
- **Data Priority**: Handle > Tags > SKU Mappings
- **IMPORTANT**: Always read progress file at start of session for context #memory

### Title Rules #memory
- **ALWAYS remove `-SAMPLE` from titles** - Never include "-SAMPLE" suffix in product titles
- **No hardcoded vendor tags** - Use actual vendor name, not "Koroseal" for all products
- **No automatic Type II tags** - Only add Type II if confirmed from product specs
- **NEVER REMOVE `display_variant` TAG** - This tag controls variant display on the website. If `display_variant` exists in product tags, NEVER delete it! Example: https://www.designerwallcoverings.com/products/yeeha-1950s-cowboy-wallpaper-pine-green-1?variant=44019770458163

### Koroseal SKU Patterns
- `AL21-15`, `C921-10`, `2R15-21`, `TAK-DA01-01`

### Handle Formats
- LIVE: `dwk-XXXXX-vendor-...-sku-pattern-color`
- SANDBOX: `pattern-color-wallcovering-sku-sample-...`

## Room Setting App
- **URL**: http://45.61.58.125:3075
- **PM2**: room-setting-app
- **Directory**: /root/Projects/Designer-Wallcoverings/DW-Programming/room-setting-app
- **Skill**: ~/.claude/skills/room-setting-generator/SKILL.md
- **IMPORTANT**: When updating the Room Setting App, ALWAYS update the skill file too #memory

## Ralph Automation
- **UI**: http://45.61.58.125:7300 (Letsbegin app)
- **IMPORTANT**: Always CONTINUE Ralph from where it left off - never restart from scratch #memory
- **Continue API**: POST /api/ralph/continue - only runs PENDING stories
- **Start API**: POST /api/ralph/run - restarts everything (use sparingly)
- **Buttons**: CONTINUE (primary) = resume pending, RESTART (secondary) = start fresh

---

## Git Repository Privacy Guidelines

**CRITICAL: All new repositories MUST be set to private by default**

### GitHub Privacy Settings

When creating repositories on GitHub:

1. **Set to Private During Creation**:
   ```bash
   # Using GitHub CLI
   gh repo create my-repo --private --source=. --remote=origin

   # Or set after creation
   gh repo edit owner/repo --visibility private
   ```

2. **Verify Privacy Status**:
   ```bash
   # Check if repository is private
   gh repo view owner/repo --json isPrivate

   # Should return: {"isPrivate": true}
   ```

3. **Repository Settings Checklist**:
   - Repository visibility: Private ✅
   - Branch protection rules: Enabled for main/master
   - Require pull request reviews: Enabled
   - Disable force pushes: Enabled
   - Disable repository deletion: Enabled

### GitLab Privacy Settings

When creating repositories on GitLab:

1. **Set to Private During Creation**:
   ```bash
   # Using GitLab CLI
   glab repo create my-repo --private

   # Or using API
   curl --request POST "https://gitlab.com/api/v4/projects" \
     --header "PRIVATE-TOKEN: <your_access_token>" \
     --data "name=my-repo&visibility=private"
   ```

2. **Verify Privacy Status**:
   ```bash
   # Check repository visibility
   glab repo view owner/repo --json visibility

   # Should return: {"visibility": "private"}
   ```

3. **Repository Settings Checklist**:
   - Visibility level: Private ✅
   - Merge request approvals: Required
   - Protected branches: Enabled for main/master
   - Push rules: Configured
   - Repository mirroring: Disabled (unless needed)

### Verification Commands

**Pre-Push Verification Script**:
```bash
#!/bin/bash
# Save as: /root/scripts/verify-repo-privacy.sh

REPO_URL=$(git config --get remote.origin.url)

if [[ $REPO_URL == *"github.com"* ]]; then
  REPO=$(echo $REPO_URL | sed 's/.*github.com[:/]\(.*\)\.git/\1/')
  PRIVACY=$(gh repo view $REPO --json isPrivate -q .isPrivate)
  if [ "$PRIVACY" != "true" ]; then
    echo "❌ ERROR: Repository is PUBLIC! Aborting push."
    exit 1
  fi
  echo "✅ Repository is private"
elif [[ $REPO_URL == *"gitlab.com"* ]]; then
  REPO=$(echo $REPO_URL | sed 's/.*gitlab.com[:/]\(.*\)\.git/\1/')
  VISIBILITY=$(glab repo view $REPO --json visibility -q .visibility)
  if [ "$VISIBILITY" != "private" ]; then
    echo "❌ ERROR: Repository is PUBLIC! Aborting push."
    exit 1
  fi
  echo "✅ Repository is private"
fi
```

**Git Pre-Push Hook** (automatic verification):
```bash
# Add to .git/hooks/pre-push in each repository
#!/bin/bash
/root/scripts/verify-repo-privacy.sh || exit 1
```

### Security Best Practices

1. **Never Commit Sensitive Data**:
   - API keys, tokens, passwords
   - Database credentials
   - SSL certificates
   - Customer data
   - Use `.gitignore` and environment variables

2. **Audit Repository Access**:
   ```bash
   # GitHub: List collaborators
   gh api repos/owner/repo/collaborators

   # GitLab: List project members
   glab api projects/:id/members
   ```

3. **Regular Privacy Audits**:
   - Review repository visibility monthly
   - Check collaborator access levels
   - Verify branch protection rules
   - Scan for exposed secrets: `git-secrets --scan`

4. **Emergency Response**:
   - If repository accidentally made public: Make private immediately
   - Rotate all credentials that may have been exposed
   - Review commit history for sensitive data
   - Use `git filter-repo` to remove secrets from history if needed

### Default Repository Template

Create new repositories using this template:

```bash
#!/bin/bash
# Usage: ./create-private-repo.sh <repo-name>

REPO_NAME=$1

# Create private GitHub repository
gh repo create $REPO_NAME --private --source=. --remote=origin

# Add .gitignore
cat > .gitignore << 'EOF'
.env
.env.local
*.key
*.pem
node_modules/
.DS_Store
secrets/
EOF

# Set up branch protection
gh api repos/$(gh repo view --json nameWithOwner -q .nameWithOwner)/branches/main/protection \
  -X PUT \
  -f required_status_checks='null' \
  -f enforce_admins=true \
  -f required_pull_request_reviews='{"required_approving_review_count":1}' \
  -f restrictions='null'

echo "✅ Private repository created with security settings"
```

### Verification Checklist

Before pushing code to any repository, verify:

- [ ] Repository is set to private
- [ ] No API keys or credentials in code
- [ ] `.gitignore` includes sensitive files
- [ ] Branch protection enabled
- [ ] Only authorized collaborators have access
- [ ] Pre-push hook configured (optional but recommended)

**REMEMBER: Security is not optional. Always verify privacy settings before the first push.**

### Pattern Name Extraction Rule #memory
- **Pattern name is often the first word in the old title** - For single-word patterns like "Damier", extract from existing title before "-Wallcovering"
- Example: "Damier Wallcovering | Innovationsusa" → Pattern = "Damier"

### Title Color Rules (CRITICAL) #memory
- **AI colors go to TAGS ONLY, never in titles** - If no spreadsheet/PDF color exists, title has NO color
- **NEVER tag a title as "Page Not Found"** - Always validate pattern names are real names, not error messages
- **Color Priority Chain**: SPREADSHEET → Scraped → PDF → (NO AI in titles!)
- **AI colors**: Add to product tags as "Background Color X" and individual color tags
- **Invalid patterns**: page not found, 404, error, not found, undefined, null → Use "Wallcovering" fallback
- **Add "Review" tag if SKU not found on Koroseal site** - Flags products needing manual review
- **Save skipped/failed products to PostgreSQL** - Table: `koroseal_update_logs` with timestamp