`DEPLOYMENT.md`

387 lines
# Deployment Guide - Handbag Authentication Platform

Complete deployment instructions for production.

---

## Quick Start

```bash
# 1. Install dependencies
npm install
cd python-matcher && python3 -m pip install -r requirements.txt && cd ..

# 2. Setup PostgreSQL
sudo -u postgres createdb handbags
sudo -u postgres psql handbags < python-matcher/scripts/setup_db.sql

# 3. Configure environment
cp .env.example .env.local
# Edit .env.local with your keys

# 4. Build Next.js
npm run build

# 5. Start services
DATABASE_URL="file:./data/handbags.db" npx prisma generate
PORT=7991 pm2 start npm --name handbag-auth-nextjs -- start
pm2 start python-matcher/api/server.py --name handbag-matcher --interpreter python3
pm2 save
```

---

## Environment Variables

### Required

```bash
# Next.js
DATABASE_URL="file:./data/handbags.db"
PORT=7991

# eBay API (get from https://developer.ebay.com/)
EBAY_APP_ID="YourEbayAppId"

# Python Matcher
DATABASE_URL="postgresql://localhost/handbags"
API_PORT=8000
```

### Optional

```bash
# OpenAI for AI analysis
OPENAI_API_KEY="sk-..."

# Email notifications (future)
SMTP_HOST="smtp.gmail.com"
SMTP_USER="your@email.com"
SMTP_PASS="password"
```

---

## Services

### 1. Next.js Application (Port 7991)

**Start:**
```bash
PORT=7991 DATABASE_URL="file:./data/handbags.db" pm2 start npm --name handbag-auth-nextjs -- start
```

**Check:**
```bash
curl -I http://localhost:7991
pm2 logs handbag-auth-nextjs
```

### 2. Python Matching API (Port 8000)

**Start:**
```bash
cd python-matcher
DATABASE_URL="postgresql://localhost/handbags" pm2 start api/server.py --name handbag-matcher --interpreter python3
```

**Check:**
```bash
curl http://localhost:8000/stats
pm2 logs handbag-matcher
```

### 3. Firewall

```bash
sudo ufw allow 7991/tcp  # Next.js
sudo ufw allow 8000/tcp  # Python API
sudo ufw status
```

---

## Database Setup

### PostgreSQL (Vector Matching)

```bash
# Install PostgreSQL and pgvector
sudo apt-get update
sudo apt-get install -y postgresql-14 postgresql-14-pgvector

# Create database
sudo -u postgres createdb handbags

# Enable pgvector extension
sudo -u postgres psql handbags -c "CREATE EXTENSION IF NOT EXISTS vector;"

# Create schema
sudo -u postgres psql handbags < python-matcher/scripts/setup_db.sql

# Verify
sudo -u postgres psql handbags -c "SELECT COUNT(*) FROM handbags;"
```

### SQLite (Price History)

```bash
# Already exists from crawler
ls -lh data/handbags.db

# Run migrations
DATABASE_URL="file:./data/handbags.db" npx prisma generate
DATABASE_URL="file:./data/handbags.db" npx prisma db push
```

---

## Initial Data Loading

### 1. Embed Handbag Catalog

Process images and generate CLIP vectors:

```bash
cd python-matcher
export DATABASE_URL="postgresql://localhost/handbags"
python3 scripts/embed_handbags.py
```

This will:
- Fetch all handbags with `embedding IS NULL`
- Download product images
- Generate 768-dim vectors
- Update PostgreSQL database
- Process ~1000 images/hour (CPU)

### 2. Import Existing Data

If you have existing catalog data:

```bash
# Example: Import from CSV
psql handbags << EOF
COPY handbags(sku, brand, model_name, image_url, price_usd)
FROM '/path/to/catalog.csv'
DELIMITER ','
CSV HEADER;
EOF
```

### 3. Run Deal Detection

Analyze listings with improved algorithm:

```bash
cd /root/Projects/handbag-authentication
DATABASE_URL="file:data/handbags.db" node ../handbag-auth-nextjs/scripts/improve-deal-detection.js
```

---

## Cron Jobs

### Daily Crawler (6 AM)

```cron
0 6 * * * /root/Projects/handbag-authentication/run-crawler.sh >> /root/Projects/handbag-authentication/logs/handbag-crawler.log 2>&1
```

### Deal Detection (7 AM, after crawler)

```cron
0 7 * * * cd /root/Projects/handbag-auth-nextjs && DATABASE_URL="file:data/handbags.db" /usr/bin/node scripts/improve-deal-detection.js >> logs/deal-detection.log 2>&1
```

### Embed New Images (8 AM)

```cron
0 8 * * * cd /root/Projects/handbag-auth-nextjs/python-matcher && DATABASE_URL="postgresql://localhost/handbags" /usr/bin/python3 scripts/embed_handbags.py >> logs/embedding.log 2>&1
```

---

## Monitoring

### PM2 Status

```bash
pm2 list
pm2 monit
pm2 logs handbag-auth-nextjs --lines 100
pm2 logs handbag-matcher --lines 100
```

### Database Health

```bash
# PostgreSQL
psql handbags -c "
SELECT
  COUNT(*) as total,
  COUNT(embedding) as embedded,
  (COUNT(embedding)::float / COUNT(*)::float * 100) as coverage_pct
FROM handbags
WHERE is_active = true;
"

# SQLite
DATABASE_URL="file:./data/handbags.db" npx prisma studio
```

### Application Health

```bash
# Next.js
curl http://localhost:7991/api/stats

# Python API
curl http://localhost:8000/stats

# eBay Integration (after approval)
curl "http://localhost:7991/api/ebay-sold?brand=Hermes&model=Birkin"
```

---

## Troubleshooting

### Next.js won't start

```bash
# Check port
lsof -i :7991

# Check build
npm run build

# Check database
ls -lh data/handbags.db
```

### Python API fails

```bash
# Check dependencies
python3 -c "import torch; import transformers; print('OK')"

# Check database
psql handbags -c "SELECT version();"

# Check GPU (optional)
python3 -c "import torch; print(f'CUDA: {torch.cuda.is_available()}')"
```

### pgvector not found

```bash
# Install extension
sudo apt-get install postgresql-14-pgvector

# Enable in database
sudo -u postgres psql handbags -c "CREATE EXTENSION vector;"
```

### Out of memory during embedding

```bash
# Use smaller batch size
# Edit python-matcher/models/handbag_embedder.py
# Change: def embed_batch(images: list, batch_size: int = 4):  # Reduce from 8 to 4
```

---

## Performance Optimization

### 1. Vector Index Tuning

```sql
-- For 10K-100K vectors
CREATE INDEX idx_handbags_embedding
ON handbags
USING ivfflat (embedding vector_l2_ops)
WITH (lists = 100);

-- For 100K-1M vectors
REINDEX INDEX idx_handbags_embedding
WITH (lists = 500);
```

### 2. Enable GPU (if available)

```bash
# Install CUDA
sudo apt-get install nvidia-cuda-toolkit

# Verify
python3 -c "import torch; print(torch.cuda.is_available())"

# Speed improvement: 5-10x faster embeddings
```

### 3. Caching

Add Redis for API response caching:

```bash
sudo apt-get install redis-server
# Cache eBay responses for 6 hours
# Cache vector search for 1 hour
```

---

## Backup Strategy

### PostgreSQL

```bash
# Daily backup
pg_dump handbags | gzip > backups/handbags-$(date +%Y%m%d).sql.gz

# Restore
gunzip -c backups/handbags-20251114.sql.gz | psql handbags
```

### SQLite

```bash
# Copy database
cp data/handbags.db backups/handbags-$(date +%Y%m%d).db

# Restore
cp backups/handbags-20251114.db data/handbags.db
```

---

## URLs

- **Main App**: http://45.61.58.125:7991
- **Price Tracker**: http://45.61.58.125:7991/price-tracker
- **Camera**: http://45.61.58.125:7991/camera
- **Python API**: http://45.61.58.125:8000
- **API Docs**: http://45.61.58.125:8000/docs

---

## Security Checklist

- [ ] Change default PostgreSQL password
- [ ] Restrict CORS origins in production
- [ ] Enable HTTPS with Let's Encrypt
- [ ] Rate limit API endpoints
- [ ] Sanitize user inputs
- [ ] Keep eBay API key secret (use environment variables)
- [ ] Regular security updates: `apt-get update && apt-get upgrade`

---

## Support

- **Logs**: `pm2 logs`
- **Database**: `psql handbags` or `npx prisma studio`
- **Documentation**: See CHANGELOG.md, EBAY_INTEGRATION.md, python-matcher/README.md