`DEPLOYMENT_CHECKLIST.md`

292 lines
# Deployment Checklist

## Pre-Deployment

- [x] Install Node.js dependencies
- [x] Initialize SQLite database
- [ ] Add OpenAI API key to `.env`
- [ ] Configure Slack webhook (optional)
- [ ] Open firewall port 7989

## Deployment Steps

### 1. Configure API Keys
```bash
nano .env
```
Add:
- `OPENAI_API_KEY=sk-your-key-here`
- `SLACK_WEBHOOK_URL=https://hooks.slack.com/...` (optional)

### 2. Open Firewall
```bash
sudo bash scripts/configure-firewall.sh
```

### 3. Test Run
```bash
# Test server
npm start
# Visit http://your-ip:7989

# In another terminal, test crawler (CTRL+C after a few seconds)
npm run crawler
```

### 4. Production Setup with PM2
```bash
# Install PM2
npm install -g pm2

# Start services
pm2 start server.js --name handbags-server
pm2 start scripts/setup-cron.js --name handbags-cron

# Save configuration
pm2 save

# Enable startup on boot
pm2 startup
# Follow the command it outputs
```

### 5. Run Initial Crawl
```bash
# This will take 10-20 minutes
npm run crawler

# Then run AI analysis
node ai/analyzer.js

# Check for deals and send notifications
node scripts/deal-notifier.js check
```

### 6. Verify Deployment
```bash
# Check processes
pm2 status

# Check server
curl http://localhost:7989/api/stats

# Check database
sqlite3 data/handbags.db "SELECT COUNT(*) FROM listings;"

# View logs
pm2 logs handbags-server
pm2 logs handbags-cron
```

---

## Post-Deployment

### Daily Operations
- **Automatic**: Crawls run every 6 hours (00:00, 06:00, 12:00, 18:00 UTC)
- **Check deals**: `node scripts/deal-notifier.js check`
- **Export data**: `node scripts/export-spreadsheet.js deals 30`

### Monitoring
```bash
# Server status
pm2 status

# View logs
pm2 logs

# Check stats
curl http://localhost:7989/api/stats

# Database queries
sqlite3 data/handbags.db
```

### Maintenance
```bash
# Restart services
pm2 restart all

# Update code
git pull  # if using git
pm2 restart all

# Clean old data (optional)
sqlite3 data/handbags.db "DELETE FROM listings WHERE crawled_at < date('now', '-30 days');"
```

---

## Quick Reference

### URLs
- **Web Interface**: `http://your-ip:7989`
- **API Stats**: `http://your-ip:7989/api/stats`
- **API Listings**: `http://your-ip:7989/api/listings`
- **Export CSV**: `http://your-ip:7989/api/export/csv`

### Common Commands
```bash
# Start everything
pm2 start all

# Stop everything
pm2 stop all

# View logs
pm2 logs

# Manual crawl
npm run crawler

# Export deals
node scripts/export-spreadsheet.js deals 30
```

---

## Troubleshooting

### Server won't start
```bash
# Check if port is in use
sudo lsof -i :7989

# Kill process if needed
sudo kill -9 <PID>

# Restart
pm2 restart handbags-server
```

### Crawl fails
```bash
# Install Chromium dependencies (Ubuntu)
sudo apt-get update
sudo apt-get install -y chromium-browser

# Or install all dependencies
sudo apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 \
libcups2 libdbus-1-3 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 \
libgtk-3-0 libnss3 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 \
libxrandr2 libxss1 libxtst6 fonts-liberation
```

### No deals showing
```bash
# Check if crawl completed
sqlite3 data/handbags.db "SELECT COUNT(*) FROM listings;"

# Check if AI analysis ran
sqlite3 data/handbags.db "SELECT COUNT(*) FROM deal_analysis;"

# Run AI analysis manually
node ai/analyzer.js
```

### Slack notifications not working
```bash
# Test webhook directly
curl -X POST -H 'Content-type: application/json' \
--data '{"text":"Test notification"}' \
YOUR_SLACK_WEBHOOK_URL

# Check .env file
cat .env | grep SLACK

# Run notifier manually
node scripts/deal-notifier.js check
```

---

## Performance Optimization

### Database
```bash
# Vacuum database periodically
sqlite3 data/handbags.db "VACUUM;"

# Analyze for query optimization
sqlite3 data/handbags.db "ANALYZE;"
```

### Crawling
- Reduce `maxPages` in crawler files for faster crawls
- Disable sites in `.env` if not needed
- Increase delay between requests to avoid blocks

### Memory
```bash
# Monitor memory usage
pm2 monit

# Restart if memory grows too large
pm2 restart handbags-cron
```

---

## Security Recommendations

1. **Add reverse proxy with authentication**
```bash
# Install nginx
sudo apt-get install nginx

# Configure basic auth
sudo htpasswd -c /etc/nginx/.htpasswd admin
```

2. **Restrict firewall to specific IPs**
```bash
# Instead of 0.0.0.0/0, use your IP
sudo ufw delete allow 7989/tcp
sudo ufw allow from YOUR_IP to any port 7989
```

3. **Enable HTTPS with Let's Encrypt**
```bash
sudo apt-get install certbot python3-certbot-nginx
sudo certbot --nginx
```

4. **Regular backups**
```bash
# Backup database daily
crontab -e
# Add: 0 2 * * * cp /root/WebsitesMisc/handbags/data/handbags.db /root/backups/handbags-$(date +\%Y\%m\%d).db
```

---

## Success Indicators

✅ Server accessible at http://your-ip:7989
✅ Stats showing total listings > 0
✅ Deals showing in web interface
✅ PM2 shows both processes running
✅ Slack notifications received (if configured)
✅ CSV export working
✅ Automatic crawls running every 6 hours

---

## Next Steps

1. **Monitor first 24 hours** - Check logs, verify crawls complete
2. **Review deals** - Assess AI accuracy, adjust thresholds if needed
3. **Set up alerts** - Daily summary emails/Slack
4. **Optimize** - Fine-tune crawl frequency, deal thresholds
5. **Scale** - Add more brands, sites, or features

---

## Support & Resources

- Setup Guide: `SETUP.md`
- Data Structure: `DATA_STRUCTURE.md`
- README: `README.md`
- PM2 Docs: https://pm2.keymetrics.io/docs/usage/quick-start/
- OpenAI API: https://platform.openai.com/docs