← back to Wine Finder

WINE_LABEL_AUTHENTICATION_GUIDE.md

534 lines

# Wine Label Authentication & Verification - Implementation Guide

## Executive Summary

This guide catalogs image datasets, machine learning models, and technologies available for building a wine label authentication system that allows users to upload wine label photos and verify authenticity.

**Problem**: Wine counterfeiting costs European businesses €1.3 billion annually. Fake wines damage brands and deceive consumers.

**Solution**: Computer vision + ML models trained on wine label images to verify authenticity in real-time.

---

## 🗄️ Wine Label Image Datasets

### 1. WineSensed Dataset (LARGEST - 897,000 Images!)

**The Gold Standard for Wine Label Recognition**

- **Size**: 897,000 wine label images
- **Unique Vintages**: 350,000+
- **Reviews**: 824,000 wine reviews
- **Source**: Vivino platform
- **Annotations**:
  - Year/Vintage
  - Region
  - Rating
  - Alcohol percentage
  - Price
  - Grape composition
  - Flavor annotations (5,000+ pairwise distances from 256 participants)

**Download**: https://data.dtu.dk/articles/dataset/WineSensed_Learning_to_Taste_A_Multimodal_Wine_Dataset/23376560

**License**: CC BY-NC-ND 4.0

**Project**: https://thoranna.github.io/learning_to_taste/

**Published**: NeurIPS 2023 (UC Davis)

**Use Case**: Train deep learning models for wine recognition with massive, diverse dataset

---

### 2. Roboflow Wine Label Detection (5,464 Images)

**Pre-Trained Model Ready to Deploy**

- **Size**: 5,464 annotated images
- **Format**: Object detection format (bounding boxes)
- **Pre-trained Model**: ✅ Available
- **API Access**: ✅ Hosted Inference API
- **Annotations Detect**:
  1. Maker/Vineyard name
  2. Vintage/Year
  3. Sustainability labels
  4. Organic certification
  5. Alcohol level
  6. Appellation (AVA/AOC/DOC)
  7. Established year
  8. Country of origin
  9. Wine type
  10. Brand logo/picture
  11. Sweetness level
  12. Quality ratings

**Access**: https://universe.roboflow.com/wine-label/wine-label-detection

**License**: Open source

**Deployment Options**:
- Web API
- Python SDK
- Mobile deployment
- Edge deployment

**Use Case**: Production-ready wine label detection, no training required

---

### 3. WinesLabelDataset (264 Images)

**Small but Well-Annotated Dataset**

- **Size**: 264 images
- **Format**: Pascal VOC annotation format
- **GitHub**: https://github.com/qanastek/WinesLabelDataset
- **Best Use**: Fine-tuning existing models
- **Recommended**: Transfer learning from larger models

**Use Case**: Academic research, model fine-tuning

---

### 4. Roboflow 100 - Wine Labels

**Additional Labeled Dataset**

- **Platform**: Roboflow Universe
- **URL**: https://universe.roboflow.com/roboflow-100/wine-labels
- **Format**: Object detection
- **Pre-trained Model**: Available

---

### 5. HuggingFace Wine Labels Dataset

**Community Dataset**

- **Platform**: HuggingFace
- **URL**: https://huggingface.co/datasets/Francesco/wine-labels
- **Format**: Image classification/detection
- **Integration**: Easy with transformers library

---

### 6. Wine Bottle Image Classification Dataset

**Bottle-Level Detection**

- **URL**: https://images.cv/dataset/wine-bottle-image-classification-dataset
- **Size**: 1,700+ annotated bottle images
- **Focus**: Entire bottle + label combined
- **Use Case**: Detect wine bottles in retail settings

---

## 🤖 Machine Learning Models & GitHub Projects

### 1. OpenWines OCR

**Label Text Extraction**

- **GitHub**: https://github.com/OpenWines/OpenWinesOCR
- **Technology**: OCR (Optical Character Recognition)
- **Purpose**: Extract text from wine label images
- **Output**: Structured wine data from labels
- **Use Case**: Feed into OpenWines open-data database

---

### 2. Wine Label Reader Toolkit

**Complete CV Pipeline**

- **GitHub**: https://github.com/AntoninLeroy/wine_label_reader_toolkit
- **Technologies**:
  - TensorFlow
  - OpenCV
  - Tesseract OCR
- **Features**:
  - Label detection
  - Text extraction
  - Wine information parsing
- **Tutorial**: https://towardsdatascience.com/how-to-read-a-label-on-a-wine-bottle-using-computer-vision-part-3-e4729e5a8140

**Use Case**: End-to-end label reading pipeline

---

### 3. SVM Fraud Detection in Wine

**Authentication via Chemical Analysis**

- **GitHub**: https://github.com/eiliaJafari/SVM-for-fraud-detection-in-wine
- **Method**: Support Vector Machine (SVM)
- **Input**: Chemical properties
- **Output**: Fraud probability
- **Problem**: Detects relabeling of inferior wines as premium brands

**Use Case**: Secondary verification via chemical composition (if available)

---

### 4. Wine Label Unwrapping

**Label Flattening Algorithm**

- **GitHub**: Referenced in HackerNoon article
- **Technology**: U-Net for shape detection
- **Purpose**: "Unwrap" curved labels from bottle photos
- **Process**: 6-point manual positioning → flat label image
- **Benefit**: Better OCR accuracy on curved surfaces

**Use Case**: Pre-processing step before OCR/classification

---

## 🔐 Authentication Technologies

### Blockchain-Based Solutions

**How It Works**:
1. Each bottle gets unique digital signature
2. Signature stored on blockchain (immutable)
3. NFC/QR code on bottle links to blockchain record
4. Scan bottle → verify against blockchain

**Key Players**:
- Authena.io
- Everledger
- Identiv (NFC + Blockchain)

**Technologies**:
- Blockchain ledgers
- NFC tags
- IoT sensors
- Digital passports

---

### AI Chemical Fingerprinting

**Spectral Analysis**:
- AI analyzes wine's chemical composition
- Creates unique molecular fingerprint
- Compares to authentic reference samples
- Detects subtle variations from counterfeits

**Soil/Terroir Signatures**:
- Unique chemical signature from soil, air, environment
- Stored in blockchain
- Non-invasive content testing confirms origin

---

### Image Recognition + AI

**Pattern Recognition**:
- ML algorithms identify genuine vs. fake labels
- Analyzes capsules, corks, packaging
- Detects printing inconsistencies
- Compares to reference database

**Current Implementation**: Wine Label Recognition API (Zyla API Hub)
- Trained on 400,000+ wine labels
- High accuracy recognition
- Commercial API available

---

## 🏗️ Proposed Architecture for Wine Tracker

### Feature: "Verify Wine Authenticity"

#### User Flow:
1. User uploads wine label photo
2. System detects label in image (bounding box)
3. System classifies wine (name, vintage, winery)
4. System checks against database of authentic wines
5. System returns:
   - ✅ **Authentic** (matches known wine)
   - ⚠️ **Unknown** (not in database)
   - ❌ **Suspicious** (inconsistencies detected)

---

### Technical Stack

#### Frontend (Upload Interface)
```javascript
// React component for image upload
- Drag-and-drop upload
- Mobile camera capture
- Image preview
- Real-time feedback
```

#### Backend API
```
POST /api/wine/verify-label
```

**Request**:
```json
{
  "image": "base64_encoded_image",
  "confidence_threshold": 0.7
}
```

**Response**:
```json
{
  "success": true,
  "verification": {
    "status": "authentic",
    "confidence": 0.92,
    "wine": {
      "name": "Château Margaux",
      "vintage": "2015",
      "winery": "Château Margaux",
      "region": "Bordeaux, France",
      "appellation": "Margaux AOC"
    },
    "details": {
      "label_detected": true,
      "text_extracted": ["Château Margaux", "2015", "Grand Vin"],
      "visual_match_score": 0.95,
      "known_counterfeits": 0
    }
  }
}
```

---

### ML Model Pipeline

#### Option 1: Pre-Trained Model (Fast Implementation)

**Use Roboflow Wine Label Detection**:
```python
from roboflow import Roboflow

rf = Roboflow(api_key="YOUR_API_KEY")
project = rf.workspace().project("wine-label-detection")
model = project.version(1).model

# Upload image and get prediction
prediction = model.predict("wine_label.jpg", confidence=40)
```

**Pros**:
- ✅ No training required
- ✅ 5,464 images already trained
- ✅ API available
- ✅ Fast deployment

**Cons**:
- ⚠️ Limited to 5,464 wine types
- ⚠️ May not recognize rare/new wines

---

#### Option 2: Custom Model with WineSensed (Best Accuracy)

**Train on 897,000 Images**:
```python
# Download WineSensed dataset
# Train CNN (ResNet50, EfficientNet, Vision Transformer)
# Fine-tune on wine classification task

import torch
import torchvision.models as models

# Load pre-trained model
model = models.resnet50(pretrained=True)
model.fc = torch.nn.Linear(2048, 350000)  # 350k unique wines

# Train on WineSensed dataset
# ...
```

**Pros**:
- ✅ Massive dataset (897k labels)
- ✅ High accuracy potential
- ✅ Covers most wines globally
- ✅ Includes pricing, ratings, regions

**Cons**:
- ⚠️ Requires GPU training (expensive)
- ⚠️ Large dataset download (~100GB+ with images)
- ⚠️ Training time: days/weeks

---

#### Option 3: Hybrid Approach (Recommended)

**Combine Multiple Techniques**:

1. **Label Detection**: Roboflow API (detect label in photo)
2. **OCR Text Extraction**: Tesseract/OpenWines OCR
3. **Image Classification**: Pre-trained model on subset
4. **Database Matching**: Cross-reference with existing wine databases
5. **Confidence Scoring**: Combine all signals

```python
def verify_wine_label(image):
    # Step 1: Detect label
    label_bbox = roboflow_detect(image)

    # Step 2: Extract text
    label_crop = crop_image(image, label_bbox)
    text = extract_text_ocr(label_crop)

    # Step 3: Parse wine info
    wine_info = parse_wine_text(text)

    # Step 4: Image classification
    visual_match = classify_image(label_crop)

    # Step 5: Database lookup
    db_wines = search_database(wine_info)

    # Step 6: Combine confidence
    confidence = calculate_confidence(text, visual_match, db_wines)

    return {
        "status": determine_status(confidence),
        "confidence": confidence,
        "wine": wine_info
    }
```

**Pros**:
- ✅ Best of all approaches
- ✅ Graceful degradation
- ✅ Higher confidence scores
- ✅ Works even if one method fails

---

## 📊 Database Integration

### Sources to Cross-Reference Against:

1. **WineSensed** (350k wines)
2. **X-Wines** (100k wines)
3. **INAO AOC/AOP** (French appellations)
4. **Vivino API** (current wines)
5. **Total Wine catalog**
6. **K&L Wine Merchants**
7. **UC Davis AVA** (US regions)

### Verification Logic:
```
IF wine_name + vintage + winery matches database:
  ✅ AUTHENTIC (high confidence)
ELSE IF wine_name matches but vintage unknown:
  ⚠️ LIKELY AUTHENTIC (medium confidence)
ELSE IF no matches found:
  ❌ UNKNOWN or SUSPICIOUS (low confidence)
```

---

## 🚀 Implementation Roadmap

### Phase 1: MVP (2-4 weeks)
- [ ] Integrate Roboflow Wine Label Detection API
- [ ] Create upload interface (drag-and-drop)
- [ ] Implement label detection
- [ ] Add OCR text extraction
- [ ] Build database matching logic
- [ ] Deploy verification endpoint

### Phase 2: Enhanced ML (1-2 months)
- [ ] Download WineSensed dataset (subset)
- [ ] Train custom classification model
- [ ] Implement label unwrapping
- [ ] Add confidence scoring
- [ ] Create verification history

### Phase 3: Advanced Features (2-3 months)
- [ ] Mobile app integration
- [ ] Real-time camera detection
- [ ] Blockchain verification (optional)
- [ ] Counterfeit database
- [ ] User reporting system

---

## 💰 Cost Estimates

### Using Roboflow API (Fastest)
- **Setup**: $0 (free tier available)
- **API Calls**: $0.0002 per prediction
- **1,000 verifications/month**: ~$0.20
- **10,000 verifications/month**: ~$2.00

### Custom Model Training
- **GPU Training**: $100-500 (cloud GPUs)
- **Dataset Storage**: $20-50/month
- **Inference**: Free (self-hosted) or $0.0001/call

### Hybrid Approach
- **Initial**: $200-300 (model training)
- **Monthly**: $50-100 (API + hosting)
- **Per verification**: ~$0.0005

---

## 📚 Resources & Links

### Datasets
- WineSensed: https://data.dtu.dk/articles/dataset/WineSensed_Learning_to_Taste_A_Multimodal_Wine_Dataset/23376560
- Roboflow: https://universe.roboflow.com/wine-label/wine-label-detection
- HuggingFace: https://huggingface.co/datasets/Francesco/wine-labels

### GitHub Projects
- OpenWines OCR: https://github.com/OpenWines/OpenWinesOCR
- Wine Label Reader: https://github.com/AntoninLeroy/wine_label_reader_toolkit
- WinesLabelDataset: https://github.com/qanastek/WinesLabelDataset

### APIs
- Roboflow API: https://roboflow.com/
- Zyla Wine Recognition: https://zylalabs.com/api-marketplace/machine+learning/wine+label+recognition+api/825

### Research
- NeurIPS 2023 Paper: https://thoranna.github.io/learning_to_taste/
- Wine Authentication Tech: https://authena.io/using-blockchain-to-verify-the-authenticity-of-wines-and-spirits-in-the-web3-world/

---

## 🎯 Recommended Next Steps

**For Red Thunder Wine Tracker:**

1. **Quick Win** (1 week):
   - Integrate Roboflow API
   - Add "Verify Label" button to UI
   - Basic upload + detection

2. **Medium Term** (1 month):
   - Download WineSensed subset (10k images)
   - Train basic classifier
   - Add to existing wine search

3. **Long Term** (3 months):
   - Full WineSensed integration
   - Mobile camera support
   - Counterfeit reporting

**ROI**:
- Unique differentiator vs. competitors
- Builds trust with users
- Prevents counterfeit purchases
- Educational value for wine collectors

---

*Document compiled: November 2025*
*For: Red Thunder Wine Tracker - Wine Label Authentication Feature*