📸 Receipt OCR System

Transform receipt photos into structured data automatically

Snap a photo with your iPhone → Instantly populated Google Sheet with all items, prices, and details.

---

🎯 What It Does

This system automatically extracts every item from receipts and organizes them in Google Sheets:

Before (Manual Entry - 2-3 minutes):

❌ Type each item manually
❌ Prone to typos and missed items  
❌ Tedious for 10+ item receipts

After (Automated - 5 seconds):

✅ One tap to capture and process
✅ All items extracted automatically
✅ Organized in searchable spreadsheet

---

🌟 Key Features

• 📱 One-Tap Workflow - iPhone Shortcuts integration
• 🏪 Universal Store Support - Works with Target, CVS, Walmart, Trader Joe's, etc.
• 🎯 High Accuracy - 94% item extraction rate
• ⚡ Fast - 3-5 second processing time
• 🔍 Complete Data - Store name, address, items, quantities, prices, taxes
• ☁️ Cloud-Based - No server management required
• 💰 Cost-Effective - ~$0.006 per receipt

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• Google Cloud account
• Anthropic API key (Claude)
• Google Sheets API credentials

Installation

1. Clone the repository

git clone https://github.com/DebugJedi/ReceiptOCR
cd receipt-ocr

2. Install dependencies

pip install -r requirements.txt

3. Set up environment variables

cp .env.example .env
# Edit .env with your API keys

Required environment variables:

ANTHROPIC_API_KEY=sk-ant-xxxxx
GOOGLE_CREDS_JSON={"type":"service_account",...}
spreadsheet_id=your-google-sheet-id
system_API=your-optional-auth-key

4. Run locally

python OCR_app.py
# Server starts at http://localhost:8000

---

📱 iPhone Setup

Using iPhone Shortcuts

1. Create a new Shortcut
2. Add these actions:

1. Receive [Images] input from Share Sheet
2. Set variable [Image] to [Shortcut Input]
3. Get Contents of URL
   - URL: https://your-cloud-run-url.run.app/receipt
   - Method: POST
   - Headers:
     * Content-Type: multipart/form-data
   - Request Body: Form
     * file: [Image]
4. Get Dictionary from [Contents of URL]
5. Show Notification
   - Title: Receipt Processed!
   - Body: [store_name] - $[total] - [item_count] items

3. Use the shortcut:
   • Open Photos app
   • Select receipt image
   • Tap Share → Your Shortcut name
   • Wait 3-5 seconds
   • Get notification with results

---

🏗️ Architecture

┌─────────────┐
│   iPhone    │
│   Camera    │
└──────┬──────┘
       │ Photo
       ▼
┌─────────────────┐
│ iPhone Shortcut │
│  (Automation)   │
└────────┬────────┘
         │ HTTPS POST
         ▼
┌──────────────────────┐
│  Google Cloud Run    │
│    FastAPI Server    │
└─────┬──────────┬─────┘
      │          │
      ▼          ▼
┌───────────┐  ┌──────────────┐
│ Claude AI │  │Google Sheets │
│  Vision   │  │     API      │
└───────────┘  └──────────────┘

Flow:

1. iPhone captures receipt photo
2. Shortcut sends image to Cloud Run API
3. FastAPI validates and compresses image
4. Claude Vision API extracts structured data
5. Data is appended to Google Sheets
6. iPhone receives confirmation notification

---

🛠️ Deployment

Deploy to Google Cloud Run

1. Build Docker image

docker build -t receipt-ocr .

2. Push to Google Container Registry

docker tag receipt-ocr gcr.io/YOUR-PROJECT/receipt-ocr
docker push gcr.io/YOUR-PROJECT/receipt-ocr

3. Deploy to Cloud Run

gcloud run deploy receipt-ocr \
  --image gcr.io/YOUR-PROJECT/receipt-ocr \
  --platform managed \
  --region us-central1 \
  --allow-unauthenticated \
  --memory 512Mi \
  --set-env-vars ANTHROPIC_API_KEY=xxx,GOOGLE_CREDS_JSON=xxx

4. Get your Cloud Run URL

gcloud run services describe receipt-ocr --format='value(status.url)'

5. Update iPhone Shortcut with the Cloud Run URL

---

🧪 Testing

Test with cURL

curl -X POST https://your-url.run.app/receipt \
  -F "file=@test_receipt.jpg" \
  -H "Authorization: Bearer your-api-key"

Test with Python

import requests

url = "https://your-url.run.app/receipt"
files = {"file": open("receipt.jpg", "rb")}
headers = {"Authorization": "Bearer your-api-key"}

response = requests.post(url, files=files, headers=headers)
print(response.json())

Expected Response

{
  "status": "success",
  "message": "Receipt processed successfully",
  "data": {
    "receipt_id": "3743",
    "store_name": "CVS PHARMACY",
    "date": "2023-01-10",
    "total": $0.00,
    "payment_method": "VISA",
    "card_last_4": "**84",
    "item_count": 11
  },
  "sheet_update": {
    "rows_added": 12,
    "cells_updated": 144
  }
}

---

📝 API Documentation

POST /receipt

Process a receipt image and store results in Google Sheets.

Request:

• Method: POST
• Content-Type: multipart/form-data
• Body:
  • file: Image file (JPEG, PNG, HEIC)
• Headers (optional):
  • Authorization: Bearer token

Response:

{
  "status": "success",
  "data": {
    "receipt_id": "string",
    "store_name": "string",
    "address": "string",
    "date": "YYYY-MM-DD",
    "subtotal": 0.00,
    "tax": 0.00,
    "total": 0.00,
    "payment_method": "string",
    "card_last_4": "string",
    "item_count": 0
  },
  "sheet_update": {
    "rows_added": 0,
    "cells_updated": 0
  }
}

Error Responses:

• 400: Invalid file type or corrupted image
• 401: Invalid authorization token
• 500: Internal server error

---

🏪 Supported Stores

Tested and working with:

• ✅ Pharmacies: CVS, Walgreens, Rite Aid
• ✅ Grocery: Trader Joe's, Whole Foods, Safeway, Kroger
• ✅ Big Box: Target, Walmart, Costco
• ✅ Convenience: 7-Eleven, Circle K
• ✅ Restaurants: Various formats
• ✅ Gas Stations: Shell, Chevron, BP

Note: The system uses AI-powered OCR and works with most printed receipts. If you encounter issues with a specific store format, please open an issue.

---

💡 Use Cases

Personal Finance

• Track grocery spending
• Monitor budget categories
• Tax preparation (itemized deductions)
• Price comparison across stores

Small Business

• Expense reporting automation
• Receipt organization for accounting
• Vendor spending analysis
• Mileage and per diem tracking

Data Analysis

• Spending trends over time
• Category breakdown
• Store comparison
• Product price tracking

---

🔧 Configuration

Environment Variables

Variable	Description	Required	Example
ANTHROPIC_API_KEY	Claude API key	Yes	sk-ant-xxxxx
GOOGLE_CREDS_JSON	Service account JSON	Yes	{"type":"service_account",...}
spreadsheet_id	Google Sheets ID	Yes	1BxiMVs0XRA5nFMdKvBdBZjgm...
system_API	Optional auth key	No	your-secret-key

Google Sheets Setup

1. Create a new Google Sheet
2. Enable Google Sheets API in Google Cloud Console
3. Create a service account and download JSON credentials
4. Share the sheet with the service account email
5. Copy the sheet ID from the URL

Anthropic API Setup

1. Sign up at Anthropic Console
2. Generate an API key
3. Add to .env file

---

📊 Performance

Metric	Value
Average Processing Time	3.2 seconds
Item Extraction Accuracy	94%
Success Rate	99.4%
Cost Per Receipt	$0.006
Supported Image Formats	JPG, PNG, HEIC
Max Image Size	10MB (compressed to 4MB)

Accuracy measured across 150 test receipts spanning 12 store formats including grocery, pharmacy, and big box retailers.

---

🐛 Troubleshooting

Issue: "No items extracted"

Causes:

• Receipt image is blurry or low quality
• Receipt is folded or crumpled
• Lighting is too dark

Solutions:

• Retake photo with better lighting
• Flatten receipt before photographing
• Ensure text is clearly readable

---

Issue: "Wrong quantities extracted"

Causes:

• Size indicators confused with quantities (e.g., "3oz" vs "3 items")

Solutions:

• System should handle this automatically
• If persistent, report the issue

---

Issue: "API timeout"

Causes:

• Large image files
• Network connectivity issues

Solutions:

• Image is automatically compressed
• Check Cloud Run logs for errors
• Increase timeout in Cloud Run settings

---

Development Setup

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

Testing Locally

# Run tests
pytest tests/

# Test with a sample receipt
python parser.py tests/test_receipts/sample_cvs.jpg

# Start dev server with auto-reload
uvicorn OCR_app:app --reload

---

📜 License

This project is licensed under the MIT License - see LICENSE file for details.

---

🙏 Acknowledgments

• Anthropic Claude for the Vision API
• FastAPI for the excellent web framework
• Google Cloud Run for serverless hosting

---

📧 Contact

Priyank - debugjedi@gmail.com

Project Link: ReceiptOCR

---

📈 Roadmap

• Basic receipt OCR
• Google Sheets integration
• iPhone Shortcuts support
• Universal store format support
• Duplicate detection
• Multi-currency support
• Analytics dashboard
• Receipt image archive
• Batch processing
• Mobile app

See ROADMAP.md for detailed plans.