A full-stack Expense Tracker app with:
Flutter frontend for input, visualization, and receipt upload
FastAPI backend with ML-powered text classification
SQLite database for local expense storage
PyTorch model for automatic category prediction
Donut model for receipt understanding (structured parsing, not just OCR)
(Optional) Neon Postgres integration — scaffolded but not yet active
🚀 Features
Enter expenses in natural language (e.g., "Spent 500 at KFC").
Upload receipts → Donut model extracts text + structured fields → auto-adds expenses.
ML model detects amounts and predicts categories (Food, Transport, Education, Housing, Healthcare, Others).
View daily totals and category breakdowns.
Delete entries and update totals instantly.
Frontend visualization with charts and calendar.
🧾 Receipt Handling (Donut Model)
Instead of raw OCR, receipts are parsed using Donut (Document Understanding Transformer).
Donut directly outputs structured JSON (e.g., { "items": [...], "total": 600, "merchant": "KFC" }).
Extracted data is mapped into date, category, amount, and text before storing in DB.
Example Flow
Upload receipt (e.g., "KFC_Bill.jpg").
Donut model parses →
{ "merchant": "KFC", "items": [ {"name": "Burger", "price": 500}, {"name": "Coke", "price": 100} ], "total": 600, "date": "2025-09-04" }
App saves expense: Food - 600 Rs (Spent at KFC).
Entry appears in daily summary & category breakdown.
⚙️ Backend Setup
-
Install dependencies pip install fastapi uvicorn torch scikit-learn joblib sqlite3 spacy pip install transformers timm # Donut dependencies python -m spacy download en_core_web_sm
-
Initialize DB python backend/add_to_db.py
-
Run backend uvicorn backend.app:app --reload
🔑 API Routes
POST /predict Add text-based expense → classify + save.
POST /receipt Upload receipt image → Donut parses → auto-save expense.
GET /expenses List all expenses (filter by date if needed).
DELETE /expenses/{id} Delete expense by ID.
GET /stats/day/{date} Category breakdown for a date.
✅ TODO
Integrate Neon Postgres (currently scaffolded, not active).
Improve Donut fine-tuning for regional receipt formats.
Deploy backend + connect Flutter app.
DEMO:
- Add a note
Updated to database.
The model classifies it into a category.
Graph for viewing purposes:
Calendar picker to go to chosen dates and edit the expenses there:
Upload receipt : can edit items, cost and category before adding to database
Adding soon : Smart AI suggested budgeting Expense prediction for next months expense based on previous 3 months : model created and tested, need to integrate