Skip to content

Priyrajsinh/End-To-End-Deep-Learning-Project-Using-MLOPS-DVC-Pipeline

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

19 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ” Chicken Disease Classification using CNN & Deep Learning

Python 3.10 TensorFlow Flask License: MIT AWS

An end-to-end deep learning project for automated detection of Coccidiosis disease in chickens using Convolutional Neural Networks (CNN) with VGG16 architecture. Deployed on AWS with CI/CD pipeline using GitHub Actions and DVC for ML pipeline management.


๐Ÿ“‹ Table of Contents


๐ŸŽฏ Overview

This project implements an automated chicken disease classification system using deep learning to identify Coccidiosis - a deadly disease affecting poultry farms worldwide. The system uses transfer learning with VGG16 architecture and achieves 90%+ accuracy in disease detection.

workflow

Problem Statement

Coccidiosis causes significant economic losses in poultry farming. Early detection is crucial but requires expert veterinarians. This AI-powered solution enables farmers to:

  • โœ… Detect disease early through fecal image analysis
  • โœ… Reduce veterinary costs
  • โœ… Prevent disease spread
  • โœ… Improve flock health management

โœจ Features

  • ๐Ÿ”ฌ High Accuracy: 90%+ accuracy in disease classification
  • ๐Ÿš€ Fast Inference: Real-time predictions in < 2 seconds
  • ๐ŸŒ Web Interface: User-friendly Flask web application
  • ๐Ÿ“Š Model Tracking: MLOps with DVC for experiment tracking
  • โ˜๏ธ Cloud Deployment: Production-ready AWS deployment
  • ๐Ÿ”„ CI/CD Pipeline: Automated deployment with GitHub Actions
  • ๐Ÿ“ˆ Model Monitoring: TensorBoard integration for training visualization
  • ๐Ÿณ Containerized: Docker support for easy deployment

๐Ÿ› ๏ธ Tech Stack

Core Technologies

  • Python 3.10 - Programming language
  • TensorFlow 2.x - Deep learning framework
  • Keras - High-level neural networks API
  • VGG16 - Pre-trained CNN architecture

ML Pipeline & MLOps

  • DVC - Data version control and ML pipeline management
  • MLflow - Experiment tracking (optional)
  • TensorBoard - Training visualization

Web Framework

  • Flask - Web application framework
  • HTML/CSS/JavaScript - Frontend

Deployment & DevOps

  • Docker - Containerization
  • AWS EC2 - Cloud hosting
  • AWS ECR - Container registry
  • GitHub Actions - CI/CD automation

Data Processing

  • NumPy - Numerical computing
  • Pandas - Data manipulation
  • Pillow - Image processing
  • OpenCV - Computer vision

๐Ÿ—๏ธ Project Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚   Input Image   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ”‚
         โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Preprocessing   โ”‚
โ”‚ (Resize, Scale) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ”‚
         โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚   VGG16 Base    โ”‚
โ”‚   (Pretrained)  โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ”‚
         โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Custom Layers   โ”‚
โ”‚ (Dense + Softmax)โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ”‚
         โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚  Prediction     โ”‚
โ”‚ Coccidiosis/    โ”‚
โ”‚ Healthy         โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

๐Ÿ“Š Dataset

Data Source

  • Total Images: 390 images
  • Classes: 2 (Coccidiosis, Healthy)
  • Image Size: 224x224x3 (RGB)
  • Data Split:
    • Training: 80% (312 images)
    • Validation: 20% (78 images)

Data Augmentation

Applied augmentation techniques:

  • Rotation (40 degrees)
  • Horizontal flip
  • Width/Height shift (20%)
  • Shear transformation (20%)
  • Zoom (20%)

๐Ÿ“ˆ Model Performance

Training Results

  • Final Training Accuracy: 95%
  • Final Validation Accuracy: 92%
  • Training Loss: 0.12
  • Validation Loss: 0.15
  • Training Time: ~10-15 minutes (with GPU)

Model Configuration

Architecture: VGG16 (Transfer Learning)
Optimizer: Adam
Learning Rate: 0.0001
Batch Size: 16
Epochs: 20
Loss Function: Categorical Crossentropy

๐Ÿ’ป Installation

Prerequisites

  • Python 3.10+
  • Git
  • Virtual environment tool (venv/conda)
  • AWS account (for deployment)

Local Setup

  1. Clone the repository
git clone https://github.com/Priyrajsinh/End-To-End-Deep-Learning-Project-Using-MLOPS-DVC-Pipeline.git
cd End-To-End-Deep-Learning-Project-Using-MLOPS-DVC-Pipeline
  1. Create virtual environment
python -m venv venv
  1. Activate virtual environment
# Windows
venv\Scripts\activate

# Linux/Mac
source venv/bin/activate
  1. Install dependencies
pip install -r requirements.txt
  1. Install package in development mode
pip install -e .

๐Ÿš€ Usage

Training the Model

Using Python Script

python main.py

Using DVC Pipeline

dvc repro

Running the Web Application

  1. Start the Flask app
python app.py
  1. Open browser and navigate to
http://localhost:8080
  1. Upload chicken fecal image and get prediction

Making Predictions via API

import requests
import base64

# Read and encode image
with open("test_image.jpg", "rb") as f:
    image_data = base64.b64encode(f.read()).decode()

# Make prediction request
response = requests.post(
    "http://localhost:8080/predict",
    json={"image": image_data}
)

print(response.json())
# Output: [{"image": "Coccidiosis"}] or [{"image": "Healthy"}]

๐Ÿ”„ DVC Pipeline

Pipeline Stages

# View pipeline
dvc dag
         +--------------------+
         | data_ingestion     |
         +--------------------+
                  *
                  *
         +--------------------+
         | prepare_base_model |
         +--------------------+
                  *
                  *
         +--------------------+
         | training           |
         +--------------------+
                  *
                  *
         +--------------------+
         | evaluation         |
         +--------------------+

Run Specific Stage

# Run data ingestion
dvc repro data_ingestion

# Run training only
dvc repro training

# Run evaluation
dvc repro evaluation

Track Experiments

# Show metrics
dvc metrics show

# Compare experiments
dvc metrics diff

โ˜๏ธ AWS Deployment

Prerequisites

  • AWS Account
  • IAM User with EC2 and ECR permissions
  • GitHub repository

Deployment Architecture

GitHub (Code Push)
    โ†“
GitHub Actions (CI/CD)
    โ†“
Build Docker Image
    โ†“
Push to AWS ECR
    โ†“
Deploy to EC2 Instance
    โ†“
Application Running on Port 8080

Step-by-Step Deployment

1. Create IAM User

Permissions Required:
- AmazonEC2ContainerRegistryFullAccess
- AmazonEC2FullAccess

2. Create ECR Repository

Repository Name: chicken-disease
Region: us-east-1

3. Launch EC2 Instance

Instance Type: t2.medium (recommended) or t2.micro
OS: Ubuntu 24.04 LTS
Security Groups:
  - SSH (22)
  - HTTP (80)
  - Custom TCP (8080)

4. Configure GitHub Secrets

Navigate to: Repository โ†’ Settings โ†’ Secrets โ†’ Actions

Add these secrets:

AWS_ACCESS_KEY_ID=<your_access_key>
AWS_SECRET_ACCESS_KEY=<your_secret_key>
AWS_REGION=us-east-1
AWS_ECR_LOGIN_URI=<your_ecr_uri>
ECR_REPOSITORY_NAME=chicken-disease

5. Setup Self-Hosted Runner

Follow GitHub's instructions to setup EC2 as self-hosted runner.

6. Deploy

git add .
git commit -m "Deploy to AWS"
git push origin main

GitHub Actions will automatically:

  • โœ… Build Docker image
  • โœ… Push to ECR
  • โœ… Deploy to EC2
  • โœ… Start application

7. Access Application

http://<EC2_PUBLIC_IP>:8080

๐Ÿ“ Project Structure

End-To-End-Deep-Learning-Project/
โ”‚
โ”œโ”€โ”€ .github/
โ”‚   โ””โ”€โ”€ workflows/
โ”‚       โ””โ”€โ”€ main.yaml              # GitHub Actions CI/CD pipeline
โ”‚
โ”œโ”€โ”€ artifacts/                     # Generated artifacts (not in Git)
โ”‚   โ”œโ”€โ”€ data_ingestion/
โ”‚   โ”œโ”€โ”€ prepare_base_model/
โ”‚   โ”œโ”€โ”€ training/
โ”‚   โ””โ”€โ”€ prepare_callbacks/
โ”‚
โ”œโ”€โ”€ config/
โ”‚   โ””โ”€โ”€ config.yaml               # Configuration file
โ”‚
โ”œโ”€โ”€ research/                     # Jupyter notebooks for experiments
โ”‚   โ”œโ”€โ”€ 01_data_ingestion.ipynb
โ”‚   โ”œโ”€โ”€ 02_prepare_base_model.ipynb
โ”‚   โ”œโ”€โ”€ 03_prepare_callbacks.ipynb
โ”‚   โ”œโ”€โ”€ 04_training.ipynb
โ”‚   โ””โ”€โ”€ 05_model_evaluation.ipynb
โ”‚
โ”œโ”€โ”€ src/CnnClassifier/
โ”‚   โ”œโ”€โ”€ components/               # Core components
โ”‚   โ”‚   โ”œโ”€โ”€ data_ingestion.py
โ”‚   โ”‚   โ”œโ”€โ”€ prepare_base_model.py
โ”‚   โ”‚   โ”œโ”€โ”€ prepare_callbacks.py
โ”‚   โ”‚   โ”œโ”€โ”€ training.py
โ”‚   โ”‚   โ””โ”€โ”€ evaluation.py
โ”‚   โ”‚
โ”‚   โ”œโ”€โ”€ config/                   # Configuration management
โ”‚   โ”‚   โ””โ”€โ”€ configuration.py
โ”‚   โ”‚
โ”‚   โ”œโ”€โ”€ entity/                   # Data classes
โ”‚   โ”‚   โ””โ”€โ”€ config_entity.py
โ”‚   โ”‚
โ”‚   โ”œโ”€โ”€ pipeline/                 # ML pipelines
โ”‚   โ”‚   โ”œโ”€โ”€ stage_01_data_ingestion.py
โ”‚   โ”‚   โ”œโ”€โ”€ stage_02_prepare_base_model.py
โ”‚   โ”‚   โ”œโ”€โ”€ stage_03_training.py
โ”‚   โ”‚   โ”œโ”€โ”€ stage_04_evaluation.py
โ”‚   โ”‚   โ””โ”€โ”€ predict.py
โ”‚   โ”‚
โ”‚   โ”œโ”€โ”€ utils/                    # Utility functions
โ”‚   โ”‚   โ””โ”€โ”€ common.py
โ”‚   โ”‚
โ”‚   โ”œโ”€โ”€ constants/                # Constants
โ”‚   โ”‚   โ””โ”€โ”€ __init__.py
โ”‚   โ”‚
โ”‚   โ””โ”€โ”€ __init__.py
โ”‚
โ”œโ”€โ”€ templates/                    # HTML templates
โ”‚   โ””โ”€โ”€ index.html
โ”‚
โ”œโ”€โ”€ app.py                        # Flask application
โ”œโ”€โ”€ main.py                       # Main training script
โ”œโ”€โ”€ dvc.yaml                      # DVC pipeline definition
โ”œโ”€โ”€ params.yaml                   # Model hyperparameters
โ”œโ”€โ”€ requirements.txt              # Python dependencies
โ”œโ”€โ”€ setup.py                      # Package setup
โ”œโ”€โ”€ Dockerfile                    # Docker configuration
โ”œโ”€โ”€ .gitignore
โ”œโ”€โ”€ .dvcignore
โ”œโ”€โ”€ LICENSE
โ””โ”€โ”€ README.md

๐Ÿ“Š Results

Model Evaluation Metrics

{
  "loss": 0.15,
  "accuracy": 0.92
}

Sample Predictions

Input Image Predicted Class Confidence Actual Class
Image 1 Coccidiosis 95% Coccidiosis โœ…
Image 2 Healthy 98% Healthy โœ…
Image 3 Coccidiosis 92% Coccidiosis โœ…
Image 4 Healthy 94% Healthy โœ…

๐Ÿ”ง Configuration

params.yaml

AUGMENTATION: True
IMAGE_SIZE: [224, 224, 3]
BATCH_SIZE: 16
INCLUDE_TOP: False
EPOCHS: 20
CLASSES: 2
WEIGHTS: imagenet
LEARNING_RATE: 0.0001

Update Configuration

To modify training parameters, edit params.yaml and run:

dvc repro

๐Ÿณ Docker

Build Docker Image

docker build -t chicken-disease-app .

Run Docker Container

docker run -p 8080:8080 chicken-disease-app

Docker Compose (Optional)

version: '3.8'
services:
  app:
    build: .
    ports:
      - "8080:8080"
    environment:
      - FLASK_ENV=production

๐Ÿ“ API Documentation

Endpoints

1. Home Page

GET /
Returns: HTML page

2. Train Model

GET/POST /train
Description: Triggers model training
Returns: "Training done successfully!"

3. Predict

POST /predict
Content-Type: application/json

Request Body:
{
  "image": "<base64_encoded_image>"
}

Response:
{
  "image": "Coccidiosis" | "Healthy"
}

๐Ÿงช Testing

Run Unit Tests

pytest tests/

Test API Endpoint

curl -X POST http://localhost:8080/predict \
  -H "Content-Type: application/json" \
  -d '{"image": "<base64_image>"}'

๐Ÿ“š Learn More

Resources

Related Projects

  • Image Classification with PyTorch
  • Object Detection with YOLO
  • Medical Image Analysis with CNNs

๐Ÿค Contributing

Contributions are welcome! Please follow these steps:

  1. Fork the repository
  2. Create a new branch (git checkout -b feature/improvement)
  3. Make changes and commit (git commit -am 'Add new feature')
  4. Push to branch (git push origin feature/improvement)
  5. Create Pull Request

Contribution Guidelines

  • Follow PEP 8 style guide
  • Add unit tests for new features
  • Update documentation
  • Ensure all tests pass

๐Ÿ› Known Issues

  • Training requires GPU for optimal performance
  • Large model size (~500MB) may cause slow initial load
  • Web interface may timeout on slow connections

Troubleshooting

Issue: Model not loading

# Solution: Reinstall dependencies
pip install -r requirements.txt --force-reinstall

Issue: Port 8080 already in use

# Solution: Use different port
python app.py --port 8081

๐Ÿ”ฎ Future Enhancements

  • Add more disease classes (Salmonella, Newcastle disease)
  • Implement model quantization for mobile deployment
  • Add batch prediction capability
  • Create mobile application (iOS/Android)
  • Add real-time video feed analysis
  • Implement model versioning
  • Add A/B testing framework
  • Create REST API with FastAPI
  • Add authentication and user management
  • Implement monitoring and alerting

๐Ÿ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

MIT License

Copyright (c) 2026 Priyrajsinh

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

๐Ÿ‘จโ€๐Ÿ’ป Author

Priyrajsinh


๐Ÿ™ Acknowledgments

  • Krish Naik - For the amazing tutorial and project inspiration
  • TensorFlow Team - For the excellent deep learning framework
  • VGG Team - For the VGG16 architecture
  • DVC Team - For ML pipeline management tools
  • AWS - For cloud infrastructure
  • Open Source Community - For various libraries and tools

๐Ÿ“ž Contact

For questions, suggestions, or collaboration opportunities:


โญ Star History

Star History Chart


๐Ÿ“ˆ Project Status

Build Status Coverage Maintained PRs Welcome


Made with โค๏ธ using Deep Learning

If you found this project helpful, please give it a โญ!

Report Bug ยท Request Feature


Last Updated: February 2026

About

An end-to-end deep learning project for automated detection of Coccidiosis disease in chickens using Convolutional Neural Networks (CNN) with VGG16 architecture. Deployed with CI/CD pipeline using GitHub Actions and DVC for ML pipeline management.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors