Skip to content

Commit 14a6671

Browse files
sidmohan0sidmohan0
committed
Extended .gitignore
Reorganized and expanded to properly handle build artifacts, caches, and development files Added patterns for common editor files, OS-specific files, and documentation build artifacts Pinned dependency versions Updated requirements.txt with exact versions for all dependencies Updated requirements-dev.txt with exact versions and added pytest-benchmark and ruff Added CI workflows Updated the existing benchmark workflow with modern GitHub Actions versions Created new workflows: tests.yml for running unit tests on Python 3.10, 3.11, and 3.12 lint.yml for running flake8, ruff, and mypy type checking Updated documentation Added new CI workflow badges to the README Updated the CHANGELOG with the new features and improvements in 4.1.0
1 parent 3f8f172 commit 14a6671

File tree

11 files changed

+137
-50
lines changed

11 files changed

+137
-50
lines changed

‎.github/workflows/benchmark.yml‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@ name: Performance Benchmarks
22

33
on:
44
push:
5-
branches: [main, develop]
5+
branches: [main, dev]
66
pull_request:
7-
branches: [main, develop]
7+
branches: [main, dev]
88
# Schedule benchmarks to run weekly
99
schedule:
1010
- cron: "0 0 * * 0" # Run at midnight on Sundays
@@ -13,12 +13,12 @@ jobs:
1313
benchmark:
1414
runs-on: ubuntu-latest
1515
steps:
16-
- uses: actions/checkout@v3
16+
- uses: actions/checkout@v4
1717
with:
1818
fetch-depth: 0 # Fetch all history for proper comparison
1919

2020
- name: Set up Python
21-
uses: actions/setup-python@v4
21+
uses: actions/setup-python@v5
2222
with:
2323
python-version: "3.10"
2424
cache: "pip"
@@ -28,10 +28,9 @@ jobs:
2828
python -m pip install --upgrade pip
2929
pip install -e .
3030
pip install -r requirements-dev.txt
31-
pip install pytest-benchmark
3231
3332
- name: Restore benchmark data
34-
uses: actions/cache@v3
33+
uses: actions/cache@v4
3534
with:
3635
path: .benchmarks
3736
key: benchmark-${{ runner.os }}-${{ hashFiles('**/requirements*.txt') }}
@@ -41,7 +40,7 @@ jobs:
4140
- name: Run benchmarks and save baseline
4241
run: |
4342
# Run benchmarks and save results
44-
pytest tests/benchmark_text_service.py -v --benchmark-autosave
43+
python -m pytest tests/benchmark_text_service.py -v --benchmark-autosave --benchmark-json=benchmark-results.json
4544
4645
- name: Check for performance regression
4746
run: |
@@ -71,10 +70,12 @@ jobs:
7170
fi
7271
7372
- name: Upload benchmark results
74-
uses: actions/upload-artifact@v3
73+
uses: actions/upload-artifact@v4
7574
with:
7675
name: benchmark-results
77-
path: .benchmarks/
76+
path: |
77+
.benchmarks/
78+
benchmark-results.json
7879
7980
- name: Alert on regression
8081
if: failure()

‎.github/workflows/lint.yml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Lint
2+
3+
on:
4+
push:
5+
branches: [main, dev]
6+
pull_request:
7+
branches: [main, dev]
8+
9+
jobs:
10+
lint:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- uses: actions/checkout@v4
14+
15+
- name: Set up Python
16+
uses: actions/setup-python@v5
17+
with:
18+
python-version: "3.10"
19+
cache: "pip"
20+
21+
- name: Install dependencies
22+
run: |
23+
python -m pip install --upgrade pip
24+
pip install -r requirements-dev.txt
25+
26+
- name: Lint with flake8
27+
run: |
28+
# stop the build if there are Python syntax errors or undefined names
29+
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
30+
# exit-zero treats all errors as warnings
31+
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
32+
33+
- name: Lint with ruff
34+
run: |
35+
ruff check .
36+
37+
- name: Type check with mypy
38+
run: |
39+
mypy datafog/

‎.github/workflows/tests.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [main, dev]
6+
pull_request:
7+
branches: [main, dev]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
python-version: ["3.10", "3.11", "3.12"]
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Set up Python ${{ matrix.python-version }}
20+
uses: actions/setup-python@v5
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
cache: "pip"
24+
25+
- name: Install dependencies
26+
run: |
27+
python -m pip install --upgrade pip
28+
pip install -e .
29+
pip install -r requirements-dev.txt
30+
31+
- name: Run tests with pytest
32+
run: |
33+
python -m pytest tests/ --cov=datafog --cov-report=xml
34+
35+
- name: Upload coverage report
36+
uses: codecov/codecov-action@v4
37+
with:
38+
file: ./coverage.xml
39+
fail_ci_if_error: true
40+
token: ${{ secrets.CODECOV_TOKEN }}

‎CHANGELOG.MD‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ChangeLog
22

3-
## [2025-05-02]
3+
## [2025-05-05]
44

55
### `datafog-python` [4.1.0]
66

@@ -13,6 +13,11 @@
1313
- Added 'When do I need spaCy?' guidance to documentation
1414
- Created scripts for running benchmarks locally and comparing results
1515
- Improved documentation with performance metrics and engine selection guidance
16+
- Extended .gitignore to better handle build artifacts and development files
17+
- Added GitHub Actions workflows for testing, linting, and benchmarking
18+
- Pinned all dependency versions in requirements.txt and requirements-dev.txt for reproducible builds
19+
- Added mypy type checking to CI pipeline
20+
- Added ruff linting to development dependencies
1621

1722
## [2024-03-25]
1823

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@
1111
<a href="https://pypi.org/project/datafog/"><img src="https://img.shields.io/pypi/pyversions/datafog.svg?style=flat-square" alt="PyPI pyversions"></a>
1212
<a href="https://github.com/datafog/datafog-python"><img src="https://img.shields.io/github/stars/datafog/datafog-python.svg?style=flat-square&logo=github&label=Stars&logoColor=white" alt="GitHub stars"></a>
1313
<a href="https://pypistats.org/packages/datafog"><img src="https://img.shields.io/pypi/dm/datafog.svg?style=flat-square" alt="PyPi downloads"></a>
14+
<a href="https://github.com/datafog/datafog-python/actions/workflows/tests.yml"><img src="https://github.com/datafog/datafog-python/actions/workflows/tests.yml/badge.svg" alt="Tests"></a>
15+
<a href="https://github.com/datafog/datafog-python/actions/workflows/lint.yml"><img src="https://github.com/datafog/datafog-python/actions/workflows/lint.yml/badge.svg" alt="Lint"></a>
16+
<a href="https://github.com/datafog/datafog-python/actions/workflows/benchmark.yml"><img src="https://github.com/datafog/datafog-python/actions/workflows/benchmark.yml/badge.svg" alt="Benchmarks"></a>
1417
<a href="https://discord.gg/bzDth394R4"><img src="https://img.shields.io/discord/1173803135341449227?style=flat" alt="Discord"></a>
1518
<a href="https://github.com/psf/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square" alt="Code style: black"></a>
1619
<a href="https://codecov.io/gh/datafog/datafog-python"><img src="https://img.shields.io/codecov/c/github/datafog/datafog-python.svg?style=flat-square" alt="codecov"></a>

‎datafog/__about__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "4.1.0"
1+
__version__ = "4.1.0b3"

‎notes/story-1.8-tkt.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
– Delete committed artifacts, extend .gitignore.
2+
– Add CI jobs for lint (ruff/flake8), mypy, tests, bench.
3+
– Pin exact versions in requirements*.txt; keep full dependency set.
4+
– Update docs / README badges.

‎requirements-dev.txt‎

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,16 @@
22
-r requirements.txt
33

44
# Development and testing dependencies
5-
just
6-
isort
7-
black
8-
blacken-docs
9-
certifi
10-
flake8
11-
prettier
12-
tox
13-
pytest==7.4.0
14-
pytest-asyncio==0.21.0
15-
pytest-cov
16-
mypy
17-
autoflake
18-
pre-commit
5+
just==0.8.162
6+
isort==5.13.2
7+
black==24.3.0
8+
blacken-docs==1.13.0
9+
certifi==2025.4.26
10+
flake8==7.0.0
11+
prettier==0.0.7
12+
tox==4.14.2
13+
pytest-benchmark==4.0.0
14+
mypy==1.9.0
15+
autoflake==2.2.1
16+
pre-commit==3.7.0
17+
ruff==0.3.4

‎requirements.txt‎

Lines changed: 19 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,23 @@
1-
pandas
1+
# Core dependencies with pinned versions
2+
pandas==2.2.3
23
requests==2.32.3
34
spacy==3.7.5
4-
pydantic>=2.8.2,<3.0.0
5-
Pillow
6-
sentencepiece
7-
protobuf
8-
pytesseract
9-
aiohttp
10-
pytest-asyncio
11-
numpy
12-
fastapi
13-
asyncio
14-
setuptools
5+
pydantic==2.11.4
6+
Pillow==11.2.1
7+
sentencepiece==0.2.0
8+
protobuf==6.30.2
9+
pytesseract==0.3.13
10+
aiohttp==3.11.18
11+
numpy==1.26.4
12+
fastapi==0.115.12
13+
asyncio==3.4.3
14+
setuptools>=68.0.0
1515
pydantic-settings==2.3.4
1616
typer==0.12.3
17-
sphinx
18-
cryptography
17+
sphinx==7.2.6
18+
cryptography==44.0.2
19+
20+
# Testing dependencies
21+
pytest==7.4.0
22+
pytest-asyncio==0.21.0
23+
pytest-cov==4.1.0

‎setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
long_description = f.read()
66

77
# Use a single source of truth for the version
8-
__version__ = "4.1.0"
8+
__version__ = "4.1.0b3"
99

1010
project_urls = {
1111
"Homepage": "https://datafog.ai",

0 commit comments

Comments
 (0)