Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 52 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ on:
- "v*"
pull_request:
jobs:
test:
lib_test:
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down Expand Up @@ -63,11 +63,60 @@ jobs:
cd $GITHUB_WORKSPACE
pytest tests

docs:
examples_test:
runs-on: ubuntu-latest
needs: lib_test
strategy:
matrix:
python-version: [ "3.10", "3.11", "3.12", "3.13" ]
example_app: ["todos"]

services:
mongodb:
image: mongo
ports:
- 27017:27017
options: >-
--health-cmd "echo 'db.runCommand("ping").ok' | mongosh --quiet"
--health-interval 10s
--health-timeout 5s
--health-retries 5
--name mongo_container

redis:
image: redis/redis-stack-server
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5

steps:
- name: Checkout the commit
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Test app '[${{ matrix.example_app }}]'
run: |
cd $GITHUB_WORKSPACE/examples/${{ matrix.example_app }}
python -m pip install --upgrade pip
python --version
pip install -r requirements.txt
pip install -U ../.."[all]"
black --check .
pytest .

release:
name: Release
runs-on: ubuntu-latest
if: "startsWith(github.ref, 'refs/tags/')"
needs: test
needs: examples_test
steps:
- name: Checkout the commit
uses: actions/checkout@v4
Expand Down
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,29 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased]

### Added

- Added [todos](./examples/todos) example application

### Changed

- Replaced the second required argument of `MongoStore.find()`,
`MongoStore.update()`, `MongoStore.delete()`with their first optional
key-word argument `query`.
- Changed the `limit` parameter of the `MongoStore.find()` to be by default 0.
- Removed the `Beanie-odm`-specific arguments passed to `MongoStore.insert()`,
`MongoStore.find()`, `MongoStore.update()`, `MongoStore.delete()`.

### Fixed

- Fixed error: "got Future <Future pending> attached to a different loop" when
`MongoStore.insert()`, `MongoStore.find()`, `MongoStore.update()`, `MongoStore.delete()`
are called especially during tests that create different event loops.
- Fixed `SQLStore.delete()` to return any nested items that were deleted.
- Fixed error: "got Future <Future pending> attached to a different loop" when
`RedisStore.insert()`, `RedisStore.find()`, `RedisStore.update()`, `RedisStore.delete()`
are called especially during tests that create different event loops.

## [0.1.4] - 2025-02-15

### Fixed
Expand Down
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,13 @@ support your favourite database technology.
- [RedisOM (_optional_)](https://redis.io/docs/latest/integrate/redisom-for-python/) - only required for [redis](https://redis.io/)
- [Beanie (_optional_)](https://beanie-odm.dev/) - only required for [MongoDB](https://www.mongodb.com/)

## Examples

See the [`examples`](/examples) folder for some example applications.
Hopefully more examples will be added with time. Currently, we have the following:

- [todos](./examples/todos)

## Quick Start

### Install NQLStore from Pypi
Expand Down Expand Up @@ -328,7 +335,6 @@ libraries = await redis_store.delete(
## TODO

- [ ] Add documentation site
- [ ] Add example applications

## Contributions

Expand Down
5 changes: 4 additions & 1 deletion examples/todos/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -182,4 +182,7 @@ renv/
menv/

# pyenv
.python-version
.python-version

# sqlite test db
test.db
23 changes: 14 additions & 9 deletions examples/todos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,19 +33,24 @@ pip install -r requirements.txt
- To use with redis, install [redis stack](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)
and start its server in another terminal.

- Start the application, selecting which database(s) to use.
- Start the application, set the URL's for the database(s) to use.
- _SQL_URL = f"sqlite+aiosqlite:///{_SQL_DB}"
_REDIS_URL = "redis://localhost:6379/0"
_MONGO_URL = "mongodb://localhost:27017"
_MONGO_DB = "testing"
Options are:
- `--sql` for [SQLite](https://www.sqlite.org/). This the default when no option is passed
- `--mongo` for [MongoDB](https://www.mongodb.com/products/self-managed/community-edition)
- `--redis` for [Redis](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)
- `SQL_URL` for [SQLite](https://www.sqlite.org/).
- `MONGO_URL` (required) and `MONGO_DB` (default: "todos") for [MongoDB](https://www.mongodb.com/products/self-managed/community-edition)
- `REDIS_URL` for [Redis](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/).

_It is possible to use multiple databases at the same time. Just pass multiple options_
_It is possible to use multiple databases at the same time. Just set multiple environment variables_

```shell
python main.py --sql # for SQL
# python main.py --redis # for redis
# python main.py --mongo # for mongoDB
# python main.py --sql --mongo # for SQL and mongoDB at the sametime
export SQL_URL="sqlite+aiosqlite:///test.db"
#export MONGO_URL="mongodb://localhost:27017"
#export MONGO_DB="testing"
export REDIS_URL="redis://localhost:6379/0"
fastapi dev main.py
```

## License
Expand Down
143 changes: 76 additions & 67 deletions examples/todos/conftest.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
"""Fixtures for tests"""

import os
from dataclasses import dataclass
from typing import Any

import pytest
import pytest_asyncio
from fastapi.testclient import TestClient
from models import (
MongoTodo,
MongoTodoList,
RedisTodo,
RedisTodoList,
SqlTodo,
SqlTodoList,
)

from nqlstore import MongoStore, RedisStore, SQLStore

Expand All @@ -22,62 +29,91 @@
{"name": "Boo", "todos": [{"title": "Talk endlessly till daybreak"}]},
]

_SQL_DB = "test.db"
_SQL_URL = f"sqlite+aiosqlite:///{_SQL_DB}"
_REDIS_URL = "redis://localhost:6379/0"
_MONGO_URL = "mongodb://localhost:27017"
_MONGO_DB = "testing"


@pytest.fixture
def sql_store():
def client_with_sql():
"""The fastapi test client when SQL is enabled"""
_reset_env()
os.environ["SQL_URL"] = _SQL_URL

from main import app

yield TestClient(app)
_reset_env()


@pytest_asyncio.fixture
async def client_with_redis():
"""The fastapi test client when redis is enabled"""
_reset_env()
os.environ["REDIS_URL"] = _REDIS_URL

from main import app

yield TestClient(app)
_reset_env()


@pytest.fixture
def client_with_mongo():
"""The fastapi test client when mongodb is enabled"""
_reset_env()

os.environ["MONGO_URL"] = _MONGO_URL
os.environ["MONGO_DB"] = _MONGO_DB

from main import app

yield TestClient(app)
_reset_env()


@pytest_asyncio.fixture()
async def sql_store():
"""The sql store stored in memory"""
sql_url = "sqlite+aiosqlite:///:memory:"
os.environ["SQL_URL"] = sql_url
store = SQLStore(uri=_SQL_URL)

store = SQLStore(uri=sql_url)
await store.register([SqlTodoList, SqlTodo])
yield store

# cleanup
os.environ["SQL_URL"] = ""
# clean up
os.remove(_SQL_DB)


@pytest.fixture
def mongo_store():
@pytest_asyncio.fixture()
async def mongo_store():
"""The mongodb store. Requires a running instance of mongodb"""
import pymongo

mongo_url = "mongodb://localhost:27017"
mongo_db = "testing"
os.environ["MONGO_URL"] = mongo_url
os.environ["MONGO_DB"] = mongo_db
mongo_store = MongoStore(uri=_MONGO_URL, database=_MONGO_DB)
await mongo_store.register([MongoTodoList, MongoTodo])

store = MongoStore(uri=mongo_url, database=mongo_db)
yield store
yield mongo_store

# clean up after the test
client = pymongo.MongoClient("mongodb://localhost:27017") # type: ignore
client.drop_database("testing")
os.environ["MONGO_URL"] = ""
# clean up
client = pymongo.MongoClient(_MONGO_URL) # type: ignore
client.drop_database(_MONGO_DB)


@pytest.fixture
def redis_store():
@pytest_asyncio.fixture
async def redis_store():
"""The redis store. Requires a running instance of redis stack"""
import redis

redis_url = "redis://localhost:6379/0"
os.environ["REDIS_URL"] = redis_url
store = RedisStore(_REDIS_URL)
await store.register([RedisTodoList, RedisTodo])

store = RedisStore(uri=redis_url)
yield store

# clean up after the test
# clean up
client = redis.Redis("localhost", 6379, 0)
client.flushall()
os.environ["REDIS_URL"] = ""


@pytest.fixture
def client():
"""The fastapi test client"""
from main import app

yield TestClient(app)


@pytest_asyncio.fixture()
Expand Down Expand Up @@ -107,36 +143,9 @@ async def redis_todolists(redis_store: RedisStore):
yield records


@dataclass(frozen=True)
class LazyFixture:
"""A fixture to be resolved lazily."""

name: str


def lazy_fixture(name: str) -> LazyFixture:
"""Create a lazy fixture."""
return LazyFixture(name)


@pytest.hookimpl(tryfirst=True)
def pytest_fixture_setup(
fixturedef: pytest.FixtureDef,
request: pytest.FixtureRequest,
) -> object | None:
"""Pytest hook to load lazy fixtures during setup

https://docs.pytest.org/en/latest/reference/reference.html#pytest.hookspec.pytest_fixture_setup
Stops at first non-None result

Args:
fixturedef: fixture definition object.
request: fixture request object.

Returns:
fixture value or None.
"""
param = getattr(request, "param", None)
if isinstance(param, LazyFixture):
request.param = request.getfixturevalue(param.name)
return None
def _reset_env():
"""Resets the environment variables available to the app"""
os.environ["SQL_URL"] = ""
os.environ["REDIS_URL"] = ""
os.environ["MONGO_URL"] = ""
os.environ["MONGO_DB"] = "testing"
Loading