Skip to content

Commit 9fcf0e6

Browse files
whummerclaudepurcell
authored
Add initial version of ParadeDB extension (#122)
* add initial version of ParadeDB extension LocalStack extension for ParadeDB (PostgreSQL-based search and analytics). Features: - Runs paradedb/paradedb Docker container - Exposes PostgreSQL port 5432 for direct connections - Configurable via PARADEDB_POSTGRES_USER/PASSWORD/DB env vars - Integration tests for basic SQL and pg_search BM25 functionality - CI workflow for automated testing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix ParadeDB tests to use v2 API syntax Update tests to use the new ParadeDB v2 API: - Use CREATE INDEX ... USING bm25 instead of CALL paradedb.create_bm25() - Use ||| operator for disjunction (OR) search - Use &&& operator for conjunction (AND) search - Use pdb.score() for relevance scoring - Use DROP INDEX instead of paradedb.drop_bm25() Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * [paradedb] Prefer more distinct connection credentials * [paradedb] Also allow port to be overridden This can be helpful to avoid clashing with another locally-running postgres, for example. * [paradedb] Additional project keywords * [paradedb] Remove hallucinated project description * [paradedb] Fix hallucinated example * [paradedb] Just use the quickstart example as a test No need to exhaustively verify or showcase different aspects of paradedb here. * [paradedb] Apply formatter * [paradedb] Add to main list of available extensions --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Steve Purcell <steve@sanityinc.com>
1 parent a497cbc commit 9fcf0e6

File tree

13 files changed

+578
-0
lines changed

13 files changed

+578
-0
lines changed

.github/workflows/paradedb.yml

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: LocalStack ParadeDB Extension Tests
2+
3+
on:
4+
push:
5+
paths:
6+
- paradedb/**
7+
branches:
8+
- main
9+
pull_request:
10+
paths:
11+
- .github/workflows/paradedb.yml
12+
- paradedb/**
13+
workflow_dispatch:
14+
15+
env:
16+
LOCALSTACK_DISABLE_EVENTS: "1"
17+
LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }}
18+
19+
jobs:
20+
integration-tests:
21+
name: Run Integration Tests
22+
runs-on: ubuntu-latest
23+
timeout-minutes: 10
24+
steps:
25+
- name: Checkout
26+
uses: actions/checkout@v4
27+
28+
- name: Setup LocalStack and extension
29+
run: |
30+
cd paradedb
31+
32+
docker pull localstack/localstack-pro &
33+
docker pull paradedb/paradedb &
34+
pip install localstack
35+
36+
make install
37+
make lint
38+
make dist
39+
localstack extensions -v install file://$(ls ./dist/localstack_extension_paradedb-*.tar.gz)
40+
41+
DEBUG=1 localstack start -d
42+
localstack wait
43+
44+
- name: Run integration tests
45+
run: |
46+
cd paradedb
47+
make test
48+
49+
- name: Print logs
50+
if: always()
51+
run: |
52+
localstack logs
53+
localstack stop

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ You can install the respective extension by calling `localstack extensions insta
7676
| [Stripe](https://github.com/localstack/localstack-extensions/tree/main/stripe) | localstack-extension-stripe | 0.2.0 | Stable |
7777
| [Terraform Init](https://github.com/localstack/localstack-extensions/tree/main/terraform-init) | localstack-extension-terraform-init | 0.2.0 | Experimental |
7878
| [TypeDB](https://github.com/localstack/localstack-extensions/tree/main/typedb) | localstack-extension-typedb | 0.1.3 | Experimental |
79+
| [ParadeDB](https://github.com/localstack/localstack-extensions/tree/main/paradedb) | localstack-extension-paradedb | 0.1.0 | Experimental |
7980

8081

8182
## Developing Extensions

flake.nix

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
description = "localstack-extensions";
3+
4+
inputs = {
5+
nixpkgs.url = "nixpkgs/nixpkgs-unstable";
6+
};
7+
8+
outputs = { self, nixpkgs }@inputs:
9+
(
10+
let
11+
forAllSystems = nixpkgs.lib.genAttrs nixpkgs.lib.platforms.all;
12+
in
13+
{
14+
devShell = forAllSystems (system:
15+
let pkgs = import nixpkgs { inherit system; }; in
16+
pkgs.mkShell {
17+
buildInputs = with pkgs; [ uv python311 python311Packages.pip ty ];
18+
}
19+
);
20+
}
21+
);
22+
}

paradedb/.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.venv
2+
dist
3+
build
4+
**/*.egg-info
5+
.eggs

paradedb/Makefile

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
VENV_BIN = python3 -m venv
2+
VENV_DIR ?= .venv
3+
VENV_ACTIVATE = $(VENV_DIR)/bin/activate
4+
VENV_RUN = . $(VENV_ACTIVATE)
5+
TEST_PATH ?= tests
6+
7+
usage: ## Shows usage for this Makefile
8+
@cat Makefile | grep -E '^[a-zA-Z_-]+:.*?## .*$$' | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-15s\033[0m %s\n", $$1, $$2}'
9+
10+
venv: $(VENV_ACTIVATE)
11+
12+
$(VENV_ACTIVATE): pyproject.toml
13+
test -d .venv || $(VENV_BIN) .venv
14+
$(VENV_RUN); pip install --upgrade pip setuptools plux
15+
$(VENV_RUN); pip install -e .[dev]
16+
touch $(VENV_DIR)/bin/activate
17+
18+
clean:
19+
rm -rf .venv/
20+
rm -rf build/
21+
rm -rf .eggs/
22+
rm -rf *.egg-info/
23+
24+
install: venv ## Install dependencies
25+
$(VENV_RUN); python -m plux entrypoints
26+
27+
dist: venv ## Create distribution
28+
$(VENV_RUN); python -m build
29+
30+
publish: clean-dist venv dist ## Publish extension to pypi
31+
$(VENV_RUN); pip install --upgrade twine; twine upload dist/*
32+
33+
entrypoints: venv ## Generate plugin entrypoints for Python package
34+
$(VENV_RUN); python -m plux entrypoints
35+
36+
format: ## Run ruff to format the codebase
37+
$(VENV_RUN); python -m ruff format .; make lint
38+
39+
lint: ## Run ruff to lint the codebase
40+
$(VENV_RUN); python -m ruff check --output-format=full .
41+
42+
test: ## Run integration tests (requires LocalStack running with the Extension installed)
43+
$(VENV_RUN); pytest $(PYTEST_ARGS) $(TEST_PATH)
44+
45+
clean-dist: clean
46+
rm -rf dist/
47+
48+
.PHONY: clean clean-dist dist install publish usage venv format test

paradedb/README.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
ParadeDB on LocalStack
2+
======================
3+
4+
This repo contains a [LocalStack Extension](https://github.com/localstack/localstack-extensions) that facilitates developing [ParadeDB](https://www.paradedb.com)-based applications locally.
5+
6+
ParadeDB is an Elasticsearch alternative built on Postgres. It provides full-text search with BM25 scoring, hybrid search combining semantic and keyword search, and real-time analytics capabilities.
7+
8+
After installing the extension, a ParadeDB server instance will become available and can be accessed using standard PostgreSQL clients.
9+
10+
## Connection Details
11+
12+
Once the extension is running, you can connect to ParadeDB using any PostgreSQL client with the following default credentials:
13+
14+
- **Host**: `localhost` (or the Docker host if running in a container)
15+
- **Port**: `5432` (mapped from the container)
16+
- **Database**: `mydatabase`
17+
- **Username**: `myuser`
18+
- **Password**: `mypassword`
19+
20+
Example connection using `psql`:
21+
```bash
22+
psql -h localhost -p 5432 -U myuser -d mydatabase
23+
```
24+
25+
Example connection using Python:
26+
```python
27+
import psycopg2
28+
29+
conn = psycopg2.connect(
30+
host="localhost",
31+
port=5432,
32+
database="mydatabase",
33+
user="myuser",
34+
password="mypassword"
35+
)
36+
```
37+
38+
## ParadeDB Features
39+
40+
ParadeDB includes the **pg_search** extension, for both search and
41+
analytics workloads.
42+
43+
Example of BM25 scoring, from the excellent [quickstart guide](https://docs.paradedb.com/documentation/getting-started/quickstart):
44+
45+
```sql
46+
CALL paradedb.create_bm25_test_table(
47+
schema_name => 'public',
48+
table_name => 'mock_items'
49+
);
50+
51+
CREATE INDEX search_idx ON mock_items
52+
USING bm25 (id, description, category, rating, in_stock, created_at, metadata, weight_range)
53+
WITH (key_field='id');
54+
55+
SELECT description, pdb.score(id)
56+
FROM mock_items
57+
WHERE description ||| 'running shoes' AND rating > 2
58+
ORDER BY score DESC
59+
LIMIT 5;
60+
```
61+
62+
## Configuration
63+
64+
The following environment variables can be passed to the LocalStack container to configure the extension:
65+
66+
* `PARADEDB_POSTGRES_USER`: PostgreSQL username (default: `myuser`)
67+
* `PARADEDB_POSTGRES_PASSWORD`: PostgreSQL password (default: `mypassword`)
68+
* `PARADEDB_POSTGRES_DB`: Default database name (default: `mydatabase`)
69+
70+
## Prerequisites
71+
72+
* Docker
73+
* LocalStack Pro (free trial available)
74+
* `localstack` CLI
75+
* `make`
76+
77+
## Install from GitHub repository
78+
79+
This extension can be installed directly from this Github repo via:
80+
81+
```bash
82+
localstack extensions install "git+https://github.com/localstack/localstack-extensions.git#egg=localstack-extension-paradedb&subdirectory=paradedb"
83+
```
84+
85+
## Install local development version
86+
87+
Please refer to the docs [here](https://github.com/localstack/localstack-extensions?tab=readme-ov-file#start-localstack-with-the-extension) for instructions on how to start the extension in developer mode.
88+
89+
## Change Log
90+
91+
* `0.1.0`: Initial version of the extension
92+
93+
## License
94+
95+
The code in this repo is available under the Apache 2.0 license.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
name = "localstack_paradedb"
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
import os
2+
import logging
3+
4+
from localstack_paradedb.utils.docker import DatabaseDockerContainerExtension
5+
6+
LOG = logging.getLogger(__name__)
7+
8+
# Environment variables for configuration
9+
ENV_POSTGRES_USER = "PARADEDB_POSTGRES_USER"
10+
ENV_POSTGRES_PASSWORD = "PARADEDB_POSTGRES_PASSWORD"
11+
ENV_POSTGRES_DB = "PARADEDB_POSTGRES_DB"
12+
ENV_POSTGRES_PORT = "PARADEDB_POSTGRES_PORT"
13+
14+
# Default values
15+
DEFAULT_POSTGRES_USER = "myuser"
16+
DEFAULT_POSTGRES_PASSWORD = "mypassword"
17+
DEFAULT_POSTGRES_DB = "mydatabase"
18+
DEFAULT_POSTGRES_PORT = 5432
19+
20+
21+
class ParadeDbExtension(DatabaseDockerContainerExtension):
22+
name = "paradedb"
23+
24+
# Name of the Docker image to spin up
25+
DOCKER_IMAGE = "paradedb/paradedb"
26+
27+
def __init__(self):
28+
# Get configuration from environment variables
29+
postgres_user = os.environ.get(ENV_POSTGRES_USER, DEFAULT_POSTGRES_USER)
30+
postgres_password = os.environ.get(
31+
ENV_POSTGRES_PASSWORD, DEFAULT_POSTGRES_PASSWORD
32+
)
33+
postgres_db = os.environ.get(ENV_POSTGRES_DB, DEFAULT_POSTGRES_DB)
34+
postgres_port = int(os.environ.get(ENV_POSTGRES_PORT, DEFAULT_POSTGRES_PORT))
35+
36+
# Environment variables to pass to the container
37+
env_vars = {
38+
"POSTGRES_USER": postgres_user,
39+
"POSTGRES_PASSWORD": postgres_password,
40+
"POSTGRES_DB": postgres_db,
41+
}
42+
43+
super().__init__(
44+
image_name=self.DOCKER_IMAGE,
45+
container_ports=[postgres_port],
46+
env_vars=env_vars,
47+
)
48+
49+
# Store configuration for connection info
50+
self.postgres_user = postgres_user
51+
self.postgres_password = postgres_password
52+
self.postgres_db = postgres_db
53+
self.postgres_port = postgres_port
54+
55+
def get_connection_info(self) -> dict:
56+
"""Return connection information for ParadeDB."""
57+
info = super().get_connection_info()
58+
info.update(
59+
{
60+
"database": self.postgres_db,
61+
"user": self.postgres_user,
62+
"password": self.postgres_password,
63+
"port": self.postgres_port,
64+
"connection_string": (
65+
f"postgresql://{self.postgres_user}:{self.postgres_password}"
66+
f"@{self.container_host}:{self.postgres_port}/{self.postgres_db}"
67+
),
68+
}
69+
)
70+
return info

paradedb/localstack_paradedb/utils/__init__.py

Whitespace-only changes.

0 commit comments

Comments
 (0)