Skip to content

chore: move dev extras to dependency-groups #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tokoko merged 5 commits into from
Jan 5, 2026
+670 −338
Merged

chore: move dev extras to dependency-groups #135

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d45ebc1
chore: move dev extras to dependency-groups
tokoko Dec 24, 2025
04c22c7
fix: remove test extra from ci
tokoko Dec 24, 2025
bc547cd
fix: remove test extra from codegen-check
tokoko Dec 24, 2025
5d1d6ee
fix: remove extras from devcontainer
tokoko Dec 24, 2025
0da849c
chore: create gen_extensions group
tokoko Dec 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "uv venv --clear && uv sync --extra test --extra gen_proto",
"postCreateCommand": "uv venv --clear && uv sync --group gen_proto",

// Configure tool-specific properties.
"customizations": {
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/codegen-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ jobs:
runCmd: |
# fetch submodule tags since actions/checkout does not
git submodule foreach 'git fetch --unshallow || true'
# Ensure dependencies are installed
uv sync --extra test --extra gen_proto
# Run all code generation steps
make codegen
Expand Down
5 changes: 1 addition & 4 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,6 @@ jobs:
uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python }}
- name: Install package and test dependencies
run: |
uv sync --frozen --extra test
- name: Run tests
run: |
uv run pytest
uv run --frozen pytest
4 changes: 1 addition & 3 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ make codegen
Run the upgrade script to upgrade the submodule and regenerate the protobuf stubs.

```
uv sync --extra gen_proto
uv run ./update_proto.sh <version>
uv run --group gen_proto ./update_proto.sh <version>
```

Or run the proto codegen without updating the Substrait Git submodule:
Expand Down Expand Up @@ -71,6 +70,5 @@ make format
# Test
Run tests in the project's root dir.
```
uv sync --extra test
uv run pytest
```
4 changes: 2 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ codegen-version:
&& echo '"' >> src/substrait/gen/version.py

codegen-proto:
./gen_proto.sh
uv run --group gen_proto ./gen_proto.sh

codegen-extensions:
uv run --with datamodel-code-generator datamodel-codegen \
uv run --group gen_extensions datamodel-codegen \
--input-file-type jsonschema \
--input third_party/substrait/text/simple_extensions_schema.yaml \
--output src/substrait/gen/json/simple_extensions.py \
Expand Down
9 changes: 6 additions & 3 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description = "A python package for Substrait."
authors = [{name = "Substrait contributors", email = "substrait@googlegroups.com"}]
license = {text = "Apache-2.0"}
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.10,<3.14"
dependencies = ["protobuf >=3.19.1,<6"]
dynamic = ["version"]

Expand All @@ -13,9 +13,12 @@ write_to = "src/substrait/_version.py"

[project.optional-dependencies]
extensions = ["antlr4-python3-runtime", "pyyaml"]
Copy link
Member

@nielspardon nielspardon Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering whether we still need this set of optional dependencies given that they are also part of the dev dependency group. Further, shouldn't those be default dependencies since we can assume most people with non-trivial use cases will want to use the extensions in some form?

Copy link
Contributor Author

@tokoko tokoko Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the idea was that it's likely only necessary for producers, not consumers, see here. I don't have strong opinions. but starting from extra felt safer.

gen_proto = ["protobuf >=3.19.1,<6", "protoletariat >= 3.0.0"]
sql = ["sqloxide", "deepdiff"]
test = ["pytest >= 7.0.0", "antlr4-python3-runtime", "pyyaml", "sqloxide", "deepdiff", "duckdb<=1.2.2", "datafusion"]

[dependency-groups]
dev = ["pytest >= 7.0.0", "antlr4-python3-runtime", "pyyaml", "sqloxide", "deepdiff", "duckdb<=1.2.2", "datafusion"]
gen_proto = ["protobuf >=3.19.1,<6", "protoletariat >= 3.0.0"]
gen_extensions = ["datamodel-code-generator"]

[tool.pytest.ini_options]
pythonpath = "src"
Expand Down
146 changes: 73 additions & 73 deletions src/substrait/gen/json/simple_extensions.py
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading