Skip to content

Commit e9f099b

Browse files
BrandonLWhiteBrandonLWhite
authored
Projects by command (#34)
* Implement dynamic `commands` * Add projects-by-command * Remove hard-coded command fields * Update README. Lint. * Update example workflow * Process example projects in test workflow * Fix JSON parse * Install poetry and pdm * Install poe * Update snapshot * Fix package name * Fix poe run prefix * Add empty tests * Install poetry-plugin-bundle * Fix bundle command test * Update snapshot * Touch up README
1 parent 935f5e4 commit e9f099b

File tree

21 files changed

+582
-205
lines changed

21 files changed

+582
-205
lines changed

‎.github/workflows/examples/example.yml‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,22 +11,21 @@ jobs:
1111

1212
- name: Discover Projects
1313
id: discover
14-
uses: BrandonLWhite/find-python-projects-action@main
14+
uses: BrandonLWhite/find-python-projects-action@vX.Y.Z
1515

1616
- name: Print Paths
1717
run: echo ${{steps.discover.outputs.paths }}
1818

1919
outputs:
20-
testable-projects: ${{ steps.discover.outputs.testable-projects }}
21-
packageable-projects: ${{ steps.discover.outputs.packageable-projects }}
20+
projects-by-command: ${{ steps.discover.outputs.projects-by-command }}
2221

2322
build-python-projects:
2423
name: "Test: ${{ matrix.project.name }}"
2524
runs-on: ubuntu-latest
2625
needs: discover-python-projects
2726
strategy:
2827
matrix:
29-
project: ${{ fromJson(needs.discover-python-projects.outputs.testable-projects) }}
28+
project: ${{ fromJson(needs.discover-python-projects.outputs.projects-by-command.install) }}
3029

3130
defaults:
3231
run:
@@ -47,12 +46,12 @@ jobs:
4746
python-version-file: ${{ matrix.project.path }}
4847

4948
- name: Install project
50-
if: matrix.project.installCommand
51-
run: ${{ matrix.project.installCommand }}
49+
if: matrix.project.commands.install
50+
run: ${{ matrix.project.commands.install }}
5251

5352
- name: Test project
54-
run: ${{ matrix.project.testCommand }}
53+
run: ${{ matrix.project.commands.test }}
5554

5655
- name: Package project
57-
if: matrix.project.packageCommand
58-
run: ${{ matrix.project.packageCommand }}
56+
if: matrix.project.commands.package
57+
run: ${{ matrix.project.commands.package }}

‎.github/workflows/test.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,5 +21,50 @@ jobs:
2121
steps:
2222
- uses: actions/checkout@v4
2323
- uses: ./
24+
id: discover
2425
with:
2526
root-dir: ./test-fixtures/multi-project
27+
outputs:
28+
projects-by-command: ${{ steps.discover.outputs.projects-by-command }}
29+
30+
build-python-projects:
31+
name: "Test: ${{ matrix.project.name }}"
32+
runs-on: ubuntu-latest
33+
needs: test
34+
strategy:
35+
matrix:
36+
project: ${{ fromJson(needs.test.outputs.projects-by-command).install }}
37+
38+
defaults:
39+
run:
40+
shell: bash
41+
working-directory: ${{ matrix.project.directory }}
42+
43+
steps:
44+
- name: Check out code
45+
uses: actions/checkout@v4
46+
47+
- name: Dump project object
48+
run: |
49+
echo '${{ toJson(matrix.project) }}'
50+
51+
- name: Setup Python
52+
uses: actions/setup-python@v5
53+
with:
54+
python-version-file: ${{ matrix.project.path }}
55+
56+
- run: pipx install poetry==1.8.2
57+
- run: pipx inject poetry poetry-plugin-bundle==1.3.0
58+
- run: pipx install pdm==2.12.3
59+
- run: pipx install poethepoet==0.25.0
60+
61+
- name: Install project
62+
if: matrix.project.commands.install
63+
run: ${{ matrix.project.commands.install }}
64+
65+
- name: Test project
66+
run: ${{ matrix.project.commands.test }}
67+
68+
- name: Package project
69+
if: matrix.project.commands.package
70+
run: ${{ matrix.project.commands.package }}

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,3 +65,8 @@ typings/
6565

6666
# next.js build output
6767
.next
68+
69+
__pycache__/
70+
*.py[cod]
71+
*$py.class
72+
.venv

‎README.md‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@ Python projects contained in this directory or lower will be discovered. Defaul
1818

1919
- **projects**: JSON array of all found projects ([project object](#project-object-output-shape))
2020

21-
- **testable-projects**: JSON array of all found projects ([project object](#project-object-output-shape)) that implement a `test` command in `pyproject.toml` (See [Project Commands](#project-commands))
22-
23-
- **packageable-projects**: JSON array of all found projects ([project object](#project-object-output-shape)) that implement a `package` command in `pyproject.toml` (See [Project Commands](#project-commands))
21+
- **projects-by-command**: JSON object with keys corresponding to the name of discovered commands
22+
(eg `install`, `test`, `package`, etc.) in all projects, with an array value containing each
23+
[project object](#project-object-output-shape) that implements the command.
2424

2525

2626
## Project object output shape
@@ -36,21 +36,19 @@ These are the fields for `project` objects in the output:
3636

3737
- **pythonVersion**: Value of `[project.requires-python]` or `[tool.poetry.dependencies.python]` from `pyproject.toml`
3838

39-
- **installCommand**: The shell command to run to create and install dependencies into the virtual environment.
40-
This can either be specified explicitly in `[tool.tasks.install]` or deduced from the `[build-system.build-backend]`
41-
42-
- **testCommand**: The shell command to run to execute the tests for the project.
43-
44-
- **packageCommand**: The shell command to run to execute packaging operations for the project.
39+
- **commands**: Object with keys for each discovered command and the shell command as the value.
40+
This is dynamically constructed from the `pyproject.toml` content.
4541

4642

4743
## Project Commands
4844
In the absence of Python standards for expressing internal project CI/CD/Dev operations, this action tries to unify the various known ways in the wild.
4945

50-
This action recognizes the following typical CI related shell commands:
46+
This action will surface any command you specify in the pyproject.toml files. Typically you'll
47+
want to define one or more of the following for your CI/CD system::
5148

5249
- `test`
5350
- `package`
51+
- `publish`
5452

5553
In order to make these commands available in the action output, you'll need to define them in `pyproject.toml` using a section appropriate for the particular tools you are using in the project. You can specify all, some, or none, depending on what you need available.
5654

@@ -59,6 +57,9 @@ This action will pull the command from the first entry it finds in any of the fo
5957
- `[tool.pdm.scripts]`: [PDM](https://pdm-project.org/latest/usage/scripts/)
6058
- `[tool.poe.tasks]`: [poethepoet](https://github.com/nat-n/poethepoet)
6159

60+
### The `install` command
61+
This action perfoms special treatment for surfacing the `install` command. Namely, if it is not explicitly specified, the action will attempt to generate a default based on the packaging backend (eg Poetry, PDM) that it discovers.
62+
The intent is that you do not need to specify an `install` command in your pyproject.toml, but you can if necessary.
6263

6364
### Where is the support for `[tool.poetry.scripts]`?
6465
Unfortunately, [Poetry scripts](https://python-poetry.org/docs/pyproject/#scripts) is for specifying commands that are made available to consumers of a package. It isn't meant for CI/CD or developer operations and doesn't meet those needs, primarily because any scripts you define in this section will be added as executable shortcuts to your virtual environment or any virtual environment your package is installed into.

‎__snapshots__/find-python-projects.test.js.snap‎

Lines changed: 137 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,6 @@
22

33
exports[`find-python-projects finds projects 1`] = `
44
{
5-
"packageable-projects": [
6-
{
7-
"buildBackend": "poetry.core.masonry.api",
8-
"directory": "test-fixtures/multi-project/project-2",
9-
"installCommand": "poetry install",
10-
"name": "sub-project-2",
11-
"packageCommand": "poetry bundle venv",
12-
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
13-
"pythonVersion": "^3.9",
14-
"testCommand": "poetry run pytest",
15-
},
16-
],
175
"paths": [
186
"test-fixtures/multi-project/project-1/pyproject.toml",
197
"test-fixtures/multi-project/project-2/pyproject.toml",
@@ -24,85 +12,174 @@ exports[`find-python-projects finds projects 1`] = `
2412
"projects": [
2513
{
2614
"buildBackend": "poetry.core.masonry.api",
15+
"commands": {
16+
"install": "poetry install",
17+
},
2718
"directory": "test-fixtures/multi-project/project-1",
28-
"installCommand": "poetry install",
2919
"name": "sub-project-1",
30-
"packageCommand": null,
3120
"path": "test-fixtures/multi-project/project-1/pyproject.toml",
3221
"pythonVersion": "^3.9",
33-
"testCommand": null,
3422
},
3523
{
3624
"buildBackend": "poetry.core.masonry.api",
25+
"commands": {
26+
"install": "poetry install",
27+
"package": "poetry bundle venv .package/",
28+
"something-arbitrary": "echo 'Example of arbitrary shell command'",
29+
"test": "poetry run pytest",
30+
},
3731
"directory": "test-fixtures/multi-project/project-2",
38-
"installCommand": "poetry install",
3932
"name": "sub-project-2",
40-
"packageCommand": "poetry bundle venv",
4133
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
4234
"pythonVersion": "^3.9",
43-
"testCommand": "poetry run pytest",
4435
},
4536
{
37+
"commands": {
38+
"install": null,
39+
},
4640
"directory": "test-fixtures/multi-project/project-4",
47-
"installCommand": null,
4841
"name": "sub-project-4",
49-
"packageCommand": null,
5042
"path": "test-fixtures/multi-project/project-4/pyproject.toml",
5143
"pythonVersion": "^3.9",
52-
"testCommand": null,
5344
},
5445
{
5546
"buildBackend": "pdm.backend",
47+
"commands": {
48+
"install": "pdm install",
49+
"test": "pdm run test",
50+
},
5651
"directory": "test-fixtures/multi-project/project-5",
57-
"installCommand": "pdm install",
5852
"name": "project-5",
59-
"packageCommand": null,
6053
"path": "test-fixtures/multi-project/project-5/pyproject.toml",
6154
"pythonVersion": "==3.12.*",
62-
"testCommand": "pdm run test",
6355
},
6456
{
6557
"buildBackend": "poetry.core.masonry.api",
58+
"commands": {
59+
"install": "poetry install",
60+
"test": "poe test",
61+
},
6662
"directory": "test-fixtures/multi-project/project-6",
67-
"installCommand": "poetry install",
68-
"name": "sub-project-1",
69-
"packageCommand": null,
70-
"path": "test-fixtures/multi-project/project-6/pyproject.toml",
71-
"pythonVersion": "^3.9",
72-
"testCommand": "poe run test",
73-
},
74-
],
75-
"testable-projects": [
76-
{
77-
"buildBackend": "poetry.core.masonry.api",
78-
"directory": "test-fixtures/multi-project/project-2",
79-
"installCommand": "poetry install",
80-
"name": "sub-project-2",
81-
"packageCommand": "poetry bundle venv",
82-
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
83-
"pythonVersion": "^3.9",
84-
"testCommand": "poetry run pytest",
85-
},
86-
{
87-
"buildBackend": "pdm.backend",
88-
"directory": "test-fixtures/multi-project/project-5",
89-
"installCommand": "pdm install",
90-
"name": "project-5",
91-
"packageCommand": null,
92-
"path": "test-fixtures/multi-project/project-5/pyproject.toml",
93-
"pythonVersion": "==3.12.*",
94-
"testCommand": "pdm run test",
95-
},
96-
{
97-
"buildBackend": "poetry.core.masonry.api",
98-
"directory": "test-fixtures/multi-project/project-6",
99-
"installCommand": "poetry install",
100-
"name": "sub-project-1",
101-
"packageCommand": null,
63+
"name": "sub-project-6",
10264
"path": "test-fixtures/multi-project/project-6/pyproject.toml",
10365
"pythonVersion": "^3.9",
104-
"testCommand": "poe run test",
10566
},
10667
],
68+
"projects-by-command": {
69+
"install": [
70+
{
71+
"buildBackend": "poetry.core.masonry.api",
72+
"commands": {
73+
"install": "poetry install",
74+
},
75+
"directory": "test-fixtures/multi-project/project-1",
76+
"name": "sub-project-1",
77+
"path": "test-fixtures/multi-project/project-1/pyproject.toml",
78+
"pythonVersion": "^3.9",
79+
},
80+
{
81+
"buildBackend": "poetry.core.masonry.api",
82+
"commands": {
83+
"install": "poetry install",
84+
"package": "poetry bundle venv .package/",
85+
"something-arbitrary": "echo 'Example of arbitrary shell command'",
86+
"test": "poetry run pytest",
87+
},
88+
"directory": "test-fixtures/multi-project/project-2",
89+
"name": "sub-project-2",
90+
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
91+
"pythonVersion": "^3.9",
92+
},
93+
{
94+
"buildBackend": "pdm.backend",
95+
"commands": {
96+
"install": "pdm install",
97+
"test": "pdm run test",
98+
},
99+
"directory": "test-fixtures/multi-project/project-5",
100+
"name": "project-5",
101+
"path": "test-fixtures/multi-project/project-5/pyproject.toml",
102+
"pythonVersion": "==3.12.*",
103+
},
104+
{
105+
"buildBackend": "poetry.core.masonry.api",
106+
"commands": {
107+
"install": "poetry install",
108+
"test": "poe test",
109+
},
110+
"directory": "test-fixtures/multi-project/project-6",
111+
"name": "sub-project-6",
112+
"path": "test-fixtures/multi-project/project-6/pyproject.toml",
113+
"pythonVersion": "^3.9",
114+
},
115+
],
116+
"package": [
117+
{
118+
"buildBackend": "poetry.core.masonry.api",
119+
"commands": {
120+
"install": "poetry install",
121+
"package": "poetry bundle venv .package/",
122+
"something-arbitrary": "echo 'Example of arbitrary shell command'",
123+
"test": "poetry run pytest",
124+
},
125+
"directory": "test-fixtures/multi-project/project-2",
126+
"name": "sub-project-2",
127+
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
128+
"pythonVersion": "^3.9",
129+
},
130+
],
131+
"something-arbitrary": [
132+
{
133+
"buildBackend": "poetry.core.masonry.api",
134+
"commands": {
135+
"install": "poetry install",
136+
"package": "poetry bundle venv .package/",
137+
"something-arbitrary": "echo 'Example of arbitrary shell command'",
138+
"test": "poetry run pytest",
139+
},
140+
"directory": "test-fixtures/multi-project/project-2",
141+
"name": "sub-project-2",
142+
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
143+
"pythonVersion": "^3.9",
144+
},
145+
],
146+
"test": [
147+
{
148+
"buildBackend": "poetry.core.masonry.api",
149+
"commands": {
150+
"install": "poetry install",
151+
"package": "poetry bundle venv .package/",
152+
"something-arbitrary": "echo 'Example of arbitrary shell command'",
153+
"test": "poetry run pytest",
154+
},
155+
"directory": "test-fixtures/multi-project/project-2",
156+
"name": "sub-project-2",
157+
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
158+
"pythonVersion": "^3.9",
159+
},
160+
{
161+
"buildBackend": "pdm.backend",
162+
"commands": {
163+
"install": "pdm install",
164+
"test": "pdm run test",
165+
},
166+
"directory": "test-fixtures/multi-project/project-5",
167+
"name": "project-5",
168+
"path": "test-fixtures/multi-project/project-5/pyproject.toml",
169+
"pythonVersion": "==3.12.*",
170+
},
171+
{
172+
"buildBackend": "poetry.core.masonry.api",
173+
"commands": {
174+
"install": "poetry install",
175+
"test": "poe test",
176+
},
177+
"directory": "test-fixtures/multi-project/project-6",
178+
"name": "sub-project-6",
179+
"path": "test-fixtures/multi-project/project-6/pyproject.toml",
180+
"pythonVersion": "^3.9",
181+
},
182+
],
183+
},
107184
}
108185
`;

0 commit comments

Comments
 (0)