Skip to content

Add serial test execution support#125

Open
giordano wants to merge 16 commits intomainfrom
mg/serial-test
Open

Add serial test execution support#125
giordano wants to merge 16 commits intomainfrom
mg/serial-test

Conversation

@giordano
Copy link
Copy Markdown
Collaborator

@giordano giordano commented Apr 2, 2026

This is my (first?) attempt to fix #77 (CC @christiangnrd).

I'm not married to this implementation, I was originally thinking of changing the values in the testsuite dictionary to be a struct wrapping Expr, and with a field to designate a serial test, but then I had a friendly chat with Claude and it suggested as a less breaking alternative adding a keyword argument to runtests. I let Claude write the first implementation, I didn't like it and I changed it. I'm happy to consider other options.

See section about serial execution in the docs preview.

PR best reviewed by ignoring whitespace changes.

giordano and others added 4 commits April 2, 2026 19:52
Allow designating specific tests to run sequentially instead of in
parallel, useful for memory-hungry tests that cannot safely overlap.

New `serial` and `serial_position` keyword arguments on `runtests`
control which tests run one-at-a-time and whether the serial phase
runs before (default) or after the parallel batch. A new exported
`partition_tests` helper splits the ordered test list into serial
and parallel groups.

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
@giordano giordano added the enhancement New feature or request label Apr 2, 2026
@christiangnrd
Copy link
Copy Markdown
Contributor

I like this interface. The only reason I could see for a more complicated interface is for designating testsets to be run on the same worker (mapreduce and mapreduce_large as a concrete example) to save on compilation but I don't think that's really worth the complexity.

@giordano
Copy link
Copy Markdown
Collaborator Author

giordano commented Apr 3, 2026

The only reason I could see for a more complicated interface is for designating testsets to be run on the same worker (mapreduce and mapreduce_large as a concrete example) to save on compilation

I pushed a change to always run the serial tests on a single worker because that was something I had in mind anyway: it's a bit silly and wasteful to spawn one worker per test, and when there are multiple serial tests the other workers remain sitting lazy after they're started.

I also added a few more tests, to try and cover all corner cases I could think of (and I'm sure there are more!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow designating testsets to be run serially

2 participants