ci(test): shard Windows e2e_snapshots into 5 parallel jobs#398
Merged
Conversation
Windows is the long pole of CI: PTY steps timeout at 60s vs 20s on Unix, and `vite_task_bin` e2e fixtures expand to 156 trials that run serially inside one job. Linux/macOS finish well before Windows, so each PR waits on Windows alone. The e2e harness now reads VT_SHARD_INDEX and VT_SHARD_TOTAL and applies round-robin partitioning at the Trial level (not the fixture level) so that cases inside a heavy fixture scatter across shards. A separate VT_SKIP_E2E knob lets a sibling Windows job run the full workspace `cargo test` while the e2e_snapshots binary self-skips. CI matrix gains six Windows entries: five e2e shards (~31 trials each) plus one non-e2e job that covers plan_snapshots, fspy tests, and unit tests. Linux/macOS/musl are unchanged. All entries run the same three `cargo test` commands, parameterized by per-shard `scope` and `run_env` matrix fields.
Round-robin (i % total) scattered consecutive trials across shards, so a fixture's cases would land in N different shards. Contiguous chunks (i * total / count) keep neighbours together: a fixture's cases stay in one shard unless the fixture straddles a shard boundary, and even then only at the boundary. Same 32/31/31/31/31 distribution for 156 trials, same disjoint+exhaustive guarantees, but locality is preserved.
The non-e2e Windows shard previously set VT_SKIP_E2E=1 so the e2e harness could self-skip, but vite_task_bin has no lib tests, so excluding the whole crate via cargo's --exclude flag is equivalent and lets us skip compiling the e2e binary on that shard too.
Reverts the previous switch to --exclude vite_task_bin. That exclusion happened to be equivalent today (vite_task_bin has no lib/bin/doc tests), but it isn't verifiable from the scope alone: adding a lib test to vite_task_bin later would silently drop it from CI. The non-e2e Windows shard now scopes to the full workspace and the e2e harness self-skips when VT_SKIP_E2E is set. Coverage is now legible from the scopes: e2e_snapshots is sharded across 5 jobs, --workspace catches everything else (current and future).
The non-e2e Windows scope is now `--workspace --exclude vite_task_bin` and the e2e scope stays `-p vite_task_bin --test e2e_snapshots`. Together they cover the whole workspace, looking only at the scope fields. The risk that --exclude vite_task_bin silently drops future tests is closed by tightening vite_task_bin's Cargo.toml: lib/doc were already test=false, and the two [[bin]] entries now set test=false too. A comment on [[test]] flags that adding another test target there would escape CI, so any future contributor has to either update the workflow or add the test to a different crate.
The two scopes are now `-p vite_task_bin` (e2e shards) and `--workspace --exclude vite_task_bin` (non-e2e). Their union is the workspace by definition, so coverage of every test target follows from the scope fields alone without inspecting Cargo.toml or the harness. Reverts the [[bin]] test=false / [[test]] caveat comments added to lock down vite_task_bin's test surface — they're no longer load-bearing since any future test in the crate is automatically picked up by the e2e shards. Such a test would run on all 5 shards until it opts into VT_SHARD_INDEX/VT_SHARD_TOTAL, but it's covered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Half the CI time by shading Windows e2e_snapshots
Windows is the long pole of CI: per-step PTY timeout is 60s on Windows vs 20s on Unix, and
vite_task_bine2e fixtures expand to 156 trials that run serially inside a single job. Linux/macOS finish well before Windows on every PR.This PR shards only the Windows
e2e_snapshotstest binary 5 ways while leaving every other platform (Linux GNU, macOS arm64, macOS x64, musl) untouched.Approach
crates/vite_task_bin/tests/e2e_snapshots/main.rsreads two new env vars:VT_SHARD_INDEX(1..=total) +VT_SHARD_TOTAL→ round-robin partition the generatedVec<Trial>at the case level (not the fixture level), so heavy fixtures' cases split across shards.VT_SKIP_E2E=1→ early-return frommainso the sibling non-e2e Windows job can run the full workspacecargo testwhile this binary self-skips.CI matrix
Was 4 entries → now 9: 3 non-Windows (unchanged) + 6 Windows (5 e2e shards + 1 non-e2e). All entries run the same three
cargo testcommands, parameterized by per-shardscopeandrun_envmatrix fields — single source of truth, no per-shard branching in the steps.e2e-N-p vite_task_bin --test e2e_snapshotsVT_SHARD_INDEX=N VT_SHARD_TOTAL=5non-e2eVT_SKIP_E2E=1Test plan
testmatrix entries successfully on this PRwindows-non-e2elog runs plan_snapshots + fspy + unit tests, with e2e_snapshots reporting no workWill monitor CI timings after first run and adjust shard count or strategy if balance is poor.
Generated by Claude Code