perf(ci): add caching, path filtering, and nextest to speed up CI#723
Open
wpfleger96 wants to merge 6 commits into
Open
perf(ci): add caching, path filtering, and nextest to speed up CI#723wpfleger96 wants to merge 6 commits into
wpfleger96 wants to merge 6 commits into
Conversation
wpfleger96
force-pushed
the
wpfleger/ci-perf-optimization
branch
from
935bd05 to
854f9cb
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
wpfleger96
added a commit
that referenced
this pull request
May 22, 2026
Path filter gaps (missing desktop-rust, wrong Justfile case, missing rust-toolchain.toml/deny.toml/ci.yml triggers), hardcoded pnpm store path that missed the v11 directory, cache-poisoning from writing caches on PR events, Docker tar caching that's slower than direct pulls, and a diverged unit-test command between CI and Justfile. Split pnpm caches into restore/save with push-only writes, add save-if to new rust-cache instances, pin nextest to 0.9.136, and scope docker compose to the 3 services CI actually needs.
CI wall-clock time is bottlenecked by uncached Rust builds and jobs running unnecessarily on unrelated changes. Server cross-compile (~7-8 min × 2 targets) and Desktop Build macOS (~7-10 min) had zero cargo caching; all jobs ran on every PR regardless of what changed. - Add dorny/paths-filter gating so jobs only run when relevant files change (rust, desktop, web, mobile groups); push to main still runs everything - Add Swatinem/rust-cache to server-cross-compile and desktop-build-macos (the two most expensive uncached jobs) - Move docker compose up earlier in desktop-e2e-integration so containers boot during pnpm/Playwright install instead of after - Add Docker image caching for postgres/redis/typesense in E2E job - Add pnpm store cache to web, desktop, and desktop-e2e-integration - Switch unit tests to cargo-nextest for parallel test execution - Update Justfile test-unit to auto-detect nextest with script fallback
Path filter gaps (missing desktop-rust, wrong Justfile case, missing rust-toolchain.toml/deny.toml/ci.yml triggers), hardcoded pnpm store path that missed the v11 directory, cache-poisoning from writing caches on PR events, Docker tar caching that's slower than direct pulls, and a diverged unit-test command between CI and Justfile. Split pnpm caches into restore/save with push-only writes, add save-if to new rust-cache instances, pin nextest to 0.9.136, and scope docker compose to the 3 services CI actually needs.
Add save-if to the 4 pre-existing Swatinem/rust-cache steps (rust-lint, unit-tests, desktop, desktop-e2e-integration) so cache writes are restricted to push events across the board, not just the 2 new instances. Fix dorny/paths-filter version comment: the pinned SHA is v3.0.2, not the floating v3 tag.
…tags zizmor's cache-poisoning rule only recognizes lookup-only as a mitigation for Swatinem/rust-cache, not save-if. All 6 instances already have save-if set to prevent cache writes on PRs, but zizmor can't introspect this parameter. Add inline ignore comments as a stopgap until upstream fixes this (to be filed as a zizmor bug). Fix taiki-e/install-action version comments: the pinned SHA is v2.79.3, not the floating v2 tag.
The inline zizmor: ignore[cache-poisoning] comments appended to the version comment didn't suppress the findings — zizmor may require the directive as the first token after #. Switch to a .github/zizmor.yml config file which suppresses cache-poisoning for ci.yml at the file level, with a reference to the upstream bug (zizmorcore/zizmor#2051). Add minio and minio-init to the selective docker compose up command in desktop-e2e-integration — the relay now requires S3/MinIO for the git object-store conformance probe added in #726.
wpfleger96
force-pushed
the
wpfleger/ci-perf-optimization
branch
from
5884832 to
89e9dcd
Compare
The zizmor scanner runs as an org-level GHAS integration, not from a repo checkout — it never reads .github/zizmor.yml. Dismissed the cache-poisoning alerts (#10-13, #32, #33) as false positives via the code-scanning API instead, with references to the upstream bug (zizmorcore/zizmor#2051).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Several CI jobs were compiling Rust from scratch on every push with no caching, and all jobs ran regardless of what changed — a frontend-only PR would still wait for two full musl cross-compile runs.
server-cross-compile(~7-8 min x 2 targets) anddesktop-build-macos(~7-10 min) had zero cargo caching, making them the biggest uncached time sinks. Meanwhile, all 9 heavy jobs ran on every PR even when the relevant code hadn't changed.dorny/paths-filtergating so jobs only run when their source trees change; all jobs still run on push tomain/release. Filters includerust-toolchain.toml,deny.toml,.github/workflows/ci.yml, andpnpm-lock.yamlto avoid false negatives. Add missingdesktop-rusttrigger todesktop-e2e-integration.Swatinem/rust-cachetoserver-cross-compile(with per-target key) anddesktop-build-macos; addsave-if: ${{ github.event_name != 'pull_request' }}to all 6 rust-cache instances to prevent cache writes on PR events (PRs still restore from cache for build speed)docker compose up -dearlier indesktop-e2e-integrationso containers boot during pnpm/Playwright install; scope topostgres redis typesense minio minio-init(the services CI actually needs); remove Docker image tar caching (direct pulls from GitHub's proxy are faster than restoring 600MB+ tarballs)store/v3path (stale for pnpm 11 which usesstore/v11) to dynamicpnpm store path --silent, and split intoactions/cache/restore+actions/cache/savewith push-only writesunit-teststojust test-unit(which auto-detectscargo-nextestwith fallback); pin nextest to0.9.136dorny/paths-filter(v3->v3.0.2),taiki-e/install-action(v2->v2.79.3),Justfile->justfilecase.github/zizmor.ymlto suppresscache-poisoningfalse positives onSwatinem/rust-cache— zizmor only recognizeslookup-onlyas a mitigation, notsave-if(upstream bug filed)