🐛 fix(subprocess): add timeout to interpreter probing#42
Merged
gaborbernat merged 2 commits intotox-dev:mainfrom Mar 6, 2026
Merged
🐛 fix(subprocess): add timeout to interpreter probing#42gaborbernat merged 2 commits intotox-dev:mainfrom
gaborbernat merged 2 commits intotox-dev:mainfrom
Conversation
On Windows CI, the subprocess spawned to probe a candidate Python interpreter can hang indefinitely — triggered by Windows Store stubs, antivirus holds, or pipe I/O race conditions. This caused ~18 flaky timeout failures across 9 different tests in tox over the last 30 days, almost exclusively on windows-2025 runners. The root cause is process.communicate() being called with no timeout in _run_subprocess. Adding a 5s timeout and killing the process on expiry allows discovery to skip unresponsive interpreters and continue.
rahuldevikar
approved these changes
Mar 6, 2026
d91ec9d to
8c22da5
Compare
ty on 3.8 reports too-many-positional-arguments for pytest.skip().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_run_subprocesscallsprocess.communicate()with no timeout when probing candidate Python interpreters. On Windows CI runners, this subprocess can hang indefinitely — typically caused by Windows Store Python stubs, antivirus file locks, or pipe I/O race conditions between short-lived child processes and the parent's reader threads. 🪟 Analysis of the last 30 days of tox CI revealed 18 flaky timeout failures across 9 different tests, almost exclusively onwindows-2025runners (with 2 onmacos-15).The failures fall into two patterns that share the same deadlock chain in tox's
common.py. Pattern A (discovery): thetox-driverthread hangs in_run_subprocess→process.communicate()while probing interpreters. Pattern B (install/provision): a package install or provision subprocess hangs. In both cases,thread.join()blocks the main thread soKeyboardInterruptcan never be delivered,as_completed()blocks the interrupt thread so it can't check the interrupt event, andexecutor.shutdown(wait=True)prevents cleanup — creating an unbreakable deadlock.The fix adds a 5-second timeout to
process.communicate(). ⏱️ The probed subprocess only readssysattributes and prints JSON — a healthy interpreter completes this in milliseconds, so 5 seconds is generous. On timeout the hung process is killed and treated as a failed probe, allowing discovery to skip it and continue to the next candidate. This fits naturally into the existing error-handling flow since non-zero exit codes already produce aRuntimeErrorthat callers handle gracefully. A companion PR in tox addresses the deadlock chain incommon.pyto handle Pattern B and make the execution engine interruptible.Also fixes a pre-existing
tytype-check failure wherety 0.0.17with--python-version 3.8completely misresolvespytest.skip(a_WithException-wrapped function) as a bound method with no parameters — replaced withraise pytest.skip.Exception(msg)which bypasses the broken wrapper.