feat: migrate asyncio usage to anyio across the codebase#682
Draft
raballew wants to merge 21 commits into
Draft
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Replace asyncio.sleep with anyio.sleep, asyncio.create_subprocess_exec with anyio.run_process/open_process, asyncio.Event/Queue/Task with anyio equivalents, asyncio.run with anyio.run, and asyncio.wait_for with anyio.fail_after/move_on_after. Some asyncio imports remain where underlying libraries require them: - grpc.aio raises asyncio.InvalidStateError - pysnmp uses asyncio event loop and Event - asyncio.DatagramProtocol for TFTP UDP server - asyncio.get_running_loop for vsock transport Generated-By: Forge/20260513_094204_2078695_cb9ee2d7 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The profile parameter accepted None as a default value but was annotated as str instead of str | None. Generated-By: Forge/20260513_094204_2078695_cb9ee2d7
The _try_connect_and_extract_cert function had its own fail_after(timeout) scope using the same timeout value as the outer scope in _ssl_channel_credentials_insecure. This meant a slow first IP could consume the entire budget, preventing other IPs from being tried. The outer fail_after scope already enforces the total time limit. Generated-By: Forge/20260513_094204_2078695_cb9ee2d7
The Handler._ensure_task_group entered a task group via __aenter__ but never called __aexit__, bypassing structured concurrency and risking lost exceptions from spawned tasks. Added a done() lifecycle method that cancels all active scopes and properly exits the task group. Generated-By: Forge/20260513_094204_2078695_cb9ee2d7
raballew
force-pushed
the
488-asyncio-anyio
branch
from
77aa31a to
0938967
Compare
Fix import sorting (I001), line length (E501), unused imports (F401), undefined CancelledError (F821), and missing anyio.abc import for type checker. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
anyio.open_process() returns a Process object that does not have a communicate() method. Use anyio.run_process() which accepts input and handles stdin/stdout/stderr collection, preventing an AttributeError at runtime when applying Jumpstarter CRs. Generated-By: Forge/20260513_140607_2468630_62d9427a
anyio ByteReceiveStream does not have a readline() method. Wrap the stream with BufferedByteReceiveStream and use receive_until(b"\n") for line-delimited reading. Also adds type annotations to _read_pipe parameters and properly closes memory object streams using async with. Updates test mocks to use spec=ByteReceiveStream to catch API mismatches. Generated-By: Forge/20260513_140607_2468630_62d9427a
Replace all 180 remaining @pytest.mark.asyncio markers with @pytest.mark.anyio across 14 test files to complete the migration from asyncio to anyio test infrastructure. Generated-By: Forge/20260513_140607_2468630_62d9427a
Replace asyncio.subprocess.create_subprocess_exec with anyio.open_process and use BufferedByteReceiveStream for line reading. Update test mocks to match anyio ByteReceiveStream API and replace asyncio.run with anyio.run in test helpers. Generated-By: Forge/20260513_140607_2468630_62d9427a
When read_all=True, loop reading chunks until EndOfStream instead of reading only one chunk. This matches the previous asyncio behavior where process.stdout.read() read until EOF. Also narrows exception handling from bare except Exception to specific anyio exceptions (EndOfStream, ClosedResourceError). Generated-By: Forge/20260513_140607_2468630_62d9427a
Add try/except BaseExceptionGroup to done() so that exceptions from cancelled task group tasks are logged instead of propagating unhandled. Document the mitmproxy addon lifecycle constraint that prevents using a proper async with block for the task group. Generated-By: Forge/20260513_140607_2468630_62d9427a
Rename test_timeout_captures_output to test_timeout_discards_output to accurately reflect that anyio.run_process cancellation discards partial output on timeout, and document this as a deliberate trade-off. Generated-By: Forge/20260513_140607_2468630_62d9427a
sniffio is imported directly in jumpstarter.config.client but was only available as a transitive dependency of anyio. Declare it explicitly to prevent breakage if anyio ever changes its dependency tree. Generated-By: Forge/20260513_140607_2468630_62d9427a
…ation serial_asyncio is built on asyncio transports and protocols with no anyio equivalent. Document that the test helpers intentionally use asyncio.StreamReader/StreamWriter for this reason. Generated-By: Forge/20260513_140607_2468630_62d9427a
Verify that exactly one final result with a returncode is produced and that stdout data was actually received, ensuring the test exercises the complete path through the streaming subprocess. Generated-By: Forge/20260513_140607_2468630_62d9427a
There is no public Python API for extracting the full unverified certificate chain. Document this dependency on CPython internals and the incompatibility with alternative Python implementations. Generated-By: Forge/20260513_140607_2468630_62d9427a
When the stream ends without a trailing newline, receive_until raises IncompleteRead instead of EndOfStream. Catch IncompleteRead and flush any remaining buffered data in the QEMU driver, and treat it as an end-of-stream condition in the dut-network driver. Generated-By: Forge/20260513_140607_2468630_62d9427a
Raise RuntimeError after the task group exits cleanly instead of inside it, preventing anyio from wrapping it in a BaseExceptionGroup. Also fix test assertions that assumed positional args for anyio.open_process (which takes a list) and handle GeneratorExit in the early-exit test. Generated-By: Forge/20260513_140607_2468630_62d9427a
Extract _read_stream helper from _read_process_output to reduce cyclomatic complexity below the C901 threshold. Fix import ordering in QEMU driver to satisfy ruff I rules. Generated-By: Forge/20260513_140607_2468630_62d9427a
Replace bare `except Exception` with `except (anyio.EndOfStream, anyio.ClosedResourceError)` in the shell driver streaming loop to match the pattern used in the QEMU driver. The overly broad handler silently swallowed real errors like RuntimeError and OSError. Wrap the process lifecycle in try/finally to ensure the subprocess is killed and waited on if an unexpected exception escapes the loop, preventing zombie processes. Generated-By: Forge/20260513_140607_2468630_62d9427a Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add anyio_backend fixture to network driver conftest to prevent pytest-anyio from auto-discovering trio (which is not installed). Add ty: ignore for create_memory_object_stream subscript call that the ty type checker misidentifies as non-callable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
asyncioprimitives withanyioequivalentsasyncio.sleep,asyncio.create_subprocess_exec,asyncio.run,asyncio.create_task/gather,asyncio.wait_for,asyncio.Queue,asyncio.open_connection, andasyncio.get_running_loop().getaddrinfoto their anyio counterparts@pytest.mark.asyncioto@pytest.mark.anyioand adjust mock paths accordinglyasyncioimports for APIs with no anyio equivalent:InvalidStateError(gRPC),DatagramProtocol/DatagramTransport(TFTP UDP),Event/get_running_loop/new_event_loop(pysnmp),