Skip to content

Implement exec() in containers#6457

Open
gabivlj wants to merge 3 commits intomainfrom
gv/exec
Open

Implement exec() in containers#6457
gabivlj wants to merge 3 commits intomainfrom
gv/exec

Conversation

@gabivlj
Copy link
Copy Markdown
Contributor

@gabivlj gabivlj commented Mar 30, 2026

Makes this.ctx.container.exec() available under experimental.

The API is designed to be very extensible around ReadableStream/WritableStream, and familiar for the JS crowd.
I believe most of the bulk in container.c++ is defining the schemas.

Local dev:
It just calls the /exec APIs from Docker, but it turned out a not so easy task to keep it dead-simple. The reasoning is that Docker uses a multiplexed stream with streamIds that we have to parse. Not only that, this stream starts when we call /exec/<id>/start, which needs a Upgrade: tcp header (why not WebSockets...?). This makes us unable to use the KJ HttpClient, and hijack ourselves the connection by writing the headers and reading them with various KJ utils.

@gabivlj gabivlj requested review from a team as code owners March 30, 2026 16:01
Copy link
Copy Markdown
Contributor

@ask-bonk ask-bonk bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm Bonk, and I've done a quick review of your PR.

This PR adds exec() to the container API, enabling short-lived process execution inside DO-attached containers with stdin/stdout/stderr streaming, combined output mode, and a buffered output() helper.

Issues (by severity)

  1. [MEDIUM] demuxDockerExecOutput silently swallows DISCONNECTED exceptions without ending stdout/stderr streams, which can leave downstream ReadableStream consumers hanging forever.
  2. [MEDIUM] output() passes kj::maxValue to readAllBytes(), meaning a malicious or buggy container process can produce unlimited stdout/stderr and OOM the worker.
  3. [LOW] ExecProcess::kill() accepts signal 0, which has no kill effect on POSIX — but the RPC schema uses UInt32 while the JS API uses int, creating a mismatch where kill(0) is rejected on the JS side but would be valid on the capnp side. Consider harmonizing.
  4. [LOW] In demuxDockerExecOutput, stdout/stderr Maybe owners have already been moved into the catch block's scope when using try/catch with coroutines — the KJ_IF_SOME in the catch block accesses the same locals from the outer scope. This works because the catch is in the same scope, but is fragile.

This review was generated by an AI assistant and may contain inaccuracies.

@ask-bonk
Copy link
Copy Markdown
Contributor

ask-bonk bot commented Mar 30, 2026

Review posted successfully on PR #6457. Here's a summary of the findings:

5 inline comments posted as a batch review:

  1. [MEDIUM] demuxDockerExecOutput silently swallows DISCONNECTED exceptions without ending stdout/stderr streams — downstream consumers (like output()) will hang forever waiting for EOF. Suggestion provided to call detachEnd() before returning.

  2. [MEDIUM] output() calls readAllBytes(js, kj::maxValue) with no upper bound. A malicious container process writing infinite data could OOM the worker. Should be capped similar to how other APIs use getLimitEnforcer().getBufferingLimit().

  3. [LOW] Bug in error message at container.c++:51 — the env value validation failure message displays name instead of value. Suggestion provided.

  4. [LOW] kill() is fire-and-forget via addTask(sendIgnoringResult()) — errors are silently swallowed. Comment explaining the rationale would be helpful.

  5. [LOW] Note about JSG_LAZY_READONLY_INSTANCE_PROPERTY vs JSG_READONLY_PROTOTYPE_PROPERTY choice for ExecOutput — just flagging the semantics for awareness.

github run

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 30, 2026

Codecov Report

❌ Patch coverage is 0% with 682 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.58%. Comparing base (8007f84) to head (487ae18).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/workerd/server/container-client.c++ 0.00% 444 Missing ⚠️
src/workerd/api/container.c++ 0.00% 227 Missing ⚠️
src/workerd/api/container.h 0.00% 11 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6457      +/-   ##
==========================================
- Coverage   70.98%   70.58%   -0.40%     
==========================================
  Files         429      429              
  Lines      120725   121401     +676     
  Branches    19008    19104      +96     
==========================================
- Hits        85692    85689       -3     
- Misses      23740    24417     +677     
- Partials    11293    11295       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gabivlj gabivlj marked this pull request as draft March 30, 2026 17:24
@gabivlj gabivlj force-pushed the gv/exec branch 2 times, most recently from 1b71a48 to 97aaf22 Compare March 30, 2026 22:12
@gabivlj gabivlj marked this pull request as ready for review March 30, 2026 22:50
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 30, 2026

The generated output of @cloudflare/workers-types matches the snapshot in types/generated-snapshot 🎉

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Mar 31, 2026

Merging this PR will improve performance by 25.85%

⚡ 1 improved benchmark
✅ 69 untouched benchmarks
⏩ 129 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
bodyWithHeaders[Response] 38.9 µs 30.9 µs +25.85%

Comparing gv/exec (487ae18) with main (08aae7c)

Open in CodSpeed

Footnotes

  1. 129 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Copy link
Copy Markdown
Member

@gpanders gpanders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

container-client.c++ implementation looks good to me (with a couple small comments).

@kentonv
Copy link
Copy Markdown
Member

kentonv commented Mar 31, 2026

API LGTM, did not review implementation

…h stdio

We also have to rename to (stdout|stderr|stdin)Writer all capnp
variables and methods
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants