test: add package installation integration tests by Mossaka · Pull Request #1052 · github/gh-aw-firewall

Mossaka · 2026-02-25T20:12:45Z

Summary

Adds a "Package Installation" describe block to chroot-package-managers.test.ts that verifies actual package installation through the AWF proxy
Tests pip install (requests), npm install (chalk@4), and cargo build (cfg-if dependency)
Each test verifies the installed package works (import/require/build succeeds)

Test plan

CI integration tests pass for the new "Package Installation" describe block
pip install test installs requests and verifies import requests works
npm install test installs chalk@4 and verifies require('chalk') works
cargo build test builds a project with cfg-if dependency

🤖 Generated with Claude Code

…xy image The Dockerfile only copied server.js but server.js now requires logging.js, metrics.js, and rate-limiter.js. Without these files the container exits immediately on startup, causing all agentic workflow CI jobs to fail. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add a "Package Installation" describe block to chroot-package-managers tests that verifies actual package installation through the AWF proxy: - pip install requests with pypi.org + files.pythonhosted.org allowed - npm install chalk@4 with registry.npmjs.org allowed - cargo build with a dependency (cfg-if) with crates.io domains allowed Each test verifies the package was actually installed (import/require/build). Closes #1044 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-02-25T20:13:35Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	82.39%	82.54%	📈 +0.15%
Statements	82.32%	82.46%	📈 +0.14%
Functions	82.74%	82.74%	➡️ +0.00%
Branches	74.55%	74.65%	📈 +0.10%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	83.6% → 84.1% (+0.56%)	82.8% → 83.4% (+0.54%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Adds integration coverage for real package installation/build workflows in chroot mode (via AWF’s domain allowlisting), intended to validate the most common “download + install” paths rather than only registry queries.

Changes:

Add a new “Package Installation” describe block to tests/integration/chroot-package-managers.test.ts covering pip, npm, and cargo installs/builds.
Update containers/api-proxy/Dockerfile to copy additional JS modules alongside server.js.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
tests/integration/chroot-package-managers.test.ts	Adds new integration tests that perform actual installs/builds through the proxy and verify the results.
containers/api-proxy/Dockerfile	Modifies the image build to copy additional JS files into the api-proxy image.

Comments suppressed due to low confidence (2)

tests/integration/chroot-package-managers.test.ts:394

Temporary directory cleanup only happens if every prior command succeeds (... && rm -rf $TESTDIR). If cargo init/cargo build fails, the directory is leaked. Use an EXIT capture / trap pattern (as done elsewhere in this file) so cleanup happens on both success and failure.

        'TESTDIR=$(mktemp -d) && cd $TESTDIR && ' +
        'cargo init --name awftest 2>&1 && ' +
        // Add a small dependency (cfg-if is tiny with no transitive deps)
        'echo \'cfg-if = "1"\' >> Cargo.toml && ' +
        'cargo build 2>&1 && echo "cargo_build_ok" && ' +
        'rm -rf $TESTDIR',

tests/integration/chroot-package-managers.test.ts:358

This test writes into a fixed path (/tmp/pip-test) and never cleans it up. That can cause cross-run contamination (already-installed packages) and disk bloat on CI runners. Use a per-test mktemp directory (and remove it afterward) instead of a shared /tmp path, and consider disabling pip cache if you want to ensure a real download occurs.

      const result = await runner.runWithSudo(
        'pip3 install --target /tmp/pip-test requests 2>&1 && ' +
        'PYTHONPATH=/tmp/pip-test python3 -c "import requests; print(requests.__version__)"',

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T20:16:03Z

tests/integration/chroot-package-managers.test.ts

+      const result = await runner.runWithSudo(
+        'cd /tmp && mkdir -p npm-test && cd npm-test && npm init -y 2>&1 && ' +
+        'npm install chalk@4 2>&1 && ' +


This test creates /tmp/npm-test and leaves it behind. That can make later runs faster/flakier (install may not hit the network) and can grow disk usage over time. Prefer a unique mktemp directory (and ensure it’s removed even if npm/node fails).

This issue also appears in the following locations of the same file:

line 389

line 356

Copilot · 2026-02-25T20:16:04Z

tests/integration/chroot-package-managers.test.ts

+      const result = await runner.runWithSudo(
+        'cd /tmp && mkdir -p npm-test && cd npm-test && npm init -y 2>&1 && ' +
+        'npm install chalk@4 2>&1 && ' +
+        'node -e "require(\'/tmp/npm-test/node_modules/chalk\')" && echo "npm_install_ok"',


PR description/issue acceptance criteria mention verifying require('chalk') works, but this uses an absolute path require into /tmp/npm-test/node_modules. Running the require via Node’s module resolution from within the project dir (or via NODE_PATH) would better match the stated behavior being tested.

Copilot · 2026-02-25T20:16:04Z

tests/integration/chroot-package-managers.test.ts

+        {
+          allowDomains: ['crates.io', 'static.crates.io', 'index.crates.io'],
+          logLevel: 'debug',


This relies on Cargo using the sparse index (index.crates.io). On environments where Cargo still uses the git index, it will attempt to reach GitHub and this test will fail under the current allowDomains. To make the test deterministic, explicitly force sparse protocol (e.g., via CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse) or expand the domain allowlist to cover the git index host your Cargo version uses.

Copilot · 2026-02-25T20:16:05Z

containers/api-proxy/Dockerfile


 # Copy application files
-COPY server.js ./
+COPY server.js logging.js metrics.js rate-limiter.js ./


Docker build will fail: this COPY lists logging.js/metrics.js/rate-limiter.js but those files don’t exist in containers/api-proxy/ (only server.js, package*.json, README). Either add these files to the image context or revert the COPY to only include server.js (or copy the whole directory).

Suggested change

COPY server.js logging.js metrics.js rate-limiter.js ./

COPY server.js ./

Copilot · 2026-02-25T20:16:05Z

tests/integration/chroot-package-managers.test.ts

+
+      expect(result).toSucceed();
+      // The last line of output should be the version string
+      expect(result.stdout).toMatch(/\d+\.\d+\.\d+/);


The comment says the last line should be the version string, but the assertion only checks that somewhere in stdout there is an x.y.z pattern (which could match pip’s own output). Consider asserting against the last non-empty line (or anchoring the regex to the end) so the check matches the stated intent.

Suggested change

expect(result.stdout).toMatch(/\d+\.\d+\.\d+/);

const lines = result.stdout.split('\n').map(line => line.trim()).filter(line => line.length > 0);

const lastLine = lines[lines.length - 1] || '';

expect(lastLine).toMatch(/^\d+\.\d+\.\d+$/);

- Use mktemp dirs instead of fixed /tmp paths (avoid cross-run contamination) - Add --no-cache-dir to pip install (ensure real download) - Assert pip version on last line specifically (not just anywhere in stdout) - Use NODE_PATH for npm require instead of absolute path - Force CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse for deterministic behavior - Ensure temp dir cleanup via trap pattern (EC=$?; rm -rf; exit $EC) - Revert unrelated api-proxy Dockerfile change Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Mossaka · 2026-02-25T20:27:18Z

Addressed all review feedback in 1c0791d:

Use mktemp for all temp dirs — pip, npm, and cargo tests now use mktemp -d instead of fixed /tmp paths, avoiding cross-run contamination
Precise pip version assertion — Now checks the last non-empty line matches ^\d+\.\d+\.\d+$ instead of just searching anywhere in stdout
NODE_PATH for npm require — Uses NODE_PATH=$NPMDIR/node_modules for proper module resolution instead of absolute require path
Force sparse protocol for cargo — Sets CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse for deterministic behavior across Cargo versions
Cleanup on failure — Cargo test uses EC=$?; rm -rf $TESTDIR; exit $EC pattern so temp dir is cleaned up even on failure; pip/npm use && rm -rf with --no-cache-dir
Reverted Dockerfile — Removed the unrelated api-proxy Dockerfile change from this PR

github-actions · 2026-02-25T20:30:54Z

Smoke Test Results

✅ GitHub MCP: docs: add integration test coverage guide with gap analysis #1036 docs: add integration test coverage guide with gap analysis, feat: group --help flags by category, hide dev-only options #1035 feat: group --help flags by category, hide dev-only options
✅ Playwright: Page title contains "GitHub"
✅ File write: /tmp/gh-aw/agent/smoke-test-claude-22414574176.txt created
✅ Bash: File verified via cat

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1052

github-actions · 2026-02-25T20:31:02Z

Build Test: C++ Results

Project	CMake	Build	Status
fmt	✅	✅	PASS
json	✅	✅	PASS

Overall: PASS ✅

Generated by Build Test C++ for issue #1052

github-actions · 2026-02-25T20:31:09Z

🦀 Rust Build Test Results

Project	Build	Tests	Status
fd	✅	1/1	PASS
zoxide	✅	1/1	PASS

Overall: PASS ✅

Generated by Build Test Rust for issue #1052

github-actions · 2026-02-25T20:31:14Z

Deno Build Test Results

Project	Tests	Status
oak	1/1	✅ PASS
std	1/1	✅ PASS

Overall: ✅ PASS

Deno version: 2.7.1

Generated by Build Test Deno for issue #1052

github-actions · 2026-02-25T20:31:20Z

Smoke test results for @Mossaka:

✅ GitHub MCP — Last 2 merged PRs: #1036 "docs: add integration test coverage guide with gap analysis", #1035 "feat: group --help flags by category, hide dev-only options"
✅ Playwright — github.com title contains "GitHub"
✅ File write — /tmp/gh-aw/agent/smoke-test-copilot-22414574162.txt created
✅ Bash verify — file content confirmed

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1052

github-actions · 2026-02-25T20:31:41Z

Node.js Build Test Results

Project	Install	Tests	Status
clsx	✅	PASS	✅ PASS
execa	✅	PASS	✅ PASS
p-limit	✅	PASS	✅ PASS

Overall: ✅ PASS

Generated by Build Test Node.js for issue #1052

github-actions · 2026-02-25T20:31:41Z

Go Build Test Results

Project	Download	Tests	Status
color	✅	1/1	PASS
env	✅	1/1	PASS
uuid	✅	1/1	PASS

Overall: ✅ PASS

Generated by Build Test Go for issue #1052

github-actions · 2026-02-25T20:31:43Z

Merged PRs: docs: add integration test coverage guide with gap analysis; feat: group --help flags by category, hide dev-only options
PR List: test: add DNS restriction enforcement tests; test: add --env-all integration tests
GitHub MCP review ✅; safeinputs-gh ✅; Playwright ✅; Tavily search ❌ (tool missing); file write ✅; cat verify ✅; discussion comment ✅; build ✅
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1052

github-actions · 2026-02-25T20:31:46Z

🧪 Build Test: Bun Results

Project	Install	Tests	Status
elysia	✅	1/1	PASS
hono	✅	1/1	PASS

Overall: ✅ PASS

Bun version: 1.3.9

Generated by Build Test Bun for issue #1052

github-actions · 2026-02-25T20:32:21Z

Java Build Test Results

Project	Compile	Tests	Status
gson	✅	1/1	PASS
caffeine	✅	1/1	PASS

Overall: ✅ PASS

Both projects compiled and all tests passed successfully against PR #1052.

Generated by Build Test Java for issue #1052

Mossaka and others added 2 commits February 25, 2026 20:12

Copilot AI review requested due to automatic review settings February 25, 2026 20:12

Copilot started reviewing on behalf of Mossaka February 25, 2026 20:13 View session

Copilot AI reviewed Feb 25, 2026

View reviewed changes

github-actions bot added smoke-claude build-test-cpp labels Feb 25, 2026

github-actions bot added the build-test-rust label Feb 25, 2026

github-actions bot added the build-test-deno label Feb 25, 2026

github-actions bot added smoke-copilot build-test-go labels Feb 25, 2026

github-actions bot added the build-test-node label Feb 25, 2026

github-actions bot added the build-test-bun label Feb 25, 2026

github-actions bot added the build-test-java label Feb 25, 2026

github-actions bot mentioned this pull request Feb 25, 2026

[agentics] No-Op Runs #769

Open

Mossaka merged commit 5be8e96 into main Feb 25, 2026
84 checks passed

Mossaka deleted the test/package-install-tests branch February 25, 2026 22:45

github-actions bot mentioned this pull request Feb 25, 2026

feat(proxy): add observability and rate limiting to API proxy #1038

Merged

9 tasks

	COPY server.js logging.js metrics.js rate-limiter.js ./
	COPY server.js ./

-      expect(result.stdout).toMatch(/\d+\.\d+\.\d+/);
+      const lines = result.stdout.split('\n').map(line => line.trim()).filter(line => line.length > 0);
+      const lastLine = lines[lines.length - 1] || '';
+      expect(lastLine).toMatch(/^\d+\.\d+\.\d+$/);

Conversation

Mossaka commented Feb 25, 2026

Summary

Test plan

Uh oh!

github-actions bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Mossaka commented Feb 25, 2026

Uh oh!

github-actions bot commented Feb 25, 2026

Smoke Test Results

Uh oh!

github-actions bot commented Feb 25, 2026

Build Test: C++ Results

Uh oh!

github-actions bot commented Feb 25, 2026

🦀 Rust Build Test Results

Uh oh!

github-actions bot commented Feb 25, 2026

Deno Build Test Results

Uh oh!

github-actions bot commented Feb 25, 2026

Uh oh!

github-actions bot commented Feb 25, 2026

Node.js Build Test Results

Uh oh!

github-actions bot commented Feb 25, 2026

Go Build Test Results

Uh oh!

github-actions bot commented Feb 25, 2026

Uh oh!

github-actions bot commented Feb 25, 2026

🧪 Build Test: Bun Results

Uh oh!

github-actions bot commented Feb 25, 2026

Java Build Test Results

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Feb 25, 2026 •

edited

Loading