|
| 1 | +# Investigation: UI automation tools unavailable with Smithery install (issue #163) |
| 2 | + |
| 3 | +## Summary |
| 4 | +Smithery installs ship only the compiled entrypoint, while the server hard-requires a bundled `bundled/axe` path derived from `process.argv[1]`. This makes UI automation (and simulator video capture) fail even when system `axe` exists on PATH, and Doctor can report contradictory statuses. |
| 5 | + |
| 6 | +## Symptoms |
| 7 | +- UI automation tools (`describe_ui`, `tap`, `swipe`, etc.) fail with "Bundled axe tool not found. UI automation features are not available." |
| 8 | +- `doctor` reports system axe present, but UI automation unavailable due to missing bundled binary. |
| 9 | +- Smithery cache lacks `bundled/axe` directory; only `index.cjs`, `manifest.json`, `.metadata.json` present. |
| 10 | + |
| 11 | +## Investigation Log |
| 12 | + |
| 13 | +### 2026-01-06 - Initial Assessment |
| 14 | +**Hypothesis:** Smithery packaging omits bundled binaries and server does not fallback to system axe. |
| 15 | +**Findings:** Issue report indicates bundled path is computed relative to `process.argv[1]` and Smithery cache lacks `bundled/`. |
| 16 | +**Evidence:** GitHub issue #163 body (Smithery cache contents; bundled path logic). |
| 17 | +**Conclusion:** Needs code and packaging investigation. |
| 18 | + |
| 19 | +### 2026-01-06 - AXe path resolution and bundled-only assumption |
| 20 | +**Hypothesis:** AXe resolution is bundled-only, so missing `bundled/axe` disables tools regardless of PATH. |
| 21 | +**Findings:** `getAxePath()` computes `bundledAxePath` from `process.argv[1]` and returns it only if it exists; otherwise `null`. No PATH or env override. |
| 22 | +**Evidence:** `src/utils/axe-helpers.ts:15-36` |
| 23 | +**Conclusion:** Confirmed. Smithery layout lacking `bundled/` will always return null. |
| 24 | + |
| 25 | +### 2026-01-06 - UI automation and video capture gating |
| 26 | +**Hypothesis:** UI tools and video capture preflight fail when `getAxePath()` returns null. |
| 27 | +**Findings:** UI tools call `getAxePath()` and throw `DependencyError` if absent; `record_sim_video` preflights `areAxeToolsAvailable()` and `isAxeAtLeastVersion()`; `startSimulatorVideoCapture` returns error if `getAxePath()` is null. |
| 28 | +**Evidence:** `src/mcp/tools/ui-testing/describe_ui.ts:150-164`, `src/mcp/tools/simulator/record_sim_video.ts:80-88`, `src/utils/video_capture.ts:92-99` |
| 29 | +**Conclusion:** Confirmed. Missing bundled binary blocks all UI automation and simulator video capture. |
| 30 | + |
| 31 | +### 2026-01-06 - Doctor output inconsistency |
| 32 | +**Hypothesis:** Doctor uses different checks for dependency presence vs feature availability. |
| 33 | +**Findings:** Doctor uses `areAxeToolsAvailable()` (bundled-only) for UI automation feature status, while dependency check can succeed via `which axe` when bundled is missing. |
| 34 | +**Evidence:** `src/mcp/tools/doctor/doctor.ts:49-68`, `src/mcp/tools/doctor/lib/doctor.deps.ts:100-132` |
| 35 | +**Conclusion:** Confirmed. Doctor can report `axe` dependency present but UI automation unsupported. |
| 36 | + |
| 37 | +### 2026-01-06 - Packaging/Smithery artifact mismatch |
| 38 | +**Hypothesis:** NPM releases include `bundled/`, Smithery builds do not. |
| 39 | +**Findings:** `bundle:axe` creates `bundled/` and npm packaging includes it, but Smithery config has no asset inclusion hints. Release workflow bundles AXe before publish. |
| 40 | +**Evidence:** `package.json:21-44`, `.github/workflows/release.yml:48-55`, `smithery.yaml:1-3`, `smithery.config.js:1-6` |
| 41 | +**Conclusion:** Confirmed. Smithery build output likely omits bundled artifacts unless explicitly configured. |
| 42 | + |
| 43 | +### 2026-01-06 - Smithery local server deployment flow |
| 44 | +**Hypothesis:** Smithery deploys local servers from GitHub pushes and expects build-time packaging to include assets. |
| 45 | +**Findings:** README install flow uses Smithery CLI; `smithery.yaml` targets `local`. `bundled/` is gitignored, so it must be produced during Smithery’s deployment build. Current `npm run build` does not run `bundle:axe`. |
| 46 | +**Evidence:** `README.md:11-74`, `smithery.yaml:1-3`, `.github/workflows/release.yml:48-62`, `.gitignore:66-68` |
| 47 | +**Conclusion:** Confirmed. Smithery deploy must run `bundle:axe` and explicitly include `bundled/` in the produced bundle. |
| 48 | + |
| 49 | +### 2026-01-06 - Smithery config constraints and bundling workaround |
| 50 | +**Hypothesis:** Adding esbuild plugins in `smithery.config.js` overrides Smithery’s bootstrap plugin. |
| 51 | +**Findings:** Smithery CLI merges config via spread and replaces `plugins`, causing `virtual:bootstrap` resolution to fail when custom plugins are supplied. Side-effect bundling in `smithery.config.js` avoids plugin override and can copy `bundled/` into `.smithery/`. |
| 52 | +**Evidence:** `node_modules/@smithery/cli/dist/index.js:~2716600-2717500`, `smithery.config.js:1-47` |
| 53 | +**Conclusion:** Confirmed. Bundling must run outside esbuild plugins; Linux builders must skip binary verification. |
| 54 | + |
| 55 | +## Root Cause |
| 56 | +Two coupled assumptions break Smithery installs: |
| 57 | +1) `getAxePath()` is bundled-only and derives the path from `process.argv[1]`, which points into Smithery’s cache (missing `bundled/axe`), so it always returns null. |
| 58 | +2) Smithery packaging does not include the `bundled/` directory, so the bundled-only resolver can never succeed under Smithery even if AXe is installed system-wide. |
| 59 | + |
| 60 | +## Recommendations |
| 61 | +1. Add a robust AXe resolver: allow explicit env override and PATH fallback; keep bundled as preferred but not exclusive. |
| 62 | +2. Distinguish bundled vs system AXe in UI tools and video capture; only apply bundled-specific env when the bundled binary is used. |
| 63 | +3. Align Doctor output: show both bundled availability and PATH availability, and use that in the UI automation supported status. |
| 64 | +4. Update Smithery build to run `bundle:axe` and copy `bundled/` into the Smithery bundle output; skip binary verification on non-mac builders to avoid build failures. |
| 65 | + |
| 66 | +## Preventive Measures |
| 67 | +- Add tests for AXe resolution precedence (bundled, env override, PATH) and for Doctor output consistency. |
| 68 | +- Document Smithery-specific install requirements and verify `bundled/` presence in Smithery artifacts during CI. |
0 commit comments