Conversation
Contract tests that validate which VSCode and Positron APIs the extension calls, with what arguments, and what it expects back. Tests mock the vscode and positron modules, import actual extension code against those mocks, and assert API interactions. Includes compile-time conformance checks that verify mock properties exist in the real @types/vscode and positron type definitions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
727db02 to
6cf3d08
Compare
dotNomad
left a comment
There was a problem hiding this comment.
Overall this idea is great. I had a couple ideas that I think are worth digging into a bit to simplify
I wonder if we can simplify this a bit by using a __mocks__ folder
If there is a mocks folder alongside a file that you are mocking, and the factory is not provided, Vitest will try to find a file with the same name in the mocks subfolder and use it as an actual module. If you are mocking a dependency, Vitest will try to find a mocks folder in the root of the project (default is process.cwd()). You can tell Vitest where the dependencies are located through the deps.moduleDirectories config option.
There is an example of __mocks__ in the File System example: https://vitest.dev/guide/mocking/file-system.html#example
| "devDependencies": { | ||
| "@types/vscode": "^1.87.0", | ||
| "typescript": "^5.7.0", | ||
| "vitest": "^3.0.0" |
There was a problem hiding this comment.
Could bump up to the latest vitest here.
There was a problem hiding this comment.
Do we use this tsconfig.json at all? I only see the conformance one being used in the package.json
| export enum TreeItemCollapsibleState { | ||
| None = 0, | ||
| Collapsed = 1, | ||
| Expanded = 2, | ||
| } |
There was a problem hiding this comment.
For these enum should we just import them from vscode? I can't see a reason for us to duplicate them here.
There was a problem hiding this comment.
According to claude, we shouldn't / can't do this:
No, you shouldn't import enums from vscode directly in the mock file. Here's why:
The vscode module is not a real npm package — it's a virtual module provided by the VS Code runtime at extension load time. In Vitest (which runs outside VS Code), there is no real vscode module to import
from. That's exactly why every test file uses vi.mock("vscode", () => { ... }) to provide a fake implementation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove the unused tsconfig.json from extension-contract-tests (only tsconfig.conformance.json and vitest.config.ts are referenced). Bump vitest from ^3.0.0 to ^4.1.0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ibility Vitest 4.x passes `new` calls through to the implementation function. Arrow functions cannot be constructors, so `new vi.fn(() => ...)` throws TypeError. Switch to regular function expressions for all mocks that are instantiated with `new`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
test/extension-contract-tests/— a contract test suite that validates which VSCode and Positron APIs the extension calls, with what arguments, and what it expects backvscodeandpositronmodules, import actual extension source code against those mocks, and assert API interactionstsc) that verify mock properties exist in the real@types/vscodeand Positron type definitionsjust test-extension-contractsandjust check-extension-contract-conformancerecipesextension-contract-testsworkflow to CI (pull request, main, nightly)What's tested
activationextension.tsactivate()wiring: trust, URI handler, commands, contextsauth-providerauthProvider.tsauthentication.registerAuthenticationProviderconnect-filesystemconnect_content_fs.tsworkspace.registerFileSystemProvider,FileSystemErrordialogsdialogs.tswindow.showInformationMessage(modal),l10n.tdocument-trackerentrypointTracker.tscommands.executeCommand("setContext")extension-settingsextension.tsworkspace.getConfiguration("positPublisher")file-watcherswatchers.tsworkspace.createFileSystemWatcher,RelativePatterninterpreter-discoveryutils/vscode.tscommands.executeCommand,workspace.getConfiguration, Positron runtimellm-toolsllm/index.tslm.registerToolopen-connectopen_connect.tswindow.showInputBox,workspace.updateWorkspaceFolderspositron-settingsutils/positronSettings.tsworkspace.getConfiguration("positron.r")window-utilsutils/window.tswindow.showErrorMessage,withProgress,createTerminalTest plan
just test-extension-contractspassesjust check-extension-contract-conformancepassesextension-contract-testsworkflow passes🤖 Generated with Claude Code