Tests Run Even Though Global Setup Fails

[keithyipkw]

Describe the bug

When a global setup fails, the tests that depend on it do not run, which is expected. However, the tests start if they are run the second time, using the side effects of the failed global setup. It seems that the extension treats the failed global setup as if it succeeds, and combines the behaviour of not re-running a global setup every time as described in #671.

Reproduction

Clone the global-setup branch in https://github.com/keithyipkw/vitest-vscode
pnpm i && pnpm dev
Run and Debug: Run Extension Global Setup Error
In the new VS Code instance, run the test "should fail or do not execute"
The global setup fails, and the test does not run
Run the test again
The test passes

Output

[INFO 3:52:18 PM] [v0.0.0] Vitest extension is activated because Vitest is installed or there is a Vite/Vitest config file in the workspace.
[INFO 3:52:19 PM] [API] Resolving configs: global-setup-error/vitest.config.ts
[INFO 3:52:19 PM] [API] Running Vitest v4.0.0-beta.12 (global-setup-error/vitest.config.ts) with "C:\nvm4w\nodejs\node.EXE C:/Users/user/Workspace/vitest-vscode/dist/worker.js"
[INFO 3:52:19 PM] [API] Watching vitest.config.ts
[INFO 3:52:19 PM] [VSCODE] Watching global-setup-error with pattern **/*
[INFO 3:52:20 PM] [API] Collecting tests: test/run.test.ts
[INFO 3:52:56 PM] Running 1 file(s) with name pattern: ^\s?testing should fail or not execute$
[3:52:56 PM] Starting a test run because test/run.test.ts triggered a watch rerun event
[3:52:56 PM] Enqueuing "should fail or not execute"
[INFO 3:52:56 PM] [Worker] Global setup ac669f6
[INFO 3:52:56 PM] [Worker] Global setup ac669f6
[Error 3:52:56 PM] Failed to run tests [Error Error] Global setup error ac669f6e
Error: Global setup error ac669f6e
    at Object.__vite_ssr_export_default__ [as setup] (C:/Users/user/Workspace/vitest-vscode/samples/global-setup-error/global_setup.ts:12:8)
    at TestProject._initializeGlobalSetup (file:///C:/Users/user/Workspace/vitest-vscode/node_modules/.pnpm/vitest@4.0.0-beta.12_@types_7e4aad2cc04bd3d40bea143e1c26500f/node_modules/vitest/dist/chunks/cli-api.CpywZzJV.js:7252:51)
    at Vitest.initializeGlobalSetup (file:///C:/Users/user/Workspace/vitest-vscode/node_modules/.pnpm/vitest@4.0.0-beta.12_@types_7e4aad2cc04bd3d40bea143e1c26500f/node_modules/vitest/dist/chunks/cli-api.CpywZzJV.js:9495:35)
    at file:///C:/Users/user/Workspace/vitest-vscode/node_modules/.pnpm/vitest@4.0.0-beta.12_@types_7e4aad2cc04bd3d40bea143e1c26500f/node_modules/vitest/dist/chunks/cli-api.CpywZzJV.js:9420:5
    at Vitest.runFiles (file:///C:/Users/user/Workspace/vitest-vscode/node_modules/.pnpm/vitest@4.0.0-beta.12_@types_7e4aad2cc04bd3d40bea143e1c26500f/node_modules/vitest/dist/chunks/cli-api.CpywZzJV.js:9441:7)
    at Vitest.rerunTestSpecifications (file:///C:/Users/user/Workspace/vitest-vscode/node_modules/.pnpm/vitest@4.0.0-beta.12_@types_7e4aad2cc04bd3d40bea143e1c26500f/node_modules/vitest/dist/chunks/cli-api.CpywZzJV.js:9411:18)
    at ExtensionWorkerRunner.runTests (C:\Users\user\Workspace\vitest-vscode\dist\workerNew.js:667:7)
    at _ExtensionWorker.runTests (C:\Users\user\Workspace\vitest-vscode\dist\workerNew.js:831:5)
    at _WebSocket.onMessage2 (C:\Users\user\Workspace\vitest-vscode\dist\worker.js:3780:20)
[3:52:56 PM] Ending test run test/run.test.ts
[3:52:56 PM] Test run promise is finished, the queue is 0
[3:52:56 PM] Ending test run <none>
[INFO 3:53:00 PM] Running 1 file(s) with name pattern: ^\s?testing should fail or not execute$
[3:53:00 PM] Starting a test run because test/run.test.ts triggered a watch rerun event
[3:53:00 PM] Enqueuing "should fail or not execute"
[3:53:01 PM] No task result for "run.test.ts", ignoring
[3:53:01 PM] No task result for "testing", ignoring
[3:53:01 PM] Enqueuing "should fail or not execute" because it was just collected
[3:53:01 PM] No errors found for "run.test.ts"
[3:53:01 PM] No errors found for "testing"
[3:53:01 PM] Marking "should fail or not execute" as passed
[3:53:01 PM] No errors found for "run.test.ts"
[3:53:01 PM] Ending test run test/run.test.ts
[3:53:01 PM] Test run promise is finished, the queue is 0
[3:53:01 PM] [VSCODE] Ignoring file: node_modules\.vite\vitest\da39a3ee5e6b4b0d3255bfef95601890afd80709\results.json

Extension Version

Commit 77d7e17 (the current main)

Vitest Version

4.0.0-beta.12

Validations

• Check that you are using the latest version of the extension
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.