Tests fail when using fish (and possibly other shells)

[artlogic]

Description

When attempting to run tests under fish, the following errors are incorrectly thrown:

--- FAIL: TestDeployCommand_DeployHook (11.06s)
    --- FAIL: TestDeployCommand_DeployHook/fails_to_execute_an_unknown_script_path (2.31s)
        deploy_test.go:294:
                Error Trace:    github.com/slackapi/slack-cli/cmd/platform/deploy_test.go:294
                Error:          Should be true
                Test:           TestDeployCommand_DeployHook/fails_to_execute_an_unknown_script_path
    --- FAIL: TestDeployCommand_DeployHook/echos_stderr_output_to_standard_err (2.20s)
        deploy_test.go:296:
                Error Trace:    github.com/slackapi/slack-cli/cmd/platform/deploy_test.go:296
                Error:          Received unexpected error:
                                Error running 'Deploy' command: exit status 127 (sdk_hook_invocation_failed)

                                Suggestion
                                   Run `platform.test doctor` to check that your system dependencies are up-to-date.
                Test:           TestDeployCommand_DeployHook/echos_stderr_output_to_standard_err
FAIL
FAIL    github.com/slackapi/slack-cli/cmd/platform      11.953s

This seems to be due to a dependency on particular shell syntax/messages in the tests.

Version

Using slack-cli v3.15.0

OS Info

Arch Linux (under WSL2)

Steps to reproduce:

Share the commands to run, source code, and project settings:

1. Build.
2. go test ./...

Expected result:

I expected the tests to pass.

Actual result:

The tests failed, as shown above.

Requirements

• I've read and understood the Contributing guidelines and have done my best effort to follow them.
• I've read and agree to the Code of Conduct.
• I've searched for any related issues and avoided creating a duplicate issue.

[zimeg]

👋 Hey @artlogic! Thank you for sharing these findings with us! 🧪

We test our production builds on Windows with stable findings alongside ongoing developments, but hadn't run the unit tests so recent! I can confirm this is an issue perhaps specific to Windows since the fish shell didn't cause issue for me at this time.

Let's keep #464 as grounds to improve these checks - our tests are strict for expected outputs that might be different on outputs of pwsh as well.

[artlogic]

👋 Hey @artlogic! Thank you for sharing these findings with us! 🧪

We test our production builds on Windows with stable findings alongside ongoing developments, but hadn't run the unit tests so recent! I can confirm this is an issue perhaps specific to Windows since the fish shell didn't cause issue for me at this time.

Let's keep #464 as grounds to improve these checks - our tests are strict for expected outputs that might be different on outputs of pwsh as well.

Thanks! One point of clarification - I'm actually running tests on Arch Linux (under WSL2) so this would specifically be a Linux issue. What version of fish did you test with? I'm using 4.6.0 - it's possible older versions might not exhibit the behavior.

Also, you may already know this, but testing this behavior requires running tests like so: SHELL=/usr/sbin/fish go test ./... or changing your default shell. The tests appear to use the default (not your current).

[zimeg]

@artlogic Ahhaha thanks for the note that the test command has to use the fish shell itself 🎣 ✨

I can confirm these errors and think the matrix of tests above might be added to! AFAICT, with the SHELL variable set, the run and the deploy command still work as expected so these errors might be limited to tests? 🧪