feat(cli): add --check flag to goose info for provider testing#8289
feat(cli): add --check flag to goose info for provider testing#8289mvanhorn wants to merge 5 commits intoblock:mainfrom
Conversation
…ting Adds a --check flag to `goose info` that tests the configured provider connection and reports auth status, initialization time, and available model count. On failure, prints the error with an actionable hint. Users currently discover broken provider configurations only after starting a session. This flag lets them verify setup works before committing to a session. Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
The initial implementation used config.get_param("provider") which
reads from the PROVIDER env var. The correct accessors are
get_goose_provider() and get_goose_model() which read from
GOOSE_PROVIDER/GOOSE_MODEL and the config file.
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e61936675d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "Auth:", | ||
| &style("ok").green().to_string(), | ||
| label_padding, | ||
| ); |
There was a problem hiding this comment.
This marks Auth: ok immediately after provider construction, before any authenticated request is made, so invalid credentials can still be reported as successful. For providers like OpenAI-compatible ones, construction can succeed without validating the key, and the first real auth check happens in fetch_supported_models(), which may then return 401 while the output still says auth is ok. Since --check is intended to validate provider setup, this is a false positive and can mislead users into thinking their auth is healthy when it is not.
Useful? React with 👍 / 👎.
Addresses Codex review feedback: provider construction can succeed without validating credentials, so Auth: ok was a false positive. Now fetch_supported_models() runs first, and auth status reflects whether the credentialed call actually passed. Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52e7c8e72a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| print_aligned( | ||
| "Auth:", | ||
| &style("ok").green().to_string(), | ||
| label_padding, | ||
| ); |
There was a problem hiding this comment.
When fetch_supported_models() returns an error that does not match the string checks for 401/Authentication/Unauthorized (for example NetworkError, RateLimitExceeded, or other request failures), this branch still prints Auth: ok (and immediately after, Connection: ok). That creates a false positive for goose info --check even though the verification call failed, so users can be told their provider is healthy during outages or connectivity failures.
Useful? React with 👍 / 👎.
Non-auth errors (network, rate limit) from fetch_supported_models() were still printing Auth: ok. Now any verification failure shows Check: FAILED with the error message. Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 44e140d551
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| Ok(model_config) => { | ||
| match goose::providers::create(&provider, model_config, Vec::new()).await { | ||
| Ok(p) => { | ||
| let elapsed = start.elapsed(); |
There was a problem hiding this comment.
The Connection: ok (verified in …) value is computed before fetch_supported_models().await, so the reported verification time excludes the credentialed network call that actually validates connectivity. In slow or degraded networks this can print a near-zero "verified" duration even though the check took much longer, which makes the new health signal inaccurate for troubleshooting.
Useful? React with 👍 / 👎.
| match goose::providers::create(&provider, model_config, Vec::new()).await { | ||
| Ok(p) => { | ||
| let elapsed = start.elapsed(); | ||
| match p.fetch_supported_models().await { |
There was a problem hiding this comment.
this sounds like the right way to do it, since it is free for most providers, but it is not. for custom providers that specify their own models and have fetching disabled, this will always work. for openrouter, this will also always work. you should probably just send an actual test message
There was a problem hiding this comment.
Good call -- switched from fetch_supported_models() to a minimal complete() call (Message::user("Say 'ok'") with no tools). This actually exercises the auth path, so it'll catch bad keys on custom providers and OpenRouter where model listing always succeeds.
Replace fetch_supported_models() with a minimal completion call (Message::user 'Say ok' with max 1 response). This catches auth failures on custom providers with fetching disabled and on OpenRouter, where model listing always succeeds regardless of credentials. Addresses review feedback from @DOsinga on PR block#8289.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f83fc853f3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| _ => { | ||
| println!( | ||
| " {} No provider configured. Run '{}' first.", | ||
| style("⚠").yellow(), | ||
| style("goose configure").cyan() | ||
| ); | ||
| } |
There was a problem hiding this comment.
The fallback arm for match (provider_name, model_name) collapses every error into the same "No provider configured" message, so goose info --check misreports malformed/unreadable config files (or other retrieval failures) as missing setup. In those cases users are directed to reconfigure instead of seeing the real failure, which makes troubleshooting provider checks much harder. Please only use this message for actual NotFound cases and print other errors explicitly.
Useful? React with 👍 / 👎.
2 participants
Summary
Adds a
--checkflag togoose infothat tests the configured provider connection. Reports auth status, initialization time, and available model count. On failure, prints the error with a hint to rungoose configure.Demo
Three scenes: (1)
goose infobefore this change (paths only), (2)goose info --checkwith no provider configured (warning + hint), (3)goose info --checkwith invalid credentials (auth failure correctly detected after credentialed API call).Why this matters
Users discover broken provider configurations only after starting a session. The auth error appears after the session banner has already printed, and the error message gives no guidance on how to fix it. Several issues trace back to this first-use friction:
This flag provides a quick pre-flight check without starting a full session.
Changes
Two files changed in
crates/goose-cli/:src/cli.rs(+2 lines): Addedcheck: boolfield to theInfocommand variant. Updated dispatch to passcheckand.awaitthe now-asynchandle_info.src/commands/info.rs(+93 lines): Madehandle_infoasync. When--checkis passed, reads provider/model fromConfig::get_goose_provider()/get_goose_model(), creates the provider viagoose::providers::create(), then callsfetch_supported_models()as the credentialed API call. Auth status is reported based on whether that call succeeds or returns a 401/auth error. Non-auth errors (network, rate limit) are reported separately under Models.Three output paths:
goose configurehintAuth: FAILEDwith actionable hintTesting
cargo fmt --checkpassescargo clippy -p goose-cli --all-targets -- -D warningspasses (zero warnings)cargo test -p goose-clipassesRelated Issues
Relates to #6287, #6893, #7903, #8029
This contribution was developed with AI assistance (Codex + Claude Code).