-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Describe the bug
There seems to be some unexpected timeout when using maxCompletionTokens. I have two version of a project one in python one for java. In python making a simple chat request with 800 max completion tokens works, but in java it fails in various way depending on how big of a value I give.
In Java at 800 max completion tokens I get a finishReason of length. With no text response.
In Java at 2000 max completion tokens I've gotten everything from the text response to an exception being thrown that the connection was terminated.
Exception or Stack Trace
Add the exception log and stack trace if available
To Reproduce
Steps to reproduce the behavior:
See code snippet.
Code Snippet
public static void main(String[] args) {
String endpoint = System.getenv("AZURE_AI_FOUNDRY_ENDPOINT");
String model = "gpt-5";
OpenAIClient openAIClient = new AIProjectClientBuilder()
.endpoint(endpoint)
.credential(new DefaultAzureCredentialBuilder().build())
.buildOpenAIClient();
List<ChatCompletionMessageParam> messages = new ArrayList<>();
messages.add(ChatCompletionMessageParam.ofSystem(
ChatCompletionSystemMessageParam.builder()
.content("You are a helpful assistant.")
.build()));
messages.add(ChatCompletionMessageParam.ofUser(
ChatCompletionUserMessageParam.builder()
.content("What is Azure App Configuration?")
.build()));
ChatCompletionCreateParams params = ChatCompletionCreateParams.builder()
.model(model)
.messages(messages)
.maxCompletionTokens(800)
.build();
ChatCompletion response = openAIClient.chat().completions().create(params);
String aiResponse = response.choices().get(0).message().content().orElse("");
System.out.println("AI: " + aiResponse);
}Expected behavior
This should work, it does in other languages.
Setup (please complete the following information):
- OS: Windows
- IDE: VS Code
- Library/Libraries: azure-ai-projects 2.0.0-beta.2
- Java version: 17
- App Server/Environment: N/A
- Frameworks: N/A
If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:
- verbose dependency tree (
mvn dependency:tree -Dverbose) - exception message, full stack trace, and any available logs
Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
- Bug Description Added
- Repro Steps Added
- Setup information Added