Skip to content

[BUG] Azure AI Projects maxCompletionTokens #48452

@mrm9084

Description

@mrm9084

Describe the bug
There seems to be some unexpected timeout when using maxCompletionTokens. I have two version of a project one in python one for java. In python making a simple chat request with 800 max completion tokens works, but in java it fails in various way depending on how big of a value I give.

In Java at 800 max completion tokens I get a finishReason of length. With no text response.

In Java at 2000 max completion tokens I've gotten everything from the text response to an exception being thrown that the connection was terminated.

Exception or Stack Trace
Add the exception log and stack trace if available

To Reproduce
Steps to reproduce the behavior:

See code snippet.

Code Snippet

public static void main(String[] args) {
    String endpoint = System.getenv("AZURE_AI_FOUNDRY_ENDPOINT");

    String model = "gpt-5";

    OpenAIClient openAIClient = new AIProjectClientBuilder()
            .endpoint(endpoint)
            .credential(new DefaultAzureCredentialBuilder().build())
            .buildOpenAIClient();

    List<ChatCompletionMessageParam> messages = new ArrayList<>();

    messages.add(ChatCompletionMessageParam.ofSystem(
            ChatCompletionSystemMessageParam.builder()
                    .content("You are a helpful assistant.")
                    .build()));

    messages.add(ChatCompletionMessageParam.ofUser(
            ChatCompletionUserMessageParam.builder()
                    .content("What is Azure App Configuration?")
                    .build()));

    ChatCompletionCreateParams params = ChatCompletionCreateParams.builder()
            .model(model)
            .messages(messages)
            .maxCompletionTokens(800)
            .build();

    ChatCompletion response = openAIClient.chat().completions().create(params);

    String aiResponse = response.choices().get(0).message().content().orElse("");
    System.out.println("AI: " + aiResponse);
}

Expected behavior
This should work, it does in other languages.

Setup (please complete the following information):

  • OS: Windows
  • IDE: VS Code
  • Library/Libraries: azure-ai-projects 2.0.0-beta.2
  • Java version: 17
  • App Server/Environment: N/A
  • Frameworks: N/A

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

  • verbose dependency tree (mvn dependency:tree -Dverbose)
  • exception message, full stack trace, and any available logs

Additional context
Add any other context about the problem here.

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Bug Description Added
  • Repro Steps Added
  • Setup information Added

Metadata

Metadata

Assignees

Labels

AI Projectsneeds-triageWorkflow: This is a new issue that needs to be triaged to the appropriate team.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions