test: add unit tests for openai_llm2_python ThinkParser#2150
Open
nuthalapativarun wants to merge 1 commit into
Open
test: add unit tests for openai_llm2_python ThinkParser#2150nuthalapativarun wants to merge 1 commit into
nuthalapativarun wants to merge 1 commit into
Conversation
ThinkParser handles <think>...</think> reasoning blocks in streaming LLM responses but had no test coverage. Add 16 tests covering: - Plain text message_delta events - Full <think>...</think> block parsing (delta + done events) - Partial open/close tag buffering across chunks - Split tags across multiple chunks - finalize() flushing pending buffer and open blocks - Multiple think blocks in a single stream - process_reasoning_content() API - Legacy process() bool interface
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ThinkParserinopenai_llm2_python, covering the<think>...</think>reasoning-block parser used to handle streaming LLM responses from reasoning models (e.g. o1, DeepSeek-R1 style)conftest.pythat stubsten_runtimeandten_ai_baseso pure-Python helper tests run without a full TEN runtime installationWhat is tested
test_plain_text_produces_message_deltamessage_deltatest_think_block_produces_reasoning_eventsreasoning_delta+reasoning_done+message_deltatest_think_block_reasoning_contenttest_message_after_think_block</think>emitted asmessage_deltatest_empty_input_returns_no_eventstest_partial_open_tag_buffered<this held in_pendingtest_partial_open_tag_resolved_on_next_chunk<think>test_split_think_tag_across_chunks<thi/nk>split across calls still detectedtest_state_returns_to_normal_after_think</think>test_finalize_flushes_pending_normalfinalize()drains_pendingin NORMAL statetest_finalize_closes_open_think_blockfinalize()closes unclosed THINK blocktest_multiple_think_blocksreasoning_donetest_process_reasoning_content_emits_deltaprocess_reasoning_content()pathtest_process_reasoning_content_empty_closes_blocktest_process_returns_false_for_plain_textprocess()bool returntest_process_returns_true_on_state_changeprocess()returnsTrueon state transitionTest plan
python3 -m pytest ai_agents/agents/ten_packages/extension/openai_llm2_python/tests/ -v— 16/16 passedtask format— clean (no production code changes)