Namely, want to support OpenAI and Anthropic.
In addition to those below, the other main challenges are (1) abstracting the vector database and (2) managing model context.
| Dimension |
OpenAI Chat Completions |
OpenAI Responses |
Anthropic Messages |
| State Management |
Stateless (you manage messages array) |
Stateful (server-side via previous_response_id) |
Stateless (you manage messages array) |
| Execution |
Sync, Async, Streaming |
Sync, Async, Streaming |
Sync, Async, Streaming |
| Structured Output |
Yes (response_format, function calling) |
Yes (same mechanisms) |
Yes (tool use with JSON schema) |
| Tool Calls |
Yes (functions/tools) |
Yes (built-in tools like web search, file search, computer use) |
Yes (tool use) |
| Extended Reasoning |
No |
Yes (reasoning parameter) |
Yes (extended_thinking parameter) |
| Built-in Tools |
No |
Yes (web search, file search, code interpreter, computer use) |
No (tools must be defined) |
| Primary Use Case |
General chat, most integrations |
Agentic workflows, multi-turn with state |
General chat, agentic workflows |
Namely, want to support OpenAI and Anthropic.
In addition to those below, the other main challenges are (1) abstracting the vector database and (2) managing model context.