Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
[flake8]
max-line-length = 88
exclude = .git,.github,.chglog,__pycache__,docs,venv,env,mypy_cache
max-complexity = 10
max-complexity = 10
extend-ignore = E203
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,24 @@ For more detailed examples and integration patterns, check out:

---

## Streaming

The SDK auto-detects the response wire format from `Content-Type`:

- `application/vnd.amazon.eventstream` — AWS Bedrock event-stream framing (used by `invoke-with-response-stream` and `converse-stream`). Frames are parsed via `botocore.eventstream`; Bedrock's `{"bytes": "<base64-json>"}` envelope is unwrapped automatically.
- `text/event-stream` / `application/x-ndjson` — SSE (OpenAI, Anthropic direct, Gemini).

Each route configures a `stream_response_path` (JSONPath) that extracts the per-chunk text from the model's native chunk schema. Common values:

| Model family | `stream_response_path` |
| ------------------------ | -------------------------- |
| Anthropic Claude | `delta.text` |
| Meta Llama (Bedrock) | `generation` |
| Cohere Command (Bedrock) | `text` |
| OpenAI chat | `choices[0].delta.content` |

If `stream_response_path` doesn't match a chunk, the SDK logs a single structured WARNING with the payload keys it saw — check that warning against the `ModelSpec` on the gateway.

## Type Hints & py.typed Marker

This package includes a `py.typed` marker file, which indicates to type checkers (like `mypy`, `pyright`, `pylance`) that the package supports type checking. This allows IDEs and static analysis tools to provide better autocomplete, type checking, and refactoring support.
Expand Down
Loading