Description:
Add span events to track streaming request milestones for detailed streaming performance analysis.
Detailed Requirements:
- Add span events during streaming:
first_token - Time to first token received
chunk_received - Periodic chunk delivery (configurable sampling)
stream_complete - Stream completion with totals
- Event attributes:
ttfb_ms - Milliseconds to first token
chunk_size - Size of chunk in tokens/bytes
chunks_received - Total chunks at completion
total_tokens - Total tokens at completion
- Integrate with streaming backends:
- OpenAI streaming responses
- Other streaming-capable backends
- Make chunk event sampling configurable (avoid event spam)
Files to Modify:
mellea/telemetry/__init__.py - Add span event helpers
mellea/backends/openai.py - Add streaming events
- Other streaming backends as applicable
Environment Variables:
| Variable |
Default |
Description |
MELLEA_TRACE_STREAM_CHUNKS |
false |
Emit events for each chunk |
MELLEA_TRACE_STREAM_SAMPLE_RATE |
10 |
Sample every Nth chunk |
Event Schema:
span.add_event("first_token", {"ttfb_ms": 245})
span.add_event("chunk_received", {"chunk_index": 5, "tokens": 12})
span.add_event("stream_complete", {"total_chunks": 15, "total_tokens": 150})
Acceptance Criteria:
Description:
Add span events to track streaming request milestones for detailed streaming performance analysis.
Detailed Requirements:
first_token- Time to first token receivedchunk_received- Periodic chunk delivery (configurable sampling)stream_complete- Stream completion with totalsttfb_ms- Milliseconds to first tokenchunk_size- Size of chunk in tokens/byteschunks_received- Total chunks at completiontotal_tokens- Total tokens at completionFiles to Modify:
mellea/telemetry/__init__.py- Add span event helpersmellea/backends/openai.py- Add streaming eventsEnvironment Variables:
MELLEA_TRACE_STREAM_CHUNKSfalseMELLEA_TRACE_STREAM_SAMPLE_RATE10Event Schema:
Acceptance Criteria: