Skip to content

refactor: move tracing onto plugin/hook pattern #1045

@ajbozarth

Description

@ajbozarth

Background

Tracing is the only telemetry pillar still using pre-plugin instrumentation. Metrics (#443) and logging (#442) both migrated to the canonical plugin/hook pattern during their epics; tracing needs the same treatment to unblock the rest of epic #444.

Current state

  • mellea/telemetry/backend_instrumentation.py exposes instrument_generate_from_context, instrument_generate_from_raw, start_generate_span, record_token_usage, record_response_metadata.
  • Each of 5 backends (openai, ollama, huggingface, litellm, watsonx) calls these helpers inline (~20 call sites total).
  • The active span is stored in mot._meta["_telemetry_span"] so it can survive the asyncio.create_task boundary and be closed in post_processing() / error path.
  • Tracing reads raw provider responses for attribute extraction instead of mot.generation.

Scope

  • Create mellea/telemetry/tracing_plugins.py following mellea/telemetry/metrics_plugins.py as the reference. Example structure:
    • BackendTracingPlugin (priority ~50) — hooks generation_post_call in FIRE_AND_FORGET mode, reads payload.model_output.generation, emits the backend generation span with gen_ai.* attributes.
    • Register plugins through a _TRACING_PLUGIN_CLASSES tuple inside tracing.py at import time when tracing is enabled.
  • Remove all direct tracing imports from mellea/backends/*.py. Delete the ~20 inline instrumentation calls.
  • Remove the mot._meta["_telemetry_span"] stash. Spans now live inside the plugin and don't need to round-trip through the MOT.
  • Keep mellea/telemetry/tracing.py as the low-level setup module (tracer provider, exporters, Tracer creation). Plugins consume get_application_tracer() / get_backend_tracer().
  • Application-level spans (session, aact) stay as a separate question — tracked in {{NEW-D}}.
  • Must preserve all attributes feat(telemetry): close five OTel GenAI semantic convention emission gaps (#1035) #1036 adds (gen_ai.provider.name, gen_ai.conversation.id, llm.prompt_template.*, gen_ai.request.{temperature, top_p, top_k, frequency_penalty, presence_penalty}, error.type, cache/reasoning tokens).

Phase & dependencies

Phase 1 (foundation). No internal deps. Blocks all of Phase 2 and Phase 3. Sequencing within Phase 1: this issue → {{NEW-B}} → {{NEW-C}}; all three likely land as a single PR per the epic plan.

Coordination notes

Acceptance criteria

Parent epic: #444

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions