Add monkey patching for ADK's litellm import#1660
Add monkey patching for ADK's litellm import#1660saglave wants to merge 1 commit intoNVIDIA:developfrom
Conversation
Signed-off-by: Sangharsh Aglave <aglave@synopsys.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Comment |
|
@saglave when this is ready for review could you retarget the |
|
@willkill07 Sure. Will do so. Just running some tests to make sure that the LLM calls aren't logged twice. |
Description
The
ADKProfilerHandlermonkey patch forlitellm.acompletionworks usually intercepts LLM calls made throughlitellm. But depending on how NAT and ADK are initiated, this would at times fail.Root Cause
Google ADK's
lite_llm.pyuses a direct import ofacompletionThis creates a local reference in the ADK module's namespace. When the profiler patched
litellm.acompletion, it only modified the reference in thelitellmmodule, not the local copy that ADK already had.Since
LiteLLMClient.acompletion()calls the localacompletion, it bypassed the monkey patch entirely.This fix adds monkey-patching for
google.adk.models.lite_llm.acompletionalongside the existinglitellm.acompletionto ensure that the calls are always intercepted.Closes
By Submitting this PR I confirm: