fix(llm): add retries to RAG orchestrator to handle output validation issues by vitali87 · Pull Request #399 · vitali87/code-graph-rag

vitali87 · 2026-03-01T00:37:48Z

Add retries=3 to the RAG orchestrator agent creation to handle transient output validation failures.

Dependency graph

Merge in this order:

pr-1: chore: update .gitignore with popular vibe-coding agent directories
pr-2: fix(llm): add retries to RAG orchestrator to handle output validation issues  <-- this PR
├── pr-4: feat(mcp/tools): add shell_command, document_analyzer, semantic_search, and ask_agent tools to MCP registry
│   └── pr-5: feat(mcp/client): add MCP client CLI for querying the code graph via MCP server
│       └── pr-7: docs: update README with non-interactive mode, new MCP tools, and agent tool list
└── pr-6: feat(main): add non-interactive --ask-agent mode to the CLI
    └── pr-7: docs: update README with non-interactive mode, new MCP tools, and agent tool list
pr-3: feat(mcp/server): improve MCP server logging and suppress output during tool execution
└── pr-4: feat(mcp/tools): add shell_command, document_analyzer, semantic_search, and ask_agent tools to MCP registry
    └── pr-5: feat(mcp/client): add MCP client CLI for querying the code graph via MCP server
        └── pr-7: docs: update README with non-interactive mode, new MCP tools, and agent tool list

… issues

gemini-code-assist · 2026-03-01T00:37:58Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the reliability of the RAG orchestrator by implementing a retry strategy. This change aims to mitigate transient failures that can occur during the LLM's output validation process, ensuring a more stable and resilient agent initialization without altering its core functionality.

Highlights

RAG Orchestrator Robustness: Increased the robustness of the RAG orchestrator by adding a retry mechanism to handle potential output validation issues during its initialization.

Changelog

codebase_rag/services/llm.py
- Added retries=3 to the Agent initialization for the RAG orchestrator.

Activity

No activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request adds a retry mechanism to the RAG orchestrator to improve resilience against transient LLM failures. A security audit confirmed that no new vulnerabilities were introduced by these changes. However, the current implementation hardcodes the number of retries to 3. It is recommended to make this value configurable, similar to existing patterns like thinking_budget, to enhance flexibility.

gemini-code-assist · 2026-03-01T00:39:33Z

codebase_rag/services/llm.py

            model=llm,
            system_prompt=RAG_ORCHESTRATOR_SYSTEM_PROMPT,
            tools=tools,
+            retries=3,  # Increase retries to handle output validation issues


While adding retries is a good way to handle transient LLM issues, hardcoding the value 3 makes it inflexible. It would be better to make this configurable, similar to other settings like thinking_budget.

This would involve:

Adding ORCHESTRATOR_RETRIES: int | None = None to AppConfig in codebase_rag/config.py.

Adding a retries: int | None = None field to the ModelConfig dataclass.

Populating this field from the environment variable in AppConfig._get_default_config.

Then you could use it here like this:

Suggested change

retries=3, # Increase retries to handle output validation issues

retries=config.retries or 3, # Use configured retries, default to 3

greptile-apps · 2026-03-01T00:41:38Z

Greptile Summary

This PR adds retries=3 to the RAG orchestrator agent initialization to handle transient output validation failures. The change is straightforward and correctly uses PydanticAI's built-in retry mechanism.

Key changes:

Added retry parameter to create_rag_orchestrator function's Agent initialization

Minor issue:

Inline comment needs (H) prefix per project's comment policy

Confidence Score: 4/5

This PR is safe to merge with minimal risk
Simple one-line change adding retry logic to handle transient failures; uses PydanticAI's built-in retry mechanism correctly; only minor style issue with comment formatting
No files require special attention - the change is straightforward

Important Files Changed

Filename	Overview
codebase_rag/services/llm.py	added `retries=3` to RAG orchestrator agent to handle output validation failures; minor style issue with inline comment

_{Last reviewed commit: bee8526}

greptile-apps

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-03-01T00:41:41Z

codebase_rag/services/llm.py

            model=llm,
            system_prompt=RAG_ORCHESTRATOR_SYSTEM_PROMPT,
            tools=tools,
+            retries=3,  # Increase retries to handle output validation issues


inline comment violates project's "No Comments" policy - must be prefixed with (H) or removed

Suggested change

retries=3, # Increase retries to handle output validation issues

retries=3, # (H) Increase retries to handle output validation issues

Context Used: Rule from dashboard - ## Technical Requirements

Agentic Framework

PydanticAI Only: This project uses PydanticAI... (source)

Prompt To Fix With AI

This is a comment left during a code review. Path: codebase_rag/services/llm.py Line: 114 Comment: inline comment violates project's "No Comments" policy - must be prefixed with `(H)` or removed ```suggestion retries=3, # (H) Increase retries to handle output validation issues ``` **Context Used:** Rule from `dashboard` - ## Technical Requirements ### Agentic Framework - **PydanticAI Only**: This project uses PydanticAI... ([source](https://app.greptile.com/review/custom-context?memory=d4240b05-b763-467a-a6bf-94f73e8b6859)) How can I resolve this? If you propose a fix, please make it concise.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

fix(llm): add retries to RAG orchestrator to handle output validation…

bee8526

… issues

github-project-automation bot added this to @vitali87's graph code Mar 1, 2026

gemini-code-assist bot reviewed Mar 1, 2026

View reviewed changes

greptile-apps bot reviewed Mar 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(llm): add retries to RAG orchestrator to handle output validation issues#399

fix(llm): add retries to RAG orchestrator to handle output validation issues#399
vitali87 wants to merge 1 commit intomainfrom
pr-split/add-cmd-question/pr-2

vitali87 commented Mar 1, 2026

Uh oh!

gemini-code-assist bot commented Mar 1, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Mar 1, 2026

Uh oh!

greptile-apps bot commented Mar 1, 2026

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Mar 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	retries=3, # Increase retries to handle output validation issues
	retries=config.retries or 3, # Use configured retries, default to 3

	retries=3, # Increase retries to handle output validation issues
	retries=3, # (H) Increase retries to handle output validation issues

Uh oh!

Conversation

vitali87 commented Mar 1, 2026

Dependency graph

Uh oh!

gemini-code-assist bot commented Mar 1, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 1, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot commented Mar 1, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Mar 1, 2026

Choose a reason for hiding this comment

Agentic Framework

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants