Skip to content

feat(ai-gateway): Update load balancing capabilities documentation for AI Gateway 2.0#5308

Open
tomek-labuk wants to merge 8 commits into
new-ai-gatewayfrom
feat/update-load-balancing-docs
Open

feat(ai-gateway): Update load balancing capabilities documentation for AI Gateway 2.0#5308
tomek-labuk wants to merge 8 commits into
new-ai-gatewayfrom
feat/update-load-balancing-docs

Conversation

@tomek-labuk
Copy link
Copy Markdown
Contributor

@tomek-labuk tomek-labuk commented May 20, 2026

Description

Notes:

  • I have no idea how to handle examples for load balancing algorithms for Model
  • Do we need a separate reference for load balancing?
  • Is model documentation getting too heavy?
  • Model documentation still needs to be aligned with UI, it's heave on the API spec side

Preview Links

Checklist

  • Tested how-to docs. If not, note why here.
  • All pages contain metadata.
  • Any new docs link to existing docs.
  • All autogenerated instructions render correctly (API, decK, Konnect, Kong Manager).
  • Style guide (capitalized gateway entities, placeholder URLs) implemented correctly.
  • Every page has a description entry in frontmatter.
  • Add new pages to the product documentation index (if applicable).

@tomek-labuk tomek-labuk requested a review from a team as a code owner May 20, 2026 12:17
Copilot AI review requested due to automatic review settings May 20, 2026 12:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the AI Gateway documentation to reflect the AI Gateway 2.0 load balancing workflow (Model entity-based configuration) while preserving legacy AI Proxy Advanced plugin-based guidance for existing deployments.

Changes:

  • Updates the /ai-gateway/load-balancing/ reference to describe Model entity load balancing (config.balancer / target_models) and adds a legacy algorithms section behind a <details> block.
  • Adds a new include containing the legacy (plugin-based) load balancing algorithms table.
  • Expands the Model entity reference with a clearer capabilities table, a deeper TOC, and new per-algorithm Model entity examples.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
app/ai-gateway/load-balancing.md Reframes load balancing docs around Model entities (AI Gateway 2.0) and nests legacy plugin guidance.
app/_includes/ai-gateway/load-balancing/legacy-algorithms.md Adds a legacy, plugin-focused algorithms table for older configurations.
app/_ai_gateway_entities/model.md Enhances the Model entity reference (capabilities table, TOC depth, and new algorithm examples).

Comment thread app/ai-gateway/load-balancing.md Outdated
Comment thread app/ai-gateway/load-balancing.md Outdated
Comment on lines +206 to +211
@@ -182,6 +207,8 @@ rows:

You can customize {{site.ai_gateway}} load balancer to fit different application needs, such as minimizing latency, enabling sticky sessions, or optimizing for cost. The table below maps common scenarios to key configuration options that control load balancing behavior:

#### Retry and fallback scenarios for Model entities {% new_in 2.0.0 %}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9f9206c — removed the redundant "Retry and fallback scenarios" heading and folded the intro text under the single "Retry and fallback scenarios for Model entities" heading.

Comment thread app/ai-gateway/load-balancing.md Outdated
Comment thread app/_includes/ai-gateway/load-balancing/legacy-algorithms.md Outdated
Comment thread app/_ai_gateway_entities/model.md Outdated
Comment on lines +522 to 523
{% include plugins/ai-vector-db.md name="Model entity" %}

tomek-labuk and others added 3 commits May 20, 2026 14:26
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ tomek-labuk
❌ Copilot
You have signed the CLA already but the status is still pending? Let us recheck it.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants