Add Per-Model Provider Routing Overrides

[coniferconcepts]

Add Per-Model Provider Routing Overrides

Problem

Mirrowel currently has:

• provider model inventories
• global routing weights
• model_aliases.*.default_provider

That is not enough to express model-specific routing constraints when provider availability differs by model.

Current examples:

• nemotron-3-super should route only to Ollama Cloud
• qwen3.5 should route only among Ollama Cloud and Chutes
• GO should not be considered for qwen3.5

Today, current config hints at defaults but does not clearly enforce:

• provider exclusion per model
• per-model custom weights
• strict single-provider routes

Proposed Design

Add routing.model_overrides and resolve weighted-router/<model> before provider lock-in in RotatingClient.

Precedence:

1. routing.model_overrides[model]
2. model_aliases[model].default_provider
3. global routing.weights
4. fallback tier rules

Proposed Config

routing:
  model_overrides:
    nemotron-3-super:
      strategy: "single"
      primary: ollama
      allowed_providers: [ollama]
      fallback_providers: []
      strict: true
      log_decision: true
      allow_global_fallback: false
      reason: "Only available on Ollama Cloud"

    qwen3.5:
      strategy: "weighted"
      allowed_providers: [ollama, chutes]
      weights:
        ollama: 0.85
        chutes: 0.15
      fallback_providers: [zen]
      strict: true
      log_decision: true
      allow_global_fallback: false
      reason: "Shared between Ollama Cloud and Chutes; exclude GO"

Implementation Touch Points

• add mirrowel-src/src/rotator_library/routing_policy.py
• update mirrowel-src/src/rotator_library/client.py
  • rewrite before provider lock-in at client.py:1186
  • integrate near existing model-id resolution at client.py:1236

v1 Scope

• parse and validate routing.model_overrides
• implement single
• pin nemotron-3-super to Ollama only
• add route-decision logging
• add tests

v2 Scope

• implement weighted
• add qwen3.5 override
• optionally add sequential

Acceptance Criteria

• weighted-router/nemotron-3-super always rewrites to ollama/nemotron-3-super
• invalid override config fails at load time
• weighted-router/qwen3.5 only selects ollama or chutes
• route decisions are visible in logs
• existing provider-specific retry/credential behavior remains unchanged

Suggested Tests

• single-provider rewrite for nemotron-3-super
• invalid provider/model validation failure
• no-fallback single route fails fast
• weighted qwen3.5 excludes GO
• route log contains selection_source and override_applied

[mirrobot-agent]

@coniferconcepts Thank you for submitting this issue. I am now beginning my analysis and will report back shortly.

[mirrobot-agent]

Initial Analysis Report

Summary: This is a well-specified feature request to add per-model provider routing overrides, allowing fine-grained control over which providers can be selected for specific models (e.g., routing nemotron-3-super only to Ollama Cloud, or qwen3.5 only to Ollama Cloud and Chutes).

Issue Validation: Confirmed - This is a valid enhancement request with clear requirements and implementation guidance.

Reproducibility Assessment: Not applicable (feature request, not a bug).

Root Cause Analysis:
The current architecture lacks a mechanism to enforce provider constraints at the model level. Based on my investigation:

1. Current Behavior: Provider selection occurs at line client.py:1186 where provider = model.split("/")[0] extracts the provider from the model string. There is no intermediate routing layer to override this before credential selection.

2. Missing Components:

   • No routing_policy.py module exists in src/rotator_library/
   • No routing.model_overrides configuration parsing
   • No model-level provider filtering before the credential rotation loop at client.py:1213

3. Integration Points: The issue correctly identifies that routing logic should be integrated:

   • Before provider lock-in at client.py:1186-1190
   • Near model ID resolution at client.py:1236-1240

Suggested Labels: enhancement, routing, configuration

• Justification: This adds new functionality for model-specific routing constraints, requires configuration schema changes, and affects the routing/rotation logic.

Proposed Next Steps:

1. Create src/rotator_library/routing_policy.py with:

   • RoutingPolicy class to parse/validate routing.model_overrides
   • Methods: get_allowed_providers(model), get_weights(model), is_strict(model)
   • Validation for strategy types: single, weighted, (future: sequential)

2. Update src/rotator_library/client.py:

   • Add routing policy initialization in __init__
   • Insert routing resolution between lines 1186-1190:

     # After extracting provider from model
     # Check if model has routing overrides that should rewrite provider selection
     routing_decision = self.routing_policy.resolve(model)
     if routing_decision:
         provider = routing_decision.selected_provider
         allowed_providers = routing_decision.allowed_providers

   • Filter credentials_for_provider based on allowed_providers at line 1213
   • Add route-decision logging with selection_source and override_applied fields

3. Add configuration validation in src/rotator_library/config/defaults.py:

   • Define schema for routing.model_overrides
   • Add validation for provider names against configured credentials

4. Implement tests for:

   • Single-provider rewrite for nemotron-3-super
   • Weighted selection for qwen3.5 excluding GO
   • Invalid override config fails at load time
   • Route log contains decision metadata

Missing Information (if any):

• How should weighted-router/<model> prefix be handled? Is this a special model alias pattern or literal string?
• Should routing overrides apply to streaming endpoints as well (likely yes, but worth confirming)?
• What is the expected behavior when strict: true but no allowed providers have available credentials?

Investigation Warnings

• No duplicate issues found (searched all open/closed issues)
• External documentation URLs could not be fetched (tool restriction), analysis based on codebase only

This analysis was generated by an AI assistant.

[Mirrowel]

This is a good feature to add, but it would be a special feature for fallbacks - not default. Simply because models differ per-provider, hence why you choose provider/model combo and not just model.