Module lightweight_routing 

Lightweight (cheap/fast) model routing for auxiliary features.

Structs

LightweightRouteResolution

Result of resolving which model a lightweight feature should use.

ModelRoute

A resolved provider-and-model pair that identifies a specific LLM endpoint.

Enums

LightweightFeature

Features that may use a lightweight (cheaper, faster) model instead of the primary model to reduce cost and latency.

LightweightRouteSource

Indicates how a lightweight route was resolved.

Functions

auto_lightweight_model

Automatically select a lightweight model for the given provider and active model.

create_provider_for_model_route

Instantiate an LLMProvider for the given model route.

lightweight_model_choices

Return the list of available lightweight model choices for the given provider.

main_model_route

Build a ModelRoute for the primary model from runtime configuration.

resolve_api_key_for_model_route

Resolve the API key for a model route, using the runtime key when the route targets the same provider as the main model, or falling back to environment variables otherwise.

resolve_api_key_for_model_route_with_mode

Resolve a lightweight route key using the configured secure-storage mode.

resolve_lightweight_route

Resolve the best model route for a given lightweight feature.