Enum RoutingStrategy 

pub enum RoutingStrategy {
    Fallback,
    RoundRobin,
    LeastLatency,
    LatencyWeighted(f64),
    LowestCost,
    InputDirected(Arc<dyn Fn(&str) -> usize + Send + Sync>),
}

Strategy for selecting which model a RouterLLM tries first.

Regardless of strategy, a failed model triggers fallback to the next candidate in the derived order until one succeeds or all fail.

Variants

Fallback

Always try models in registration order (primary-first). Use with RouterLLM::with_fallbacks for classic primary + backups semantics.

RoundRobin

Rotate the starting index across calls so traffic spreads evenly.

LeastLatency

Prefer the model with the lowest recent latency (exponential moving average updated after each call).

LatencyWeighted(f64)

Pick the primary model by a weighted draw over observed latency: slot weight is (1 / latency_ms).powf(beta), so beta = 1.0 makes a 100 ms model ten times as likely to lead as a 1000 ms one. Models never tried are optimistically given the best observed latency so they still receive exploration traffic; when nothing has been tried all weights are equal. Remaining slots follow the draw as a fallback chain sorted by weight. The draw is a deterministic SplitMix64 sequence seeded per router (no rand dependency).

LowestCost

Prefer the model with the lowest configured cost.

InputDirected(Arc<dyn Fn(&str) -> usize + Send + Sync>)

Pick the primary index from a user-supplied closure over the input text; remaining models are tried in registration order as fallback.