Struct LoadAwareMoERouter 

pub struct LoadAwareMoERouter { /* private fields */ }

Load-aware MoE router

Routes to the least loaded available expert, with optional affinity for consistent routing when loads are similar.

Implementations

impl LoadAwareMoERouter

pub fn new() -> Self

Create a new load-aware router

pub fn with_affinity_threshold(self, threshold: f64) -> Self

Set affinity threshold

If load difference is below this threshold, prefer consistent routing

Trait Implementations

impl Clone for LoadAwareMoERouter

fn clone(&self) -> LoadAwareMoERouter

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LoadAwareMoERouter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LoadAwareMoERouter

fn default() -> Self

Returns the “default value” for a type.

impl MoERouter for LoadAwareMoERouter

fn route<'life0, 'life1, 'async_trait>( &'life0 self, input: &'life1 str, num_experts: usize, ) -> Pin<Box<dyn Future<Output = RouteResult> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Route an input to an expert

fn route_with_experts<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, input: &'life1 str, experts: &'life2 [Expert], ) -> Pin<Box<dyn Future<Output = RouteResult> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Route with expert health information

fn name(&self) -> &str

Get router name for metrics/logging