Trait RouterTrait 

pub trait RouterTrait: Send + Sync {
    // Required methods
    fn route(
        &self,
        request: &InferenceRequest,
    ) -> BoxFuture<'_, FederationResult<RouteTarget>>;
    fn get_candidates(
        &self,
        request: &InferenceRequest,
    ) -> BoxFuture<'_, FederationResult<Vec<RouteCandidate>>>;
}

Router - selects the best node for a request

Required Methods

fn route( &self, request: &InferenceRequest, ) -> BoxFuture<'_, FederationResult<RouteTarget>>

Route a request to the best available node

fn get_candidates( &self, request: &InferenceRequest, ) -> BoxFuture<'_, FederationResult<Vec<RouteCandidate>>>

Get all possible routes for a request (for debugging/transparency)

Implementors

impl RouterTrait for Router