pub struct Router { /* private fields */ }Expand description
The main router implementation
Implementations§
Source§impl Router
impl Router
pub fn new( config: RouterConfig, catalog: Arc<ModelCatalog>, health: Arc<HealthChecker>, circuit_breaker: Arc<CircuitBreaker>, ) -> Self
Sourcepub fn with_policy(self, policy: CompositePolicy) -> Self
pub fn with_policy(self, policy: CompositePolicy) -> Self
Create router with custom policy
Trait Implementations§
Source§impl RouterTrait for Router
impl RouterTrait for Router
Source§fn route(
&self,
request: &InferenceRequest,
) -> BoxFuture<'_, FederationResult<RouteTarget>>
fn route( &self, request: &InferenceRequest, ) -> BoxFuture<'_, FederationResult<RouteTarget>>
Route a request to the best available node
Source§fn get_candidates(
&self,
request: &InferenceRequest,
) -> BoxFuture<'_, FederationResult<Vec<RouteCandidate>>>
fn get_candidates( &self, request: &InferenceRequest, ) -> BoxFuture<'_, FederationResult<Vec<RouteCandidate>>>
Get all possible routes for a request (for debugging/transparency)
Auto Trait Implementations§
impl Freeze for Router
impl !RefUnwindSafe for Router
impl Send for Router
impl Sync for Router
impl Unpin for Router
impl !UnwindSafe for Router
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more