pub struct Router { /* private fields */ }Expand description
The main router implementation
Implementations§
Source§impl Router
impl Router
pub fn new( config: RouterConfig, catalog: Arc<ModelCatalog>, health: Arc<HealthChecker>, circuit_breaker: Arc<CircuitBreaker>, ) -> Self
Sourcepub fn with_policy(self, policy: CompositePolicy) -> Self
pub fn with_policy(self, policy: CompositePolicy) -> Self
Create router with custom policy
Trait Implementations§
Source§impl RouterTrait for Router
impl RouterTrait for Router
Source§fn route(
&self,
request: &InferenceRequest,
) -> BoxFuture<'_, FederationResult<RouteTarget>>
fn route( &self, request: &InferenceRequest, ) -> BoxFuture<'_, FederationResult<RouteTarget>>
Route a request to the best available node
Source§fn get_candidates(
&self,
request: &InferenceRequest,
) -> BoxFuture<'_, FederationResult<Vec<RouteCandidate>>>
fn get_candidates( &self, request: &InferenceRequest, ) -> BoxFuture<'_, FederationResult<Vec<RouteCandidate>>>
Get all possible routes for a request (for debugging/transparency)
Auto Trait Implementations§
impl Freeze for Router
impl !RefUnwindSafe for Router
impl Send for Router
impl Sync for Router
impl Unpin for Router
impl UnsafeUnpin for Router
impl !UnwindSafe for Router
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more