pub struct RetryClient<C> { /* private fields */ }Expand description
An LLMClient wrapper that retries on transient errors.
Retries are attempted for:
- 429 Rate Limited — uses the
retry-afterheader value as the initial delay. - 5xx Server Errors — exponential backoff starting from
base_delay.
All other errors (4xx, network timeouts, deserialization failures) are returned immediately without retry.
Implementations§
Source§impl<C> RetryClient<C>
impl<C> RetryClient<C>
Trait Implementations§
Source§impl<C: LLMClient> LLMClient for RetryClient<C>
impl<C: LLMClient> LLMClient for RetryClient<C>
Source§fn stream_complete<'life0, 'async_trait>(
&'life0 self,
request: LLMRequest,
) -> Pin<Box<dyn Future<Output = Result<LLMStream>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn stream_complete<'life0, 'async_trait>(
&'life0 self,
request: LLMRequest,
) -> Pin<Box<dyn Future<Output = Result<LLMStream>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Streaming is not retried. Delegates directly to the inner client.
Source§fn complete<'life0, 'async_trait>(
&'life0 self,
request: LLMRequest,
) -> Pin<Box<dyn Future<Output = Result<LLMResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn complete<'life0, 'async_trait>(
&'life0 self,
request: LLMRequest,
) -> Pin<Box<dyn Future<Output = Result<LLMResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Send a chat completion request and return the full response.
Source§fn model_name(&self) -> &str
fn model_name(&self) -> &str
Return the model name this client is configured to use.
Auto Trait Implementations§
impl<C> Freeze for RetryClient<C>where
C: Freeze,
impl<C> RefUnwindSafe for RetryClient<C>where
C: RefUnwindSafe,
impl<C> Send for RetryClient<C>where
C: Send,
impl<C> Sync for RetryClient<C>where
C: Sync,
impl<C> Unpin for RetryClient<C>where
C: Unpin,
impl<C> UnsafeUnpin for RetryClient<C>where
C: UnsafeUnpin,
impl<C> UnwindSafe for RetryClient<C>where
C: UnwindSafe,
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
impl<T> ErasedDestructor for Twhere
T: 'static,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::RequestSource§impl<T> Pointable for T
impl<T> Pointable for T
Source§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> Read<Exclusive, BecauseExclusive> for Twhere
T: ?Sized,
Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
The inverse inclusion map: attempts to construct
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
Checks if
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
Use with care! Same as
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
The inclusion map: converts
self to the equivalent element of its superset.