pub struct RetryProvider<I> { /* private fields */ }Expand description
Retry wrapper: implements Provider and retries inner failures per
RetryPolicy.
Streaming semantics: only when stream_chat returns Err (connection
setup failure) is the whole call retried; errors inside the stream are
passed through verbatim and never retried — once the stream is
established this implementation cannot tell an “interruption before the
first event” from an “interruption after part of the output was
delivered”, and retrying would duplicate or corrupt output.
Hence “retry before the first event” only covers method-level failures;
interruptions within the stream (including after setup but before the
first event) are left to the caller (the Agent) to handle.
Timeouts are the inner provider’s (for example OpenAiProvider)
responsibility: each attempt may hit the inner Timeout error, and
Retryable::Default includes Timeout, so “timeouts are retried too”
falls out naturally.
§Errors
After retries are exhausted, the last error is returned (neither the first nor an aggregate); non-retryable errors are returned immediately on the first failure.
§Cancellation semantics
Dropping the future while waiting on backoff cancels the whole call; no further attempts are made. An already-issued inner request is cancelled together with the future (whether the network request continues at the transport layer depends on the underlying client).
Implementations§
Source§impl<I> RetryProvider<I>
impl<I> RetryProvider<I>
Sourcepub fn new(inner: I) -> RetryProvider<I>
pub fn new(inner: I) -> RetryProvider<I>
Wraps with the default policy (3 attempts / exponential backoff + jitter / default retry judgment).
Sourcepub fn with_policy(self, policy: RetryPolicy) -> RetryProvider<I>
pub fn with_policy(self, policy: RetryPolicy) -> RetryProvider<I>
Replaces the retry policy.
§Examples
For tests / local simulation: fixed short waits, at most 2 attempts,
ignoring the vendor’s Retry-After:
use std::time::Duration;
use molo::{Backoff, FakeProvider, RetryPolicy, RetryProvider};
let provider = RetryProvider::new(FakeProvider::new([])).with_policy(
RetryPolicy::default()
.with_max_attempts(2)
.with_backoff(Backoff::Fixed(Duration::from_millis(10)))
.with_respect_retry_after(false),
);