Struct LlmRetryPolicy 

pub struct LlmRetryPolicy {
    pub max_attempts_per_model: usize,
    pub initial_backoff_ms: u64,
    pub max_backoff_ms: u64,
    pub backoff_jitter_percent: u8,
    pub max_retry_window_ms: Option<u64>,
    pub retry_stream_start: bool,
    pub max_stream_event_retries: usize,
    pub stream_error_fallback_threshold: usize,
}

Retry strategy for LLM inference calls.

Fields

max_attempts_per_model: usize

Max attempts per model candidate (must be >= 1).

initial_backoff_ms: u64

Initial backoff for retries in milliseconds.

max_backoff_ms: u64

Max backoff cap in milliseconds.

backoff_jitter_percent: u8

Random jitter applied around the exponential backoff base, expressed as a percentage of the computed delay.

max_retry_window_ms: Option<u64>

Maximum wall-clock retry window in milliseconds for one retry sequence. When exceeded, no further retry backoff is scheduled.

retry_stream_start: bool

Retry stream startup failures before any output is emitted.

max_stream_event_retries: usize

Max retryable mid-stream recovery attempts across a single run.

stream_error_fallback_threshold: usize

Consecutive mid-stream failures on one model before escalating to the next fallback model.

Trait Implementations

impl Clone for LlmRetryPolicy

fn clone(&self) -> LlmRetryPolicy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlmRetryPolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LlmRetryPolicy

fn default() -> Self

Returns the “default value” for a type.