pub struct ProviderPool { /* private fields */ }Expand description
The configured profiles and their health.
Build one with ProviderPool::builder and share it behind an Arc; the
health map is behind a mutex so recording an outcome needs only &self.
Implementations§
Source§impl ProviderPool
impl ProviderPool
Sourcepub fn builder() -> ProviderPoolBuilder
pub fn builder() -> ProviderPoolBuilder
Starts building a pool.
Sourcepub fn profiles(&self) -> Vec<ModelProfile>
pub fn profiles(&self) -> Vec<ModelProfile>
Every configured profile, in declaration order.
Sourcepub fn provider(&self, model: &ModelRef) -> Option<Arc<dyn ModelProvider>>
pub fn provider(&self, model: &ModelRef) -> Option<Arc<dyn ModelProvider>>
The adapter serving model, when configured.
Sourcepub fn health(&self, model: &ModelRef) -> ProviderHealth
pub fn health(&self, model: &ModelRef) -> ProviderHealth
The recorded health of model.
Sourcepub fn is_healthy(&self, model: &ModelRef) -> bool
pub fn is_healthy(&self, model: &ModelRef) -> bool
Returns true when model is outside a cooldown window.
Sourcepub fn record_success(&self, model: &ModelRef)
pub fn record_success(&self, model: &ModelRef)
Records a successful call, clearing any cooldown.
Sourcepub fn record_failure(&self, model: &ModelRef, class: RetryClass)
pub fn record_failure(&self, model: &ModelRef, class: RetryClass)
Records a failed call.
Only classes that say something about the provider’s availability count:
Retry, RetryAfter and
Fallback. A Fatal
outcome — a refusal, a content filter, a context overflow — means the
provider answered exactly as asked, so it does not degrade its health.
Sourcepub fn reset_health(&self)
pub fn reset_health(&self)
Forgets every recorded outcome.
Trait Implementations§
Auto Trait Implementations§
impl !Freeze for ProviderPool
impl !RefUnwindSafe for ProviderPool
impl !UnwindSafe for ProviderPool
impl Send for ProviderPool
impl Sync for ProviderPool
impl Unpin for ProviderPool
impl UnsafeUnpin for ProviderPool
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more