Crate llm_circuit_breaker 

llm-circuit-breaker

Tiny circuit breaker for LLM API calls.

llm-retry handles individual transient failures with backoff. This crate handles the case where the LLM provider is clearly down for everyone: rather than retry each call (and pay for it), stop calling for a window and let one probe through after the window expires.

States

• Closed — calls pass through, failures are counted.
• Open — after failure_threshold consecutive failures, all calls reject with BreakerError::Open for reset_timeout.
• HalfOpen — after reset_timeout expires, the next call probes. If it succeeds, the breaker closes. If it fails, the breaker opens for another reset_timeout.

Example

use llm_circuit_breaker::{CircuitBreaker, BreakerConfig, BreakerError};
use std::time::Duration;

let cb = CircuitBreaker::new(BreakerConfig {
    failure_threshold: 3,
    reset_timeout: Duration::from_secs(30),
});

// first failures are counted
let r: Result<u32, _> = cb.call(|_: &&str| true, || Err("server_error"));
assert!(matches!(r, Err(BreakerError::Inner("server_error"))));

// after three failures the breaker opens
let _ = cb.call(|_: &&str| true, || Err::<u32, &str>("server_error"));
let _ = cb.call(|_: &&str| true, || Err::<u32, &str>("server_error"));
let r: Result<u32, _> = cb.call(|_: &&str| true, || Ok(42));
assert!(matches!(r, Err(BreakerError::Open)));

Counting policy

should_count(&err) -> bool decides whether a given error counts as a failure for the breaker. Typically you only count server/throttle errors and skip caller bugs (4xx, validation). Use the predicates from llm-retry to share the policy.

Structs

BreakerConfig

Configuration for a CircuitBreaker.

CircuitBreaker

Thread-safe circuit breaker.

Enums

BreakerError

What CircuitBreaker::call returns when not Ok.

BreakerState

Observable state of the breaker. Returned by CircuitBreaker::state.