Struct LlmExecutor 

pub struct LlmExecutor { /* private fields */ }

Unified executor for LLM operations.

Coordinates throttle, retry, and fallback mechanisms.

Implementations

impl LlmExecutor

pub fn new(config: LlmConfig) -> Self

Create a new executor with the given configuration.

pub fn with_defaults() -> Self

Create an executor with default configuration.

pub fn for_model(model: impl Into<String>) -> Self

Create an executor for a specific model.

pub fn with_throttle(self, controller: ConcurrencyController) -> Self

Add throttle control.

pub fn with_shared_throttle( self, controller: Arc<ConcurrencyController>, ) -> Self

Add throttle control from an existing Arc.

pub fn with_fallback(self, chain: FallbackChain) -> Self

Add fallback chain.

pub fn with_shared_fallback(self, chain: Arc<FallbackChain>) -> Self

Add fallback chain from an existing Arc.

pub fn config(&self) -> &LlmConfig

Get the configuration.

pub fn throttle(&self) -> Option<&ConcurrencyController>

Get the throttle controller (if any).

pub fn fallback(&self) -> Option<&FallbackChain>

Get the fallback chain (if any).

pub async fn complete(&self, system: &str, user: &str) -> LlmResult<String>

Execute a completion with unified coordination.

This method coordinates:

1. Throttle: Acquire permit before API call
2. Retry: Exponential backoff on transient errors
3. Fallback: Model/endpoint degradation on persistent failures

pub async fn complete_with_max_tokens( &self, system: &str, user: &str, max_tokens: u16, ) -> LlmResult<String>

Execute a completion with custom max tokens.

Trait Implementations

impl Clone for LlmExecutor

fn clone(&self) -> LlmExecutor

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlmExecutor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LlmExecutor

fn default() -> Self

Returns the “default value” for a type.