Struct RoundRobinRouter 

pub struct RoundRobinRouter { /* private fields */ }

Router that selects providers using round-robin order.

Implementations

impl RoundRobinRouter

pub fn new(providers: Vec<Arc<dyn Provider>>) -> Result<Self>

Create a new round-robin router.

Errors

Returns a routing error if no providers are supplied.

pub fn provider_count(&self) -> usize

Return the number of configured providers.

pub async fn complete( &self, request: &CompletionRequest, ) -> Result<CompletionResponse>

Execute a completion request using round-robin provider selection.

pub async fn stream( &self, request: &CompletionRequest, ) -> Result<Box<dyn Stream<Item = Result<CompletionChunk>> + Send + Unpin>>

Execute a streaming request using round-robin provider selection.