Struct Client 

pub struct Client {
    pub openrouter_client: Client,
}

Client that manages connections to all upstream providers.

Handles provider selection, BYOK key injection, and fallback between providers.

Fields

openrouter_client: Client

OpenRouter provider client.

Implementations

impl Client

pub fn new(openrouter_client: Client) -> Self

Creates a new upstream client.

pub async fn create_streaming( &self, ctx: Context<impl ContextExt + Send + Sync + 'static>, id: String, first_chunk_timeout: Duration, other_chunk_timeout: Duration, ensemble_llm: Arc<EnsembleLlm>, request: Params, ) -> Result<Option<impl Stream<Item = Result<ChatCompletionChunk, Error>> + Send + Unpin + 'static>, Error>

Creates a streaming completion, trying each upstream provider in order.

First attempts with BYOK if available, then falls back to the default key. Returns Ok(None) if no upstreams were available.

Trait Implementations

impl Clone for Client

fn clone(&self) -> Client

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Client

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.