Struct ShimClient 

pub struct ShimClient { /* private fields */ }

Implementations

impl ShimClient

pub fn new() -> Self

pub async fn warmup(&self, urls: &[&str])

Pre-establish TCP+TLS connections to provider endpoints. Call this after creating the Router to warm the connection pool.

pub async fn send(&self, req: &ProviderRequest) -> Result<Response>

pub async fn completion( &self, provider: &dyn Provider, model: &str, request: &Value, ) -> Result<Value>

pub async fn stream( &self, provider: &dyn Provider, model: &str, request: &Value, ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>>

Trait Implementations

impl Default for ShimClient

fn default() -> Self

Returns the “default value” for a type.