Struct chronicle_proxy::Proxy

pub struct Proxy { /* private fields */ }

Implementations

impl Proxy

pub fn builder() -> ProxyBuilder

pub async fn send( &self, options: ProxyRequestOptions, body: ChatRequest ) -> Result<ProxiedChatResponse, Report<Error>>

Send a request, choosing the provider based on the requested model and provider.

options.models can be used to specify a list of models and providers to use. options.model will be used next to choose a model to use. This and body[“model”] can be an alias name. options.provider can be used to choose a specific provider if the model is not an alias. body["model"] is used if options.model is empty.

pub fn set_provider(&self, provider: Arc<dyn ChatModelProvider>)

Add a provider to the system. This will replace any existing provider with the same name.

pub fn remove_provider(&self, name: &str)

Remove a provider. Any aliases that reference this provider’s name will stop working.

pub fn set_alias(&self, alias: AliasConfig)

Add an alias to the system. This will replace any existing alias with the same name.

pub fn remove_alias(&self, name: &str)

Remove an alias

pub fn set_api_key(&self, api_key: ApiKeyConfig)

Add an API key to the system. This will replace any existing API key with the same name.

pub fn remove_api_key(&self, name: &str)

Remove an API key. Any aliases that reference this API key’s name will stop working.

pub async fn shutdown(&mut self)

Shutdown the proxy, making sure to write any queued logging events

Trait Implementations

impl Debug for Proxy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.