Struct RemoteLLMClient 

pub struct RemoteLLMClient { /* private fields */ }

A client for interacting with a remote LLM.

Implementations

impl RemoteLLMClient

pub fn new(config: LLMConfig) -> Self

Creates a new RemoteLLMClient.

pub fn config(&self) -> &LLMConfig

Returns the configuration of the client.

impl RemoteLLMClient

pub async fn chat( &self, messages: Vec<ChatMessage>, tools: Option<Vec<ToolDefinition>>, temperature: Option<f32>, max_tokens: Option<u32>, stop: Option<Vec<String>>, ) -> Result<ChatMessage>

Sends a chat request to the remote LLM.

pub async fn chat_stream<F>( &self, messages: Vec<ChatMessage>, tools: Option<Vec<ToolDefinition>>, temperature: Option<f32>, max_tokens: Option<u32>, stop: Option<Vec<String>>, on_chunk: F, ) -> Result<ChatMessage>

where F: FnMut(&str) + Send,

Sends a streaming chat request to the remote LLM.

Trait Implementations

impl LLMProvider for RemoteLLMClient

fn as_any(&self) -> &dyn Any

Returns the provider as an Any type.

fn generate<'life0, 'async_trait>( &'life0 self, request: LLMRequest, ) -> Pin<Box<dyn Future<Output = Result<LLMResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generates a response from the LLM.