Struct ChatClient 

pub struct ChatClient { /* private fields */ }

👎Deprecated since 2.0.0:

The OpenAI direct clients are deprecated; use the Session API instead (ChatSession::new(&model)).

Client for OpenAI-compatible chat completions backed by a local model.

Implementations

impl ChatClient

pub fn frequency_penalty(self, v: f64) -> Self

Set the frequency penalty.

pub fn max_tokens(self, v: u32) -> Self

Set the maximum number of tokens to generate.

pub fn n(self, v: u32) -> Self

Set the number of completions to generate.

pub fn temperature(self, v: f64) -> Self

Set the sampling temperature.

pub fn presence_penalty(self, v: f64) -> Self

Set the presence penalty.

pub fn top_p(self, v: f64) -> Self

Set the nucleus sampling probability.

pub fn top_k(self, v: u32) -> Self

Set the top-k sampling parameter (Foundry extension).

pub fn random_seed(self, v: u64) -> Self

Set the random seed for reproducible results (Foundry extension).

pub fn response_format(self, v: ChatResponseFormat) -> Self

Set the desired response format.

pub fn tool_choice(self, v: ChatToolChoice) -> Self

Set the tool choice strategy.

pub async fn complete_chat( &self, messages: &[ChatCompletionRequestMessage], tools: Option<&[ChatCompletionTools]>, ) -> Result<CreateChatCompletionResponse, FoundryLocalError>

Perform a non-streaming chat completion.

pub async fn complete_streaming_chat( &self, messages: &[ChatCompletionRequestMessage], tools: Option<&[ChatCompletionTools]>, ) -> Result<ChatCompletionStream, FoundryLocalError>

Perform a streaming chat completion, returning a ChatCompletionStream.

Use the stream with futures_core::StreamExt::next() or tokio_stream::StreamExt::next().