Struct ChatClient

pub struct ChatClient {
    pub base_url: String,
    pub auth_token: Option<String>,
    pub http_client: Client,
}

The object to hold information about the model server and reqwest HTTP client.

Fields

base_url: String

auth_token: Option<String>

http_client: Client

Implementations

impl ChatClient

pub fn init(base_url: String, auth_token: Option<String>) -> Self

Create a new ChatClient instance, which hosts the basic information and reqwest client for making the requests

Args:

• base_url Base url of the API server. This URL should NOT end with ‘/’.
• auth_token Bearer authentication token. It is often called “API Key”.

pub fn create_chat_completion_request( &self, model: &str, messages: &Vec<ChatMessage>, is_stream: bool, params: &HashMap<String, Value>, ) -> RequestBuilder

Create a chat completion request.

Args:

• model Name of the model to be called.
• messages A Vec of ChatMessage as the context.
• is_stream Whether the request is stream-based.
• params Other parameters to be sent.

pub async fn chat_completion_stream( &self, model: &str, messages: &Vec<ChatMessage>, params: &HashMap<String, Value>, ) -> Result<impl Stream<Item = Result<ChatResponseChunkDelta>>>

Request chat completion in the async stream approach.

Args:

• model Name of the model to be called.
• messages A Vec of ChatMessage as the context.
• params Other parameters to be sent.

Trait Implementations

impl Debug for ChatClient

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.