Struct AnthropicChatCompletionClient

pub struct AnthropicChatCompletionClient { /* private fields */ }

AnthropicChatCompletionClient

Allows for interacting with the Anthropic models via the messages API.

Examples

use serde_json::{Map, Value};
use rag_toolchain::clients::*;
use rag_toolchain::common::*;
async fn generate_completion() {
    let model: AnthropicModel = AnthropicModel::Claude3Sonnet;
    let mut additional_config: Map<String, Value> = Map::new();
    additional_config.insert("temperature".into(), 0.5.into());

    let client: AnthropicChatCompletionClient =
        AnthropicChatCompletionClient::try_new_with_additional_config(
            model,
            4096,
            additional_config,
        )
        .unwrap();

    let system_message: PromptMessage =
        PromptMessage::SystemMessage("You only reply in a bullet point list".into());
    let user_message: PromptMessage = PromptMessage::HumanMessage("How does the water flow".into());

    // We invoke the chat client with a list of messages
    let reply = client
        .invoke(vec![system_message.clone(), user_message.clone()])
        .await
        .unwrap();
    println!("{:?}", reply.content());
}

Required Environment Variables

ANTHROPIC_API_KEY: The API key for the Anthropic API

Implementations

impl AnthropicChatCompletionClient

pub fn try_new(model: AnthropicModel, max_tokens: u32) -> Result<Self, VarError>

AnthropicChatCompletionClient::try_new

This method creates a new instance of the AnthropicChatCompletionClient. All optional inference parameters will be set to their default values on Anthropic’s end.

Arguments

• model: AnthropicModel - The model to use for the chat completion.
• max_tokens: u32 - The maximum number of tokens to generate in the response. See the API documentation for more information.

Errors

VarError - This error is returned when the ANTHROPIC_API_KEY environment variable is not set.

Returns

AnthropicChatCompletionClient - The client to interact with the Anthropic API.

pub fn try_new_with_additional_config( model: AnthropicModel, max_tokens: u32, additional_config: Map<String, Value>, ) -> Result<Self, VarError>

AnthropicChatCompletionClient::try_new_with_additional_config

This method creates a new instance of the AnthropicChatCompletionClient. All optional inference parameters will be set to their default values on Anthropic’s end.

Arguments

• model: AnthropicModel - The model to use for the chat completion.
• max_tokens: u32 - The maximum number of tokens to generate in the response. See the API documentation for more information.
• additional_config: Map<String, Value> - Additional configuration to pass to the API. See the API documentation for more information. Examples of this can be temperature, top_p, etc.

Errors

VarError - This error is returned when the ANTHROPIC_API_KEY environment variable is not set.

Returns

AnthropicChatCompletionClient - The client to interact with the Anthropic API.

Trait Implementations

impl AsyncChatClient for AnthropicChatCompletionClient

async fn invoke( &self, prompt_messages: Vec<PromptMessage>, ) -> Result<PromptMessage, Self::ErrorType>

AnthropicChatCompletionClient::invoke

Function to send a list of PromptMessage to the Anthropic API and receive a response.

Arguments

• prompt_messages: Vec<PromptMessage> - The list of messages to send to the API.

Errors

• AnthropicError - This error is returned when the API returns an error.

Returns

PromptMessage::AIMessage - The response from the API.

type ErrorType = AnthropicError