Struct CustomLlmModel

pub struct CustomLlmModel {

    pub messages: Option<Vec<OpenAiMessage>>,
    pub tools: Option<Vec<AnyscaleModelToolsInner>>,
    pub tool_ids: Option<Vec<String>>,
    pub knowledge_base: Option<CreateCustomKnowledgeBaseDto>,
    pub knowledge_base_id: Option<String>,
    pub provider: ProviderTrue,
    pub metadata_send_mode: Option<MetadataSendModeTrue>,
    pub url: String,
    pub timeout_seconds: Option<f64>,
    pub model: String,
    pub temperature: Option<f64>,
    pub max_tokens: Option<f64>,
    pub emotion_recognition_enabled: Option<bool>,
    pub num_fast_turns: Option<f64>,
}

Fields

messages: Option<Vec<OpenAiMessage>>

This is the starting state for the conversation.

tools: Option<Vec<AnyscaleModelToolsInner>>

These are the tools that the assistant can use during the call. To use existing tools, use toolIds. Both tools and toolIds can be used together.

tool_ids: Option<Vec<String>>

These are the tools that the assistant can use during the call. To use transient tools, use tools. Both tools and toolIds can be used together.

knowledge_base: Option<CreateCustomKnowledgeBaseDto>

knowledge_base_id: Option<String>

This is the ID of the knowledge base the model will use.

provider: ProviderTrue

This is the provider that will be used for the model. Any service, including your own server, that is compatible with the OpenAI API can be used.

metadata_send_mode: Option<MetadataSendModeTrue>

This determines whether metadata is sent in requests to the custom provider. - off will not send any metadata. payload will look like { messages } - variable will send assistant.metadata as a variable on the payload. payload will look like { messages, metadata } - destructured will send assistant.metadata fields directly on the payload. payload will look like { messages, ...metadata } Further, variable and destructured will send call, phoneNumber, and customer objects in the payload. Default is variable.

url: String

These is the URL we’ll use for the OpenAI client’s baseURL. Ex. https://openrouter.ai/api/v1

timeout_seconds: Option<f64>

This sets the timeout for the connection to the custom provider without needing to stream any tokens back. Default is 20 seconds.

model: String

This is the name of the model. Ex. cognitivecomputations/dolphin-mixtral-8x7b

temperature: Option<f64>

This is the temperature that will be used for calls. Default is 0 to leverage caching for lower latency.

max_tokens: Option<f64>

This is the max number of tokens that the assistant will be allowed to generate in each turn of the conversation. Default is 250.

emotion_recognition_enabled: Option<bool>

This determines whether we detect user’s emotion while they speak and send it as an additional info to model. Default false because the model is usually are good at understanding the user’s emotion from text. @default false

num_fast_turns: Option<f64>

This sets how many turns at the start of the conversation to use a smaller, faster model from the same provider before switching to the primary model. Example, gpt-3.5-turbo if provider is openai. Default is 0. @default 0

Implementations

impl CustomLlmModel

pub fn new(provider: ProviderTrue, url: String, model: String) -> CustomLlmModel

Trait Implementations

impl Clone for CustomLlmModel

fn clone(&self) -> CustomLlmModel

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CustomLlmModel

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CustomLlmModel

fn default() -> CustomLlmModel

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for CustomLlmModel

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for CustomLlmModel

fn eq(&self, other: &CustomLlmModel) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for CustomLlmModel

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for CustomLlmModel