Struct ChatRequest

pub struct ChatRequest {

    pub messages: Vec<ChatMessage>,
    pub system: Option<String>,
    pub model: Option<String>,
    pub frequency_penalty: Option<f32>,
    pub logit_bias: Option<BTreeMap<usize, f32>>,
    pub logprobs: Option<bool>,
    pub top_logprobs: Option<u8>,
    pub max_tokens: Option<u32>,
    pub n: Option<u32>,
    pub presence_penalty: Option<f32>,
    pub response_format: Option<Value>,
    pub seed: Option<i64>,
    pub stop: Vec<String>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub tools: Vec<Tool>,
    pub tool_choice: Option<Value>,
    pub user: Option<String>,
    pub stream: bool,
    pub stream_options: Option<StreamOptions>,
}

The request that can be submitted to the proxy, for transformation and submission to a provider.

Fields

messages: Vec<ChatMessage>

The messages in the chat so far.

system: Option<String>

A separate field for system message as an alternative to specifying it in messages.

model: Option<String>

The model to use. This can be omitted if the proxy options specify a model.

frequency_penalty: Option<f32>

How to penalize tokens based on their frequency in the text so far

logit_bias: Option<BTreeMap<usize, f32>>

Specific control of certain token probabilities

logprobs: Option<bool>

Return the logprobs of the generated tokens

top_logprobs: Option<u8>

If logprobs is true, how many logprobs to return per token.

max_tokens: Option<u32>

max_tokens is optional for some providers but you should include it. We don’t require it here for compatibility when wrapping other libraries that may not be aware they are using Chronicle.

n: Option<u32>

Generate multiple chat completions concurrently. Not every model provider supports this.

presence_penalty: Option<f32>

How to penalize tokens based on their existing presence in the text so far

response_format: Option<Value>

Force JSON output

seed: Option<i64>

A random seed to use when generating the response.

stop: Vec<String>

Tell the model to stop when it encounters these token sequences

temperature: Option<f32>

Temperature to use when generating the response

top_p: Option<f32>

Customize the top-P probability of tokens to consider when generating the response

tools: Vec<Tool>

Tools available for the model to use

tool_choice: Option<Value>

Customize how the model chooses tools

user: Option<String>

The “user” to send to the provider.

stream: bool

Send the response back as a stream of chunks.

stream_options: Option<StreamOptions>

For OpenAI, this lets us enable usage when streaming. Chronicle will set this automatically when appropriate.

Implementations

impl ChatRequest

pub fn transform(&mut self, options: &ChatRequestTransformation<'_>)

Transform a chat request to fit different variations on the OpenAI format.

pub fn merge_message_names(&mut self)

For providers that don’t support a name field in their message, convert messages with names to the format “{name}: {content}

pub fn move_system_to_messages(&mut self)

Move the entry in the system field to the start of messages.

pub fn move_system_message_to_top_level(&mut self)

Move a system role ChatMessage to the system field

Trait Implementations

impl Clone for ChatRequest

fn clone(&self) -> ChatRequest

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChatRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ChatRequest

fn default() -> ChatRequest

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ChatRequest

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ChatRequest

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.