Enum CacheHint 

#[non_exhaustive]
pub enum CacheHint {
    None,
    System,
    Prefix {
        messages: usize,
    },
}

A hint that a prefix of the prompt is worth caching provider-side.

Only honoured when the profile declares prompt_caching; an adapter that cannot cache ignores it silently, because caching changes cost and latency but never meaning.

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

None

No hint.

System

Cache the system prompt.

Prefix

Cache the system prompt and the first messages messages.

Fields

messages: usize

How many leading messages belong to the stable prefix.

Trait Implementations

impl Clone for CacheHint

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CacheHint

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CacheHint

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for CacheHint

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for CacheHint

impl Hash for CacheHint

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for CacheHint

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for CacheHint

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for CacheHint