Enum Strategy 

pub enum Strategy<M: BaseChatModel> {
    Truncate,
    Summarize {
        llm: Arc<M>,
        summary_prompt: String,
    },
}

Strategy for fitting messages within a token limit.

Variants

Truncate

Drop oldest messages to fit within the token limit. System messages are always preserved.

Summarize

Use an LLM to compress old messages into a summary system message.

Fields

llm: Arc<M>

The LLM used to generate summaries.

summary_prompt: String

Custom summary prompt. Must contain {conversation} placeholder.

Implementations

impl<M: BaseChatModel> Strategy<M>

pub fn summarize(llm: M) -> Self

Creates a new Summarize strategy with the given LLM and default prompt.

pub fn summarize_with_prompt(llm: M, prompt: impl Into<String>) -> Self

Creates a new Summarize strategy with a custom prompt.

The prompt must contain the {conversation} placeholder.

Trait Implementations

impl<M: Debug + BaseChatModel> Debug for Strategy<M>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.