Skip to main content

TokenCounter

traitclaw

Trait TokenCounter 

Source 
pub trait TokenCounter: Send + Sync {
    // Required methods
    fn count_messages(&self, messages: &[Message]) -> usize;
    fn count_str(&self, text: &str) -> usize;
}
Expand description

Trait for pluggable token counting backends.

Implement this trait to provide accurate token counting for specific models (e.g., tiktoken for OpenAI models).

Required Methods§

Source

fn count_messages(&self, messages: &[Message]) -> usize

Count tokens in a message list.

Source

fn count_str(&self, text: &str) -> usize

Count tokens in a single string.

Implementors§