Struct TokenCounter 

pub struct TokenCounter { /* private fields */ }

Token counter for various models with caching

Implementations

impl TokenCounter

pub fn new(model_name: &str) -> Result<Self>

Create a new token counter for the given model with caching

pub fn count_tokens(&self, text: &str) -> usize

Count tokens in the given text with caching

pub fn estimate_chat_tokens( &self, prompt: &str, system_prompt: Option<&str>, history: &[ChatEntry], ) -> usize

Estimate tokens for a chat request including system prompt and history

pub fn exceeds_context_limit( &self, prompt: &str, system_prompt: Option<&str>, history: &[ChatEntry], context_limit: u32, ) -> bool

Check if the estimated tokens exceed the context limit

pub fn truncate_to_fit( &self, prompt: &str, system_prompt: Option<&str>, history: &[ChatEntry], context_limit: u32, max_output_tokens: Option<u32>, ) -> (String, Vec<ChatEntry>)

Truncate input to fit within context limit