pub struct HeuristicTokenCounter { /* private fields */ }Expand description
Estimates token cost from UTF-8 lengths, rounded up per text-bearing part. Tool calls count name bytes plus serialized argument bytes. Attachments and message overhead have fixed costs.
use rig_core::completion::Message;
use rig_memory::HeuristicTokenCounter;
let counter = HeuristicTokenCounter::default();
assert_eq!(counter.count(&Message::user("12345678")), 2 + 4);Implementations§
Source§impl HeuristicTokenCounter
impl HeuristicTokenCounter
Sourcepub fn count_tool(&self, tool: &ToolDefinition) -> usize
pub fn count_tool(&self, tool: &ToolDefinition) -> usize
About how many tokens the definition of a tool costs as a request sends it: its name, description and parameters’ JSON.
Trait Implementations§
Source§impl Clone for HeuristicTokenCounter
impl Clone for HeuristicTokenCounter
impl Copy for HeuristicTokenCounter
Source§impl Debug for HeuristicTokenCounter
impl Debug for HeuristicTokenCounter
Source§impl Default for HeuristicTokenCounter
impl Default for HeuristicTokenCounter
Source§fn default() -> Self
fn default() -> Self
Uses four bytes per token, four tokens per message, and
ATTACHMENT_TOKENS per attachment.
Auto Trait Implementations§
impl Forget for HeuristicTokenCounter
impl Freeze for HeuristicTokenCounter
impl Move for HeuristicTokenCounter
impl RefUnwindSafe for HeuristicTokenCounter
impl Send for HeuristicTokenCounter
impl Sync for HeuristicTokenCounter
impl Unpin for HeuristicTokenCounter
impl UnsafeUnpin for HeuristicTokenCounter
impl UnwindSafe for HeuristicTokenCounter
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more