Skip to main content

TokenCount

boundgraph_slice::slicer

Trait TokenCount 

Source 
pub trait TokenCount:
    Send
    + Sync
    + Debug {
    // Required method
    fn count(&self, text: &str) -> usize;
}
Expand description

Turns text into a token count.

See the module documentation for why the default is pessimistic.

Required Methods§

Source

fn count(&self, text: &str) -> usize

How many tokens text is worth.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§