pub struct TokenizationComparison {
pub tiktoken_count: usize,
pub legacy_count: usize,
pub accuracy_ratio: f64,
pub improvement: Option<f64>,
}
Expand description
Comparison between tiktoken and legacy tokenization
Fields§
§tiktoken_count: usize
§legacy_count: usize
§accuracy_ratio: f64
§improvement: Option<f64>
Implementations§
Trait Implementations§
Source§impl Clone for TokenizationComparison
impl Clone for TokenizationComparison
Source§fn clone(&self) -> TokenizationComparison
fn clone(&self) -> TokenizationComparison
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moreAuto Trait Implementations§
impl Freeze for TokenizationComparison
impl RefUnwindSafe for TokenizationComparison
impl Send for TokenizationComparison
impl Sync for TokenizationComparison
impl Unpin for TokenizationComparison
impl UnwindSafe for TokenizationComparison
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more