Trait TokenEstimator 

pub trait TokenEstimator {
    // Required methods
    fn info(&self) -> EstimatorInfo;
    fn count_bytes(&self, bytes: &[u8]) -> usize;
}

Required Methods

fn info(&self) -> EstimatorInfo

Estimator provenance for CompressionReport.estimator.

fn count_bytes(&self, bytes: &[u8]) -> usize

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl TokenEstimator for ByteHeuristicEstimator

impl TokenEstimator for TiktokenEstimator

Available on crate feature tiktoken only.