Enum TokenEncoding 

pub enum TokenEncoding {
    Cl100kBase,
    O200kBase,
}

Supported encoding types for token counting

Variants

Cl100kBase

cl100k_base - GPT-4, GPT-4-turbo, text-embedding-3-*

O200kBase

o200k_base - GPT-4o, GPT-4o-mini

Implementations

impl TokenEncoding

pub fn as_str(&self) -> &'static str

Trait Implementations

impl Clone for TokenEncoding

fn clone(&self) -> TokenEncoding

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenEncoding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenEncoding

fn eq(&self, other: &TokenEncoding) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenEncoding

impl Eq for TokenEncoding

impl StructuralPartialEq for TokenEncoding