Enum TokenCounting 

pub enum TokenCounting {
    None,
    Local,
    Endpoint,
}

How a model can count the tokens a request costs, when it can.

LanguageModel::token_counting reports which kind of counting a model offers: Local counts in-process against a published vocabulary and is cheap enough to run on every assembly; Endpoint counts exactly over a network call and is worth its round trip only near a threshold.

Variants

None

No counting: the caller falls back to its estimate.

Local

In-process counting against an embedded vocabulary.

Endpoint

Counting through a provider endpoint.

Trait Implementations

impl Clone for TokenCounting

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for TokenCounting

impl Debug for TokenCounting

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenCounting

fn default() -> Self

Returns the “default value” for a type.

impl Eq for TokenCounting

impl PartialEq for TokenCounting

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenCounting