Enum TiktokenError 

#[non_exhaustive]
pub enum TiktokenError {
    Load {
        encoding_name: &'static str,
        message: String,
    },
}

Available on crate feature tokenizer-tiktoken only.

Errors raised when constructing a TiktokenCounter.

tiktoken-rs returns Box<dyn Error> from its loader functions; this type strips the upstream chain to a String so the error stays Send + Sync + 'static for ergonomic cross-thread propagation (downstream operators map this onto entelix_core::Error::config).

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Load

Loading the BPE tables for the requested encoding failed. In practice the upstream loaders only fail if the embedded merge / vocab tables fail to parse, which would indicate an upstream packaging bug. Variant shape mirrors HfTokenizerError::Load for cross-companion consistency.

Fields

encoding_name: &'static str

Canonical encoding name the load was attempted for (e.g. "cl100k_base").

message: String

Upstream tiktoken-rs error message (chain stripped).

Trait Implementations

impl Debug for TiktokenError

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Display for TiktokenError

fn fmt(&self, __formatter: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Error for TiktokenError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0:

use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0:

replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.