Enum BpeError 

pub enum BpeError {
    InvalidMergeEntry(String),
    InvalidVocabEntry(String),
    MissingVocabEntry(String),
}

Errors that can occur when building a Bpe tokenizer or encoding or decoding text using it.

Variants

InvalidMergeEntry(String)

There was an invalid entry in the merge list. This means that either the entry doesn’t have the expected <token> [SPACE] <token> format or the <token> is not either a single character or the concatenation of another pair in the merge list.

InvalidVocabEntry(String)

An entry in the vocab (token string to ID map) is not either a known special token or an entry in the merge list.

MissingVocabEntry(String)

An entry was not found in the vocabulary.

Trait Implementations

impl Debug for BpeError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for BpeError

fn fmt(&self, fmt: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Error for BpeError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0: use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0: replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.