Enum Special 

pub enum Special {
    Tokenize,
    Plaintext,
}

👎Deprecated since 0.1.0: This enum is a mixture of options for llama cpp providing less flexibility it only used with deprecated methods and will be removed in the future.

How to determine if we should tokenize special tokens

Variants

Tokenize

👎Deprecated since 0.1.0: This enum is a mixture of options for llama cpp providing less flexibility it only used with deprecated methods and will be removed in the future.

Allow tokenizing special and/or control tokens which otherwise are not exposed and treated as plaintext. Does not insert a leading space.

Plaintext

👎Deprecated since 0.1.0: This enum is a mixture of options for llama cpp providing less flexibility it only used with deprecated methods and will be removed in the future.

Treat special and/or control tokens as plaintext.

Trait Implementations

impl Clone for Special

fn clone(&self) -> Special

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Special

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Special

fn eq(&self, other: &Special) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for Special

impl Eq for Special

impl StructuralPartialEq for Special