Enum TokenField 

pub enum TokenField {
    Str(String),
    Obj(Map<String, Value>),
}

A special-token field: either a bare string, or an AddedToken object with a content key.

Variants

Str(String)

"bos_token": "<s>".

Obj(Map<String, Value>)

"bos_token": {"content": "<s>", "lstrip": false, …}.

Implementations

impl TokenField

pub fn as_str(&self) -> Option<&str>

The token’s string value (content for the object form).

Trait Implementations

impl Clone for TokenField

fn clone(&self) -> TokenField

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenField

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for TokenField

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for TokenField

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.