Struct Encoding 

pub struct Encoding {
    pub ids: Vec<TokenId>,
    pub attention_mask: Vec<u32>,
}

The result of encoding a single sequence with an EncodeConfig.

Fields

ids: Vec<TokenId>

The token ids, including any special/padding tokens.

attention_mask: Vec<u32>

A mask that is 1 for real (or special) tokens and 0 for padding.

Implementations

impl Encoding

pub fn len(&self) -> usize

Length of the (possibly padded) sequence.

pub fn is_empty(&self) -> bool

Whether the encoding is empty.

Trait Implementations

impl Clone for Encoding

fn clone(&self) -> Encoding

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Encoding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for Encoding

impl PartialEq for Encoding

fn eq(&self, other: &Encoding) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Encoding