Struct piston_meta::tokenizer::TokenizerState
source · pub struct TokenizerState(pub usize);
Expand description
Stores the number of tokens received.
Tuple Fields§
§0: usize
Implementations§
source§impl TokenizerState
impl TokenizerState
sourcepub fn new() -> TokenizerState
pub fn new() -> TokenizerState
Creates a new tokenizer state.
Trait Implementations§
source§impl Clone for TokenizerState
impl Clone for TokenizerState
source§fn clone(&self) -> TokenizerState
fn clone(&self) -> TokenizerState
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moresource§impl Debug for TokenizerState
impl Debug for TokenizerState
source§impl Ord for TokenizerState
impl Ord for TokenizerState
source§fn cmp(&self, other: &TokenizerState) -> Ordering
fn cmp(&self, other: &TokenizerState) -> Ordering
1.21.0 · source§fn max(self, other: Self) -> Selfwhere
Self: Sized,
fn max(self, other: Self) -> Selfwhere
Self: Sized,
Compares and returns the maximum of two values. Read more
source§impl PartialEq<TokenizerState> for TokenizerState
impl PartialEq<TokenizerState> for TokenizerState
source§fn eq(&self, other: &TokenizerState) -> bool
fn eq(&self, other: &TokenizerState) -> bool
source§impl PartialOrd<TokenizerState> for TokenizerState
impl PartialOrd<TokenizerState> for TokenizerState
source§fn partial_cmp(&self, other: &TokenizerState) -> Option<Ordering>
fn partial_cmp(&self, other: &TokenizerState) -> Option<Ordering>
1.0.0 · source§fn le(&self, other: &Rhs) -> bool
fn le(&self, other: &Rhs) -> bool
This method tests less than or equal to (for
self
and other
) and is used by the <=
operator. Read more