Trait Tokenizable

pub trait Tokenizable {
    // Required method
    fn tokenize<RNG: RngCore>(&self, rng: &mut RNG) -> Result<Vec<Token<'_>>>;

    // Provided method
    fn tokenze_no_rng(&self) -> Result<Vec<Token<'_>>> { ... }
}

Any object that can be transformed into a token stream

Required Methods

fn tokenize<RNG: RngCore>(&self, rng: &mut RNG) -> Result<Vec<Token<'_>>>

Creates a token stream with respect to self, lifetime should match self’s lifetime

Provided Methods

fn tokenze_no_rng(&self) -> Result<Vec<Token<'_>>>

Creates a token stream with respect to self, lifetime should match self’s lifetime with no rng

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl<STR> Tokenizable for STR

where STR: AsRef<str>,