Struct ArenaFactory 

pub struct ArenaFactory<'input, TF, T> { /* private fields */ }

This is a wrapper for Token factory that allows to allocate tokens in separate arena. It can allow to significantly improve performance by passing Tokens by references everywhere.

Requires &'a Tok: Default bound to produce invalid tokens, which can be easily implemented like this:

lazy_static!{ static ref INVALID_TOKEN:Box<CustomToken> = ... }
impl Default for &'_ CustomToken {
    fn default() -> Self { &**INVALID_TOKEN }
}

or if possible just

const INVALID_TOKEN:CustomToken = ...

Trait Implementations

impl<'input, TF: Debug, T> Debug for ArenaFactory<'input, TF, T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'input, TF, T> Default for ArenaFactory<'input, TF, T>

where TF: Default,

fn default() -> Self

Returns the “default value” for a type.

impl<'input, TF, Tok> TokenFactory<'input> for ArenaFactory<'input, TF, Tok>

where TF: TokenFactory<'input, Tok = Box<Tok>, Inner = Tok>, Tok: Token<Data = TF::Data> + Clone + TidAble<'input>, for<'a> &'a Tok: Default,

type Inner = Tok

Type of tokens emitted by this factory.

type Tok = &'input Tok

Ownership of the emitted tokens

type Data = <TF as TokenFactory<'input>>::Data

Type of the underlying storage

type From = <TF as TokenFactory<'input>>::From

Type of the CharStream that factory can produce tokens from

fn create<T>( &'input self, source: Option<&mut T>, ttype: isize, text: Option<<Self::Data as ToOwned>::Owned>, channel: isize, start: isize, stop: isize, line: isize, column: isize, ) -> Self::Tok

where T: CharStream<Self::From> + ?Sized,

Creates token either from sourse or from pure data in text Either source or text are not None

fn create_invalid() -> &'input Tok

Creates invalid token Invalid tokens must have TOKEN_INVALID_TYPE token type.

fn get_data(from: Self::From) -> Cow<'input, Self::Data>

Creates Self::Data representation for from for lexer to work with when it does not need to create full token

impl<'input, TF: TidAble<'input>, T: TidAble<'input>> TidAble<'input> for ArenaFactory<'input, TF, T>