Skip to main content

AnyToken

Type Alias AnyToken 

Source
pub type AnyToken<'a> = TypedToken<'a, AnyGrammar>;
Expand description

Token alias for grammar-independent tokenization pipelines.

Aliased Type§

pub struct AnyToken<'a> { /* private fields */ }