Function lexical_scanner::lexer_as_str
source · [−]Expand description
Constructs a vector of tokens. This straight forward lexical scanner is preset to support over 75 tokens. The list of tokens can be found at this sites github page. Usually, passing in a text string is used for debugging and testing. Example
pub use lexical_scanner::*;
pub use enums::*;
let text = "The number 5.0 is > 1;";
let token_list = lexical_scanner::lexer_as_str(text);