lexers 0.0.8

Various string tokenizers
Documentation

text tokenizers

Example: get a sequence of lexers::MathToken with MathTokenizer::from_str("sqrt(-(1-x^2) / (1 + x^2))");