logos-0.7.6 doesn't have any documentation.
Create ridiculously fast Lexers.
Logos works by:
- Resolving all logical branching of token definitions into a tree.
- Optimizing complex patterns into Lookup Tables.
- Always using a Lookup Table for the first byte of a token.
- Producing code that never backtracks, thus running at linear time or close to it.
In practice it means that for most grammars the lexing performance is virtually unaffected by the number
of tokens defined in the grammar. Or, in other words, it is really fast.
Example
extern crate logos;
use logos::Logos;
#[derive(Logos, Debug, PartialEq)]
enum Token {
#[end]
End,
#[error]
Error,
#[token = "fast"]
Fast,
#[token = "."]
Period,
#[regex = "[a-zA-Z]+"]
Text,
}
fn main() {
let mut lexer = Token::lexer("Create ridiculously fast Lexers.");
assert_eq!(lexer.token, Token::Text);
assert_eq!(lexer.slice(), "Create");
assert_eq!(lexer.range(), 0..6);
lexer.advance();
assert_eq!(lexer.token, Token::Text);
assert_eq!(lexer.slice(), "ridiculously");
assert_eq!(lexer.range(), 7..19);
lexer.advance();
assert_eq!(lexer.token, Token::Fast);
assert_eq!(lexer.slice(), "fast");
assert_eq!(lexer.range(), 20..24);
lexer.advance();
assert_eq!(lexer.token, Token::Text);
assert_eq!(lexer.slice(), "Lexers");
assert_eq!(lexer.range(), 25..31);
lexer.advance();
assert_eq!(lexer.token, Token::Period);
assert_eq!(lexer.slice(), ".");
assert_eq!(lexer.range(), 31..32);
lexer.advance();
assert_eq!(lexer.token, Token::End);
}