Crate erl_tokenize [−] [src]
Erlang source code tokenizer.
Examples
Tokenizes the Erlang code io:format("Hello").
:
use erl_tokenize::Tokenizer; let src = r#"io:format("Hello")."#; let tokenizer = Tokenizer::new(src); let tokens = tokenizer.collect::<Result<Vec<_>, _>>().unwrap(); assert_eq!(tokens.iter().map(|t| t.text()).collect::<Vec<_>>(), ["io", ":", "format", "(", r#""Hello""#, ")", "."]);
References
- erl_scan module
- Erlang Data Types
Modules
tokens |
Tokens. |
values |
Token values. |
Structs
Error |
This crate specific error type. |
Lexer |
Lexer. |
Position |
Position of token. |
Tokenizer |
Tokenizer. |
Enums
ErrorKind |
The list of the possible error kinds |
HiddenToken |
Hidden token. |
LexicalToken |
Lexical token. |
Token |
Token. |
Traits
PositionRange |
This trait allows to get a (half-open) range where the subject is located. |
Type Definitions
Result |
This crate specific |