Lexer
The lexical analysis component of Tusk.
About
This crate provides the Lexer
and Token
implementations used in Tusk. It allows you to provide a &str
of input and stream Token
instances on demand.
Usage
To use the crate, first add it to your Cargo.toml
:
[]
= "0.2.*"
To create a new Lexer
, import the struct
and use the Lexer::new()
method.
use Lexer;
To get the next token from the input, use the Lexer::next()
method:
use Lexer;
This method returns a Token
. This struct has 3 fields:
Contributing
For more information, please read the CONTRIBUTING document.
License
This repository is distributed under the MIT license. For more information, please read the LICENSE document.