[−][src]Crate ress
ress A crate for parsing raw JS into a token stream
The primary interfaces are the function tokenize
and
the struct Scanner
. The Scanner
struct impls Iterator
and the tokenize
function is just a wrapper
around Scanner::collect()
.
The Scanner
will provide a stream of Item
s, and Item
is
has 2 properties a Token
and a Span
. The Span
is a
representation of where the Item
exists in the original source while the Token
provides details about what JavaScript token it represents.
An example of what a token stream might look like
Re-exports
pub use crate::tokens::owned::Comment as OwnedComment; |
pub use crate::tokens::owned::Ident as OwnedIdent; |
pub use crate::tokens::owned::Number as OwnedNumber; |
pub use crate::tokens::owned::RegEx as OwnedRegEx; |
pub use crate::tokens::owned::StringLit as OwnedStringLit; |
pub use crate::tokens::owned::Template as OwnedTemplate; |
pub use crate::tokens::owned::Token as OwnedToken; |
pub use crate::tokens::refs::Comment; |
pub use crate::tokens::refs::Ident; |
pub use crate::tokens::refs::Number; |
pub use crate::tokens::refs::RegEx; |
pub use crate::tokens::refs::StringLit; |
pub use crate::tokens::refs::Template; |
pub use crate::tokens::refs::Token as RefToken; |
pub use crate::tokens::BooleanLiteral as Boolean; |
pub use crate::tokens::Keyword; |
pub use crate::tokens::Punct; |
pub use crate::tokens::Token; |
Modules
error | |
tokens |
Structs
Item | |
Position | |
Scanner | |
ScannerState | |
SourceLocation | |
Span | A location in the original source text |
Tokenizer |
Enums
OpenCurlyKind |
Functions
tokenize | a convince function for collecting a scanner into
a |