[−][src]Crate ress
ress A crate for parsing raw JS into a token stream
The primary interfaces are the function tokenize and
the struct Scanner. The Scanner struct impls Iterator
and the tokenize function is just a wrapper
around Scanner::collect().
The Scanner will provide a stream of Items, and Item is
has 2 properties a Token and a Span. The Span is a
representation of where the Item exists in the original source while the Token
provides details about what JavaScript token it represents.
An example of what a token stream might look like
Re-exports
pub use crate::tokens::owned::Comment as OwnedComment; |
pub use crate::tokens::owned::Ident as OwnedIdent; |
pub use crate::tokens::owned::Number as OwnedNumber; |
pub use crate::tokens::owned::RegEx as OwnedRegEx; |
pub use crate::tokens::owned::StringLit as OwnedStringLit; |
pub use crate::tokens::owned::Template as OwnedTemplate; |
pub use crate::tokens::owned::Token as OwnedToken; |
pub use crate::tokens::refs::Comment; |
pub use crate::tokens::refs::Ident; |
pub use crate::tokens::refs::Number; |
pub use crate::tokens::refs::RegEx; |
pub use crate::tokens::refs::StringLit; |
pub use crate::tokens::refs::Template; |
pub use crate::tokens::refs::Token as RefToken; |
pub use crate::tokens::BooleanLiteral as Boolean; |
pub use crate::tokens::Keyword; |
pub use crate::tokens::Punct; |
pub use crate::tokens::Token; |
Modules
| error | |
| tokens |
Structs
| Item | |
| Scanner | |
| ScannerState | |
| Span | A location in the original source text |
| Tokenizer |
Enums
| OpenCurlyKind |
Functions
| tokenize | a convince function for collecting a scanner into
a |