Crate ress

source ·
Expand description

js_parse A crate for parsing raw JS into a token stream

The primary interfaces are the function tokenize and the struct Scanner. The Scanner struct impls Iterator and the tokenize function is just a wrapper around Scanner::collect().

The Scanner will provide a stream of Items, and Item is has 2 properties a Token and a Span. The Span is a representation of where the Item exists in the original source while the Token provides details about what JavaScript token it represents.

An example of what a token stream might look like

Modules

Structs

An identifier
A wrapper around a token that will include the byte span of the text that it was found at
An iterator over a js token stream
A location in the original source text

Enums

The tokenized representation of true or false
A JS Keyword
The representation of a single JS token

Functions

a convince function for collecting a scanner into a Vec<Token>