Skip to main content

Crate muskitty_html5_tokenizer

Crate muskitty_html5_tokenizer 

Source
Expand description

MusKitty HTML5 Tokenizer

Implements the tokenization stage of the WHATWG HTML parsing model (§13.2.5 Tokenization).

The tokenizer consumes a stream of Unicode code points and emits tokens (start tags, end tags, comments, characters, DOCTYPEs, EOF). These tokens are consumed by the tree construction stage to build the DOM tree.

§References

Structs§

DoctypeToken
A DOCTYPE token.
HtmlTokenizer
A concrete HTML tokenizer.
TagToken
A tag token (used for both start and end tags, distinguished by TagKind).

Enums§

ParseError
Parse errors defined by WHATWG §13.2.5.
State
Tokenizer states.
TagKind
Distinguishes start tags from end tags.
Token
A token emitted by the tokenizer.

Traits§

Tokenizer
The HTML tokenizer.