Crate tl 

rustedbytes-tl

A lightweight HTML parser written in Rust. It borrows input bytes and offers allocation-free bounded parsing by default, or heap-backed DOMs and CSS queries with std. The Rust crate is imported as tl.

This README describes the current Git source. The source contains breaking changes since the published 0.2.0 API; the first release containing them must be 0.3.x or later. See Cargo.toml for the version selected for the next release. The published 0.2.0 API reference does not describe all APIs below. Generate current API docs with cargo doc --features entities --open.

Getting started

Use the current source with heap-backed parsing:

[dependencies]
rustedbytes-tl = { git = "https://github.com/RustedBytes/rustedbytes-tl", features = ["std"] }

Commit your application’s Cargo.lock to retain the resolved Git revision.

let html = "<ul><li><a href='/item'>Item</a></li></ul>";
let dom = tl::parse(html, tl::ParserOptions::default()).unwrap();
let handle = dom.query_selector("ul > li a[href]").unwrap().next().unwrap();
let node = handle.get(dom.parser()).unwrap();
assert_eq!(node.inner_text(dom.parser()), "Item");
let tag = node.as_tag().unwrap();
let href = tag.attributes().get("href").unwrap().unwrap();
assert_eq!(href.as_bytes(), b"/item");

Keep html alive for every use of the borrowed DOM. If the DOM must own its input, use the safe tl::VDomGuard::parse(String, ParserOptions) constructor. See the usage guide for owned input, attributes, decoded text, query errors and bounded parsing.

Features

Feature	Behavior
Default (none)	no_std, no allocation; explicit capacities in parse; bounded simple selectors
std	Heap-backed storage, owned DOMs, text extraction, structural CSS selectors
entities	Implies std; adds character-reference decoding through decoded_inner_text
portable-simd	Requires nightly Rust; optional SIMD parsing path, combinable with the above

__INTERNALS_DO_NOT_USE exposes instrumentation internals and is not a supported application feature. Stable builds should select features explicitly rather than use --all-features, which enables nightly-only portable-simd.

Scope and documentation

The parser does not implement the complete HTML5 tree-building algorithm. Malformed HTML can produce a different tree from a browser; omitted tbody elements are not inserted. CSS support is a subset, and text extraction does not implement browser layout or visibility rules. HTML parsing and text extraction do not sanitize untrusted content.

• Usage guide: executable examples and API behavior.
• Development guide: feature checks, docs, benchmarks and troubleshooting.
• Migration and silkworm compatibility: supported selectors and remaining compatibility work.
• Rust audit: safety fixes and measured performance, with methodology and limits.

Provenance and license

Forked from astral-tl and extended for bounded no_std parsing and RustedBytes consumers. Licensed under MIT; see LICENSE.

Usage guide

These examples target the current Git source. Import the package rustedbytes-tl as tl. Enable std for heap-backed parsing, and entities for decoded text; the latter already enables std.

Borrowed and owned input

tl::parse(&str, ParserOptions) borrows the input. Node handles must be resolved against the parser that produced them; a handle is not an independently owned node or a globally unique identifier. Do not use handles from another DOM.

For input obtained as a String, the guard keeps that input alive:

let guard = tl::VDomGuard::parse(
    String::from("<p id='message'>Hello</p>"),
    tl::ParserOptions::default(),
).unwrap();
let dom = guard.get_ref();
let handle = dom.get_element_by_id("message").unwrap();
assert_eq!(handle.get(dom.parser()).unwrap().inner_text(dom.parser()), "Hello");

DOM references and cloned nodes obtained from the guard cannot outlive it. get_mut_ref also returns a shared DOM reference; its name does not grant DOM mutation. The legacy unsafe parse_owned entry point remains available, but new code should use VDomGuard::parse.

ParserOptions::default() does not build ID/class indexes. With std, get_element_by_id and get_elements_by_class_name still work by scanning. Use ParserOptions::default().track_ids().track_classes() when repeated lookups justify building indexes. CSS query support does not require these flags.

Queries and attributes

With std, query_selector returns None for invalid or unsupported syntax. A valid query returns an iterator, which can be empty:

let dom = tl::parse("<button disabled title='Save'>Save</button>", Default::default()).unwrap();
assert!(dom.query_selector("button:unsupported").is_none());
assert_eq!(dom.query_selector("article").unwrap().count(), 0);
let button = dom.query_selector("button[disabled]").unwrap().next().unwrap();
let attributes = button.get(dom.parser()).unwrap().as_tag().unwrap().attributes();
assert!(attributes.get("missing").is_none());
assert!(matches!(attributes.get("disabled"), Some(None)));
assert_eq!(attributes.get("title").unwrap().unwrap().as_bytes(), b"Save");

The nested attribute result distinguishes missing attributes (None), present attributes without a value (Some(None)) and values (Some(Some(bytes))). Attributes retain raw character references. Bytes::as_bytes() gives bytes; try_as_utf8_str() validates UTF-8 and as_utf8_str() (with std) converts lossily. Initial spans borrow input; explicitly replacing bytes can own data.

Supported std queries include type, ID, class, universal, attribute presence and value matching (=, ~=, ^=, $=, *=), compound selectors, comma lists, descendant, child and sibling combinators, :first-child, :nth-child(an+b), :not(...), and descendant-relative :has(...). Element positions ignore text and comments. Type names match ASCII-insensitively. CSS escapes, namespaces, attribute case flags, other pseudo-classes and :has(> a) are unsupported. Recursive nesting and compound size are bounded; see the compatibility notes.

Structural queries build indexes proportional to DOM node count. :has can scan many candidates; it is not guaranteed to take linear time. Calling Selector::matches directly lacks the structural context supplied by the DOM query iterator.

Raw and decoded text

inner_text concatenates descendant text, ignores comments and leaves entities encoded. It does not insert layout spaces or filter hidden elements. With entities, opt into decoding:

let dom = tl::parse("<p>A &amp; B &#x1F980;</p>", Default::default()).unwrap();
let handle = dom.query_selector("p").unwrap().next().unwrap();
let node = handle.get(dom.parser()).unwrap();
assert_eq!(node.inner_text(dom.parser()), "A &amp; B &#x1F980;");
assert_eq!(node.decoded_inner_text(dom.parser()), "A & B 🦀");

Decoding is per text node: markup boundaries cannot form a new entity. Script/style tags preserve raw text; directly decoding a raw child uses ordinary text-context rules. Decoding does not change source bytes or attributes. Returned text is Cow<str> and may borrow or allocate depending on the content and tree. Do not assume every extraction is allocation-free.

Bounded parsing without std

Without std, parse takes six capacities in this order: nodes, parser stack, root nodes, tracked IDs, tracked classes and selector nodes. Include text and comment nodes in the node budget. Capacities also affect stack/storage size; choose them for your input rather than copying large arbitrary bounds.

let dom = tl::parse::<32, 16, 8, 8, 8, 16>(
    "<p id='message'>Hello</p>", tl::ParserOptions::default(),
).unwrap();
assert!(dom.get_element_by_id("message").is_some());
assert_eq!(dom.query_selector("p").unwrap().count(), 1);

Bounded selectors support simple matching, not the structural CSS engine available with std. Here query_selector returns Result, rather than the std API’s Option. The bounded parser also reports invalid or unsupported selector syntax as SelectorCapacityExceeded; that error does not prove that increasing capacity will make the query valid.

Insufficient storage returns ParseError: node, stack, root, ID, class and selector capacity errors identify those budgets. Without std, each tag can store 8 ordinary attributes (ID and class are stored separately) and 256 direct children. These fixed bounds are not changed by the six parse capacities. InvalidLength rejects input too large for the parser’s u32 spans. These errors are not HTML conformance validation. For no-allocation output, use write_outer_html with a caller-provided core::fmt::Write sink.

Updating consumers

The current source adds public Selector variants, changes bounded parsing APIs and makes InlineVec::inline_parts_mut and InlineHashMap::inline_parts_mut unsafe. Exhaustive enum matches and callers of raw mutable storage need review. Prefer safe collection methods such as get_mut, as_mut_slice (vector) and insert (map). Do not add an unsafe block without meeting the documented initialized-prefix and ownership invariants. The crate version has not yet been bumped; review the Git revision you consume.

Re-exports

pub use errors::ParseError;

Modules

errors

Errors that occur throughout the crate

inline

Inline data structures

queryselector

Query selector API

Structs

Attributes

Stores all attributes of an HTML tag, as well as additional metadata such as id and class

Bytes

A storage type for raw bytes, used by the parser

Children

A thin wrapper around the children of HTMLTag

ChildrenMut

A thin mutable wrapper around the children of HTMLTag

HTMLTag

Represents a single HTML element

NodeHandle

A detached, external handle to a HTML node, originally obtained from a Parser

Parser

The main HTML parser

ParserOptions

Options for the HTML Parser

VDom

VDom represents a Document Object Model

Enums

HTMLVersion

HTML Version ()

Node

An HTML Node

Functions

parse

Parses the given input string using bounded, allocation-free storage.

parse_query_selector

Parses a query selector using bounded, allocation-free storage.

Type Aliases

ClassVec

Inline class vector

InnerNodeHandle

The inner type of a NodeHandle, used to represent an index into the tags table

RawAttributesMap

The type of map for “raw” attributes

RawChildren

The type of vector for children of an HTML tag

Tree

A list of HTML nodes