Expand description
HTML → denoised Markdown units.
extract is the whole parsing story: parse once with tl, index the
arena, pick the content root, and write Markdown while dropping boilerplate.
It never touches the network and never allocates a second copy of the
document, so it is cheap enough to run over a whole result set in parallel
via extract_many.
Structs§
- Article
- A cleaned document.
- Extract
Options - Knobs for
extract. - Link
- One harvested link.
- Meta
- Everything we can learn about a page without reading its body.
- Render
Options - Options that change what ends up in the Markdown.
- Unit
- One addressable block of an article.
Enums§
- Article
Kind - Whether a page turned out to be an article or a listing.
- Index
Mode - When to fall back to index-page extraction.
- Unit
Kind - What kind of block a
Unitcame from.
Functions§
- extract
- Turn a raw HTML document into denoised Markdown units.
- extract_
many - Extract many documents in parallel across the rayon pool.
- extract_
with extractwith explicit options.