Expand description
HTML → Markdown conversion, ported from upstream Scrapling’s
Response.markdown() (v0.4.15): turn pages into clean, LLM-ready
Markdown for RAG/ingestion pipelines.
The converter always removes noise (<script>, <style>, <noscript>,
<svg>) and content commonly abused for prompt injection: CSS-hidden
elements (display:none, visibility:hidden, opacity:0, zero
font-size/height/width), aria-hidden="true" elements, <template>
tags, HTML comments, zero-width Unicode characters, and ASCII control
characters — the same cleaning upstream applies. Beyond upstream, the
hidden HTML attribute also hides an element, and the document <head>
(including <title>) is never rendered: page titles belong in metadata,
not in the converted body.
Output conventions: ATX headings (#), **bold** / *italic*,
- bullets, 1. ordered lists, fenced code blocks, > blockquotes,
and pipe tables. Link and image URLs are kept exactly as they appear in
the document (no resolution against the base URL), matching upstream.
Functions§
- to_
markdown - Convert this node’s subtree to sanitized Markdown. See the module docs for what is stripped and the output conventions.