hl-lite
A tiny, dependency-free syntax highlighter for documentation sites.
hl-lite turns source text into a flat list of classified spans. Twelve
hand-written byte-level lexers, no regex engine, no C, no dependencies at
all, #![forbid(unsafe_code)] — about 48 KB of WebAssembly (19 KB
brotli) for all twelve languages, against the ~6 MB a tree-sitter grammar set
costs, and a sub-second cold build instead of ~73 CPU-seconds.
It is deliberately lexical, not scope-aware: the fidelity of highlight.js or Prism, not of a parser. That is what a fenced code block in a documentation page needs.
Usage
use ;
let lang = from_slug.expect;
let mut html = Stringnew;
for span in highlight
The whole API:
// Keyword -> "keyword"
;
Guarantees
- Concatenating every
Span::textreproduces the input byte for byte, so a renderer can never drop or duplicate source. - Spans never split a
char; non-ASCII input cannot panic. - Unterminated strings, comments, heredocs and templates run to the end of the input — no panic, no infinite loop.
- One forward pass per input: O(n).
- Adjacent spans of the same kind are merged.
Kind::Plaincovers whitespace and ordinary identifiers.
Languages
Lang::from_slug is case-insensitive and accepts the usual fence aliases.
| Language | Slugs | Notable limitations |
|---|---|---|
| Rust | rust, rs |
Any capitalised identifier is a Type, so SCREAMING_SNAKE constants are typed too. Lifetimes are Constant. Raw identifiers (r#type) are not special-cased. |
| Bash | bash, sh, shell, zsh, console |
${…} and $(…) are single Variable spans — not lexed inside. Heredoc bodies are one String, interpolations included. Only a short list of commands is highlighted, and only in command position. A $ prompt at the start of a line is handled; a # prompt is read as a comment, as in a real shell. |
| CSS | css |
url(…) contents are not lexed. At-rule preludes are looser than declarations: a name before : is a Property, everything else plain. |
| Dockerfile | dockerfile, docker |
The shell inside RUN is not lexed as Bash. RUN <<EOF heredocs are not recognised. |
| HTML / XML / SVG | html, xml, svg |
<script> and <style> bodies stay Plain — no recursion into JS or CSS. That is also what keeps a < b inside a script from opening a tag. |
| JavaScript | javascript, js, jsx, mjs, cjs |
${…} holes in template literals are Plain. Regex detection follows the previous significant token; a / that does not close on its line is division. JSX tags lex as operators and identifiers. Any capitalised identifier is a Type. |
| TypeScript | typescript, ts, tsx |
As JavaScript, plus type-level keywords and the primitive type names. |
| JSON | json, jsonc, json5 |
Tolerates // and /* */ comments and trailing commas; it highlights, it does not validate. |
| Markdown | markdown, md, mdx |
Fenced blocks are one String and are never lexed as their language — the renderer splits fences out first. Tables, setext headings, reference links and indented code blocks stay plain. A --- frontmatter fence reads as a thematic break and the frontmatter body is lexed as Markdown. |
| Python | python, py, py3 |
f-string holes stay inside the String. Decorators are recognised at the start of a line. Only class/def names become Type/Function; other capitalised names stay plain. The builtin list is short. |
| TOML | toml |
Dotted keys are emitted as Property . Property. Date-times are recognised by shape and emitted as Number. |
| YAML | yaml, yml |
Plain scalars stay Plain (which is correct, if sparse). A , inside a block-context scalar reads as punctuation. Block scalars (|, >) take every following line indented deeper than the line that introduced them. A * opening a plain scalar reads as an alias. |
Anything else — Lang::from_slug returns None, and the caller renders the
block as plain text.
Adding a language
- Add a variant to
Lang, its slugs toSLUGS, and an arm toLang::slugand tohighlightinsrc/lib.rs. - Write
src/<lang>.rswithpub(crate) fn lex(out: &mut Emit<'_>). Scan bytes, and report only what you recognise without.push(start, end, kind)— the gaps becomePlainautomatically, which is what makes the round-trip guarantee hold. Reuse the scanners insrc/util.rs, and only ever cut on ASCII bytes. - Add a
tests/fixtures/<lang>.<ext>sample of 30–80 realistic lines, and an entry inFIXTURESandALL_LANGSintests/highlight.rs. - Generate the golden, then read it (see below).
Two rules keep the invariant tests passing: every loop iteration must advance
the cursor, and a scan that can fail must not be retried at the next byte
without a guard — see no_regex_until in src/js.rs for the pattern that
keeps a pathological line from turning into O(n²).
Updating the goldens
tests/fixtures/*.tokens holds one line per span, Kind<TAB>text, with \\,
\n, \t and \r escaped. These files are the specification, so a diff in
them is a deliberate change to the output.
HL_LITE_UPDATE=1
License
MIT