Crate arborium_highlight

Expand description

Unified syntax highlighting for arborium.

This crate provides the core highlighting engine that works with both:

Statically linked Rust grammars: For CLI tools and servers
Dynamically loaded WASM plugins: For browser contexts

§Why Async in a Highlighting Library?

You might wonder why a syntax highlighting library has async code. The answer is browser support.

Parsing is synchronous: Tree-sitter parsing cannot be async—it’s a fundamentally synchronous operation that walks the syntax tree.
Getting a grammar can be async: In browser contexts, grammar plugins are loaded from a CDN via JavaScript’s dynamic import(). This is inherently async since it involves network requests and WASM instantiation.

In native Rust, grammars are statically linked, so the provider returns immediately. But the trait is async to support both use cases with the same code.

§Architecture

The highlighting system is built around two key traits:

Grammar: What a grammar can do — parse text and return spans
GrammarProvider: How grammars are obtained — this is where sync vs async differs

§The Sync-in-Async-Clothing Pattern

The core highlighting logic (including injection handling) is written once as async code in HighlighterCore. Two wrappers provide the sync and async APIs:

SyncHighlighter: Polls the async future once and panics if it yields. This is safe for native Rust where providers return immediately.
AsyncHighlighter: Actually awaits provider calls. Use this for browser/WASM contexts where grammar loading involves network requests.

This design ensures both environments share the exact same injection-handling logic, avoiding subtle bugs from duplicated code.

§When to Use Which

Context	Highlighter	Provider Example
Native Rust	`SyncHighlighter`	`StaticProvider` (grammars compiled in)
Browser WASM	`AsyncHighlighter`	`JsGrammarProvider` (loads from CDN)

§Quick Start

use arborium_highlight::{SyncHighlighter, Grammar, GrammarProvider, ParseResult, Span};
use arborium_highlight::{HighlightConfig, HtmlFormat};

// Define your grammar (implements Grammar trait)
struct MyGrammar { /* ... */ }
impl Grammar for MyGrammar {
    fn parse(&mut self, text: &str) -> ParseResult {
        // Parse and return spans + injections
        ParseResult::default()
    }
}

// Define your provider (implements GrammarProvider trait)
struct MyProvider { /* ... */ }
impl GrammarProvider for MyProvider {
    type Grammar = MyGrammar;
    async fn get(&mut self, language: &str) -> Option<&mut Self::Grammar> {
        // Return grammar for language
        None
    }
}

// Use with default configuration (custom elements: <a-k>, <a-f>, etc.)
let mut highlighter = SyncHighlighter::new(MyProvider { /* ... */ });
let html = highlighter.highlight("rust", "fn main() {}");
// Output: <a-k>fn</a-k> <a-f>main</a-f>() {}

// Or use class-based output for compatibility with existing CSS
let config = HighlightConfig {
    html_format: HtmlFormat::ClassNames,
    ..Default::default()
};
let mut highlighter = SyncHighlighter::with_config(MyProvider { /* ... */ }, config);
let html = highlighter.highlight("rust", "fn main() {}");
// Output: <span class="keyword">fn</span> <span class="function">main</span>() {}

§HTML Output Formats

Arborium supports multiple HTML output formats via HtmlFormat:

CustomElements (default): Compact custom elements like <a-k>, <a-f>, etc.
CustomElementsWithPrefix(prefix): Custom elements with your prefix, e.g., <code-k>
ClassNames: Traditional <span class="keyword"> for compatibility
ClassNamesWithPrefix(prefix): Namespaced classes like <span class="arb-keyword">

See HtmlFormat for examples and use cases.

Structs§

AnsiOptions: Options controlling ANSI rendering behavior.
AsyncHighlighter: Asynchronous highlighter for WASM/browser contexts.
HighlightConfig: Configuration for highlighting.
Injection: An injection point for embedded languages.
ParseResult: Result of parsing a document with a grammar.
Span: A span of highlighted text.
SyncHighlighter: Synchronous highlighter for Rust contexts.
ThemedSpan: A span with a theme style index for rendering.

Enums§

HighlightError: Errors that can occur during highlighting.
HtmlFormat: HTML output format for syntax highlighting.

Traits§

Grammar: A grammar that can parse text and produce highlight spans.
GrammarProvider: Provides grammars for languages.

Functions§

html_escape: Escape HTML special characters.
spans_to_ansi: Deduplicate spans and convert to ANSI-colored text using a theme.
spans_to_ansi_with_options: ANSI rendering with additional configuration options.
spans_to_html: Deduplicate spans and convert to HTML.
spans_to_themed: Convert raw spans to themed spans by resolving capture names to theme indices.
write_spans_as_ansi: Write spans as ANSI-colored text to a writer.
write_spans_as_html: Write spans as HTML to a writer.