shiki

Fast TextMate highlighting for Rust.

shiki is the core crate of shiki-rs. It compiles TextMate grammars into a compact runtime representation, tokenizes with Oniguruma, resolves themes, and provides HTML and custom rendering APIs.

This crate intentionally contains no bundled language or theme catalog. Use shiki-langs and shiki-themes, provide runtime definitions, or generate an engine with shiki-macros.

Installation

cargo add shiki

For the usual bundled setup:

cargo add shiki shiki-langs shiki-themes

Quick start

use shiki::{Highlighter, LanguageBundle};

static LANGUAGES: LanguageBundle = shiki_langs::languages![rust];

let mut highlighter = Highlighter::builder()
    .bundle(&LANGUAGES)
    .languages(["rust"])
    .theme(&shiki_themes::CATPPUCCIN_MOCHA)
    .build()?;

let html = highlighter.code_to_html("fn main() {}", "rust")?;

Plain text

Every engine includes the reserved text language and its txt and plain aliases. It emits one token for each non-empty line, bypassing syntax matching while retaining HTML escaping and the selected theme's default foreground and background styles. This makes it suitable as the fallback for an unavailable or user-provided language:

let html = highlighter.code_to_html(
    "<script>not highlighted</script>",
    "text",
)?;

A standalone plain-text engine does not require a language bundle:

let mut highlighter = shiki::Highlighter::builder()
    .languages(["text"])
    .theme(&shiki_themes::GITHUB_DARK)
    .build()?;

ANSI terminal output

The reserved ansi language converts terminal SGR sequences to themed tokens and HTML without requiring a TextMate grammar:

let html = highlighter.code_to_html(
    "\x1b[1;32mcompleted\x1b[0m in 12ms",
    "ansi",
)?;

It supports the standard and bright theme palettes, 256-color and TrueColor sequences, foreground/background colors, reverse video, dim, bold, italic, underline, strikethrough, and state spanning multiple lines. Theme JSON terminal.ansi* colors are retained in runtime and precompiled engines; VS Code-compatible colors are used when a theme omits them. Owned ANSI tokens have no TextMate scopes and use scope ID 0; the scope-token and incremental grammar APIs remain specific to TextMate languages.

Build a shared engine when several documents need the same immutable grammar and theme data:

let engine = Highlighter::builder()
    .bundle(&LANGUAGES)
    .languages(["rust"])
    .theme(&shiki_themes::CATPPUCCIN_MOCHA)
    .build_engine()?;

let mut first = engine.session("rust")?;
let mut second = engine.session("rust")?;

Main types

• HighlighterBuilder selects definitions, themes, and regex limits.
• HighlighterEngine owns shared immutable compiled grammar and theme data.
• Highlighter lazily owns per-language tokenizers and rendering caches.
• LanguageSession owns one independent document tokenizer.
• GrammarState carries TextMate begin/end and begin/while state between lines.
• HtmlRenderer streams compact HTML without materializing owned theme tokens.
• Renderer is the extension point for other output formats.

Tokenization APIs

Highlighter exposes representations for different integration costs:

• tokenize_line for incremental documents;
• tokenize_line_into for allocation-reusing streaming tokenization;
• token_styles for borrowed multi-theme style resolution;
• code_to_scope_tokens for source ranges and compact scope IDs;
• code_to_tokens for owned tokens resolved against one theme;
• code_to_tokens_with_themes for every configured theme;
• code_to_html and code_to_html_with_options for streaming HTML;
• render for a custom Renderer implementation.

split_lines, ThemeInfo, and ansi::AnsiParser expose the same line, theme, and ANSI primitives used by the built-in renderer. Internal grammar IR and tokenizer storage stay private so custom renderers do not depend on cache layout details.

Scope names can be recovered from ScopeStackId when diagnostics or semantic inspection needs them. cache_stats, clear_language_cache, and clear_all_caches expose control over lazily accumulated runtime state.

HTML rendering

HtmlOptions independently controls explicit classes and attributes, automatic shiki/theme classes, data-themes, line wrappers, root theme variables, foreground/background styles, default theme selection, token spans, and variable-only output.

let mut options = shiki::HtmlOptions::default();
options.default_theme = Some("light".into());
options.pre_classes.push("code-block".into());
options.code_classes.push("language-rust".into());
options
    .pre_attributes
    .insert("data-language".into(), "rust".into());
options.include_line_wrapper = false;
options.include_default_theme_styles = false;

let html = highlighter.code_to_html_with_options(
    "let value = 1;",
    "rust",
    &options,
)?;

Use HtmlOptions::clean() for a <pre><code>…</code></pre> wrapper with no automatic wrapper attributes or root styles. It retains the line class and syntax-highlighted token styles.

Custom options own their Vec<String> and BTreeMap<String, String> values. Use LazyLock to create one shared global configuration:

use std::sync::LazyLock;

static HTML: LazyLock<shiki::HtmlOptions> = LazyLock::new(|| {
    let mut options = shiki::HtmlOptions::clean();
    options.pre_classes.push("code-block".into());
    options
        .code_attributes
        .insert("aria-label".into(), "source".into());
    options.include_data_themes = true;
    options.include_line_wrapper = false;
    options
});

Classes, attributes, line_class, and default_theme are public fields. Set line_class to None to keep a classless line wrapper, or set include_line_wrapper to false to remove the wrapper itself.

With multiple themes, token colors are emitted as CSS custom properties. Font and background variables are emitted only when required. Adjacent tokens with the same final visual style are merged even when their scope stacks differ; plain whitespace is left unwrapped when a span would have no visual effect.

Runtime definitions

The default json feature enables TextMate grammar and theme JSON:

let highlighter = Highlighter::builder()
    .json_language("custom", grammar_json)?
    .json_theme("custom", theme_json)?
    .build()?;

Definitions may also be constructed through RawGrammar, RawRule, RawTheme, and their nested raw types. LanguageInput adds aliases and external injection targets to runtime grammars.

Features

Compile without the JSON stack when the target only consumes snapshots generated by shiki-macros:

cargo add shiki --no-default-features

In that configuration, target-side shiki depends on onig_sys, thiserror, and the small lz4_flex snapshot codec, but not serde or serde_json.

Performance model

Compiled grammar IR and themes belong to the engine. Engines produced by shiki-macros decode each grammar IR on first use; runtime-built engines compile their selected grammars eagerly. Oniguruma scanners, dynamic regexes, scope transitions, injection results, and style rows are created lazily in a highlighter or session. HighlighterEngine::loaded_language_count exposes the number of resident grammar IRs.

• Reuse engines and sessions.
• Bundle only required roots when possible.
• Use incremental line state for editors.
• Prefer scope tokens or streaming rendering when owned tokens are unnecessary.
• Expect very long minified lines to cost more than ordinary line-oriented source.