syntect-tmlanguage

Load real TextMate (.tmLanguage/.tmLanguage.json) grammars into syntect's own SyntaxDefinition data model, so syntect-based tools gain TextMate grammar support without abandoning syntect's own regex-matching and highlighting engine.

TextMate/VS Code grammars are the most widely available syntax-highlighting grammar format. syntect only loads .sublime-syntax (YAML) natively — this crate bridges the gap.

How it works

syntect::parsing::ContextReference (and its sibling types) is #[non_exhaustive] with no public constructor, so direct struct construction from outside syntect isn't possible. Instead, this crate translates a parsed .tmLanguage.json grammar into .sublime-syntax-shaped YAML text and hands it to syntect's own SyntaxDefinition::load_from_str — the same, stable, public entry point real .sublime-syntax files go through.

Usage

use syntect::parsing::SyntaxSetBuilder;

let definition = syntect_tmlanguage::load_file("MyLanguage.tmLanguage.json")?;

let mut builder = SyntaxSetBuilder::new();
builder.add(definition);
let syntax_set = builder.build();
# Ok::<(), syntect_tmlanguage::Error>(())

Add the result to a syntect::parsing::SyntaxSetBuilder the same way you would a hand-written .sublime-syntax file — this crate doesn't wrap that step. For build-time bundling (convert once, ship a fast-loading artifact), either save to_sublime_syntax_yaml's output as a .sublime-syntax file directly, or cache a built SyntaxSet with syntect::dumps::dump_to_file/from_dump_file.

What's supported

match rules (scope, captures)
begin/end rules, including applyEndPatternLast, name/contentName, beginCaptures/endCaptures, and end-pattern backreferences to begin's own capture groups
include: #name (repository), $self/$base, and external scope references — the mechanism real structural embedding (e.g. a fenced code block including another language's grammar) relies on
Runtime loading of an arbitrary local grammar file (load_file), and in-memory loading from a string (load)

Known limitations

These are gaps in syntect itself, not just this crate — a grammar that needs them fails translation with a specific error rather than being silently mistranslated:

begin/while rules (used by e.g. VS Code's own markdown.tmLanguage.json for blockquotes and fenced code blocks) — syntect has no engine-level equivalent at all.
\G anchors — confirmed to never match at all through syntect's onig backend, in any position (see examples/g_anchor_never_matches.rs). Used by e.g. VS Code's shell-unix-bash.tmLanguage.json and html.tmLanguage.json in commonly-exercised paths.
True selector-based injections (VS Code's injectionSelector / the grammar's injections key) — syntect has no support for scope-selector-triggered pattern injection at all.

This crate uses syntect's regex-onig feature (real Oniguruma via a C binding), matching syntect's own default. This means it does not cross-compile to wasm32-unknown-unknown as-is.

License