# ANTLR4 Runtime for Rust
[](https://crates.io/crates/antlr-rust-runtime)
[](https://github.com/ophi-dev/antlr-rust-runtime/actions/workflows/antlr-runtime-testsuite.yml)
`antlr-rust-runtime` is a pure Rust runtime and metadata generator for ANTLR v4
lexers and parsers. It is a clean-room implementation written from scratch from
the public ANTLR runtime contract; it does not vendor or fork an older Rust
ANTLR runtime.
## First Steps
### 1. Install ANTLR4
Follow the ANTLR getting-started guide and install the ANTLR tool jar. The
runtime tests currently validate against ANTLR `4.13.2`.
### 2. Install the Rust ANTLR runtime tools
Each ANTLR target language needs a runtime package used by generated parsers.
For Rust projects, add the runtime crate:
```toml
[dependencies]
antlr-rust-runtime = "0.5"
```
The library crate is imported as `antlr4_runtime`:
```rust
use antlr4_runtime::{CommonTokenStream, InputStream};
```
Install the companion generator binary:
```bash
cargo install antlr-rust-runtime
```
This installs `antlr4-rust-gen`, which turns ANTLR `.interp` metadata into Rust
lexer and parser modules.
### 3. Generate your parser
The current release uses a metadata-first generation path:
1. run the official ANTLR tool to produce `.interp` files,
2. run `antlr4-rust-gen` to emit Rust modules,
3. compile those modules against `antlr4_runtime`.
For a split lexer/parser grammar:
```bash
antlr4 MyGrammarLexer.g4 MyGrammarParser.g4
antlr4-rust-gen \
--lexer MyGrammarLexer.interp \
--parser MyGrammarParser.interp \
--out-dir src/generated
```
The checked-in ANTLR `RustTarget`/StringTemplate shell is kept in `tool/` and
will be expanded around the same runtime contracts.
### Alternative: Generate metadata with antlr-ng
[`antlr-ng`](https://www.antlr-ng.org/introduction.html) is a TypeScript/npm
parser generator based on ANTLR 4.13.2. It does not currently ship a Rust
target, but it can produce the same `.interp` metadata that `antlr4-rust-gen`
uses.
Install it with npm or run it through `npx`:
```bash
npx -y antlr-ng -Dlanguage=Java -o build/antlr --exact-output-dir true JSON.g4
```
The `-Dlanguage=Java` option selects one of antlr-ng's bundled code-generation
targets only so the tool emits grammar artifacts, including `JSONLexer.interp`
and `JSON.interp`. The Java files can be ignored; Rust code still comes from
`antlr4-rust-gen`:
```bash
antlr4-rust-gen \
--lexer build/antlr/JSONLexer.interp \
--parser build/antlr/JSON.interp \
--out-dir src/generated
```
For local tooling, antlr-ng requires Node.js 20 or newer. See the
[antlr-ng getting-started guide](https://www.antlr-ng.org/getting-started.html)
for CLI installation and option details.
## Complete Example
Suppose you are using the JSON grammar from `antlr/grammars-v4/json`.
Fetch or copy `JSON.g4`, then generate ANTLR metadata:
```bash
antlr4 JSON.g4
```
Generate Rust modules:
```bash
antlr4-rust-gen \
--lexer JSONLexer.interp \
--parser JSON.interp \
--out-dir src/generated
```
Declare the generated modules in your crate:
```rust
mod generated {
#![allow(dead_code)]
pub mod json;
pub mod json_lexer;
}
```
Call the generated parser helper for the compact path:
```rust
use generated::json::{self, Json};
use generated::json_lexer::JsonLexer;
fn main() -> Result<(), antlr4_runtime::AntlrError> {
let tree = json::parse(r#"{"a":1}"#, JsonLexer::new, Json::json)?;
println!("{}", tree.text());
Ok(())
}
```
Use `parse_with_parser` when you want the compact setup path and also need the
parser afterward for diagnostics or the owned token stream:
```rust
use antlr4_runtime::Parser;
use generated::json::{self, Json};
use generated::json_lexer::JsonLexer;
fn main() -> Result<(), antlr4_runtime::AntlrError> {
let output = json::parse_with_parser(r#"{"a":1}"#, JsonLexer::new, Json::json)?;
let syntax_errors = output.parser.number_of_syntax_errors();
let tree = output.result;
let tokens = output.parser.into_token_stream();
println!("{} errors across {} tokens", syntax_errors, tokens.tokens().len());
println!("{}", tree.text());
Ok(())
}
```
Or construct each layer explicitly when you need to set source names, parser
options, or custom error handling before invoking the entry rule:
```rust
use antlr4_runtime::{CommonTokenStream, InputStream};
use generated::json::Json;
use generated::json_lexer::JsonLexer;
fn main() -> Result<(), antlr4_runtime::AntlrError> {
let lexer = JsonLexer::new(InputStream::new(r#"{"a":1}"#));
let tokens = CommonTokenStream::new(lexer);
let mut parser = Json::new(tokens);
let tree = parser.json()?;
println!("{}", tree.text());
Ok(())
}
```
### Choosing Parser Entry Rules
Generated parsers expose one public method per grammar rule. Call the method
that matches the grammar's intended top-level rule for the input; the generator
can identify rules that are not called by other rules, but it cannot infer the
semantic choice between multiple top-level forms. The generated parser rustdoc
lists likely entry methods first, followed by all rule methods.
For the JSON grammar above, `json()` is the natural entry. Larger grammars may
have several top-level forms, so confirm the intended entry rule against that
grammar's documentation. Calling the wrong rule can still recover and return a
parse tree with error nodes, so check parser diagnostics when adding a new input
form.
## Technical Notes
- Pure Rust runtime implementation.
- Written from scratch as a clean-room implementation.
- Supports ANTLR serialized ATN deserialization.
- Supports lexer and parser execution through generated Rust wrappers.
- Supports real split lexer/parser grammars, including Kotlin smoke builds.
- Passes every upstream ANTLR runtime-testsuite descriptor discovered by the
harness: `357 passed, 0 failed, 0 skipped, 357 run`.
- Licensed under BSD-3-Clause for compatibility with ANTLR's runtime licensing
pattern and downstream open-source applications.
The runtime contains:
- `IntStream` and `CharStream`
- UTF-8 input as Unicode scalar values
- `Token`, `CommonToken`, token factories, and `TokenSource`
- buffered, channel-aware `CommonTokenStream`
- `Vocabulary`
- recognizer metadata and error listener plumbing
- parse tree node types, rule contexts, terminal nodes, error nodes, and walkers
- ANTLR v4 serialized ATN deserialization
- lexer ATN recognition with longest-match/rule-priority behavior and lexer
actions
- parser ATN rule recognition with backtracking over token stream indices
- `antlr4-rust-gen`, a Rust generator that consumes ANTLR `.interp` metadata and
emits Rust modules
- `antlr4-runtime-testsuite`, a harness for running upstream ANTLR
runtime-test descriptors through the Rust metadata path
See [docs/kotlin-build.md](docs/kotlin-build.md) for the Kotlin smoke workflow.
See [docs/runtime-testsuite.md](docs/runtime-testsuite.md) for the upstream
runtime-testsuite harness.
## Runtime Testsuite
On the maintainer checkout, where the ANTLR jar and upstream runtime-testsuite
live under `/tmp/antlr-cleanroom`, run the full sweep with:
```bash
cargo run --quiet --bin antlr4-runtime-testsuite
```
Run a specific descriptor:
```bash
cargo run --bin antlr4-runtime-testsuite -- \
--antlr-jar path/to/antlr-4.13.2-complete.jar \
--descriptors path/to/antlr4/runtime-testsuite \
--case LexerExec/KeywordID
```
## Performance
`tools/parse-bench/` benchmarks parse throughput of the generated Rust parsers
against the upstream Go runtime (`github.com/antlr4-go/antlr/v4`) — and
optionally the reference Python runtime and tree-sitter — on real-world Kotlin,
C#, Java, and Trino SQL fixtures. See
[`tools/parse-bench/README.md`](tools/parse-bench/README.md) for setup (the
ANTLR jar, the `grammars-v4` sparse checkout, and the Python dependencies).
Run the Rust-vs-Go comparison across all fixture languages:
```bash
python3 tools/parse-bench/run.py \
--languages kotlin,csharp,java,trino \
--runtimes rust-antlr,go-antlr \
--quick \
--json target/parse-bench/results.json \
--markdown target/parse-bench/results.md
```
The report prints `min`/`avg` parse time and a ratio against `rust-antlr` for
every fixture. Drop `--quick` (or add `--iters`/`--warmups`) for longer, lower
variance runs; add `--runtimes rust-antlr,go-antlr,python-antlr,tree-sitter` to
include the other runtimes.
### Current results
Relative parse speed of this runtime versus the Go runtime, summarized as the
geometric mean of the per-fixture `go ÷ rust` parse-time ratios in each language
group (**> 1.0** means Rust is faster than Go; **< 1.0** means slower):
| Kotlin | 4 | ~18× faster |
| Java | 4 | ~1.8× faster |
| C# | 4 | ~1.2× faster |
| Trino SQL| 5 | ~1.1× faster |
Rust is faster than Go on every fixture in all four language groups, with
Kotlin leading dramatically (expression-ladder memoization in the generated
walker). Learned lexer and parser DFAs are shared across recognizer
instances, so repeated parses of the same grammar — the common case for a
CLI tool or language server — skip relearning entirely. Numbers are
warm-parse minimums on an Apple M3 Pro and are indicative — re-run the
benchmark on your own hardware for authoritative figures.
## Useful Information
- ANTLR: <https://www.antlr.org/>
- ANTLR documentation: <https://github.com/antlr/antlr4/blob/dev/doc/index.md>
- Grammars v4: <https://github.com/antlr/grammars-v4>