substrait-antlr

Generated Rust ANTLR parsers for the Substrait grammars, built against the antlr4rust runtime.

The grammars from which the code is generated can be found here.

Versions of this crate correspond to Substrait releases. vx.y.z of substrait-antlr contains code generated from vx.y.z of the substrait repository.

Usage

use substrait_antlr::antlr4rust::common_token_stream::CommonTokenStream;
use substrait_antlr::antlr4rust::InputStream;
use substrait_antlr::substrait_type::{SubstraitTypeLexer, SubstraitTypeParser};

let lexer = SubstraitTypeLexer::new(InputStream::new("list?<i32>"));
let tokens = CommonTokenStream::new(lexer);
let mut parser = SubstraitTypeParser::new(tokens);
let tree = parser.startRule().unwrap();

• substrait_type — lexer/parser/listeners for the Substrait type grammar (SubstraitType.g4).
• func_test_case — lexer/parser/listeners for the function test case grammar (FuncTestCaseParser.g4).

The generated code only works with the exact antlr4rust version it was generated for, so this crate re-exports that runtime as substrait_antlr::antlr4rust. Use it through the re-export rather than depending on antlr4rust directly, so that the two can never drift apart.

Generation and Publishing

Code generation and publishing is handled in the substrait-packaging repository.

Unlike the other Substrait Rust crates, the ANTLR parsers are committed to the repository, because regenerating them requires a forked ANTLR build (the Rust target is not yet part of upstream ANTLR) and Java, neither of which is a cargo build dependency.

When a new version of the Substrait specification is released, automation regenerates the parsers for that version and pushes them to GitHub with a tag formatted like rust/substrait-antlr/vx.y.z. The automation then publishes that crate to crates.io.

Local Generation

The generate_antlr.sh script can be executed locally to regenerate the parsers. It downloads the forked ANTLR JAR and requires java. Set SUBSTRAIT_HOME to a directory containing the Substrait specification (defaults to ../../substrait).