open-cypher 0.4.0

An unofficial Rust parser for the openCypher query language.
Documentation

open-cypher

Crates.io Documentation CI Scheduled fuzzing Mutation testing Benchmarks

open-cypher is an unofficial Rust lexer and parser for the openCypher query language.

The 0.2 rewrite targets the openCypher 2024.3 grammar with a lossless Logos token stream, a LALRPOP parser, a typed and spanned AST, and structured diagnostics. The crate parses syntax only: it does not resolve names, perform type checking, execute queries, or claim official TCK certification.

Status

Versions 0.2 and later are a clean break from the original Pest-based API. The 2024.3 syntax coverage is measured by a deterministic projection of every query occurrence in the pinned TCK and by executable traceability for all 377 BNF productions. This is parser evidence only, not semantic, runtime, or official TCK certification.

The checked projection contains 4,131 unique queries and 4,882 materialized occurrences: 4,880 active occurrences plus two upstream @ignore scenarios retained as supplemental evidence. All cases remain part of the release gate.

The language snapshot is pinned independently of the crate version:

  • openCypher release: 2024.3
  • upstream commit: 677cbafabb8c3c5eed458fd3b1ec0daec8d67d23

The workspace minimum supported Rust version is 1.88.

See spec/UPSTREAM.toml, the production map, and the deviation ledger for provenance and implementation coverage.

Usage

[dependencies]
open-cypher = "0.4.0"

parse is the strict entry point. It returns the syntax tree and the full token stream, or every diagnostic collected while lexing and parsing.

use open_cypher::parse;

let source = "MATCH (person:Person) WHERE person.age > 30 RETURN person.name";

match parse(source) {
    Ok(parsed) => {
        // Every node carries a byte span into `source`.
        println!("{:#?}", parsed.program);
        for token in parsed.tokens {
            println!("{:?} at {:?}", token.kind, token.span);
        }
    }
    Err(errors) => eprintln!("{}", errors.render(source)),
}

A failed parse renders like this:

error[OCY-P004]: unclosed delimiter `(`
 --> 1:7
  |
1 | MATCH (person:Person RETURN person.name
  |       ^
  = help: add the matching delimiter before byte 39

parse_recovering always returns a program root together with the diagnostics, and lex exposes every token, including whitespace and comments. The API documentation walks through each entry point, and the examples/ directory has runnable versions:

cargo run --example parse
cargo run --example recover
cargo run --example lex

The optional serde feature implements serialization for public syntax and diagnostic data types:

[dependencies]
open-cypher = { version = "0.4.0", features = ["serde"] }

Scope

Supported:

  • the complete openCypher 2024.3 grammar, with every one of its 377 BNF productions mapped to parser targets and executable witnesses;
  • a small set of deliberate extensions recorded in spec/DEVIATIONS.toml, such as COUNT {} / COLLECT {} subqueries, != and ||, GQL path modes, label expressions in SET / REMOVE, and empty or comment-only input.

Out of scope for the 0.4 series:

  • semantic analysis and execution: names are not resolved, types are not checked, and some inputs a database would reject at compile time (for example a parameter used as a MATCH property map) parse successfully so a later pass can report them;
  • multi-statement scripts: a program holds zero or one query with an optional trailing semicolon;
  • arbitrary syntax repair: recovery does not invent identifiers or close delimiters.

parse_recovering retains valid top-level clauses around malformed regions, using ClauseKind::Error placeholders with source spans. Recovery does not cross unclosed delimiters. It retains UNION branches and operator spans, including ALL and DISTINCT. Missing branches contain an empty-span error clause; valid clauses take precedence over interpreting contextual names as operators. Multi-statement inputs and inputs with no recoverable clauses retain a whole-input error statement. A shared retry budget bounds recovery across all branches; exhaustion leaves an unsplit error suffix. The complete token stream is preserved, and at most 32 diagnostics are reported per parse. Missing expression operands can be retained as empty-span ExprKind::Error nodes, preserving operators, precedence, and valid siblings. For example, RETURN 1 + retains an addition whose right operand is an error. This is insertion-only recovery: source tokens are not deleted, and unclosed delimiters or errors requiring identifier repair still fall back to clause errors. Each candidate allows at most four insertions, charged to the shared retry budget. Consumers must inspect expression errors as well as clause/statement errors. See the 0.4 recovery migration notes for before/after AST examples and recovery boundaries. Multi-statement input remains a candidate for a later minor release.

Some accepted inputs look surprising but follow the 2024.3 grammar and are not parser bugs:

  • a[1:b] parses as an index whose subscript is 1:b: a subscript is any value expression, and a comparison predicand may carry a :Label test.

Testing

The test suite is layered so that a green unit test cannot be mistaken for language conformance. docs/testing.md covers each layer in detail; in short:

  • Unit and integration tests cover the public API, lexer boundaries, diagnostics and recovery, AST shape and exact spans, historical 0.1 bugs, and a file-backed smoke corpus. Property tests generate random queries (PROPTEST_CASES=4096 for the expanded run).
  • TCK syntax projection. Every query occurrence in the pinned openCypher TCK is extracted into a checked-in JSONL file and parsed in CI, with an explicit reviewed expectation for each SyntaxError scenario. This is a syntax projection, not official TCK certification.
  • Production traceability. spec/PRODUCTION_MAP.toml and spec/WITNESSES.toml map all 377 BNF productions to positive and negative witness queries that cargo xtask spec verify executes.
  • Fuzzing. fuzz/ holds cargo-fuzz targets for lexing, strict parsing, recovery, and grammar-aware query generation. Pull requests replay the checked-in corpora; a scheduled job fuzzes each target for ten minutes. Minimized failures land as deterministic regression tests.
  • Coverage. CI enforces 85% line coverage over handwritten source, with generated parser code, fixtures, benches, and fuzz harnesses excluded.
  • Mutation testing. A weekly sharded cargo-mutants run mutates the handwritten Rust and checks that the tests notice. Mutants that were triaged by hand and found equivalent are listed in .cargo/mutants.toml; the last full sweep before 0.2.0 had no surviving mutants.

Development

The repository keeps generated LALRPOP Rust checked in so downstream builds do not need to run the parser generator. Maintenance and verification commands are provided through the workspace's cargo xtask alias.

cargo test --workspace --all-features
cargo xtask grammar verify
cargo xtask spec verify
cargo xtask tck verify
cargo xtask tck check
cargo xtask tck report
cargo xtask spec release-check
cargo bench --bench parser

Fuzz targets live in fuzz/ and use cargo-fuzz.

The upstream release workflow checks the latest published openCypher release against spec/UPSTREAM.toml on the first day of each month at 07:23 UTC. It can also be run manually from Actions. A different tag or commit fails the check and adds release notes, a comparison link, and a spec update inspection command to the run summary. This is a maintenance signal; adopting the release is a separate, reviewed change. Enable failed-workflow notifications in GitHub to receive alerts. The schedule takes effect once the workflow is on the default branch; GitHub can disable scheduled workflows after 60 days of repository inactivity.

Licensing

Project-authored source is available under MIT or Apache-2.0. Vendored openCypher material and grammar-derived files are Apache-2.0; the packaged crate therefore carries Apache-2.0 metadata. See the license and notice files beside the relevant upstream snapshot for details.