lelwel 0.5.0

LL(1) parser generator
Documentation

lelwel

Crates.io MIT/Apache 2.0 Crates.io Rust

Table of Contents

Introduction

Lelwel generates recursive descent parsers for Rust using LL(1) grammars. Conflicts are resolved with semantic predicates. Semantic actions are used for ad hoc syntax-directed translation. Unlike in other parser generators (e.g. Bison, JavaCC, or Coco/R), actions and predicates are not defined inline, which makes it easier to read the grammar.

Lelwel is written as a library, which is used by the CLI tool llw and the language server lelwel-ls. There is a plugin for Neovim that uses the language server.

Why Yet Another Parser Generator?

  • Language Server: Get instant feedback when your grammar contains conflicts or errors.
  • Easy to Read: Code for semantic actions and predicates does not clutter the grammar.
  • Easy to Debug: The generated parser is easy to understand and can be debugged with standard tools.

Grammar Examples

The parser for lelwel grammar files (*.llw) is itself generated by lelwel. There is also an example for a JSON parser. The following example shows a grammar for a basic calculator.

// calculator

token Num{f64}='<number>';
token Add='+' Sub='-' Mul='*' Div='/';
token LPar='(' RPar=')';

start{f64}:
  expr #1
;
expr{f64}:
  term #1 (
    '+' term #2
  | '-' term #3
  )* #4
;
term{f64}:
  atomic #1 (
    '*' atomic #2
  | '/' atomic #3
  )* #4
;
atomic{f64}:
  Num #1
| '(' expr ')' #2
;

start#1 { Ok(expr) }

expr#1 { let mut res = term; }
expr#2 { res += term; }
expr#3 { res -= term; }
expr#4 { Ok(res) }

term#1 { let mut res = atomic; }
term#2 { res *= atomic; }
term#3 { res /= atomic; }
term#4 { Ok(res) }

atomic#1 { Ok(Num.0) }
atomic#2 { Ok(expr) }

Quickstart

  1. Add the following to your Cargo.toml and build.rs files. 
    [dependencies]
logos = "0.13.0"
codespan-reporting = "0.11.1"

[build-dependencies]
lelwel = "0.5.0"

    fn main() {
   lelwel::build("src/your_grammar.llw");
}
  2. Start a build. This will create a minimal grammar file at the specified path and a parser.rs file next to it. The parser.rs file is supposed to be manually edited to implement the lexer and it includes the actual parser, which is written to the Cargo OUT_DIR.
  3. Optionally you can install the CLI or language server to validate your grammar file: cargo install --features="cli","lsp" lelwel.
  4. Use the parser module. 
    use codespan_reporting::diagnostic::Severity;
use codespan_reporting::files::SimpleFile;
use codespan_reporting::term::termcolor::{ColorChoice, StandardStream};
use codespan_reporting::term::{self, Config};
use logos::Logos;
use parser::*;

mod parser;

fn main() -> std::io::Result<()> {
    let args: Vec<String> = std::env::args().collect();
    if args.len() != 2 {
        std::process::exit(1);
    }
    let contents = std::fs::read_to_string(&args[1])?;

    let mut tokens = TokenStream::new(Token::lexer(&contents));
    let mut diags = vec![];
    if let Some(result) = Parser::parse(&mut tokens, &mut diags) {
        println!("{result:?}");
    }

    let writer = StandardStream::stderr(ColorChoice::Auto);
    let config = Config::default();
    let file = SimpleFile::new(&args[1], &contents);
    for diag in diags.iter() {
       term::emit(&mut writer.lock(), &config, &file, &diag).unwrap();
    }
    if diags.iter().any(|d| d.severity == Severity::Error) {
        std::process::exit(1);
    }
    Ok(())
}

Documentation

TODO

License

Lelwel and its generated code is licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.