Crate laps

source ·
Expand description

Lexer and parser collections.

With laps, you can build lexers/parsers by just defining tokens/ASTs and deriving Tokenize/Parse trait for them.

Example

Implement a lexer for S-expression:

use laps::prelude::*;

#[token_kind]
#[derive(Debug, Tokenize)]
enum TokenKind {
 // This token will be skipped.
 #[skip(r"\s+")]
 _Skip,
 /// Parentheses.
 #[regex(r"[()]")]
 Paren(char),
 /// Atom.
 #[regex(r"[^\s()]+")]
 Atom(String),
 /// End-of-file.
 #[eof]
 Eof,
}

And the parser and ASTs (or actually CSTs):

type Token = laps::token::Token<TokenKind>;

token_ast! {
 macro Token<TokenKind> {
   [atom] => { kind: TokenKind::Atom(_), prompt: "atom" },
   [lpr] => { kind: TokenKind::Paren('(') },
   [rpr] => { kind: TokenKind::Paren(')') },
   [eof] => { kind: TokenKind::Eof },
 }
}

#[derive(Parse)]
#[token(Token)]
enum Statement {
 Elem(Elem),
 End(Token![eof]),
}

#[derive(Parse)]
#[token(Token)]
struct SExp(Token![lpr], Vec<Elem>, Token![rpr]);

#[derive(Parse)]
#[token(Token)]
enum Elem {
 Atom(Token![atom]),
 SExp(SExp),
}

The above implementation is very close in form to the corresponding EBNF representation of the S-expression:

Statement ::= Elem | EOF;
SExp      ::= "(" {Elem} ")";
Elem      ::= ATOM | SExp;

More Examples

See the examples directory, which contains the following examples:

  • sexp: a S-expression parser.
  • calc: a simple expression calculator.
  • json: a simple JSON parser.
  • clike: interpreter for a C-like programming language.

Modules

  • ast
    Some common predefined AST structures that can be used in parser.
  • input
    Utilities for constructing lexers.
  • lexer
    Implementations for constructing lexers.
  • parse
    Implementations for constructing parsers.
  • prelude
    A prelude of some common traits and macros (if enabled feature macros) in laps.
  • reader
    Reader related implementations for lexers.
  • span
    Span (Span) and error (Error) related implementations.
  • token
    Token (Token) related implementations, including tokenizer (Tokenizer) and token stream (TokenStream).

Macros