Crate laps

Lexer and parser collections.

With laps, you can build lexers/parsers by just defining tokens/ASTs and deriving Tokenize/Parse trait for them.

Example

Implement a lexer for S-expression:

use laps::prelude::*;

#[token_kind]
#[derive(Debug, Tokenize)]
enum TokenKind {
 // This token will be skipped.
 #[skip(r"\s+")]
 _Skip,
 /// Parentheses.
 #[regex(r"[()]")]
 Paren(char),
 /// Atom.
 #[regex(r"[^\s()]+")]
 Atom(String),
 /// End-of-file.
 #[eof]
 Eof,
}

And the parser and ASTs (or actually CSTs):

type Token = laps::token::Token<TokenKind>;

token_ast! {
 macro Token<TokenKind> {
   [atom] => { kind: TokenKind::Atom(_), prompt: "atom" },
   [lpr] => { kind: TokenKind::Paren('(') },
   [rpr] => { kind: TokenKind::Paren(')') },
   [eof] => { kind: TokenKind::Eof },
 }
}

#[derive(Parse)]
#[token(Token)]
enum Statement {
 Elem(Elem),
 End(Token![eof]),
}

#[derive(Parse)]
#[token(Token)]
struct SExp(Token![lpr], Vec<Elem>, Token![rpr]);

#[derive(Parse)]
#[token(Token)]
enum Elem {
 Atom(Token![atom]),
 SExp(SExp),
}

The above implementation is very close in form to the corresponding EBNF representation of the S-expression:

Statement ::= Elem | EOF;
SExp      ::= "(" {Elem} ")";
Elem      ::= ATOM | SExp;

More Examples

See the examples directory, which contains the following examples:

• sexp: a S-expression parser.
• calc: a simple expression calculator.
• json: a simple JSON parser.
• clike: interpreter for a C-like programming language.

Accelerating Code Completion for IDEs

By default, Cargo does not enable optimizations for procedural macros, which may result in slower code completion if you are using laps to generate lexers. To avoid this, you can add the following configuration to Cargo.toml:

[profile.dev.build-override]
opt-level = 3

You can also try to manually enable/disable parallelization for lexer generation by adding:

#[derive(Tokenize)]
#[enable_par(true)] // or #[enable_par(false)]
enum TokenKind {
 // ...
}

Modules

ast

Some common predefined AST structures that can be used in parser.

input

Utilities for constructing lexers.

lexer

Implementations for constructing lexers.

parse

Implementations for constructing parsers.

prelude

A prelude of some common traits and macros (if enabled feature macros) in laps.

reader

Reader related implementations for lexers.

span

Span (Span) and error (Error) related implementations.

token

Token (Token) related implementations, including tokenizer (Tokenizer) and token stream (TokenStream).

Macros

log_error

Logs normal error message.

log_fatal_error

Logs fatal error message.

log_raw_error

Logs normal error with no span provided.

log_raw_fatal_error

Logs fatal error with no span provided.

log_raw_warning

Logs warning with no span provided.

log_warning

Logs warning message.

return_error

Logs error message and returns a result.