Crate char_lex

CHAR-LEX

Char-Lex is a crate for easely creating a char based lexer from multiple custom enums!

Example

use char_lex::prelude::*;

#[token]
#[derive(Debug, PartialEq)]
enum Digit {
    Zero = '0',
    One = '1',
    Two = '2',
    Three = '3',
    Four = '4',
    Five = '5',
    Six = '6',
    Seven = '7',
    Eight = '8',
    Nine = '9',
}

fn main() {
    let lexer: Lexer<Digit, Digit> = Lexer::new("189");
    let tokens: Vec<Digit> = lexer.collect();
     
    assert_eq!(vec![Digit::One, Digit::Eight, Digit::Nine], tokens);
}

Tokens can also be wrapped in anything that implements the TokenWrapper<T> trait!

Example

use char_lex::prelude::*;

#[token]
#[derive(Debug, PartialEq)]
enum Token {
    One = '1',
}

#[derive(Debug, PartialEq)]
struct Wrapper {
    token: Token,
    character: char,
}

impl TokenWrapper<Token> for Wrapper {
    fn wrap(token: Token, context: Context) -> Self {
        Self { token, character: context.character }
    }
}

fn main() {
    let lexer: Lexer<Token, Wrapper> = Lexer::new("1");
    let tokens: Vec<Wrapper> = lexer.collect();
     
    assert_eq!(vec![Wrapper { token: Token::One, character: '1' }], tokens);
}

Modules

error

Contains the Error type.

prelude

Prelude module. It renames Error to LexErr!

utils

Contains utility types!

Structs

Lexer

The main lexer type.

Traits

TokenMatch

Trait for anything that wants to match a single Token.

TokenTrait

The main trait for Tokens, it is the automatically implemented by the token attribute macro.

TokenWrapper

Trait for anything that wants to automatically wrap a Token.

Attribute Macros

token

The token attribute macro.