rex-lexer 3.9.12

Rex: A strongly-typed, pure, implicitly parallel functional programming language
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

# Rex Lexer (`rex-lexer`)

This crate tokenizes Rex source into a stream of `Token`s with precise `Span` information.

## Entry point

```rust
use rex_lexer::Token;

fn main() -> Result<(), rex_lexer::LexicalError> {
    let tokens = Token::tokenize("let x = 1 in x")?;
    let _ = tokens;
    Ok(())
}
```

## Spans

`rex_lexer::span` provides `Position` and `Span` types used throughout the workspace for diagnostics
and editor tooling.