jayce 0.3.0

jayce is a blazing fast tokenizer 🌌
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

# jayce


#### Token


```rust,ignore
pub struct Token {
    pub kind: String,
    pub value: String,
    pub line: usize,
    pub column: usize,
}
```

End of File returns `Token` kind `EoF` value `End of File`

New Lines returns `Token` kind `NewLine` value `\n`

#### Tokenizer


```rust,ignore
Jayce::new(source: &'a str, duos: &[(&'a str, &str)]);
```

#### Example


```rust
use jayce::Jayce;

let duos = &[
    ("WhiteSpace", r"^\s+"),
    ("Name", r"^[a-z][a-z_]*"),
    ("Number", r"^[0-9]+"),
    ("Operator", r"^[-+*/%]"),
];

let source = "zoe = 34";

let mut jayce = Jayce::new(source, duos);

println!("{:?}", jayce.eat());

// Token { kind: "Name", value: "zoe", line: 1, column: 4 }
```