1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
//! # klex (kujira-lexer)
//!
//! A simple lexer (tokenizer) generator for Rust.
//!
//! klex generates Rust lexer code from a single definition file. You describe token patterns
//! with regular expressions, and it outputs Rust source that includes a `Token` struct and a `Lexer` struct.
//!
//! ## Usage
//!
//! ### As a library
//!
//! ```rust
//! use klex::{generate_lexer, parse_spec};
//! use std::fs;
//!
//! // Read input file
//! let input = fs::read_to_string("tests/example.klex").expect("Failed to read input file");
//!
//! // Parse the input
//! let spec = parse_spec(&input).expect("Failed to parse input");
//!
//! // Generate Rust code
//! let output = generate_lexer(&spec, "tests/example.klex");
//!
//! // Write output
//! fs::write("output.rs", output).expect("Failed to write output");
//! ```
//!
//! ### Input file format
//!
//! An input file consists of three sections separated by `%%`:
//!
//! ```text
//! (Rust code here – e.g. use statements)
//! %%
//! (Rules here – token patterns written as regular expressions)
//! %%
//! (Rust code here – e.g. main function or tests)
//! ```
//!
//! ### Writing rules
//!
//! Write one rule per line in the following form:
//!
//! ```text
//! <regex pattern> -> <TOKEN_NAME>
//! ```
//!
//! Examples:
//! ```text
//! [0-9]+ -> NUMBER
//! [a-zA-Z_][a-zA-Z0-9_]* -> IDENTIFIER
//! \+ -> PLUS
//! \- -> MINUS
//! ```
pub use generate_lexer;
pub use ;
pub use Token;