pub enum Token {
OBra(usize),
CBra(usize),
Comma(usize),
Text(Arc<String>, usize),
Number(Arc<String>, usize),
Range(usize),
}Expand description
Defines the possible types of tokens that can be encountered during the process of tokenization.
The Token enum is used to represent different types of tokens that can be produced while tokenizing a string. Each variant of the Token enum corresponds to a specific type of token, such as opening brace, closing brace, comma, text, number, or range operator.
Variants§
OBra(usize)
Represents an opening brace { at the specified position.
CBra(usize)
Represents a closing brace } at the specified position.
Comma(usize)
Represents a comma , at the specified position.
Text(Arc<String>, usize)
Represents any non-number text at the specified position.
The associated String contains the text value.
Number(Arc<String>, usize)
Represents a number at the specified position.
The associated String contains the numeric value.
Range(usize)
Represents the range operator .. at the specified position.