Enum Token 

pub enum Token {
    Wildcard,
    TypedMatch(Grokker),
    Value(TypedToken),
}

Represents a token within a log line, which can be a wildcard, a typed match, or a specific value.

Variants

Wildcard

A wildcard token, matching any other token. Represented as “<*>” in display.

TypedMatch(Grokker)

A token that matches any value of a specific predefined type (e.g., UUID, IPv4).

Value(TypedToken)

A token containing a specific, non-wildcard value.

Implementations

impl Token

pub fn from_parse(input: &str) -> Token

Parses an input string and attempts to classify it into a Token type.

This function uses a set of regular expressions (MATCHERS) to determine if the input string matches any known Grokker patterns. It prioritizes more specific matches and handles ambiguities (e.g., a UUID also matching a hostname pattern).

Arguments

• input - The string slice to parse.

Returns

A Token representing the classification of the input string.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the Token for display.

This implementation provides a string representation of the token, using “<*>” for Wildcard tokens, the Grokker’s display for TypedMatch, and the underlying value’s string representation for Value tokens.

Arguments

• f - The formatter to write into.

Returns

A fmt::Result indicating success or failure of the formatting operation.

impl From<Token> for DefaultSymbol

fn from(tok: Token) -> DefaultSymbol

Converts a Token into its DefaultSymbol representation.

This conversion uses the global string interner to obtain a symbol for the token’s string value or a predefined symbol for Wildcard and TypedMatch tokens.

Arguments

• tok - The Token to convert.

Returns

The DefaultSymbol corresponding to the Token.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Token