Enum Token 

pub enum Token {
    Numeric(i16, u8),
    OrdinalDay(u8),
    MonthName(MonthName),
}

A single meaningful chunk produced by crate::extract::tokenise.

The tokeniser strips separator characters and noise words, leaving only tokens that could contribute to a date component. At most three tokens are returned (one per date component: day, month, year).

Each variant stores the already-parsed value rather than the raw source text, so consumers can use the token directly without re-parsing.

Variants

Numeric(i16, u8)

A parsed integer together with the number of digits in the original source string.

The digit count is required for year disambiguation: "24" (2 digits) must be expanded via TwoDigitYearExpansion, while "2024" (4 digits) is used as-is. Three-digit and five-digit numbers are never valid date components.

Uses i16 for the value because the full year range required by the spec (0–3000) fits within i16::MAX (32,767), and day/month values are far smaller.

OrdinalDay(u8)

The numeric day extracted from an ordinal like "19th" or "1st", with the suffix already stripped.

MonthName(MonthName)

A resolved MonthName variant, matched from a full name, abbreviation, unambiguous prefix, or fuzzy misspelling.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Token

impl StructuralPartialEq for Token