Enum Token 

pub enum Token<'src> {
    Date(&'src str),
    Number(&'src str),
    String(&'src str),
    Account(&'src str),
    Currency(&'src str),
    Tag(&'src str),
    Link(&'src str),
    Txn,
    Balance,
    Open,
    Close,
    Commodity,
    Pad,
    Event,
    Query,
    Note,
    Document,
    Price,
    Custom,
    Option_,
    Include,
    Plugin,
    Pushtag,
    Poptag,
    Pushmeta,
    Popmeta,
    True,
    False,
    Null,
    LDoubleBrace,
    RDoubleBrace,
    LBraceHash,
    LBrace,
    RBrace,
    LParen,
    RParen,
    AtAt,
    At,
    Colon,
    Comma,
    Tilde,
    Pipe,
    Plus,
    Minus,
    Star,
    Slash,
    Pending,
    Flag(&'src str),
    Newline,
    Comment(&'src str),
    Hash,
    PercentComment(&'src str),
    Shebang(&'src str),
    EmacsDirective(&'src str),
    MetaKey(&'src str),
    Indent(usize),
    DeepIndent(usize),
    Error(&'src str),
}

Token types produced by the Logos lexer.

Variants

Date(&'src str)

A date in YYYY-MM-DD or YYYY/MM/DD format.

Number(&'src str)

A number with optional sign, thousands separators, and decimals. Examples: 123, -456, 1,234.56, 1234.5678, .50, -.50, 1. (trailing decimal) Python beancount accepts trailing decimal (e.g., “1.” meaning “1.0”).

String(&'src str)

A double-quoted string (handles escape sequences). The slice includes the quotes.

Account(&'src str)

An account name like Assets:Bank:Checking, Aktiva:Bank:Girokonto, or Ciste-jmeni:Stav. Must start with a capitalized word (account type prefix) and have at least one sub-account. Account type prefix can contain hyphens (e.g., Ciste-jmeni for Czech “Čisté jmění”). Sub-accounts must start with uppercase letter, digit, or non-ASCII character (matching Python beancount). Supports Unicode letters, symbols, and emojis (e.g., Expenses:École, Assets:沪深300, Assets:CORP✨). Pattern matches beancount’s lexer.l: ([A-Z]|UTF-8-ONLY)([A-Za-z0-9-]|UTF-8-ONLY)*. [^\x00-\x7F] matches any non-ASCII UTF-8 character (equivalent to beancount’s UTF-8-ONLY). The account type prefix is validated later against options (name_assets, etc.).

Currency(&'src str)

A currency/commodity code like USD, EUR, AAPL, BTC. Uppercase letters, can contain digits, apostrophes, dots, underscores, hyphens. Note: This pattern is lower priority than Account, Keywords, and Flags. Currency must have at least 2 characters to avoid conflict with single-letter flags. Also supports / prefix for options/futures contracts (e.g., /ESM24, /LOX21_211204_P100.25). The / prefix requires an uppercase letter first to avoid matching /1.14 as currency.

Tag(&'src str)

A tag like #tag-name.

Link(&'src str)

A link like ^link-name.

Txn

The txn keyword for transactions.

Balance

The balance directive keyword.

Open

The open directive keyword.

Close

The close directive keyword.

Commodity

The commodity directive keyword.

Pad

The pad directive keyword.

Event

The event directive keyword.

Query

The query directive keyword.

Note

The note directive keyword.

Document

The document directive keyword.

Price

The price directive keyword.

Custom

The custom directive keyword.

Option_

The option directive keyword.

Include

The include directive keyword.

Plugin

The plugin directive keyword.

Pushtag

The pushtag directive keyword.

Poptag

The poptag directive keyword.

Pushmeta

The pushmeta directive keyword.

Popmeta

The popmeta directive keyword.

True

The TRUE boolean literal (also True, true).

False

The FALSE boolean literal (also False, false).

Null

The NULL literal.

LDoubleBrace

Double left brace {{ for cost specifications (legacy total cost).

RDoubleBrace

Double right brace }} for cost specifications.

LBraceHash

Left brace with hash {# for total cost (new syntax).

LBrace

Left brace { for cost specifications.

RBrace

Right brace } for cost specifications.

LParen

Left parenthesis ( for expressions.

RParen

Right parenthesis ) for expressions.

AtAt

Double at-sign @@ for total cost.

At

At-sign @ for unit cost.

Colon

Colon : separator.

Comma

Comma , separator.

Tilde

Tilde ~ for tolerance.

Pipe

Pipe | for deprecated payee/narration separator.

Plus

Plus + operator.

Minus

Minus - operator.

Star

Star * for cleared transactions and multiplication.

Slash

Slash / for division.

Pending

Pending flag ! for incomplete transactions.

Flag(&'src str)

Other transaction flags: P S T C U R M ? & Note: # and % are handled as comments when followed by space

Newline

Newline (significant in Beancount for directive boundaries).

Comment(&'src str)

A comment starting with semicolon. The slice includes the semicolon.

Hash

Hash token # used as separator in cost specs: {per_unit # total currency} Note: In Python beancount, # is only a comment at the START of a line. Mid-line # text is NOT a comment - it’s either a cost separator or syntax error. Start-of-line hash comments are handled in post-processing (tokenize function).

PercentComment(&'src str)

A percent comment (ledger-style). Python beancount accepts % as a comment character for ledger compatibility.

Shebang(&'src str)

Shebang line at start of file (e.g., #!/usr/bin/env bean-web). Treated as a comment-like directive to skip.

EmacsDirective(&'src str)

Emacs org-mode directive (e.g., “#+STARTUP: showall”). These are Emacs configuration lines that should be skipped.

MetaKey(&'src str)

A metadata key (identifier followed by colon). Examples: filename:, lineno:, custom-key:, nameOnCard: The slice includes the trailing colon. Keys can use camelCase or snake_case.

Indent(usize)

Indentation token (inserted by post-processing, not by Logos). Contains the number of leading spaces. This is a placeholder - actual indentation detection happens in tokenize.

DeepIndent(usize)

Deep indentation (4+ spaces) - used for posting-level metadata.

Error(&'src str)

Error token for unrecognized input. Contains the invalid source text for better error messages.

Implementations

impl Token<'_>

pub const fn is_txn_flag(&self) -> bool

Returns true if this is a transaction flag (* or !).

pub const fn is_directive_keyword(&self) -> bool

Returns true if this is a keyword that starts a directive.

Trait Implementations

impl<'src> Clone for Token<'src>

fn clone(&self) -> Token<'src>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'src> Debug for Token<'src>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Token<'_>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'s> Logos<'s> for Token<'s>

type Error = ()

Error type returned by the lexer. This can be set using #[logos(error = MyError)]. Defaults to () if not set.

type Extras = ()

Associated type Extras for the particular lexer. This can be set using #[logos(extras = MyExtras)] and accessed inside callbacks.

type Source = str

Source type this token can be lexed from. This will default to str, unless one of the defined patterns explicitly uses non-unicode byte values or byte slices, in which case that implementation will use [u8].

fn lex( lex: &mut Lexer<'s, Self>, ) -> Option<Result<Self, <Self as Logos<'s>>::Error>>

The heart of Logos. Called by the Lexer. The implementation for this function is generated by the logos-derive crate.

fn lexer(source: &'source Self::Source) -> Lexer<'source, Self>

where Self::Extras: Default,

Create a new instance of a Lexer that will produce tokens implementing this Logos.

fn lexer_with_extras( source: &'source Self::Source, extras: Self::Extras, ) -> Lexer<'source, Self>

Create a new instance of a Lexer with the provided Extras that will produce tokens implementing this Logos.

impl<'src> PartialEq for Token<'src>

fn eq(&self, other: &Token<'src>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'src> Eq for Token<'src>

impl<'src> StructuralPartialEq for Token<'src>