Expand description
Inline token types and specifications
This module defines the token types for inline parsing. Inline elements are span-based
elements that can start and end at arbitrary positions within text content. Unlike the
line-based tokens (core and line), inline tokens operate at the character level and can
be nested within each other.
For the complete inline grammar specification, see specs/v1/grammar-inline.lex.
Related token modules:
- [core](super::core) - Character and word-level tokens from the logos lexer
- [line](super::line) - Line-based tokens for the main parserInline Token Types
These are the inline element types supported by lex:
- Strong: *text* (bold/emphasis)
- Emphasis: _text_ (italic)
- Code: `text` (monospace, literal)
- Math: #formula# (mathematical notation, literal)
- Reference: [target] (links, citations, footnotes)
References support multiple subtypes including:
- Citations: [@key] or [@key1; @key2, pp. 42-45]
- Footnotes: [^label] or [42]
- Session references: [#2.1]
- URLs: [https://example.com]
- File paths: [./path/to/file]
- TK placeholders: [TK] or [TK-identifier]
- General references: [Section Title]
Inline elements have these properties:
- Clear start and end markers (single character tokens)
- Can be nested (except literal types)
- Cannot break parent element boundaries
- No space allowed between marker and contentToken Specifications
Each inline type is defined by an InlineSpec that specifies:
- The kind of inline element (from InlineKind enum)
- Start and end tokens (characters)
- Whether it's literal (no nested parsing inside)
- Optional post-processing callback for complex logicCitation Parsing
Citations are a specialized form of reference that follow academic citation format.
The reference token [target] is post-processed to detect citations starting with @.
Citation parsing is handled by [crate::lex::inlines::citations] which extracts:
- Multiple citation keys (separated by ; or ,)
- Optional page locators (p. or pp. followed by page ranges)
- Page ranges in various formats: single pages, ranges, or listsEnumsยง
- Inline
Kind - The type of inline element