Enum TokenContext

pub enum TokenContext {
    Document,
    SectionHeader,
    FieldValue,
    StyleOverride,
    DrawingCommands,
    UuEncodedData,
}

Tokenization context for state-aware parsing

Tracks current parsing context to enable context-sensitive tokenization of ASS script elements that have different lexical rules in different contexts.

Variants

Document

Top-level document parsing

Default state for processing section headers, comments, and top-level document structure.

SectionHeader

Inside section header like [Events]

Special tokenization rules for section names within square brackets.

FieldValue

Inside field definition line

Field values have different whitespace and delimiter handling than other contexts.

StyleOverride

Inside style override block like {\b1}

Override tags use backslash prefixes and have special syntax rules.

DrawingCommands

Inside drawing commands (\p1)

Drawing commands use vector graphics syntax with different coordinate and command parsing rules.

UuEncodedData

Inside UU-encoded data (fonts/graphics)

Binary data sections use different character validation and line parsing rules.

Implementations

impl TokenContext

pub const fn allows_whitespace_skipping(self) -> bool

Check if context allows whitespace skipping

pub const fn is_delimited_block(self) -> bool

Check if context is inside a delimited block

pub const fn closing_delimiter(self) -> Option<char>

Get expected closing delimiter for context

pub const fn enter_field_value(self) -> Self

Transition to field value context after colon

pub const fn reset_to_document(self) -> Self

Reset to document context (typically after newline)

Trait Implementations

impl Clone for TokenContext

fn clone(&self) -> TokenContext

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenContext

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenContext

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for TokenContext

fn eq(&self, other: &TokenContext) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TokenContext

impl Eq for TokenContext

impl StructuralPartialEq for TokenContext