Enum Token 

pub enum Token {
    ControlSymbol(char),
    ControlWord {
        name: String,
        arg: Option<i32>,
    },
    ControlBin(Vec<u8>),
    Text(Vec<u8>),
    StartGroup,
    EndGroup,
    Newline(Vec<u8>),
}

Variants

ControlSymbol(char)

ControlWord

Fields

name: String

arg: Option<i32>

ControlBin(Vec<u8>)

Text(Vec<u8>)

Text is not str because it can be in any of various encodings - it’s up to the processor to identify any encoding information in the stream, and do any encoding conversion desired

StartGroup

EndGroup

Newline(Vec<u8>)

stores the actual bytes of newline found

Implementations

impl Token

pub fn to_rtf(&self) -> Vec<u8>

pub fn token_delimiter_after(&self, next_token: &Token) -> &'static str

This function returns a control word delimiter if one is required, or an empty string if none is required

Control Word tokens must be delimited by a non-alphanumeric value, so if the subsequent content could be alphanumeric, a space (’ ’) delimiter must be inserted

pub fn token_delimiter_before(&self, prev_token: &Token) -> &'static str

This function returns a control word delimiter if one is required, or an empty string if none is required

Control Word tokens must be delimited by a non-alphanumeric value, so if the subsequent content could be alphanumeric, a space (’ ’) delimiter must be inserted

pub fn get_name(&self) -> Option<String>

pub fn get_arg(&self) -> Option<i32>

pub fn get_symbol(&self) -> Option<char>

pub fn get_bin(&self) -> Option<&[u8]>

pub fn get_text(&self) -> Option<&[u8]>

Trait Implementations

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Token

impl StructuralPartialEq for Token