Struct Token 

pub struct Token {
    pub id: u32,
    pub name_offset: u32,
    pub file_id: u16,
    pub column: u16,
    pub line: u32,
    pub kind: u8,
    pub scope_id: u32,
    /* private fields */
}

A token (identifier) occurrence in the source code

Layout (24 bytes):

• id: u32 (4)
• name_offset: u32 (4)
• file_id: u16 (2)
• column: u16 (2)
• line: u32 (4)
• kind: u8 (1)
• _padding: [u8; 3] (3)
• scope_id: u32 (4)

Fields

id: u32

Unique token ID

name_offset: u32

Offset into string table for the token name

file_id: u16

File ID where this token appears

column: u16

Column number (0-indexed)

line: u32

Line number (1-indexed)

kind: u8

Token kind

scope_id: u32

Scope ID this token belongs to

Implementations

impl Token

pub const fn new( id: u32, name_offset: u32, file_id: u16, line: u32, column: u16, kind: TokenKind, scope_id: u32, ) -> Self

Create a new token

pub fn token_kind(&self) -> TokenKind

Get the token kind

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Token

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Token

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for Token

impl Eq for Token

impl StructuralPartialEq for Token