Skip to main content

Token

Enum Token 

Source
pub enum Token {
Show 27 variants Ident(String), Function(String), AtKeyword(String), Hash(String, HashType), String(String), BadString, Url(String), BadUrl, Delim(char), Number(Numeric), Percentage(Numeric), Dimension(Numeric, String), UnicodeRange(Option<u32>, Option<u32>), Whitespace, Comment(String), Colon, Semicolon, Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, OpenBrace, CloseBrace, Cdo, Cdc, Eof,
}
Expand description

A CSS token (§4.1).

Variants correspond to the token types defined by the CSS Syntax Module §4.1 railroad diagrams. Each variant’s doc comment cites the relevant normative section.

Variants§

§

Ident(String)

<ident-token> (§4.1). An identifier such as color, auto, -webkit-flex. The value is the decoded ident sequence (escapes resolved, §4.3.12).

§

Function(String)

<function-token> (§4.1). An identifier immediately followed by (, e.g. translate(, var(. The value is the function name (without the parenthesis).

§

AtKeyword(String)

<at-keyword-token> (§4.1). @ followed by an ident sequence, e.g. @media, @import. The value is the keyword (without @).

§

Hash(String, HashType)

<hash-token> (§4.1). # followed by an ident sequence or hex digits, e.g. #main, #ff00aa. The HashType flag indicates whether the hash is an “id” hash (would-start-an-ident-sequence), an “unrestricted” hash (hex digits that don’t form an ident), or “unknown” for hashes that don’t match either pattern.

§

String(String)

<string-token> (§4.1). A quoted string ("..." or '...') with escapes resolved. The value is the string’s contents without the surrounding quotes.

§

BadString

<bad-string-token> (§4.1). Emitted when a string contains an unescaped newline (§4.3.5).

§

Url(String)

<url-token> (§4.1). The url(...) form with an unquoted or quoted URL. The value is the decoded URL contents.

§

BadUrl

<bad-url-token> (§4.1). Emitted when a URL contains an invalid escape or unterminated quoted segment (§4.3.6).

§

Delim(char)

<delim-token> (§4.1). A single code point not consumed by any other token type, e.g. >, +, ~ (when not part of ~=), ! (when not part of !=).

§

Number(Numeric)

<number-token> (§4.1). A numeric value, possibly signed, possibly with a fractional part or scientific notation. The Numeric carries the parsed value and integer/number flag.

§

Percentage(Numeric)

<percentage-token> (§4.1). A number immediately followed by %.

§

Dimension(Numeric, String)

<dimension-token> (§4.1). A number immediately followed by a unit (an ident sequence), e.g. 10px, 1.5em, -30deg. The value carries the Numeric and the unit string.

§

UnicodeRange(Option<u32>, Option<u32>)

<unicode-range-token> (§4.1). A range of Unicode code points in the U+ form, e.g. U+1234, U+12-34FF, U+12??. The two Option<u32> values are the start and end of the range. Either may be None only in malformed inputs that the spec still accepts as a unicode-range; in practice both are Some for well-formed ranges.

§

Whitespace

<whitespace-token> (§4.1). A run of one or more whitespace code points (space, tab, newline, §4.2 definition of whitespace).

§

Comment(String)

<comment-token> (§4.1). The contents of a /* ... */ comment (without the delimiters). Comment tokens are only emitted in “tokenizer-friendly” mode; the default stylesheet tokenizer discards them. This implementation emits them so callers can choose.

§

Colon

<colon-token> (§4.1). :.

§

Semicolon

<semicolon-token> (§4.1). ;.

§

Comma

<comma-token> (§4.1). ,.

§

OpenBracket

<[-token> (§4.1). [.

§

CloseBracket

<]-token> (§4.1). ].

§

OpenParen

<(-token> (§4.1). (.

§

CloseParen

<)-token> (§4.1). ).

§

OpenBrace

<{-token> (§4.1). {.

§

CloseBrace

<}-token> (§4.1). }.

§

Cdo

<CDO-token> (§4.1). The literal <!-- (Comment Declaration Open). Kept for backwards compatibility with CSS 1/2.1’s practice of wrapping stylesheets in <!-- … --> to hide them from ancient browsers. At the tokenizer level this is a single token spanning all four code points.

§

Cdc

<CDC-token> (§4.1). The literal --> (Comment Declaration Close). See Token::Cdo.

§

Eof

<EOF-token> (§5.3). Emitted once when the input stream is exhausted. [Tokenizer::next_token] returns None after this.

Trait Implementations§

Source§

impl Clone for Token

Source§

fn clone(&self) -> Token

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Token

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
Source§

impl PartialEq for Token

Source§

fn eq(&self, other: &Token) -> bool

Equality operator ==. Read more
1.0.0 (const: unstable) · Source§

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=. Read more
Source§

impl StructuralPartialEq for Token

Auto Trait Implementations§

§

impl Forget for Token

§

impl Freeze for Token

§

impl RefUnwindSafe for Token

§

impl Send for Token

§

impl Sync for Token

§

impl Unpin for Token

§

impl UnsafeUnpin for Token

§

impl UnwindSafe for Token

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.