Trait lib_ruby_parser::TokenTrait[−][src]

pub trait TokenTrait: Clone + PartialEq + Eq + Debug {

Show methods    fn new(
        token_type: i32, 
        token_value: Bytes, 
        loc: Loc, 
        lex_state_before: LexState, 
        lex_state_after: LexState
    ) -> Self;

    fn token_type(&self) -> i32;

    fn token_value(&self) -> &Bytes;

    fn set_token_value(&mut self, token_value: Bytes);

    fn into_token_value(self) -> Bytes;

    fn loc(&self) -> Loc;

    fn lex_state_before(&self) -> LexState;

    fn lex_state_after(&self) -> LexState;

    fn as_bytes(&self) -> &[u8] { ... }

    fn into_bytes(self) -> Vec<u8>
    where
        Self: Sized,
    { ... }

    fn as_str_lossy(&self) -> Result<&str, Utf8Error> { ... }

    fn to_string_lossy(&self) -> String { ... }

    fn to_string(&self) -> Result<String, FromUtf8Error> { ... }

    fn into_string(self) -> Result<String, FromUtf8Error>
    where
        Self: Sized,
    { ... }

    fn token_name(&self) -> &'static str { ... }

}
Expand description

Trait with common methods of Token (Rust- or external-based)

Required methods

Constructor

Returns type of the token

Returns value of the token

Sets token value

Consumes self, returns owned values of the token

Returns location of the token

Returns lex state before reading the token

Returns lex state after reading the token

Provided methods

Returns a byte array of the token value

Consumes a token and returns an owned byte array of the token value

Converts token value into &str

Converts token to a string, replaces unknown chars to U+FFFD

Converts token to a string

Consumes a token and converts it into a string

Returns name of the token

Implementors