Struct notmecab::ParserToken

pub struct ParserToken {
    pub surface: String,
    pub feature: String,
    pub original_id: u32,
    pub unknown: bool,
}

Fields

surface: String

Exact sequence of characters with which this token appeared in the string that was parsed.

feature: String

Description of this token’s features.

The feature string contains almost all useful information, including things like part of speech, spelling, pronunciation, etc.

The exact format of the feature string is dictionary-specific.

original_id: u32

Unique identifier of what specific lexeme realization this is, from the mecab dictionary. changes between dictionary versions.

unknown: bool

Whether this token is known in the mecab dictionary or not.

A value of true means that the character or characters under this token could not be parsed as part of any words in context.

Trait Implementations

impl Clone for ParserToken

fn clone(&self) -> ParserToken

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ParserToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.