Struct PreAnalyzedToken 

pub struct PreAnalyzedToken {
    pub form: String,
    pub tag: String,
    pub begin: Option<usize>,
    pub end: Option<usize>,
}

Pre-analyzed token element passed to crate::KiwiBuilder::add_pre_analyzed_word.

begin/end are character offsets in the given surface form (Rust str.chars() index space, not byte offsets).

Fields

form: String

Surface form.

tag: String

Part-of-speech tag.

begin: Option<usize>

Optional begin character offset.

end: Option<usize>

Optional end character offset.

Implementations

impl PreAnalyzedToken

pub fn new(form: impl Into<String>, tag: impl Into<String>) -> Self

Creates a token with only form and tag.

pub fn with_span(self, begin: usize, end: usize) -> Self

Sets explicit span offsets.

Trait Implementations

impl Clone for PreAnalyzedToken

fn clone(&self) -> PreAnalyzedToken

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PreAnalyzedToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.