Enum TokenizerMode 

pub enum TokenizerMode {
    None,
    Jieba,
}

分词模式（中英双语）。 Tokenizer mode selector (bilingual).

Variants

None

常规模式：不做中文切词，仅做轻量规范化。 Plain mode: no Chinese segmentation, only light normalization.

Jieba

Jieba 模式：使用 Jieba 与内部伴生词典进行切词。 Jieba mode: tokenize with Jieba and the internal companion dictionary.

Implementations

impl TokenizerMode

pub fn as_str(self) -> &'static str

获取稳定的字符串模式名（中英双语）。 Return the stable string representation for the tokenizer mode (bilingual).

pub fn parse(value: &str) -> Option<Self>

从字符串解析分词模式（中英双语）。 Parse tokenizer mode from string (bilingual).

Trait Implementations

impl Clone for TokenizerMode

fn clone(&self) -> TokenizerMode

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenizerMode

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenizerMode

fn default() -> TokenizerMode

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for TokenizerMode

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for TokenizerMode

fn eq(&self, other: &TokenizerMode) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for TokenizerMode

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for TokenizerMode

impl Eq for TokenizerMode

impl StructuralPartialEq for TokenizerMode