Enum SegmentationStrategy 

pub enum SegmentationStrategy {
    BPE {
        vocab_size: usize,
    },
    Character,
    Syllable,
    RuleBased {
        boundary_chars: Vec<char>,
    },
    External {
        model_path: String,
    },
}

Strategy for morphological segmentation.

Variants

BPE

Byte-Pair Encoding (BPE) based segmentation

Fields

vocab_size: usize

Target vocabulary size

Character

Character-level segmentation (fallback)

Syllable

Syllable-based segmentation (for syllabic scripts like Cherokee)

RuleBased

Rule-based segmentation using morpheme boundaries

Fields

boundary_chars: Vec<char>

Boundary markers (e.g., “-” for hyphenated morphemes)

External

External morphological analyzer (FST-based)

Fields

model_path: String

Path to analyzer model

Trait Implementations

impl Clone for SegmentationStrategy

fn clone(&self) -> SegmentationStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SegmentationStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SegmentationStrategy

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for SegmentationStrategy

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for SegmentationStrategy

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.