Struct UnicodeSentenceSplitter 

pub struct UnicodeSentenceSplitter { /* private fields */ }

Sentence splitter using Unicode UAX #29 with abbreviation-aware merging.

Implementations

impl UnicodeSentenceSplitter

pub fn new() -> Self

Create a splitter with only built-in English abbreviations.

pub fn with_extra_abbreviations(extras: &[String]) -> Self

Create a splitter with additional user-provided abbreviations.

pub fn for_lang(lang: &str, extras: &[String]) -> Self

Create a splitter for a specific language, optionally with extra abbreviations.

pub fn with_verbatim_commands(self, cmds: Vec<String>) -> Self

Extra LaTeX command names tokenized like \verb before split.

impl UnicodeSentenceSplitter

pub fn refine_segments(&self, segments: Vec<String>) -> Vec<String>

Apply abbreviation + delimiter-span merges to an already-segmented list.

Used by the neural backend so --neural shares the same post-pipeline guarantees (dialogue quotes, Dr., balanced spans) as the UAX path.

Trait Implementations

impl Default for UnicodeSentenceSplitter

fn default() -> Self

Returns the “default value” for a type.

impl SentenceSplitter for UnicodeSentenceSplitter

fn split(&self, text: &str) -> Vec<String>

Split a prose string into individual sentences.