Struct LinguaLanguageBlockSentenceSplitter 

pub struct LinguaLanguageBlockSentenceSplitter { /* private fields */ }

Combination of [LinguaLanguageBlockSplitter][crate::segmentation::LiguaLanguageBlockSplitter] and the UnicodeSentenceSplitter.

This is intended to be used near/at the start of the segmentation chain.

In case no language is detected it falls back to only splitting sentences.

Performance: There is a significant spin-up cost involved when using this struct for the first time that won’t happen on subsequent uses. This only happens once per program, independent of you keeping a specific instance or not. See the lingua documentation for RAM requirements.

Using low accuracy mode probably isn’t worth it.

Language support: Currently this crate uses whatlang::Lang to communicate languages, so the language support is the intersection of whatlang and what lingua::Language both support.

Implementations

impl LinguaLanguageBlockSentenceSplitter

pub fn new() -> Self

Create a new LiguaLanguageBlockSplitter instance that is configured to preload all languages on the first use.

pub fn new_with_builder(builder: LanguageDetectorBuilder) -> Self

Create a new LiguaLanguageBlockSplitter from a custom LanguageDetectorBuilder.

Trait Implementations

impl Default for LinguaLanguageBlockSentenceSplitter

fn default() -> Self

Returns the “default value” for a type.

impl Segmenter for LinguaLanguageBlockSentenceSplitter

type SubdivisionIter<'a> = IntoIter<SegmentedToken<'a>>

The iterator type returned by the subdivide function if it has multiple results.

fn subdivide<'a>( &self, token: SegmentedToken<'a>, ) -> UseOrSubdivide<SegmentedToken<'a>, IntoIter<SegmentedToken<'a>>>

A method that should split the given token into zero, one or more subtokens.