Struct LinguaLanguageBlockSplitter 

pub struct LinguaLanguageBlockSplitter { /* private fields */ }

Language splitter and detector.

This is intended to be used near/at the start of the segmentation chain.

Performance: There is a significant spin-up cost involved when using this struct for the first time that won’t happen on subsequent uses. This only happens once per program, independent of you keeping a specific instance or not. See the lingua documentation for RAM requirements.

Using low accuracy mode probably isn’t worth it.

Language support: Currently this crate uses whatlang::Lang to communicate languages, so the language support is the intersection of whatlang and what lingua::Language both support.

Implementations

impl LinguaLanguageBlockSplitter

pub fn new() -> Self

Create a new LiguaLanguageBlockSplitter instance that is configured to preload all languages on the first use.

pub fn new_with_builder(builder: LanguageDetectorBuilder) -> Self

Create a new LiguaLanguageBlockSplitter from a custom LanguageDetectorBuilder.

Trait Implementations

impl Default for LinguaLanguageBlockSplitter

fn default() -> Self

Returns the “default value” for a type.

impl Segmenter for LinguaLanguageBlockSplitter

type SubdivisionIter<'a> = IntoIter<SegmentedToken<'a>>

The iterator type returned by the subdivide function if it has multiple results.

fn subdivide<'a>( &self, token: SegmentedToken<'a>, ) -> UseOrSubdivide<SegmentedToken<'a>, IntoIter<SegmentedToken<'a>>>

A method that should split the given token into zero, one or more subtokens.