pub struct StreamingLanguageDetector { /* private fields */ }Expand description
Streaming language detector that wraps a [LanguageDetector] backend.
Takes ExpandUnit values one at a time and returns the detected
Language for each:
Wordunits trigger detection via a sliding context window with hysteresis — language only switches when confidence exceeds 0.5 + 0.20.NumberandMarkunits inherit the current language without detection.- Sentence boundaries (
.,?,!) should be signalled viareset_contextto clear the sliding window.
Implementations§
Source§impl StreamingLanguageDetector
impl StreamingLanguageDetector
Sourcepub fn with_lingua(languages: &[Language], default_language: Language) -> Self
pub fn with_lingua(languages: &[Language], default_language: Language) -> Self
Build a detector backed by the lingua library.
languages must contain at least two languages; default_language is
the starting assumption before any text is seen.
Sourcepub fn push(&mut self, unit: &ExpandUnit) -> Language
pub fn push(&mut self, unit: &ExpandUnit) -> Language
Push one expand unit and get back the language for that unit.
Words run through the detection algorithm; numbers and marks inherit the current language without running detection.
Sourcepub fn reset_context(&mut self)
pub fn reset_context(&mut self)
Clear the context window on sentence boundaries (., ?, !).
Auto Trait Implementations§
impl Freeze for StreamingLanguageDetector
impl !RefUnwindSafe for StreamingLanguageDetector
impl Send for StreamingLanguageDetector
impl Sync for StreamingLanguageDetector
impl Unpin for StreamingLanguageDetector
impl UnsafeUnpin for StreamingLanguageDetector
impl !UnwindSafe for StreamingLanguageDetector
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more