pub struct StatisticalAnalyzer { /* private fields */ }Expand description
Statistical language model-based analyzer for encoding detection.
Uses statistical models and language patterns to improve encoding detection accuracy beyond simple byte frequency analysis.
Implementations§
Source§impl StatisticalAnalyzer
impl StatisticalAnalyzer
Sourcepub fn new() -> Self
pub fn new() -> Self
Creates a new StatisticalAnalyzer with pre-built language models.
Initializes language models for various character encodings to enable statistical analysis of text patterns.
Sourcepub fn analyze_with_models(&self, data: &[u8]) -> Result<HashMap<Charset, f32>>
pub fn analyze_with_models(&self, data: &[u8]) -> Result<HashMap<Charset, f32>>
Analyzes byte data using language models to determine encoding likelihood.
§Arguments
data- The byte data to analyze
§Returns
Returns a HashMap mapping each charset to its likelihood score. Higher scores indicate higher likelihood that the data is encoded in that character set.
§Errors
Returns an error if the model scoring calculation fails.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for StatisticalAnalyzer
impl RefUnwindSafe for StatisticalAnalyzer
impl Send for StatisticalAnalyzer
impl Sync for StatisticalAnalyzer
impl Unpin for StatisticalAnalyzer
impl UnwindSafe for StatisticalAnalyzer
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<F, T> IntoSample<T> for Fwhere
T: FromSample<F>,
impl<F, T> IntoSample<T> for Fwhere
T: FromSample<F>,
fn into_sample(self) -> T
Source§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
The inverse inclusion map: attempts to construct
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
Checks if
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
Use with care! Same as
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
The inclusion map: converts
self to the equivalent element of its superset.