Struct StatisticalAnalyzer 

pub struct StatisticalAnalyzer { /* private fields */ }

Statistical language model-based analyzer for encoding detection.

Uses statistical models and language patterns to improve encoding detection accuracy beyond simple byte frequency analysis.

Implementations

impl StatisticalAnalyzer

pub fn new() -> Self

Creates a new StatisticalAnalyzer with pre-built language models.

Initializes language models for various character encodings to enable statistical analysis of text patterns.

pub fn analyze_with_models(&self, data: &[u8]) -> Result<HashMap<Charset, f32>>

Analyzes byte data using language models to determine encoding likelihood.

Arguments

• data - The byte data to analyze

Returns

Returns a HashMap mapping each charset to its likelihood score. Higher scores indicate higher likelihood that the data is encoded in that character set.

Errors

Returns an error if the model scoring calculation fails.

Trait Implementations

impl Default for StatisticalAnalyzer

fn default() -> Self

Returns the “default value” for a type.