Struct EntropyAnalyzer 

pub struct EntropyAnalyzer { /* private fields */ }

Analyzer that computes Shannon entropy and related information theory metrics.

Entropy measures the average information content or uncertainty in a dataset. Higher entropy indicates more randomness/diversity, while lower entropy indicates more predictability/uniformity.

Metrics Computed

• Shannon Entropy: -Σ(p_i * log2(p_i)) where p_i is the probability of each value
• Normalized Entropy: Entropy divided by log2(n) where n is the number of unique values
• Gini Impurity: 1 - Σ(p_i²), another measure of diversity
• Effective Number of Values: 2^entropy, interpretable as the effective cardinality

Example

use term_guard::analyzers::advanced::EntropyAnalyzer;
use datafusion::prelude::*;

let ctx = SessionContext::new();
// Register your data table

let analyzer = EntropyAnalyzer::new("category");
let state = analyzer.compute_state_from_data(&ctx).await?;
let metric = analyzer.compute_metric_from_state(&state)?;

if let MetricValue::Map(metrics) = metric {
    println!("Category entropy: {:?} bits", metrics.get("entropy"));
    println!("Normalized entropy: {:?}", metrics.get("normalized_entropy"));
    println!("Effective categories: {:?}", metrics.get("effective_values"));
}

Implementations

impl EntropyAnalyzer

pub fn new(column: impl Into<String>) -> Self

Creates a new entropy analyzer for the specified column.

pub fn with_max_unique_values( column: impl Into<String>, max_unique_values: usize, ) -> Self

Creates a new entropy analyzer with a custom maximum unique values limit.

pub fn column(&self) -> &str

Returns the column being analyzed.

pub fn max_unique_values(&self) -> usize

Returns the maximum unique values limit.

Trait Implementations

impl Analyzer for EntropyAnalyzer

type State = EntropyState

The state type for incremental computation.

type Metric = MetricValue

The metric type produced by this analyzer.

fn compute_state_from_data<'life0, 'life1, 'async_trait>( &'life0 self, ctx: &'life1 SessionContext, ) -> Pin<Box<dyn Future<Output = AnalyzerResult<Self::State>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Computes the state from the input data.

fn compute_metric_from_state( &self, state: &Self::State, ) -> AnalyzerResult<Self::Metric>

Computes the final metric from the accumulated state.

fn name(&self) -> &str

Returns the name of this analyzer.

fn description(&self) -> &str

Returns a description of what this analyzer computes.

fn columns(&self) -> Vec<&str>

Returns the column(s) this analyzer operates on, if any.

fn merge_states(&self, states: Vec<Self::State>) -> AnalyzerResult<Self::State>

Merges multiple states into a single state.

fn metric_key(&self) -> String

Returns the metric key for storing results.

fn is_combinable(&self) -> bool

Indicates whether this analyzer can be combined with others.

impl Clone for EntropyAnalyzer

fn clone(&self) -> EntropyAnalyzer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EntropyAnalyzer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.