Struct EnsembleDistiller 

pub struct EnsembleDistiller {
    pub weights: Vec<f32>,
    pub temperature: f32,
}

Multi-Teacher Ensemble Distillation

Distills knowledge from multiple teacher models into a single student. Combines teacher predictions via averaging or weighted averaging.

Methods

• Average: Simple mean of all teacher logits
• Weighted: Weighted combination based on teacher confidence/accuracy

Example

use entrenar::distill::EnsembleDistiller;
use ndarray::array;

let distiller = EnsembleDistiller::new(vec![1.0, 1.0], 2.0);
let teachers = vec![
    array![[2.0, 1.0, 0.5]],
    array![[1.5, 1.2, 0.8]],
];
let ensemble_logits = distiller.combine_teachers(&teachers);

Fields

weights: Vec<f32>

Weights for each teacher (normalized to sum to 1)

temperature: f32

Temperature for distillation

Implementations

impl EnsembleDistiller

pub fn new(weights: Vec<f32>, temperature: f32) -> Self

Create a new ensemble distiller with given teacher weights

Arguments

• weights - Weight for each teacher (will be normalized)
• temperature - Temperature for softening distributions

Panics

Panics if weights are empty, all zero, or temperature <= 0

pub fn uniform(num_teachers: usize, temperature: f32) -> Self

Create an ensemble with uniform weights

pub fn combine_teachers(&self, teacher_logits: &[Array2<f32>]) -> Array2<f32>

Combine multiple teacher logits into ensemble prediction

Arguments

• teacher_logits - Vector of teacher logits, each [batch_size, num_classes]

Returns

Weighted average of teacher logits [batch_size, num_classes]

pub fn combine_via_probabilities( &self, teacher_logits: &[Array2<f32>], ) -> Array2<f32>

Combine teachers via probability distribution averaging (more stable)

Converts each teacher’s logits to probabilities, averages them, then converts back to logits.

pub fn distillation_loss( &self, student_logits: &Array2<f32>, teacher_logits: &[Array2<f32>], labels: &[usize], alpha: f32, ) -> f32

Compute ensemble distillation loss

Arguments

• student_logits - Logits from student [batch_size, num_classes]
• teacher_logits - Vector of teacher logits
• labels - Ground truth labels [batch_size]
• alpha - Weight for distillation vs hard loss

Trait Implementations

impl Clone for EnsembleDistiller

fn clone(&self) -> EnsembleDistiller

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EnsembleDistiller

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.