Struct DistillationLoss 

pub struct DistillationLoss {
    pub temperature: f64,
    pub alpha: f64,
    pub hard_loss: Box<dyn Loss>,
}

Knowledge distillation loss that combines student predictions with teacher soft targets.

Based on “Distilling the Knowledge in a Neural Network” (Hinton et al., 2015).

Fields

temperature: f64

Temperature for softening probabilities (higher = softer).

alpha: f64

Weight for distillation loss (1 - alpha for hard target loss).

hard_loss: Box<dyn Loss>

Base loss function for hard targets.

Implementations

impl DistillationLoss

pub fn new( temperature: f64, alpha: f64, hard_loss: Box<dyn Loss>, ) -> TrainResult<Self>

Create a new distillation loss.

Arguments

• temperature - Temperature for softening (typically 2.0-5.0)
• alpha - Weight for soft targets (typically 0.5-0.9)
• hard_loss - Loss function for hard targets

pub fn compute_distillation( &self, student_logits: &ArrayView<'_, f64, Ix2>, teacher_logits: &ArrayView<'_, f64, Ix2>, hard_targets: &ArrayView<'_, f64, Ix2>, ) -> TrainResult<f64>

Compute distillation loss combining soft and hard targets.

Arguments

• student_logits - Raw student model outputs (before softmax)
• teacher_logits - Raw teacher model outputs (before softmax)
• hard_targets - True labels (one-hot encoded)

Returns

Combined distillation loss