Struct ResponseDistiller 

pub struct ResponseDistiller {
    pub soft_loss: DistilLoss,
    pub hard_label_weight: f32,
    pub soft_label_weight: f32,
}

Response-based knowledge distillation.

Combines a hard-label cross-entropy loss with a soft-label distillation loss.

Fields

soft_loss: DistilLoss

Distillation loss applied to soft targets.

hard_label_weight: f32

Weight for the hard-label (cross-entropy) term.

soft_label_weight: f32

Weight for the soft-label (distillation) term.

Implementations

impl ResponseDistiller

pub fn new( soft_loss: DistilLoss, hard_label_weight: f32, soft_label_weight: f32, ) -> Self

Create a response distiller.

Parameters

• soft_loss — distillation loss (e.g., KL divergence with temperature).
• hard_label_weight — weight α for cross-entropy term.
• soft_label_weight — weight β for distillation term.

pub fn pure_kl(temperature: f32) -> Self

Pure distillation (no hard labels): KL divergence at temperature tau.

pub fn balanced() -> Self

Combined distillation: 0.5 × CE + 0.5 × KL(τ=4).

pub fn compute_loss( &self, student_logits: &[f32], teacher_logits: &[f32], hard_label: usize, ) -> QuantResult<f32>

Compute the combined distillation loss.

Parameters

• student_logits — unnormalised student output (length = n_classes).
• teacher_logits — unnormalised teacher output (same length).
• hard_label — integer ground-truth class index.

Errors

• QuantError::EmptyInput — either logit slice is empty.
• QuantError::TeacherStudentMismatch — logit slices differ in length.
• QuantError::DimensionMismatch — hard_label ≥ n_classes.

pub fn compute_batch_loss( &self, student_batch: &[f32], teacher_batch: &[f32], hard_labels: &[usize], n_classes: usize, ) -> QuantResult<f32>

Compute distillation loss over a batch of examples.

Returns the average loss over all examples in the batch.

Parameters

• student_batch — [batch_size, n_classes] row-major student logits.
• teacher_batch — [batch_size, n_classes] row-major teacher logits.
• hard_labels — [batch_size] integer class labels.
• n_classes — number of output classes.

Errors

Propagates dimension and empty-input errors.

Trait Implementations

impl Clone for ResponseDistiller

fn clone(&self) -> ResponseDistiller

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ResponseDistiller

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.