Struct WandaImportance 

pub struct WandaImportance { /* private fields */ }

Wanda (Weights and Activations) importance estimator.

Combines weight magnitudes with input activation norms to identify important weights. This method is from Sun et al. (2023) and achieves strong results with no retraining needed.

Formula

importance = |w| * sqrt(activation_norm)

Where activation_norm is the L2 norm of input activations across calibration samples for each input channel.

Advantages

• No gradient computation needed
• No retraining required after pruning
• Works well at moderate sparsity (50%)
• Very fast (single forward pass for calibration)

Requirements

• Calibration data (128 samples typically sufficient)
• Activation statistics for target layer

Implementations

impl WandaImportance

pub fn new(layer_name: impl Into<String>) -> WandaImportance

Create Wanda importance estimator for a specific layer.

Arguments

• layer_name - Layer identifier to look up in CalibrationContext

pub fn with_pattern(self, pattern: SparsityPattern) -> WandaImportance

Set sparsity pattern constraint.

Arguments

• pattern - N:M pattern or other structural constraint

pub fn with_eps(self, eps: f32) -> WandaImportance

Set epsilon for numerical stability.

Arguments

• eps - Small value to prevent division by zero (default: 1e-8)

pub fn layer_name(&self) -> &str

Get the layer name.

pub fn pattern(&self) -> Option<SparsityPattern>

Get the pattern if set.

pub fn compute_from_tensors( &self, weights: &Tensor, activation_norms: &Tensor, ) -> Result<Tensor, PruningError>

Compute Wanda importance scores.

Arguments

• weights - Weight tensor of shape [out_features, in_features]
• activation_norms - L2 norms of input activations [in_features]

Returns

Importance scores with same shape as weights.

Formula

importance[i,j] = |weights[i,j]| * sqrt(activation_norms[j])

Trait Implementations

impl Clone for WandaImportance

fn clone(&self) -> WandaImportance

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for WandaImportance

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Importance for WandaImportance

fn compute( &self, module: &dyn Module, context: Option<&CalibrationContext>, ) -> Result<ImportanceScores, PruningError>

Compute importance scores for a module’s parameters.

fn name(&self) -> &'static str

Name of this importance method.

fn requires_calibration(&self) -> bool

Whether this method requires calibration data.