Struct ActivationCache 

pub struct ActivationCache { /* private fields */ }

Stores per-layer last-token activations from a forward pass.

Each tensor has shape [d_model] — the residual stream activation at the final sequence position for a given layer.

Example

use candle_mi::ActivationCache;
use candle_core::{Device, Tensor};

let mut cache = ActivationCache::with_capacity(32);
cache.push(Tensor::zeros(128, candle_core::DType::F32, &Device::Cpu).unwrap());
cache.push(Tensor::zeros(128, candle_core::DType::F32, &Device::Cpu).unwrap());
assert_eq!(cache.n_layers(), 2);

Implementations

impl ActivationCache

pub const fn new(activations: Vec<Tensor>) -> Result<Self>

Create a new cache from collected activations.

Errors

Currently infallible but returns Result for forward compatibility.

pub fn with_capacity(n_layers: usize) -> Self

Create an empty cache with capacity for n_layers layers.

pub fn push(&mut self, tensor: Tensor)

Add a layer’s activation to the cache.

pub fn get_layer(&self, layer: usize) -> Option<&Tensor>

Get the activation for a specific layer.

pub const fn n_layers(&self) -> usize

Number of cached layers.

pub const fn is_empty(&self) -> bool

Whether the cache is empty.

pub fn activations(&self) -> &[Tensor]

All cached activations as a slice.

pub fn to_f32_vecs(&self) -> Result<Vec<Vec<f32>>>

Extract activations as f32 vectors.

Returns one Vec<f32> of shape [d_model] per layer.

Errors

Returns MIError::Model if dtype conversion or flattening fails.

Trait Implementations

impl Debug for ActivationCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.