Struct TaskStatistics 

pub struct TaskStatistics { /* private fields */ }

Running gradient statistics for one task.

Both moments are exact running means over every gradient the task has observed, which is what makes the derived embedding a property of the task rather than of the particular step it was sampled at.

Implementations

impl TaskStatistics

pub fn observe(&mut self, gradient: &[f64])

Fold one gradient into the running statistics.

A gradient of a different length than the ones seen so far restarts the statistics: it can only come from a task whose parameter vector changed shape, and averaging across shapes would produce a meaningless descriptor.

pub fn observations(&self) -> usize

Number of gradients folded in.

pub fn mean_gradient(&self) -> &[f64]

Running mean gradient.

pub fn mean_squared_gradient(&self) -> &[f64]

Running mean squared gradient.

pub fn embedding(&self, dim: usize) -> Vec<f64>

Fixed-width task embedding derived from the statistics.

The descriptor is the whitened mean gradient E[g_i] / sqrt(E[g_i^2] + eps): dividing the first moment by the square root of the second is the diagonal natural-gradient direction, so a coordinate that is merely noisy (large E[g^2], small E[g]) does not dominate a coordinate that consistently pushes the same way. That vector is then reduced to dim components by signed feature hashing (Weinberger et al., “Feature Hashing for Large Scale Multitask Learning”, ICML 2009), which preserves inner products in expectation while making tasks of different parameter dimensionality comparable, and finally normalized to unit length so cosine similarity is a plain dot product.

A task whose gradients have averaged out to exactly zero has no direction; its embedding is all zeros and its similarity to everything is zero, which is the honest answer rather than an arbitrary one.

Trait Implementations

impl Clone for TaskStatistics

fn clone(&self) -> TaskStatistics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TaskStatistics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TaskStatistics

fn default() -> TaskStatistics

Returns the “default value” for a type.