Struct AxisSlices 

pub struct AxisSlices { /* private fields */ }

A tile cut into slices along axis, one for every index of its other axes, each read and written whole where the tile’s holder keeps it: the verbs an op reducing or broadcasting along one axis runs. The slices write through to the tile.

axis is the tile’s innermost, the one its cells are contiguous along; any other is refused by name.

Implementations

impl AxisSlices

pub fn mul<E: Float>(&mut self, factors: &Array<E>)

self[i, s] *= factors[i]. A plane’s fragments bounce through its scratch, met on sync_plane: every unit of the plane calls it.

pub fn probabilities<E: Float>( &mut self, scale: E, bias: &Tile, logsumexp: &Tile, )

self[c] = exp(self[c] · scale + bias[c] − logsumexp[c]): the probabilities of a softmax whose logsumexp is known, read back from it instead of walking the score again. logsumexp is a window of memory that omits the axis the softmax ran along, which it broadcasts over: along the slices where that axis is the innermost, along each slice where it is not, as a walk owned by the keys holds its scores keys first. Its other axes are this tile’s, or one wide. A cell whose logsumexp is the minimum value, every key of its softmax masked, is left at zero.

pub fn softmax_gradient<E: Float>( &mut self, probabilities: &AxisSlices, delta: &Tile, )

self[c] = probabilities[c] · (self[c] − delta[c]): a softmax’s gradient against its scores, where self holds the gradient against its probabilities and delta is, for each softmax, Σ probabilities · self along the axis it ran along. delta broadcasts as probabilities’s logsumexp does; the probabilities are cut as self is, held by the same holder.

pub fn __expand_mul<E: Float>( scope: &Scope, this: &mut <Self as CubeType>::ExpandType, factors: &<Array<E> as CubeType>::ExpandType, )

self[i, s] *= factors[i]. A plane’s fragments bounce through its scratch, met on sync_plane: every unit of the plane calls it.

pub fn __expand_probabilities<E: Float>( scope: &Scope, this: &mut <Self as CubeType>::ExpandType, scale: <E as CubeType>::ExpandType, bias: &<Tile as CubeType>::ExpandType, logsumexp: &<Tile as CubeType>::ExpandType, )

self[c] = exp(self[c] · scale + bias[c] − logsumexp[c]): the probabilities of a softmax whose logsumexp is known, read back from it instead of walking the score again. logsumexp is a window of memory that omits the axis the softmax ran along, which it broadcasts over: along the slices where that axis is the innermost, along each slice where it is not, as a walk owned by the keys holds its scores keys first. Its other axes are this tile’s, or one wide. A cell whose logsumexp is the minimum value, every key of its softmax masked, is left at zero.

pub fn __expand_softmax_gradient<E: Float>( scope: &Scope, this: &mut <Self as CubeType>::ExpandType, probabilities: &<AxisSlices as CubeType>::ExpandType, delta: &<Tile as CubeType>::ExpandType, )

self[c] = probabilities[c] · (self[c] − delta[c]): a softmax’s gradient against its scores, where self holds the gradient against its probabilities and delta is, for each softmax, Σ probabilities · self along the axis it ran along. delta broadcasts as probabilities’s logsumexp does; the probabilities are cut as self is, held by the same holder.

Trait Implementations

impl CubeType for AxisSlices

type ExpandType = AxisSlicesExpand