Struct AxisSlicesExpand 

pub struct AxisSlicesExpand { /* private fields */ }

Implementations

impl AxisSlicesExpand

pub fn __expand_mul_method<E: Float>( &mut self, scope: &Scope, factors: &<Array<E> as CubeType>::ExpandType, )

self[i, s] *= factors[i]. A plane’s fragments bounce through its scratch, met on sync_plane: every unit of the plane calls it.

pub fn __expand_probabilities_method<E: Float>( &mut self, scope: &Scope, scale: <E as CubeType>::ExpandType, bias: &<Tile as CubeType>::ExpandType, logsumexp: &<Tile as CubeType>::ExpandType, )

self[c] = exp(self[c] · scale + bias[c] − logsumexp[c]): the probabilities of a softmax whose logsumexp is known, read back from it instead of walking the score again. logsumexp is a window of memory that omits the axis the softmax ran along, which it broadcasts over: along the slices where that axis is the innermost, along each slice where it is not, as a walk owned by the keys holds its scores keys first. Its other axes are this tile’s, or one wide. A cell whose logsumexp is the minimum value, every key of its softmax masked, is left at zero.

pub fn __expand_softmax_gradient_method<E: Float>( &mut self, scope: &Scope, probabilities: &<AxisSlices as CubeType>::ExpandType, delta: &<Tile as CubeType>::ExpandType, )

self[c] = probabilities[c] · (self[c] − delta[c]): a softmax’s gradient against its scores, where self holds the gradient against its probabilities and delta is, for each softmax, Σ probabilities · self along the axis it ran along. delta broadcasts as probabilities’s logsumexp does; the probabilities are cut as self is, held by the same holder.

pub fn __expand_mul<E: Float>( scope: &Scope, this: &mut Self, factors: &<Array<E> as CubeType>::ExpandType, )

self[i, s] *= factors[i]. A plane’s fragments bounce through its scratch, met on sync_plane: every unit of the plane calls it.

pub fn __expand_probabilities<E: Float>( scope: &Scope, this: &mut Self, scale: <E as CubeType>::ExpandType, bias: &<Tile as CubeType>::ExpandType, logsumexp: &<Tile as CubeType>::ExpandType, )

self[c] = exp(self[c] · scale + bias[c] − logsumexp[c]): the probabilities of a softmax whose logsumexp is known, read back from it instead of walking the score again. logsumexp is a window of memory that omits the axis the softmax ran along, which it broadcasts over: along the slices where that axis is the innermost, along each slice where it is not, as a walk owned by the keys holds its scores keys first. Its other axes are this tile’s, or one wide. A cell whose logsumexp is the minimum value, every key of its softmax masked, is left at zero.

pub fn __expand_softmax_gradient<E: Float>( scope: &Scope, this: &mut Self, probabilities: &<AxisSlices as CubeType>::ExpandType, delta: &<Tile as CubeType>::ExpandType, )

self[c] = probabilities[c] · (self[c] − delta[c]): a softmax’s gradient against its scores, where self holds the gradient against its probabilities and delta is, for each softmax, Σ probabilities · self along the axis it ran along. delta broadcasts as probabilities’s logsumexp does; the probabilities are cut as self is, held by the same holder.

Trait Implementations

impl AsMutExpand for AxisSlicesExpand

fn __expand_ref_mut_method(&mut self, _: &Scope) -> &mut Self

fn __expand_as_mut_method(&mut self, scope: &Scope) -> &mut T

impl AsRefExpand for AxisSlicesExpand

fn __expand_ref_method(&self, _: &Scope) -> &Self

fn __expand_as_ref_method(&self, scope: &Scope) -> &T

impl CubeDebug for AxisSlicesExpand

fn set_debug_name(&self, scope: &Scope, name: &'static str)

Set the debug name of this type’s expansion. Should do nothing for types that don’t appear at runtime

impl DerefExpand for AxisSlicesExpand

where for<'__cube_deref> <AxisSlicesKind as CubeType>::ExpandType: DerefExpand<Target = <AxisSlicesKind as CubeType>::ExpandType>,

type Target = AxisSlicesExpand

fn __expand_deref_method(&self, scope: &Scope) -> Self

impl ExpandTypeClone for AxisSlicesExpand

fn clone_unchecked(&self) -> Self

Unchecked clone that only clones the conceptual runtime value. Should only be used in cases where each copy is used in a mutually exclusive branch (i.e. match, runtime enums). This is intentionally separated from Rust’s Clone semantics and should only be used for the conceptual expand values, never real data. Using two values in the same branch is undefined behaviour.

impl IntoExpand for AxisSlicesExpand

type Expand = AxisSlicesExpand

fn into_expand(self, _: &Scope) -> Self

impl IntoMut for AxisSlicesExpand

fn into_mut(self, scope: &Scope) -> Self

Convert the variable into a potentially new mutable variable in scope, copying if needed.