Trait ExpertProjectionOps 

pub trait ExpertProjectionOps: Backend {
    type ExpertProjectionError: Debug;
    type ExpertProjectionState: Clone + Send + Debug + 'static;

    // Required methods
    fn expert_projection_forward(
        input: Self::FloatTensorPrimitive,
        global_ids: Self::IntTensorPrimitive,
        weights: Self::FloatTensorPrimitive,
        options: ExpertProjectionOptions,
    ) -> Result<(Self::FloatTensorPrimitive, Self::ExpertProjectionState), Self::ExpertProjectionError>;
    fn expert_projection_backward(
        state: Self::ExpertProjectionState,
        gradient: Self::FloatTensorPrimitive,
        selection: ExpertProjectionSelection,
    ) -> Result<ExpertProjectionBackward<Self>, Self::ExpertProjectionError>;
}

Original actual grouped projection, including real trainable expert adapter matrices.

Required Associated Types

type ExpertProjectionError: Debug

Original native row/projection or first-order differentiation failure.

type ExpertProjectionState: Clone + Send + Debug + 'static

Original actual floating cube and native private COPY row mapping.

Required Methods

fn expert_projection_forward( input: Self::FloatTensorPrimitive, global_ids: Self::IntTensorPrimitive, weights: Self::FloatTensorPrimitive, options: ExpertProjectionOptions, ) -> Result<(Self::FloatTensorPrimitive, Self::ExpertProjectionState), Self::ExpertProjectionError>

Execute actual U32 assigned expert projections and restore original incoming row order.

fn expert_projection_backward( state: Self::ExpertProjectionState, gradient: Self::FloatTensorPrimitive, selection: ExpertProjectionSelection, ) -> Result<ExpertProjectionBackward<Self>, Self::ExpertProjectionError>

Native first-order selected derivatives, without allocations for omitted cube/input gradients.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors