Trait TensorParallelRoutedExpertProvider 

pub trait TensorParallelRoutedExpertProvider<B>: RoutedExpertProvider<B>
where
    B: GroupedNeuralBackend,
{
    // Required methods
    fn forward_grouped_tensor_parallel(
        &mut self,
        resident_bank: &mut B::GatedProductGroups,
        request: RoutedExpertRequest<'_, B::Tensor>,
        partitions: usize,
        context: &<B::Tensor as Tensor>::Context,
    ) -> Result<RoutedExpertTensorParallelOutput<B::Tensor>, Self::Error>;
    fn forward_relu2_routed_tensor_parallel(
        &mut self,
        resident_bank: &mut B::Relu2Groups,
        request: RoutedExpertRequest<'_, B::Tensor>,
        partitions: usize,
        context: &<B::Tensor as Tensor>::Context,
    ) -> Result<RoutedExpertTensorParallelOutput<B::Tensor>, Self::Error>;
}

Additive provider mechanism for tensor-parallel grouped partials.

Required Methods

fn forward_grouped_tensor_parallel( &mut self, resident_bank: &mut B::GatedProductGroups, request: RoutedExpertRequest<'_, B::Tensor>, partitions: usize, context: &<B::Tensor as Tensor>::Context, ) -> Result<RoutedExpertTensorParallelOutput<B::Tensor>, Self::Error>

Executes a rank-local gated-product contribution.

fn forward_relu2_routed_tensor_parallel( &mut self, resident_bank: &mut B::Relu2Groups, request: RoutedExpertRequest<'_, B::Tensor>, partitions: usize, context: &<B::Tensor as Tensor>::Context, ) -> Result<RoutedExpertTensorParallelOutput<B::Tensor>, Self::Error>

Executes a rank-local ReLU-squared contribution.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl<B, P, O, E> TensorParallelRoutedExpertProvider<B> for ObservedExpertProvider<'_, P, O, E>

where B: GroupedNeuralBackend, P: TensorParallelRoutedExpertProvider<B>, O: ActivationObserver<B::Tensor, E> + ?Sized,

impl<B> TensorParallelRoutedExpertProvider<B> for ResidentExpertProvider

where B: TensorParallelGroupedNeuralBackend,