Enum MoeBackend 

pub enum MoeBackend {
    Offload,
    Hybrid,
    Cpu,
    Fused,
}

Which MoE execution backend a machine should serve with.

Variants

Offload

Stream every missing expert over PCIe; the GPU does all the multiplying.

Hybrid

Fetch some misses, compute the rest on the CPU, overlapped.

Cpu

Ship activations to the CPU and compute every expert there.

Fused

Experts are resident in VRAM; no cache, no streaming.

Trait Implementations

impl Clone for MoeBackend

fn clone(&self) -> MoeBackend

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for MoeBackend

impl Debug for MoeBackend

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for MoeBackend

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for MoeBackend

impl PartialEq for MoeBackend

fn eq(&self, other: &MoeBackend) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for MoeBackend

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for MoeBackend