Struct DeviceMemoryHyperParams 

pub struct DeviceMemoryHyperParams {
    pub n_gpu_layers: u32,
    pub n_ctx_train: u32,
    pub n_expert: u32,
}

Hyper-parameters discovered while estimating device memory.

Fields

n_gpu_layers: u32

Model n_gpu_layers hyper-parameter used for the estimate.

n_ctx_train: u32

Model training context length.

n_expert: u32

Number of MoE experts (0 when dense).

Trait Implementations

impl Clone for DeviceMemoryHyperParams

fn clone(&self) -> DeviceMemoryHyperParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for DeviceMemoryHyperParams

impl Debug for DeviceMemoryHyperParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for DeviceMemoryHyperParams

impl PartialEq for DeviceMemoryHyperParams

fn eq(&self, other: &DeviceMemoryHyperParams) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for DeviceMemoryHyperParams