Skip to main content

WeightLoader

ferrum_quantization::loader

Trait WeightLoader 

Source 
pub trait WeightLoader<B: Backend>: Send + Sync {
    // Required methods
    fn load_tensor(&self, name: &str) -> Result<B::Buffer>;
    fn load_linear(&self, name: &str) -> Result<Box<dyn Linear<B>>>;
    fn has_tensor(&self, name: &str) -> bool;
    fn quant_config(&self) -> Option<&QuantConfig>;

    // Provided method
    fn load_stacked_gptq_experts(
        &self,
        expert_prefix_fmt: &str,
        num_experts: usize,
        proj_names: &[&str],
    ) -> Result<(Arc<dyn MarlinExpertStack<B>>, usize, usize)> { ... }
}

Required Methods§

Source

fn load_tensor(&self, name: &str) -> Result<B::Buffer>

Load a single tensor by fully qualified name (e.g. "model.embed_tokens.weight").

Source

fn load_linear(&self, name: &str) -> Result<Box<dyn Linear<B>>>

Load a projection as a Linear<B>. The concrete implementation (DenseLinear / GptqLinear / AwqLinear / GgufLinear) depends on the loader’s file format and quant config.

name is the module path without the .weight suffix, e.g. "model.layers.0.self_attn.qkv_proj".

Source

fn has_tensor(&self, name: &str) -> bool

Whether a tensor with this name exists in the source.

Source

fn quant_config(&self) -> Option<&QuantConfig>

Quantization metadata (parsed from quantize_config.json or a GGUF header). None means the source is dense.

Provided Methods§

Source

fn load_stacked_gptq_experts( &self, expert_prefix_fmt: &str, num_experts: usize, proj_names: &[&str], ) -> Result<(Arc<dyn MarlinExpertStack<B>>, usize, usize)>

Load per-expert GPTQ projections into one backend-native stacked expert tile. Backends/loaders that do not expose native stacked GPTQ return an explicit unsupported error.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§