Struct Model 

pub struct Model {
    pub cfg: ModelConfig,
    pub embd: EmbedHost,
    pub output_norm: GpuTensor,
    pub output: GpuTensor,
    pub layers: Vec<Layer>,
}

Fields

cfg: ModelConfig

embd: EmbedHost

output_norm: GpuTensor

output: GpuTensor

layers: Vec<Layer>

Implementations

impl Model

pub fn forward( &self, e: &Engine, tokens: &[u32], ) -> Result<Vec<f32>, Box<dyn Error>>

Run prefill over tokens, return logits [T, n_vocab] (host f32). positions = 0..T.

pub fn forward_last( &self, e: &Engine, tokens: &[u32], ) -> Result<Vec<f32>, Box<dyn Error>>

Logits for just the last token (the decode-relevant row).

impl Model

pub fn load_dense(e: &Engine, g: &GgufFile) -> Result<Self, Box<dyn Error>>

Load a dense (vanilla-transformer) model from GGUF. Thin wrapper over load_dense_from_source. Panics if the arch has SSM/MoE layers.

pub fn load_dense_from_source( e: &Engine, src: &dyn TensorSource, ) -> Result<Self, Box<dyn Error>>

Load a dense-attention model from any TensorSource — GGUF or a safetensors HF checkpoint. The whole loop speaks ggml names; the source maps them. The FFN is dense SwiGLU OR routed MoE (OLMoE: dense full-attention + MoE FFN). Panics on hybrid (SSM) arches — use the hybrid path.

pub fn embed_tokens( &self, e: &Engine, tokens: &[u32], ) -> Result<CudaSlice<f32>, Box<dyn Error>>

Gather embedding rows into f32 [T, n_embd] (token-major) by dequantizing only the needed rows from the host-side embedding bytes (token_embd is [n_embd, n_vocab], row per token).