Struct LlamaModelParams 

pub struct LlamaModelParams { /* private fields */ }

Parameters controlling how a model is loaded.

Starts from llama_model_default_params(); builder methods override the fields v1 needs. ik’s llama_model_params carries many fork-specific fields (mla, ncmoe, fit, repack_tensors, per-layer K/V types, …); those keep their defaults unless exposed here.

Implementations

impl LlamaModelParams

pub fn with_n_gpu_layers(self, n: u32) -> Self

Number of layers to offload to the GPU (0 = CPU only).

Takes u32 to match llama-cpp-2; clamped into ik’s i32 field.

pub fn with_use_mmap(self, use_mmap: bool) -> Self

Whether to memory-map the model file (default true).

pub fn with_use_mlock(self, use_mlock: bool) -> Self

Force the model into RAM (mlock).

pub fn with_vocab_only(self, vocab_only: bool) -> Self

Load only the vocabulary (no weights).

pub fn with_mtp(self, mtp: bool) -> Self

Load the MTP / NextN prediction layers if present in the model.

pub fn as_raw(&self) -> &llama_model_params

Access the raw params (advanced/escape hatch).

Trait Implementations

impl Clone for LlamaModelParams

fn clone(&self) -> LlamaModelParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlamaModelParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LlamaModelParams

fn default() -> Self

Returns the “default value” for a type.