Struct LlamaContextParams 

pub struct LlamaContextParams { /* private fields */ }

Parameters controlling a crate::LlamaContext.

Starts from llama_context_default_params(). Note ik keeps a seed field in the context params (stock removed it) and uses a bool flash_attn plus the MTP fields mtp / mtp_op_type. There is no ctx_type in ik.

Implementations

impl LlamaContextParams

pub fn with_n_ctx(self, n_ctx: Option<NonZeroU32>) -> Self

Context size (tokens); None means “take it from the model” (0).

Takes Option<NonZeroU32> to match llama-cpp-2.

pub fn with_n_batch(self, n_batch: u32) -> Self

Logical batch size.

pub fn with_n_ubatch(self, n_ubatch: u32) -> Self

Physical (micro) batch size.

pub fn with_n_rs_seq(self, n_rs_seq: u32) -> Self

Maximum number of sequences (distinct recurrent states).

llama-cpp-2’s NextN fork exposes this as n_rs_seq; ik’s equivalent is n_seq_max, which this sets.

pub fn with_context_type(self, context_type: LlamaContextType) -> Self

Select the context kind. LlamaContextType::Mtp enables ik’s MTP path (equivalent to Self::with_mtp(true)).

pub fn with_seed(self, seed: u32) -> Self

RNG seed (ik retains this in the context params).

pub fn with_n_threads(self, n_threads: u32) -> Self

Threads used for generation.

pub fn with_n_threads_batch(self, n_threads_batch: u32) -> Self

Threads used for batch/prompt processing.

pub fn with_flash_attn(self, flash_attn: bool) -> Self

Enable flash attention.

pub fn with_mtp(self, mtp: bool) -> Self

Activate the MTP path (requires a model loaded with .with_mtp(true)).

pub fn with_embeddings(self, embeddings: bool) -> Self

Produce embeddings on decode (sets params.embeddings).

Required for embedding and reranker models; usually paired with Self::with_pooling_type.

pub fn with_pooling_type(self, pooling_type: llama_pooling_type) -> Self

Set the pooling strategy for embeddings (sets params.pooling_type).

Takes the raw sys::llama_pooling_type (e.g. LLAMA_POOLING_TYPE_MEAN, LLAMA_POOLING_TYPE_CLS, or LLAMA_POOLING_TYPE_LAST) to avoid introducing a new enum.

pub fn as_raw(&self) -> &llama_context_params

Access the raw params (advanced/escape hatch).

Trait Implementations

impl Clone for LlamaContextParams

fn clone(&self) -> LlamaContextParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlamaContextParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LlamaContextParams

fn default() -> Self

Returns the “default value” for a type.