Struct CheckpointParams 

#[non_exhaustive]
pub struct CheckpointParams {
    pub max_checkpoints: u32,
    pub every_n_tokens: i32,
    pub min_tokens: u32,
    pub min_gap: u32,
}

Tunable parameters for the in-memory state-checkpoint cache used to preserve KV/recurrent state across chat turns for hybrid models.

Hybrid architectures (Qwen 3.5, Jamba, etc.) interleave Mamba-style recurrent layers with transformer layers. The recurrent state can’t be rolled back to an arbitrary earlier position, so a partial KV trim fails whenever the next prompt diverges deep into the conversation. To work around this, we periodically snapshot the partial seq state (recurrent + SWA, via LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY) during prompt prefill and restore the closest snapshot when the next prompt arrives. Mirrors the mechanism used by upstream llama-server.

For non-hybrid models (Qwen 2.5, Llama 3, Gemma, …) checkpoints are created but never used because the cheaper partial-trim path succeeds.

Marked #[non_exhaustive]; build via Default::default() and chain the with_* setters.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

max_checkpoints: u32

Maximum number of checkpoints retained per persistent context. 0 disables checkpointing entirely. Each checkpoint is a few MB for typical hybrid models.

every_n_tokens: i32

Approximate spacing between checkpoints during prompt prefill, in tokens. The last 4..=4 + n_ubatch tokens always get a checkpoint regardless. <= 0 means “only checkpoint near the end of the prompt”.

min_tokens: u32

Don’t checkpoint the very start of a prompt — saves space for no benefit because we’d have to re-decode that prefix anyway if it’s the entire reuse window.

min_gap: u32

Don’t take two checkpoints closer than this many tokens apart.

Implementations

impl CheckpointParams

pub fn with_max_checkpoints(self, max_checkpoints: u32) -> Self

Override the maximum number of checkpoints retained per context.

pub fn with_every_n_tokens(self, every_n_tokens: i32) -> Self

Override the approximate spacing between checkpoints (in tokens).

pub fn with_min_tokens(self, min_tokens: u32) -> Self

Override the minimum prompt length before checkpoints are taken.

pub fn with_min_gap(self, min_gap: u32) -> Self

Override the minimum spacing between two consecutive checkpoints.

Trait Implementations

impl Clone for CheckpointParams

fn clone(&self) -> CheckpointParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CheckpointParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CheckpointParams

fn default() -> Self

Returns the “default value” for a type.

impl Copy for CheckpointParams