Struct FitParams 

#[non_exhaustive]
pub struct FitParams {
    pub margins: Option<Vec<usize>>,
    pub n_ctx_min: u32,
}

Configuration for automatic GPU/CPU layer fitting.

Passed to crate::Client::builder (or crate::Client::from_gguf) so llama.cpp can probe available device memory and pick the optimal number of layers to offload to GPU automatically, instead of requiring a manual n_gpu_layers value.

Marked #[non_exhaustive]; build via Default::default() and chain the with_* setters.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

margins: Option<Vec<usize>>

Memory margin per device in bytes. If None, defaults to 1 GiB per device.

n_ctx_min: u32

Minimum context size to preserve during fitting (default: 4096).

Implementations

impl FitParams

pub fn with_margins(self, margins: Option<Vec<usize>>) -> Self

Override the per-device memory margin in bytes.

pub fn with_n_ctx_min(self, n_ctx_min: u32) -> Self

Override the minimum context size to preserve during fitting.

Trait Implementations

impl Clone for FitParams

fn clone(&self) -> FitParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FitParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for FitParams

fn default() -> Self

Returns the “default value” for a type.