Struct InferenceConfig 

pub struct InferenceConfig {
    pub backend: Backend,
    pub model_path: String,
    pub n_gpu_layers: i32,
    pub n_ctx: u32,
    pub app_name: String,
    pub explicit_dll_path: Option<PathBuf>,
    pub dll_version: Option<String>,
    pub cache_dir: Option<PathBuf>,
    pub chat_template: Option<String>,
}

Configuration for loading a model.

Fields

backend: Backend

Compute backend.

model_path: String

Path to the GGUF model file.

n_gpu_layers: i32

Number of layers to offload to GPU (-1 = all).

n_ctx: u32

Context window size in tokens (default for contexts created from this engine).

app_name: String

Application name (used for DLL cache directory).

explicit_dll_path: Option<PathBuf>

Explicit DLL path (skips auto-download).

dll_version: Option<String>

DLL version tag to download.

cache_dir: Option<PathBuf>

DLL cache directory.

chat_template: Option<String>

Optional chat template (Jinja). If not provided, uses model metadata.

Trait Implementations

impl Clone for InferenceConfig

fn clone(&self) -> InferenceConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferenceConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for InferenceConfig

fn default() -> Self

Returns the “default value” for a type.