Skip to main content

fit_params

Function fit_params 

Source
pub fn fit_params(
    _backend: &LlamaBackend,
    path_model: &Path,
    options: FitParams,
) -> Result<FitParamsResult, FitParamsError>
Expand description

Adjust model and context parameters to fit available device memory.

Wraps common_fit_params. Requires an initialized LlamaBackend. The model is probed with no_alloc internally; nothing is kept loaded on return.

Only model fields still equal to LlamaModelParams::default are modified (except n_gpu_layers on macOS where the default is -1). Context n_ctx is adjusted only when it is 0 — use LlamaContextParams::with_n_ctx with None.

§Errors

Returns FitParamsError::InvalidPath for bad paths, FitParamsError::CouldNotFit when no allocation fits, and FitParamsError::Failed on hard errors (missing model, incompatible params, …).