pub fn fit_params(
_backend: &LlamaBackend,
path_model: &Path,
options: FitParams,
) -> Result<FitParamsResult, FitParamsError>Expand description
Adjust model and context parameters to fit available device memory.
Wraps common_fit_params. Requires an initialized LlamaBackend. The model
is probed with no_alloc internally; nothing is kept loaded on return.
Only model fields still equal to LlamaModelParams::default are modified
(except n_gpu_layers on macOS where the default is -1). Context n_ctx
is adjusted only when it is 0 — use LlamaContextParams::with_n_ctx with None.
§Errors
Returns FitParamsError::InvalidPath for bad paths,
FitParamsError::CouldNotFit when no allocation fits, and
FitParamsError::Failed on hard errors (missing model, incompatible params, …).