Skip to main content

get_device_memory_data

Function get_device_memory_data 

Source
pub fn get_device_memory_data(
    path_model: &Path,
    mparams: &LlamaModelParams,
    cparams: &LlamaContextParams,
    log_level: ggml_log_level,
) -> Result<DeviceMemoryReport, DeviceMemoryError>
Expand description

Estimate per-device memory for a model path and parameter set.

This wraps common_get_device_memory_data through ext_shim. The model is loaded with no_alloc and freed before returning; no context is kept alive.

ยงErrors

Returns DeviceMemoryError when the path is invalid or llama.cpp cannot produce an estimate.