Skip to main content

baseRT_suggest_max_context

Function baseRT_suggest_max_context 

Source
pub unsafe extern "C" fn baseRT_suggest_max_context(
    model_path: *const c_char,
    max_batch: c_int,
    kv_bits: c_int,
    paged_kv: c_int,
) -> c_int
Expand description

A max_context sized for this device and this bundle. Metadata-only, so it is safe to call before loading. 0 = could not size.