Skip to main content

plan_speculation

Function plan_speculation 

Source
pub fn plan_speculation<'catalog>(
    catalog: &'catalog Catalog,
    model: &CatalogModel,
    backend: &str,
    requested: Option<CacheProfile>,
    flag_given: bool,
) -> Result<SpeculationPlan<'catalog>, BackendError>
Expand description

Resolves auto (and the llama.cpp default when no flag was given) to a concrete mode.