Module runtime 

Speculative-type introspection, llama.cpp’s logger, and model resolution.

Three small surfaces that share one thing: they are things you do around inference rather than during it.

Modules

download

Resolving models from Hugging Face and Docker, through llama.cpp’s own cache.

log

Controls for llama.cpp’s own logger — the output the C++ library produces, which is separate from anything this crate emits through tracing.

Structs

SpeculativeType

A speculative-decoding strategy, as llama.cpp names it.

Functions

speculative_types_from_gguf

Which speculative strategies a draft GGUF supports, read from its metadata.

Type Aliases

RuntimeError

Errors from this module.