Skip to main content

Module cache

Module cache 

Source

Structs§

CacheFlags
Cache options a user asked for; unset fields inherit from the base profile.
CacheProfile
NativeCacheProvider
Cache settings each backend already exposes (verified: llama.cpp b10090, Ollama 0.32.5).
SpawnDelta
Arguments and environment a cache provider adds to a backend launch.

Enums§

CacheError
FlashAttention
PromptReuse
Speculative
Speculative decoding: draft uses the catalog drafter, ngram drafts from the context itself, and auto resolves to draft or off before launch.

Constants§

MAX_PARALLEL

Traits§

CacheProvider

Functions§

effective
The profile worth recording and applying: None when absent or equal to the default.
launch_delta
Launch settings for backend, refusing profiles it cannot apply before anything starts.
provider
Providers are compiled in; names never load code or pass raw arguments through.
requested
The profile a launch applies: flags win, then the running profile for switch or the user default for up, then the backend default (None).