Struct LlamaKVCacheInterface 

pub struct LlamaKVCacheInterface { /* private fields */ }

Interface for interacting with llama.cpp’s KV cache via the llama-server HTTP API.

Construct with LlamaKVCacheInterface::with_backend(url) when the server URL is known. The default constructor creates an instance with no URL, which falls back to safe defaults for all read operations and no-ops for mutating operations.

Implementations

impl LlamaKVCacheInterface

pub fn new() -> Self

pub fn with_backend(backend_url: String) -> Self

Create an interface pre-wired to a running llama-server.

pub async fn get_current_cache_state(&self) -> Result<LlamaKVCacheState>

Get current KV cache state from llama-server.

Queries GET /slots and returns data for slot 0 (the default interactive slot). Falls back to a zero-filled state when the server is unreachable.

pub async fn extract_current_kv_entries(&self) -> Result<Vec<KVEntry>>

Extract current KV cache metadata from llama-server.

The llama-server HTTP API does not expose raw tensor data; this method constructs representative KVEntry descriptors from slot state data so the rest of the cache management pipeline has something to work with. Importance scores are derived from token position — early tokens (system prompt, opening context) score higher than later ones.

pub async fn inject_kv_entries(&self, entries: &[KVEntry]) -> Result<()>

Restore the KV cache for slot 0 from a previously saved file.

Uses llama-server’s POST /slots/0 with {"action": "restore", "filename": "..."}.

pub async fn clear_cache_entries( &self, layer_indices: &[i32], _head_indices: &[Option<i32>], ) -> Result<()>

Erase the KV cache for slot 0 via POST /slots/0 with {"action": "erase"}.

layer_indices and head_indices are accepted for API compatibility but the llama-server erase action clears the entire slot (partial-layer erase is not supported via HTTP).

pub async fn get_cache_memory_usage(&self) -> Result<usize>

Estimate memory used by the KV cache from slot state.

Uses the same heuristic as get_current_cache_state: 32 KB per token.

pub async fn estimate_cache_capacity(&self) -> Result<f32>

Estimate what fraction of the context window is filled (0.0–1.0).

Trait Implementations

impl Default for LlamaKVCacheInterface

fn default() -> Self

Returns the “default value” for a type.