Struct ServerManager 

pub struct ServerManager { /* private fields */ }

Backend process lifecycle manager.

Handles subprocess spawning for llamaserver/llamafile, budget resolution, VRAM tier detection, and health polling.

Implementations

impl ServerManager

pub fn resolve_budget( &self, mode: BudgetMode, manual_tokens: Option<i64>, n_slots: Option<i64>, kv_unified: bool, ) -> Result<i64, BudgetResolutionError>

Resolve the context budget based on the given mode.

pub fn ollama_vram_budget() -> i64

VRAM tier budget for ollama.

pub fn query_props_context(&self) -> Result<i64, BudgetResolutionError>

Query the backend /props endpoint for the actual context length.

impl ServerManager

pub fn start( &self, model: &str, gguf_path: &Path, mode: &str, extra_flags: &[String], ctx_override: Option<i64>, cache_type_k: Option<&str>, cache_type_v: Option<&str>, n_slots: Option<i64>, kv_unified: bool, ) -> Result<bool, BackendError>

Start the backend process. No-op for ollama. For llamaserver/llamafile, spawns a subprocess with the given config. Returns true if a new process was spawned, false if reused.

pub fn stop(&self) -> Result<(), BackendError>

Stop the backend process. For ollama, runs ‘ollama stop’. Waits up to 10s for termination, then forces kill. Sleeps 3s after termination for VRAM clearing.

impl ServerManager

pub fn new(backend: &str, port: i64, models_dir: Option<&Path>) -> Self

Creates a new ServerManager tracking the specified backend and port.

pub fn with_llamafile_runtime(self, path: impl AsRef<Path>) -> Self

Sets the executable path for the llamafile runtime.

Trait Implementations

impl Drop for ServerManager

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.