Module backend 

Resolving a spec’s model reference into something that can generate.

Inference will come from several places over this project’s life: a local Ollama, an OpenAI-compatible HTTP endpoint (which covers llama.cpp’s own server, vLLM, LM Studio, and most hosted providers), an embedded llama.cpp, and others. InferBackend is the interface they share; this module is how a spec picks one without anything in the call chain knowing the list.

Why a registry rather than a match

The obvious implementation is a match on the provider name in the daemon. It works, and it means every new backend edits the daemon, the parser, and an enum — three places that have nothing to do with the new backend, touched only because they enumerate. That is the shape that makes a fourth or fifth provider progressively less attractive to add.

Instead a backend supplies a BackendFactory, registers it under a provider name, and nothing else changes. The spec parser already accepts any Provider "target"; the runner only ever sees InferBackend. Adding one is genuinely additive.

A backend that needs heavy or platform-specific dependencies — embedded llama.cpp being the obvious case — can live behind a cargo feature and register itself only when enabled. Nothing here has to change to allow that.

use cuttlefish_core::spec::ModelRef;
use cuttlefish_host::backend::Registry;

let registry = Registry::with_builtins();

let backend = registry.resolve(&ModelRef::new("stub", "anything")).unwrap_or_else(|e| {
    panic!("stub is always registered: {e}")
});
assert_eq!(backend.model_name(), "stub");

// An unknown provider explains what is available rather than panicking.
let err = registry.resolve(&ModelRef::new("nope", "x")).err().unwrap();
assert!(err.to_string().contains("stub"));

Structs

Registry

The providers this host knows how to serve.

Traits

BackendFactory

Builds one kind of InferBackend from a spec’s model target.