Module infer 

Inference-server adapters (card mesh-placement-18; ADR-101 section 4).

• spec: InferenceSpec and VerifiedWorkload::inference.
• config: adopted or managed construction, restart backoff.
• probe: loopback health, model listing, tiny completion.
• capabilities: the probes behind provides().
• runtime / lifecycle: InferRuntime, the super::WorkloadRuntime for infer.llamacpp, infer.mlx-lm and infer.ollama, plus the API the stable-address proxy builds on (InferRuntime::endpoint, InferRuntime::health, InferRuntime::reconcile).
• launch / ollama: how a managed server is started or driven.

Adopted mode never controls the server. Managed mode launches only a configured launcher script, only on a port nothing else holds, with weights resolved from the model registry, and stops only the process it spawned.

Re-exports

pub use config::InferConfig;

pub use config::InferMode;

pub use config::ManagedConfig;

pub use config::RestartPolicy;

pub use config::lab_serve_program;

pub use lifecycle::Reconcile;

pub use probe::Health;

pub use probe::ServerClient;

pub use probe::ServerReport;

pub use residency::CoResidency;

pub use residency::GB;

pub use residency::ResidencyLedger;

pub use roster::ImportedRoster;

pub use roster::ImportedSpec;

pub use roster::RosterOverlay;

pub use roster::import_roster;

pub use runtime::InferRuntime;

pub use spec::InferFlavor;

pub use spec::InferenceSpec;

pub use spec::KIND_INFERENCE;

pub use spec::LatencyClass;

pub use spec::MemoryBudget;

pub use spec::ServeArgs;

Modules

capabilities

Capability probes: what a node may advertise after checking the runtime is really there (ADR-101 section 4).

config

Adapter configuration: adopted or managed, and what managed needs.

exposure

Is a managed server reachable beyond loopback? The adapter passes --host 127.0.0.1, but the launcher and the server are not ours, so the binding is checked, not assumed: connect to every non-loopback address of this machine on the server’s port.

launch

Managed-mode launch plan: argv from the spec and the resolved model, environment, working directory, and the “is something already on this port” guard that keeps the adapter from ever starting over a server it does not own.

lifecycle

Start, stop, unload, status, restart and reconcile for InferRuntime.

ollama

Ollama is driven through its API: it owns its process and decides when weights are resident, so the adapter asks it to load or unload a model and reports what /api/ps says. It never starts, stops or signals the Ollama server itself.

probe

Loopback HTTP probes of model servers: health, listed models, and an optional tiny completion. Never follows redirects, ignores proxy environment, caps response size, and only ever talks to the address the caller built from a validated loopback spec.

residency

Unified-memory budget and co-residency (card mesh-placement-20; ADR-101 section 6). One ledger is shared by every managed adapter on a node: an instance reserves its weights plus KV budget when it starts and releases them when it stops or is unloaded. A start that would exceed the budget, or that excludes (or is excluded by) a running role, is refused as unplaceable, with the reason.

roster

Import of the model lab’s roster (~/llm/docs/models/queue.yaml, card mesh-placement-20; ADR-101 section 8 step 2) into InferenceSpecs.

runtime

InferRuntime: the infer.llamacpp, infer.mlx-lm and infer.ollama adapters (ADR-101 section 4) behind one type.

spec

The inference workload spec (ADR-101 section 1) and its boundary validation.