Struct InferRuntime 

pub struct InferRuntime { /* private fields */ }

The adapter.

Implementations

impl InferRuntime

pub async fn reset_restarts(&self, h: &InstanceHandle)

Give a managed instance a fresh restart budget. Only an explicit operator start calls this: an automatic restart or drive never does, so a server that keeps dying stays given up.

pub async fn restart( &self, h: &InstanceHandle, grace: Duration, ) -> Result<(), RuntimeError>

Stop (when running) then start again.

pub async fn reconcile( &self, h: &InstanceHandle, ) -> Result<Reconcile, RuntimeError>

Restart a managed server that died while it should be running, under the configured backoff and budget, and stop one found listening beyond loopback. Meant to be called on a timer; each call does at most one restart.

impl InferRuntime

pub fn new(cfg: InferConfig) -> Self

A new adapter. It claims no capabilities until probe_capabilities has run.

pub async fn probe_capabilities(&self) -> Vec<Capability>

Check the runtime is there and refresh what provides() returns.

impl InferRuntime

pub async fn health( &self, h: &InstanceHandle, ) -> Result<ServerReport, RuntimeError>

Full probe of an instance’s server.

pub async fn endpoint(&self, h: &InstanceHandle) -> Option<String>

Base URL of an instance (http://127.0.0.1:<port>): what the stable-address proxy forwards to.

pub async fn served_model(&self, h: &InstanceHandle) -> Option<String>

The model id the server itself knows the instance by (the Ollama tag, the registry name for a launched server, the spec’s model when adopted). The mesh-serving side pins requests to it.

pub async fn spec_of(&self, h: &InstanceHandle) -> Option<InferenceSpec>

The spec of a loaded instance.

pub async fn deep_health(&self, h: &InstanceHandle) -> Result<(), RuntimeError>

Health plus one real token (ADR-101: “a tiny completion succeeds”). Costs compute, so it is separate from status.

Trait Implementations

impl WorkloadRuntime for InferRuntime

fn network_exposure(&self) -> NetworkPolicy

A managed server binds loopback only (a wider bind is refused at validation). An adopted server’s own binding cannot be observed from here (the model lab starts its servers with --host 0.0.0.0), so it is reported as LAN-reachable rather than assumed private.

fn id(&self) -> &str

Adapter id (native, container.docker, remote.api.cognitum-seed).

fn provides(&self) -> Vec<Capability>

Capabilities a node advertises when this adapter works.

fn admit<'life0, 'life1, 'async_trait>( &'life0 self, w: &'life1 VerifiedWorkload, ) -> Pin<Box<dyn Future<Output = Result<Admission, RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Self-check: can this node run w? Reserves nothing it cannot undo.

fn load<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, w: &'life1 VerifiedWorkload, cfg: &'life2 WorkloadConfig, ) -> Pin<Box<dyn Future<Output = Result<InstanceHandle, RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Stage the payload and config; returns a handle for the instance.

fn start<'life0, 'life1, 'async_trait>( &'life0 self, h: &'life1 InstanceHandle, ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Start the instance.

fn stop<'life0, 'life1, 'async_trait>( &'life0 self, h: &'life1 InstanceHandle, grace: Duration, ) -> Pin<Box<dyn Future<Output = Result<RunEvidence, RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stop it (terminate, then kill after grace) and return evidence.

fn unload<'life0, 'async_trait>( &'life0 self, h: InstanceHandle, ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Remove staged payload and state.

fn status<'life0, 'life1, 'async_trait>( &'life0 self, h: &'life1 InstanceHandle, ) -> Pin<Box<dyn Future<Output = InstanceStatus> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Current status.

fn control_mode(&self) -> ControlMode

Managed or adopted.

fn adopt<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, h: &'life1 InstanceHandle, w: &'life2 VerifiedWorkload, ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Re-attach an instance this adapter loaded before the controller restarted (its in-memory table is gone but the device still holds it). The adapter verifies the instance is still there and still the pinned version, so stop / unload work again; it starts and changes nothing on the device.

fn console<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, h: &'life1 InstanceHandle, _c: &'life2 str, ) -> Pin<Box<dyn Future<Output = Result<RunEvidence, RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Run one console cycle ([console] limits apply). command must be one of the cog’s [console].allowed_commands.

fn staged_payload<'life0, 'life1, 'async_trait>( &'life0 self, _h: &'life1 InstanceHandle, ) -> Pin<Box<dyn Future<Output = Option<PathBuf>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

The file an instance’s binary was staged to, when the adapter runs it from disk (the licence run gate rehashes it at each start). None: not run from a file this node can read (a container image is tagged by the binary’s BLAKE3; a remote device holds its own copy).

fn link_security(&self) -> LinkSecurity

How the link to a remote device is authenticated. Fails closed: only a remote adapter with a pinned transport says Pinned.

fn console_preemptions<'life0, 'life1, 'async_trait>( &'life0 self, _h: &'life1 InstanceHandle, ) -> Pin<Box<dyn Future<Output = Result<Vec<Preemption>, RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Instances that must stop before a console run of h. The host gates each one as workload.stop before calling Self::preempt.

fn preempt<'life0, 'life1, 'async_trait>( &'life0 self, p: &'life1 Preemption, ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stop one preempted instance.

fn resume<'life0, 'life1, 'async_trait>( &'life0 self, p: &'life1 Preemption, ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Restart one preempted instance after the console run.