Trait InferenceRunner 

pub trait InferenceRunner: Send + Sync {
    // Required method
    fn run<'life0, 'async_trait>(
        &'life0 self,
        request: GenerateRequest,
        emitter: EventEmitter,
    ) -> Pin<Box<dyn Future<Output = Result<RunnerResult, RunnerError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}

The trait foreign hosts implement to handle delegated inference.

Implementations are typically thin shims around a host-owned SDK (Vercel AI SDK on Node, Anthropic’s Python client, etc.). They translate the supplied GenerateRequest to a wire-format call, stream chunks back through the EventEmitter, and return a RunnerResult with the aggregated final text.

Required Methods

fn run<'life0, 'async_trait>( &'life0 self, request: GenerateRequest, emitter: EventEmitter, ) -> Pin<Box<dyn Future<Output = Result<RunnerResult, RunnerError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Drive the request to completion. The runner MUST emit at least a final StreamEvent::Done event before returning, for parity with native backends — consumers of the resulting stream rely on Done as the terminal marker.

Implementors