Struct ProcessRunner 

pub struct ProcessRunner { /* private fields */ }

Local subprocess-backed ModelRunner.

Implementations

impl ProcessRunner

pub fn new( runner: Symbol, model: impl Into<String>, command: impl Into<String>, protocol: ProcessProtocol, timeout: Duration, max_output_bytes: usize, ) -> Self

Builds a runner that invokes command using protocol, bounded by timeout and max_output_bytes.

Trait Implementations

impl Clone for ProcessRunner

fn clone(&self) -> ProcessRunner

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ProcessRunner

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl ModelRunner for ProcessRunner

fn card(&self) -> ModelCard

Returns metadata describing the runner’s advertised model surface.

fn infer(&self, cx: &mut Cx, request: ModelRequest) -> Result<ModelResponse>

Executes one non-streaming inference request.

fn infer_stream( &self, cx: &mut Cx, request: ModelRequest, sink: &mut dyn ModelEventSink, ) -> Result<ModelResponse>

Executes one streaming inference request.

fn bid(&self, _request: &ModelRequest) -> Result<ModelBid, Error>

Returns an availability and score hint for market-style routing.