Struct RunHandle 

pub struct RunHandle { /* private fields */ }

The result of ModelRunner::execute. Local runtimes typically return Streaming even for unary calls (one final chunk); remote runtimes return Streaming for SSE responses and a single-chunk stream otherwise. Callers always treat it as a stream.

Implementations

impl RunHandle

pub fn streaming(inner: BoxStream<'static, InferenceResult<TokenChunk>>) -> Self

pub fn into_stream(self) -> BoxStream<'static, InferenceResult<TokenChunk>>

Trait Implementations

impl Debug for RunHandle

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.