Struct StreamingModelInvocation 

#[non_exhaustive]
pub struct StreamingModelInvocation {
    pub inner: ModelInvocation,
}

Streaming-side counterpart to ModelInvocation — the same request + ctx payload but a distinct request type so the tower::Service trait’s associated Response can resolve to ModelStream for the streaming spine while ModelInvocation keeps resolving to ModelResponse for the one-shot spine.

Rust’s Service<Request> carries Response as an associated type — one trait impl per (Self, Request) pair. The wrapper here is the cleanest way to expose two response types from one leaf service: the same InnerChatModel<C, T> implements Service<ModelInvocation, Response = ModelResponse> and Service<StreamingModelInvocation, Response = ModelStream>, and layers stack onto each independently.

#[non_exhaustive] to keep room for streaming-only knobs (chunk size hints, partial-output buffers) post-1.0 without breaking callers.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

inner: ModelInvocation

The wrapped one-shot invocation. Streaming layers read request and ctx through this — same fields, same semantics, different Service shape.

Implementations

impl StreamingModelInvocation

pub const fn new(inner: ModelInvocation) -> Self

Wrap a ModelInvocation.

pub const fn request(&self) -> &ModelRequest

Borrow the request — read-side shortcut so layers don’t have to write invocation.inner.request.

pub const fn ctx(&self) -> &ExecutionContext

Borrow the context — read-side shortcut.

Trait Implementations

impl Clone for StreamingModelInvocation

fn clone(&self) -> StreamingModelInvocation

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StreamingModelInvocation

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<ModelInvocation> for StreamingModelInvocation

fn from(inner: ModelInvocation) -> Self

Converts to this type from the input type.

impl Retryable for StreamingModelInvocation

fn ctx(&self) -> &ExecutionContext

Borrow the ExecutionContext the retry loop checks for cancellation between attempts.

fn ctx_mut(&mut self) -> &mut ExecutionContext

Mutable handle so RetryService can stamp an idempotency key on first entry — every clone of the request that follows shares the stamped key, so vendor-side dedupe sees one logical call across N attempts.