Module streaming 

wasi:tensor/host streaming surface (roadmap feature #2).

Lets guests emit chunks of output that the host gateway flushes to the HTTP client as SSE or chunked-transfer bytes. v0.3.7 lands the WIT contract + a buffered in-memory channel; v0.4 lands actual streaming through the axum response path.

Surface

StreamingContext owns an optional Tokio mpsc::Sender<Vec<u8>> channel. The host functions wrapped by add_streaming_to_linker call into StreamingContext::emit_chunk and StreamingContext::flush; the API gateway holds the matching mpsc::Receiver and forwards the chunks into the axum SSE / chunked response body.

Caps

Two hard caps are enforced before any chunk is forwarded:

• MAX_CHUNK_BYTES (64 KiB) — single-chunk size cap. Currently the parser-side check is left for the v0.4 host-fn implementation (the wasi-tensor WIT signature does not yet carry a per-call cap distinct from the total cap); the constant is exported so the host-fn wrapper and tests share a single source of truth.
• MAX_TOTAL_STREAM_BYTES (64 MiB) — total bytes a single invocation may emit before the host returns the documented -2 = cap exceeded code from emit-chunk.

Caps are intentionally conservative for the scaffold so a runaway guest cannot exhaust gateway memory while v0.4 is in flight.

Error codes

Mirrors the WIT contract in wit/wasi-tensor.wit. Acceptance is all-or-nothing — the host forwards the whole chunk or none of it, so success is a flat 0 rather than a byte count:

• 0 — chunk fully accepted (the entire payload was forwarded).
• -1 — streaming not enabled for this invocation.
• -2 — guest tried to emit past the documented size cap.
• -3 — downstream client disconnected (receiver dropped).

Tests in tests/streaming_scaffold.rs exercise every branch.

Structs

InputContext

Per-invocation input context. Owns the bytes the host staged for the guest to pull via the wasi:tensor/host input-len / read-input host functions.

StreamingContext

Per-invocation streaming context. Owns the producer end of the mpsc::Sender<Vec<u8>> channel into which guest-emitted chunks are pushed; the API gateway holds the matching receiver and forwards the bytes onto the wire.

Constants

FN_EMIT_CHUNK

Host-function name for emit-chunk.

FN_FLUSH

Host-function name for flush.

FN_INPUT_LEN

Host-function name for input-len.

FN_READ_INPUT

Host-function name for read-input.

MAX_CHUNK_BYTES

Maximum size, in bytes, of a single emit-chunk call. 64 KiB matches typical HTTP chunk-encoder buffer sizes; guests producing larger payloads should call emit-chunk repeatedly. Enforced by the v0.4 host-fn wrapper; exported here so wrapper and tests share a constant.

MAX_TOTAL_STREAM_BYTES

Maximum total bytes a single invocation may emit across all emit-chunk calls before the host starts returning the documented -2 = cap exceeded code. 64 MiB is generous for LLM-style token streams while keeping the worst-case per-invocation memory footprint bounded.

STREAMING_MODULE

Module name used to register the wasi-tensor host functions: wasi:tensor/host@0.1.0.

Traits

HasInput

Trait implemented by store data types that can hand out an InputContext. Parallels HasStreaming; the executor’s InstanceState implements it so the input-len / read-input host functions reach the per-instance staged buffer.

HasStreaming

Trait implemented by store data types that can hand out a StreamingContext. Parallels crate::host::HasWasiCuda; the executor’s InstanceState will implement this once the v0.4 wiring lands.

Functions

add_input_to_linker

Register the wasi:tensor/host guest-input host functions (input-len, read-input) on a wasmtime Linker.

add_streaming_to_linker

Register the wasi-tensor host functions on a wasmtime Linker.