Skip to main content

invoke_function_stream

Function invoke_function_stream 

Source
pub async fn invoke_function_stream(
    __arg0: State<Arc<AppState>>,
    __arg1: Path<Uuid>,
    tenant: Option<Extension<TenantId>>,
    auth: Option<Extension<AuthContext>>,
    headers: HeaderMap,
    payload: Result<Json<InvokeRequest>, JsonRejection>,
) -> Result<Response, ApiError>
Expand description

POST /functions/{id}/invoke-stream — streaming invocation (roadmap feature #2; T34 wired this through end-to-end in v0.4).

Mirrors the body / auth surface of invoke_function. The response shape is chosen from the request’s Accept header:

  • Accept: text/event-stream — Server-Sent Events. Each chunk the guest emits via wasi:tensor/host.emit-chunk becomes one event: chunk frame. A keep-alive comment is injected on idle so intermediate proxies don’t reap the connection. The stream terminates with an event: done frame ({"status":"ok"}) on guest success or an event: error frame on guest failure.
  • Anything else — Content-Type: application/octet-stream, chunked-transfer encoding. Each guest chunk is forwarded verbatim as one HTTP chunk frame, followed by the same done / error framing for terminal status.

§Wiring

The handler builds a tokio::sync::mpsc::channel::<Vec<u8>> of depth STREAMING_CHANNEL_BUFFER, wraps the sender in a StreamingContext via StreamingContext::with_channel, and passes it to the executor through SpawnConfig::with_streaming. The executor’s spawn_instance path then builds a wasmtime Linker registering wasi:tensor/host.emit-chunk / flush against the context so guest emits land on the matching receiver.

The receiver is then converted into a futures::stream::Stream via stream::unfold and either:

  • wrapped in axum::response::sse::Sse (SSE branch) — each Vec<u8> becomes one event: chunk frame, terminated by a final event: done / event: error,
  • collected into a Body::from_stream (chunked branch) — each Vec<u8> becomes one HTTP chunk frame.

The guest call runs concurrently with the SSE writer via tokio::spawn. A oneshot::channel carries the terminal status (success / error / deadline-elapsed) so the writer can emit the final done / error event.

§Cancellation

If the HTTP client disconnects, axum drops the response future which drops the SSE writer which drops the mpsc::Receiver. The guest’s next emit-chunk then returns -3 (receiver dropped) and the existing deadline / epoch interrupt tears the instance down. Per docs/STREAMING.md, this is the documented disconnect path.

§Security

Per docs/STREAMING.md, the host does NOT sanitise chunk payloads — the bytes flow guest→client verbatim. Sanitisation (control-byte / ANSI-escape stripping) is the client’s responsibility; the CLI’s T18 sanitisation handles received text. The host’s contribution is the per-stream byte cap (MAX_TOTAL_STREAM_BYTES) enforced inside StreamingContext::emit_chunk, which bounds the per-invocation memory footprint independent of the guest’s intent.

§Body handling

The request body matches InvokeRequest (optional export and args), parsed the same way the synchronous invoke_function route does. An empty body is treated as the all-defaults case so the historical “fire-and-forget no body” wire contract still works.