pub type ProviderStreamRequestFn = extern "C" fn(req_json: StbStringRef, emit: Option<ProviderStreamEmitFn>, emit_user_data: *mut c_void, control: *const ProviderStreamControl, completion: Option<ProviderStreamCompletionFn>, completion_user_data: *mut c_void, out: *mut StbString, user_data: *mut c_void) -> i32;Expand description
A streaming provider-injection fn (for register_streaming_provider).
Same request envelope and final-response contract as ProviderRequestFn,
plus a per-chunk channel and the cancellation protocol:
req_json— borrowed{ model, context, options }envelope (never freed by the plugin).emit+emit_user_data— the host’s per-chunk callback.emitisNoneonly if the host cannot accept chunks; a plugin then degrades to buffering and returning the message throughout. While the caller is cancelled the host acknowledges structural cleanup chunks and rejects content chunks (nonzero rc). Cleanup chunks are discarded after the aborted terminal.control— pointer to the host’sProviderStreamControl, present on every call. It isnullonly against a host that cannot describe its control surface; a plugin then treats the request as not cancellable and behaves as if cancellation never happens. The plugin must not retain it: it is valid only for the duration of the request fn.completion— reserved; alwaysNoneon this host (seeProviderStreamCompletionFn). A plugin must return its final message throughoutand may not expect to keep streaming after returning.out— owning, plugin-owned, plugin-freed response holding the finalAssistantMessageJSON (identical to the one-shot contract).user_data— the opaque value the plugin registered with.
Call order: zero or more emit chunks, then exactly one return. The host
derives the terminal Done/Error event from out (or from a nonzero rc),
so a plugin that emits deltas still must fill out.
Cancellation is opt-in per chunk, not per registration. There is one
streaming registration and one signature; whether a plugin checks control is
up to it. A plugin that ignores control behaves exactly like the simpler
fire-and-forget shape (the host still stops forwarding content and terminates
the run on Aborted) — it just keeps generating until it finishes on its own.
Chunk envelope (JSON object with a type discriminator). contentIndex is the
index of the block inside the final message; each index may be started once,
but blocks may be opened in any order:
{"type":"start"}
{"type":"text_start","contentIndex":0}
{"type":"text_delta","contentIndex":0,"delta":"hello "}
{"type":"text_end","contentIndex":0,"content":"hello world"}
{"type":"thinking_start","contentIndex":1}
{"type":"thinking_delta","contentIndex":1,"delta":"considering"}
{"type":"thinking_end","contentIndex":1,"content":"considering..."}
{"type":"toolcall_start","contentIndex":2,"id":"call_1","name":"echo"}
{"type":"toolcall_delta","contentIndex":2,"delta":"{\"text\""}
{"type":"toolcall_end","contentIndex":2,"id":"call_1","name":"echo","arguments":{}}
{"type":"usage","usage":{"input":1,"output":2}}Unknown type values are ignored (forward compatibility). A *_delta may
carry an optional partial object (a full assistant message) to override the
host-side accumulated snapshot; otherwise the host accumulates deltas itself.