Module streaming 

This module provides functionality for working with streaming completion models. It provides traits and types for generating streaming completion requests and handling streaming completion responses.

The main traits defined in this module are:

• StreamingPrompt: Defines a high-level streaming LLM one-shot prompt interface
• StreamingChat: Defines a high-level streaming LLM chat interface with history
• StreamingCompletion: Defines a low-level streaming LLM completion interface

Structs

PauseControl

Control for pausing and resuming a streaming response

RawStreamingToolCall

Describes a streaming tool call response (in its entirety)

StreamingCompletionResponse

The response from a streaming completion request; message and response are populated at the end of the inner stream.

Enums

RawStreamingChoice

Enum representing a streaming chunk from the model

StreamedAssistantContent

Describes responses from a streamed provider response which is either text, a tool call or a final usage response.

StreamedUserContent

Streamed user content. This content is primarily used to represent tool results from tool calls made during a multi-turn/step agent prompt.

ToolCallDeltaContent

The content of a tool call delta - either the tool name or argument data

Traits

StreamingChat

Trait for high-level streaming chat interface

StreamingCompletion

Trait for low-level streaming completion interface

StreamingPrompt

Trait for high-level streaming prompt interface

Functions

stream_to_stdout

A helper function to stream a completion request to stdout. Tool call deltas are ignored as tool calls are generally much easier to handle when received in their entirety rather than using deltas.

Type Aliases

StreamingResultNot (wasm and WebAssembly)