Skip to main content

Module streaming

Module streaming 

Source
Expand description

Streaming support for messenger channels.

Provides utilities for streaming model responses to messenger channels in real-time, rather than waiting for the full response. Different messengers support different streaming strategies:

  • Edit-based: Send an initial message, then edit it as tokens arrive (Telegram, Discord, Slack).
  • Chunked: Send partial messages at intervals (IRC, generic).
  • Draft: Use platform-specific draft/typing APIs (where available).

Structs§

StreamBuffer
Buffer that accumulates streaming tokens and decides when to flush.
StreamConfig
Configuration for messenger streaming.

Enums§

FlushAction
Action to perform after flushing the stream buffer.
StreamStrategy
Streaming strategy for a messenger.

Functions§

recommended_strategy
Get the recommended stream strategy for a messenger type.