Expand description
Streaming support for messenger channels.
Provides utilities for streaming model responses to messenger channels in real-time, rather than waiting for the full response. Different messengers support different streaming strategies:
- Edit-based: Send an initial message, then edit it as tokens arrive (Telegram, Discord, Slack).
- Chunked: Send partial messages at intervals (IRC, generic).
- Draft: Use platform-specific draft/typing APIs (where available).
Structs§
- Stream
Buffer - Buffer that accumulates streaming tokens and decides when to flush.
- Stream
Config - Configuration for messenger streaming.
Enums§
- Flush
Action - Action to perform after flushing the stream buffer.
- Stream
Strategy - Streaming strategy for a messenger.
Functions§
- recommended_
strategy - Get the recommended stream strategy for a messenger type.