Expand description
Streaming text processors for model output: separating thinking spans from visible text, and detecting stop sequences. Both hold back a partial-tag suffix so a delimiter split across chunk boundaries is still recognized.
Re-exports§
pub use chat::AttachmentKind;pub use chat::ChatAttachment;pub use chat::ChatMessage;pub use chat::ChatMlPrompt;pub use chat::ChatRole;pub use chat::ChatWireError;pub use chat::decode_tool_specs;pub use chunk::AudioFrame;pub use chunk::CapabilityChunk;pub use chunk::GenerationStats;pub use stop_matcher::StopMatcher;pub use stop_matcher::stop_strings;pub use think_splitter::Piece;pub use think_splitter::TagPair;pub use think_splitter::ThinkSplitter;pub use tools::ToolCall;pub use tools::ToolSpec;
Modules§
- chat
- The chat-wire layer: the
ChatMessage/ChatAttachmentrequest model, the lenient and strict parsers that turn an incoming OpenAI/Ollama payload into messages, the outboundpayload_valuerendering, tool-spec decoding, and the generic ChatML prompt fallback. This is what a gateway uses to bridge a wire request to the kernel’s dispatch surface and back. - chunk
- The streamed output vocabulary: one
CapabilityChunkper thing a runtime emits — text, separated thinking, timestamped segments, audio, embedding vectors, free-text status, and a terminalDonecarrying generation stats. - stop_
matcher - Streaming stop-sequence detection. Text is emitted as it arrives, except for a trailing suffix that could be the start of a stop sequence, which is held back until the next chunk resolves it.
- think_
splitter - Separating a model’s “thinking” spans (delimited by tags like
<think>…</think>) from its visible text as the stream arrives. - tools
- Tool-calling wire types: a
ToolSpeca model may call and aToolCallit emits. TheChatMessage-facing parsing layer (request parsing, transcript inlining) lands with the chat-wire unit; this is the core the runtime adapters and the tool-call chunk need.