Skip to main content

Module capabilities

Module capabilities 

Source
Expand description

Streaming text processors for model output: separating thinking spans from visible text, and detecting stop sequences. Both hold back a partial-tag suffix so a delimiter split across chunk boundaries is still recognized.

Re-exports§

pub use chat::AttachmentKind;
pub use chat::ChatAttachment;
pub use chat::ChatMessage;
pub use chat::ChatMlPrompt;
pub use chat::ChatRole;
pub use chat::ChatWireError;
pub use chat::decode_tool_specs;
pub use chunk::AudioFrame;
pub use chunk::CapabilityChunk;
pub use chunk::GenerationStats;
pub use stop_matcher::StopMatcher;
pub use stop_matcher::stop_strings;
pub use think_splitter::Piece;
pub use think_splitter::TagPair;
pub use think_splitter::ThinkSplitter;
pub use think_splitter::has_visible_tags;
pub use tools::ToolCall;
pub use tools::ToolSpec;

Modules§

chat
The chat-wire layer: the ChatMessage/ChatAttachment request model, the lenient and strict parsers that turn an incoming OpenAI/Ollama payload into messages, the outbound payload_value rendering, tool-spec decoding, and the generic ChatML prompt fallback. This is what a gateway uses to bridge a wire request to the kernel’s dispatch surface and back.
chunk
The streamed output vocabulary: one CapabilityChunk per thing a runtime emits — text, separated thinking, timestamped segments, audio, embedding vectors, free-text status, and a terminal Done carrying generation stats.
stop_matcher
Streaming stop-sequence detection. Text is emitted as it arrives, except for a trailing suffix that could be the start of a stop sequence, which is held back until the next chunk resolves it.
think_splitter
Separating a model’s “thinking” spans (delimited by tags like <think>…</think>) from its visible text as the stream arrives.
tools
Tool-calling wire types: a ToolSpec a model may call and a ToolCall it emits. The ChatMessage-facing parsing layer (request parsing, transcript inlining) lands with the chat-wire unit; this is the core the runtime adapters and the tool-call chunk need.