Expand description
Unified LLM interface types and traits.
Provides the shared types used across all LLM providers:
Message, Response, StreamChunk, Tool, Request, and the Model trait.
Structs§
- Choice
- A completion choice (used for both streaming and non-streaming responses).
- Completion
Meta - Common metadata shared between streaming and non-streaming completions
- Completion
Tokens Details - Detailed breakdown of completion tokens
- Delta
- Message content in a completion response
- Function
Call - A function call within a tool call
- Message
- A message in the chat
- Message
Builder - A builder for messages
- Request
- A chat completion request.
- Response
- A chat completion response from the LLM
- Stream
Chunk - A streaming chat completion chunk
- Tool
- A tool for the LLM
- Tool
Call - A tool call made by the model
- Usage
- Token usage statistics
Enums§
- Finish
Reason - The reason the model stopped generating
- Role
- The role of a message
- Tool
Choice - Controls which tool is called by the model
Traits§
- Model
- Unified LLM provider trait.
Functions§
- default_
context_ limit - Returns the default context limit (in tokens) for a known model ID.
- estimate_
tokens - Estimate total tokens across a slice of messages.