pub trait LanguageModel:
Sized
+ Send
+ Sync {
type Error: Error + Send + Sync + 'static;
// Required methods
fn respond(
&self,
request: LLMRequest,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send;
fn profile(&self) -> impl Future<Output = Profile> + Send;
// Provided methods
fn respond_with_tools(
&self,
request: LLMRequestWithTools<'_>,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send { ... }
fn generate<T: JsonSchema + DeserializeOwned + 'static>(
&self,
request: LLMRequest,
) -> impl Future<Output = Result<T, GenerateError<Self::Error>>> + Send { ... }
fn complete(
&self,
prefix: &str,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send { ... }
fn summarize(
&self,
text: &str,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send { ... }
fn categorize<T: JsonSchema + DeserializeOwned + 'static>(
&self,
text: &str,
) -> impl Future<Output = Result<T, GenerateError<Self::Error>>> + Send { ... }
}Expand description
Language models for text generation and conversation.
The respond method returns a stream of Events. Tool calls are emitted
as events and are NOT automatically executed - this allows higher-level
abstractions (like agents) to control tool execution.
See the module documentation for examples and usage patterns.
Required Associated Types§
Required Methods§
Sourcefn respond(
&self,
request: LLMRequest,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send
fn respond( &self, request: LLMRequest, ) -> impl Stream<Item = Result<Event, Self::Error>> + Send
Generates a streaming response to a conversation.
Returns a stream of Events including:
Event::Text- Visible text chunksEvent::Reasoning- Internal reasoning (for reasoning models)Event::ToolCall- Requests to execute tools (NOT auto-executed)Event::BuiltInToolResult- Results from provider’s built-in tools
Provided Methods§
Sourcefn respond_with_tools(
&self,
request: LLMRequestWithTools<'_>,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send
fn respond_with_tools( &self, request: LLMRequestWithTools<'_>, ) -> impl Stream<Item = Result<Event, Self::Error>> + Send
Generates a streaming response with a mutable tool registry.
This is for providers that support built-in tool execution.
The default implementation ignores the tools and delegates to respond.
Sourcefn generate<T: JsonSchema + DeserializeOwned + 'static>(
&self,
request: LLMRequest,
) -> impl Future<Output = Result<T, GenerateError<Self::Error>>> + Send
fn generate<T: JsonSchema + DeserializeOwned + 'static>( &self, request: LLMRequest, ) -> impl Future<Output = Result<T, GenerateError<Self::Error>>> + Send
Generates structured output conforming to JSON schema.
§Note for Implementors
By default, we use a system prompt to instruct the model to generate structured output based on the provided JSON schema. However, that is not efficient enough, if supported, provider should override this method to provide native structured generation support. Native structured generation can apply decode rules in token-level, ensuring the output is always valid JSON, and reducing parsing errors.
Sourcefn complete(
&self,
prefix: &str,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send
fn complete( &self, prefix: &str, ) -> impl Stream<Item = Result<Event, Self::Error>> + Send
Completes given text prefix.
Sourcefn summarize(
&self,
text: &str,
) -> impl Stream<Item = Result<Event, Self::Error>> + Send
fn summarize( &self, text: &str, ) -> impl Stream<Item = Result<Event, Self::Error>> + Send
Summarizes text.
§Note for Implementors
By default, we provide a generic summarization prompt. However, some model provider, like Apple Intelligence, may have native summarization support. It would load a summarization-specific lora at runtime, providing better quality.
Sourcefn categorize<T: JsonSchema + DeserializeOwned + 'static>(
&self,
text: &str,
) -> impl Future<Output = Result<T, GenerateError<Self::Error>>> + Send
fn categorize<T: JsonSchema + DeserializeOwned + 'static>( &self, text: &str, ) -> impl Future<Output = Result<T, GenerateError<Self::Error>>> + Send
Categorizes text.
§Note for Implementors
By default, we use a system prompt to instruct the model to categorize text based on the provided JSON schema. However, that is not efficient enough, if supported, provider should override this method to provide native categorization support.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".