Crate kalosm_language_model

Expand description

§Language Model

This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.

§Usage (with the kalosm-llama implementation crate)

use kalosm::language::*;

#[tokio::main]
async fn main() {
    let mut model = Llama::phi_3().await.unwrap();
    let prompt = "The capital of France is ";
    let mut result = model.complete(prompt);

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Re-exports§

pub use kalosm_sample;

Structs§

AnthropicCompatibleChatModel: An chat model that uses Anthropic’s API for the a remote chat model.
AnthropicCompatibleChatModelBuilder: A builder for an Anthropic compatible chat model.
AnthropicCompatibleChatSession: A chat session for the Anthropic compatible chat model.
AnthropicCompatibleClient: A client for making requests to an OpenAI compatible API.
BoxedChatConstraintsForType: A constraints for CreateDefaultChatConstraintsForType that work with boxed StructuredChatModels.
BoxedChatModel: A boxed ChatModel.
BoxedChatSession: A boxed ChatSession.
BoxedCompletionConstraintsForType: A constraints for CreateDefaultCompletionConstraintsForType that work with boxed TextCompletionModels.
BoxedStructuredChatModel: A boxed StructuredChatModel.
BoxedStructuredTextCompletionModel: A boxed BoxedStructuredTextCompletionModel.
BoxedTextCompletionModel: A boxed TextCompletionModel.
BoxedTextCompletionSession: A boxed TextCompletionSession.
CachedEmbeddingModel: Embedding models can be expensive to run. This struct wraps an embedding model with a cache that stores embeddings that have been computed before.
Chat: Chat is a chat interface that builds on top of crate::ChatModel and crate::StructuredChatModel. It makes it easy to create a chat session with streaming responses, and constraints. Let’s start with a simple chat application:
ChatMessage: A single item in the chat history.
ChatResponseBuilder: A builder for a chat response. This is returned by Chat::add_message and can be modified until you start awaiting the response.
DynEmbedder: A trait object for an embedder.
Embedding: Embeddings
EmbeddingInput: The input to an embedding model. This includes the text to be embedded and the type of embedding to output.
GenerationParameters: Parameters to use when generating text.
NoAnthropicAPIKeyError: An error that can occur when building a remote Anthropic model without an API key.
NoConstraints: A marker type that indicates that no constraints were supplied.
NoOpenAIAPIKeyError: An error that can occur when building a remote OpenAI model without an API key.
OpenAICompatibleChatModel: An chat model that uses OpenAI’s API for the a remote chat model.
OpenAICompatibleChatModelBuilder: A builder for an openai compatible chat model.
OpenAICompatibleChatSession: A chat session for the OpenAI compatible chat model.
OpenAICompatibleClient: A client for making requests to an OpenAI compatible API.
OpenAICompatibleEmbeddingModel: An embedder that uses OpenAI’s API for the a remote embedding model.
OpenAICompatibleEmbeddingModelBuilder: A builder for an openai compatible embedding model.
SchemaParser: A parser for any type that implements the Schema trait and Deserialize.
Task: A task session lets you efficiently run a task with a model. The task session will reuse the model’s cache to avoid re-feeding the task prompt repeatedly.
TextCompletionBuilder: A builder for a text completion response. This is returned by TextCompletionModelExt::complete and can be modified with TextCompletionBuilder::with_sampler and TextCompletionBuilder::with_constraints until you start awaiting the response.

Enums§

AnthropicCompatibleChatModelError: An error that can occur when running a AnthropicCompatibleChatModel.
AnthropicCompatibleChatResponseError: An error that can occur when receiving a stream from the Anthropic API.
EmbeddingVariant: The type of embedding the model should output. For models that output different embeddings for queries and documents, this
MessageType: The type of a chat message
OpenAICompatibleChatModelError: An error that can occur when running a OpenAICompatibleChatModel.
OpenAICompatibleEmbeddingModelError: An error that can occur when running an OpenAICompatibleEmbeddingModel.

Traits§

ChatModel: A trait for unstructured chat models. This trait is required for any chat models even if they do not support structured generation. While this trait is implemented for all chat models, most methods to use models that implement this trait are implemented in the ChatModelExt trait.
ChatModelExt: An extension trait for chat models with helpers for handling chat sessions. This trait is implemented automatically for all crate::ChatModels.
ChatSession: Chat Session
CreateChatSession: A trait for creating a chat session. While it the core trait every chat session implementation implements, most methods to use models that implement this trait are implemented in the ChatModelExt trait.
CreateDefaultChatConstraintsForType: A trait that defines the default constraints for a type with this chat model.
CreateDefaultCompletionConstraintsForType: A trait that defines the default constraints for a type with this model.
CreateTextCompletionSession: A trait for creating a text completion session for a model. While it the core trait every text completion model must implement, most methods to use models that implement this trait are implemented in the TextCompletionModelExt trait.
Embedder: A model that can be used to embed text. This trait is generic over the vector space that the model uses to help keep track of what embeddings came from which model.
EmbedderCacheExt: An extension trait for Embedder that allows for caching embeddings.
EmbedderExt: An extension trait for Embedder with helper methods for iterators, and types that can be converted into a string.
IntoChatMessage: A trait for types that can be converted into a chat message.
IntoEmbedding: Convert a type into an embedding with an embedding model.
ModelBuilder: A builder that can create a model asynchronously.
ModelConstraints: A type that can constrain the output of a model into a specific output type.
StreamExt: An extension trait for Streams that provides a variety of convenient combinator functions.
StructuredChatModel: A trait for unstructured chat models that support structured generation. While this trait is implemented for all structured chat models, most methods to use models that implement this trait are implemented in the ChatModelExt trait.
StructuredTextCompletionModel: A trait for text completion models that support structured generation. While this trait is implemented for all structured text completion models, most methods to use models that implement this trait are implemented in the TextCompletionModelExt trait.
TextCompletionModel: A trait for unstructured text completion models. This trait is required for any text completion models even if they do not support structured generation. While this trait is implemented for all text completion models, most methods to use models that implement this trait are implemented in the TextCompletionModelExt trait.
TextCompletionModelExt: Text Completion Models
TextCompletionSession: Text Completion Session

Functions§

prompt_input: A simple helper function for prompting the user for input.

Crate kalosm_language_modelCopy item path

§Language Model

§Usage (with the kalosm-llama implementation crate)

Re-exports§

Structs§

Enums§

Traits§

Functions§

Crate kalosm_language_model