Crate kalosm_language_model

Language Model

This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.

Usage (with the RPhi implementation crate)

use kalosm::language::*;

#[tokio::main]
async fn main() {
    let mut model = Llama::phi_3().await.unwrap();
    let prompt = "The capital of France is ";
    let mut result = model.stream_text(prompt).await.unwrap();

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Re-exports

• pub use kalosm_sample;

Structs

• AdaEmbedder
  An embedder that uses OpenAI’s API for the Ada embedding model.
• AdaEmbedderBuilder
  A builder for the Ada embedder.
• AdaEmbedding
  The embedding space for the Ada embedding model.
• AnySession
  A type-erased session.
• CachedEmbeddingModel
  Embedding models can be expensive to run. This struct wraps an embedding model with a cache that stores embeddings that have been computed before.
• ChatMarkers
  The chat markers to use for the model.
• Embedding
  Embeddings
• EmbeddingInput
  The input to an embedding model. This includes the text to be embedded and the type of embedding to output.
• GenerateTextBuilder
  A builder for the ModelExt::generate_text method.
• GenerationParameters
  Parameters to use when generating text.
• Gpt4
  A model that uses OpenAI’s API.
• Gpt3_5
  A model that uses OpenAI’s API.
• Gpt3_5Builder
  A builder for gpt-3.5-turbo
• Gpt4Builder
  A builder for text-davinci-003
• StreamTextBuilder
  A builder for the ModelExt::stream_text method.
• StructureParserResult
  The result of a structured parser stream.
• SyncModelNotSupported
  A marker type for models that do not support synchronous generation.
• TokenOutputStream
  This is a wrapper around a tokenizer to ensure that tokens can be returned to the user in a streaming way rather than having to wait for the full decoding.
• UnknownVectorSpace
  An untyped vector space that is not associated with a model. This can be used to erase the vector type from an embedding.

Enums

• CacheIOError
  An error that can occur when loading or saving a cache.
• EmbeddingVariant
  The type of embedding the model should output. For models that output different embeddings for queries and documents, this
• ModelFeedback
  Feedback to give to the model when generating text.

Traits

• AnyModelExt
  An extension trait for models that can be converted into a trait object.
• Embedder
  A model that can be used to embed text. This trait is generic over the vector space that the model uses to help keep track of what embeddings came from which model.
• EmbedderCacheExt
  An extension trait for Embedder that allows for caching embeddings.
• EmbedderExt
  An extension trait for Embedder with helper methods for iterators, and types that can be converted into a string.
• IntoEmbedding
  Convert a type into an embedding with an embedding model.
• Model
  Text Generation Models
• ModelBuilder
  A builder that can create a model asynchronously.
• ModelExt
  Text Generation Models
• Session
  A session for a model.
• StreamExt
  An extension trait for Streams that provides a variety of convenient combinator functions.
• SyncModel
  A raw interface for a model that can be used to generate text synchronously. This provides a very low level interface to a model’s session:
• SyncModelExt
  An extension trait for sync models.
• VectorSpace
  The type of a vector space marks what model the vector space is from. You should only combine vector spaces that come from the same model.

Type Aliases

• BoxedSyncModel
  A trait object for a sync model.
• DynEmbedder
  A trait object for an embedder.
• DynModel
  A trait object for a model.