Crate kalosm_language_model

Language Model

This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.

Usage (with the RPhi implementation crate)

use rphi::prelude::*;

#[tokio::main]
async fn main() {
    let mut model = Phi::default();
    let prompt = "The capital of France is ";
    let mut result = model.stream_text(prompt).await.unwrap();

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Re-exports

• pub use kalosm_sample;

Structs

• AdaEmbedder
  An embedder that uses OpenAI’s API for the Ada embedding model.
• AdaEmbedderBuilder
  A builder for the Ada embedder.
• AdaEmbedding
  The embedding space for the Ada embedding model.
• AnySession
  A type-erased session.
• ChatMarkers
  The chat markers to use for the model.
• DollySevenBSpace
  A vector space for the DollySevenBSpace model.
• Embedding
  An embedding represents something about the meaning of data. It can be used to compare the meaning of different pieces of data, cluster data, or as input to a machine learning model.
• GenerateTextBuilder
  A builder for the ModelExt::generate_text method.
• GenerationParameters
  Parameters to use when generating text.
• Gpt3_5
  A model that uses OpenAI’s API.
• Gpt3_5Builder
  A builder for gpt-3.5-turbo
• Gpt4
  A model that uses OpenAI’s API.
• Gpt4Builder
  A builder for text-davinci-003
• GuanacoSpace
  A vector space for the GuanacoSpace model.
• LargePythiaSpace
  A vector space for the LargePythiaSpace model.
• LlamaSevenChatSpace
  A vector space for the LlamaSevenChatSpace model.
• LlamaThirteenChatSpace
  A vector space for the LlamaThirteenChatSpace model.
• MptBaseSpace
  A vector space for the MptBaseSpace model.
• MptChatSpace
  A vector space for the MptChatSpace model.
• MptInstructSpace
  A vector space for the MptInstructSpace model.
• MptStorySpace
  A vector space for the MptStorySpace model.
• OrcaSpace
  A vector space for the OrcaSpace model.
• StableLmSpace
  A vector space for the StableLmSpace model.
• StreamTextBuilder
  A builder for the ModelExt::stream_text method.
• StructureParserResult
  The result of a structured parser stream.
• SyncModelNotSupported
  A marker type for models that do not support synchronous generation.
• TinyPythiaSpace
  A vector space for the TinyPythiaSpace model.
• TokenOutputStream
  This is a wrapper around a tokenizer to ensure that tokens can be returned to the user in a streaming way rather than having to wait for the full decoding.
• UnknownVectorSpace
  An untyped vector space that is not associated with a model. This can be used to erase the vector type from an embedding.
• VicunaSpace
  A vector space for the VicunaSpace model.
• WizardLmSpace
  A vector space for the WizardLmSpace model.

Enums

• GptNeoXType
  The type of GPT-NeoX model to use.
• LlamaType
  The type of Llama model to use.
• ModelFeedback
  Feedback to give to the model when generating text.
• ModelType
  The type of model to use.
• MptType
  The type of MPT model to use.

Traits

• AnyModelExt
  An extension trait for models that can be converted into a trait object.
• CreateModel
  A model that can be created asynchronously.
• Embedder
  A model that can be used to embed text. This trait is generic over the vector space that the model uses to help keep track of what embeddings came from which model.
• Model
  A model that can be used to generate text with an associated tokenizer.
• ModelExt
  An extension trait for models.
• Session
  A session for a model.
• StreamExt
  An extension trait for Streams that provides a variety of convenient combinator functions.
• SyncModel
  A raw interface for a model that can be used to generate text synchronously. This provides a very low level interface to a model’s session:
• SyncModelExt
  An extension trait for sync models.
• VectorSpace
  The type of a vector space marks what model the vector space is from. You should only combine vector spaces that come from the same model.

Type Aliases

• BoxedSyncModel
  A trait object for a sync model.
• DynEmbedder
  A trait object for an embedder.
• DynModel
  A trait object for a model.