Trait llm_weaver::Llm

pub trait Llm<T: Config>: Default + Sized + PartialEq + Eq + Clone + Debug + Copy + Send + Sync {
    type Tokens: Copy + ToRedisArgs + FromStr + Display + Debug + ToString + Serialize + Default + TryFrom<usize> + Unsigned + FromPrimitive + ToPrimitive + CheckedAdd + SaturatingAdd + SaturatingSub + SaturatingMul + CheckedDiv + Ord + Sync + Send;
    type Request: Clone + From<ContextMessage<T>> + Send;
    type Response: Clone + Into<Option<String>> + Send;
    type Parameters: Debug + Clone + Send + Sync;

    const TOKEN_WORD_RATIO: BoundedU8<0, 100> = _;

    // Required methods
    fn max_context_length(&self) -> Self::Tokens;
    fn name(&self) -> &'static str;
    fn count_tokens(content: String) -> Result<Self::Tokens>;
    fn prompt<'life0, 'life1, 'async_trait>(
        &'life0 self,
        msgs: Vec<Self::Request>,
        params: &'life1 Self::Parameters
    ) -> Pin<Box<dyn Future<Output = Result<Self::Response>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;

    // Provided methods
    fn get_max_token_limit(&self) -> Self::Tokens { ... }
    fn ctx_msgs_to_prompt_requests(
        &self,
        msgs: &[ContextMessage<T>]
    ) -> Vec<Self::Request> { ... }
    fn convert_tokens_to_words(&self, tokens: Self::Tokens) -> Self::Tokens { ... }
}

Required Associated Types

type Tokens: Copy + ToRedisArgs + FromStr + Display + Debug + ToString + Serialize + Default + TryFrom<usize> + Unsigned + FromPrimitive + ToPrimitive + CheckedAdd + SaturatingAdd + SaturatingSub + SaturatingMul + CheckedDiv + Ord + Sync + Send

Tokens are an LLM concept which represents pieces of words. For example, each ChatGPT token represents roughly 75% of a word.

This type is used primarily for tracking the number of tokens in a TapestryFragment and counting the number of tokens in a string.

This type is configurable to allow for different types of tokens to be used. For example, u16 can be used to represent the number of tokens in a string.

type Request: Clone + From<ContextMessage<T>> + Send

Type representing the prompt request.

type Response: Clone + Into<Option<String>> + Send

Type representing the response to a prompt.

type Parameters: Debug + Clone + Send + Sync

Type representing the parameters for a prompt.

Provided Associated Constants

const TOKEN_WORD_RATIO: BoundedU8<0, 100> = _

Token to word ratio.

Defaults to 75%

Required Methods

fn max_context_length(&self) -> Self::Tokens

The maximum number of tokens that can be processed at once by an LLM model.

fn name(&self) -> &'static str

Get the model name.

This is used for logging purposes but also can be used to fetch a specific model based on &self. For example, the model passed to Loom::weave can be represented as an enum with a multitude of variants, each representing a different model.

fn count_tokens(content: String) -> Result<Self::Tokens>

Calculates the number of tokens in a string.

This may vary depending on the type of tokens used by the LLM. In the case of ChatGPT, can be calculated using the tiktoken-rs crate.

fn prompt<'life0, 'life1, 'async_trait>( &'life0 self, msgs: Vec<Self::Request>, params: &'life1 Self::Parameters ) -> Pin<Box<dyn Future<Output = Result<Self::Response>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Prompt LLM with the supplied messages and parameters.

Provided Methods

fn get_max_token_limit(&self) -> Self::Tokens

Get the maximum number of tokens allowed for the current Config::PromptModel.

fn ctx_msgs_to_prompt_requests( &self, msgs: &[ContextMessage<T>] ) -> Vec<Self::Request>

ContextMessages to Llm::Request conversion.

fn convert_tokens_to_words(&self, tokens: Self::Tokens) -> Self::Tokens

Convert tokens to words.

In the case of ChatGPT, each token represents roughly 75% of a word.

Object Safety

This trait is not object safe.

Implementors