Trait Config

pub trait Config:
    Debug
    + Sized
    + Clone
    + Default
    + Send
    + Sync
    + 'static {
    type PromptModel: Llm<Self>;
    type SummaryModel: Llm<Self>;
    type Chest: TapestryChestHandler<Self>;

    const MINIMUM_RESPONSE_LENGTH: u64;
    const TOKEN_THRESHOLD_PERCENTILE: BoundedU8<0, 100> = _;

    // Required method
    fn convert_prompt_tokens_to_summary_model_tokens(
        tokens: PromptModelTokens<Self>,
    ) -> SummaryModelTokens<Self>;
}

A trait consisting of the main configuration needed to implement [Loom].

Required Associated Constants

const MINIMUM_RESPONSE_LENGTH: u64

Ensures that the maximum completion tokens is at least the minimum response length.

If the maximum completion tokens is less than the minimum response length, a summary will be generated and a new tapestry fragment will be created.

Provided Associated Constants

const TOKEN_THRESHOLD_PERCENTILE: BoundedU8<0, 100> = _

Number between 0 and 100. Represents the percentile of the maximum number of tokens allowed for the current Config::PromptModel before a summary is generated.

Defaults to 85%

Required Associated Types

type PromptModel: Llm<Self>

The LLM to use for generating responses to prompts.

type SummaryModel: Llm<Self>

The LLM to use for generating summaries of the current TapestryFragment instance.

This is separate from Config::PromptModel to allow for a larger model to be used for generating summaries.

type Chest: TapestryChestHandler<Self>

Storage handler interface for storing and retrieving tapestry fragments.

You can optionally enable the redis or rocksdb features to use the default storage implementations for these storage backends.

Required Methods

fn convert_prompt_tokens_to_summary_model_tokens( tokens: PromptModelTokens<Self>, ) -> SummaryModelTokens<Self>

Convert Config::PromptModel to Config::SummaryModel tokens.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors