Trait llm_weaver::Config

pub trait Config: Debug + Sized + Clone + Default + Send + Sync + 'static {
    type PromptModel: Llm<Self>;
    type SummaryModel: Llm<Self>;
    type Chest: TapestryChestHandler<Self> = TapestryChest;

    const MINIMUM_RESPONSE_LENGTH: u64;
    const TOKEN_THRESHOLD_PERCENTILE: BoundedU8<0, 100> = _;

    // Required method
    fn convert_prompt_tokens_to_summary_model_tokens(
        tokens: PromptModelTokens<Self>
    ) -> SummaryModelTokens<Self>;
}

A trait consisting of the main configuration needed to implement Loom.

Required Associated Types

type PromptModel: Llm<Self>

The LLM to use for generating responses to prompts.

type SummaryModel: Llm<Self>

The LLM to use for generating summaries of the current TapestryFragment instance.

This is separate from Config::PromptModel to allow for a larger model to be used for generating summaries.

Provided Associated Types

type Chest: TapestryChestHandler<Self> = TapestryChest

Storage handler interface for storing and retrieving tapestry fragments.

Defaults to TapestryChest. Using this default requires you to supply the hostname, port and credentials to connect to your instance.

Required Associated Constants

const MINIMUM_RESPONSE_LENGTH: u64

Ensures that the maximum completion tokens is at least the minimum response length.

If the maximum completion tokens is less than the minimum response length, a summary will be generated and a new tapestry fragment will be created.

Provided Associated Constants

const TOKEN_THRESHOLD_PERCENTILE: BoundedU8<0, 100> = _

Number between 0 and 100. Represents the percentile of the maximum number of tokens allowed for the current Config::PromptModel before a summary is generated.

Defaults to 85%

Required Methods

fn convert_prompt_tokens_to_summary_model_tokens( tokens: PromptModelTokens<Self> ) -> SummaryModelTokens<Self>

Convert Config::PromptModel to Config::SummaryModel tokens.

Object Safety

This trait is not object safe.

Implementors