Trait llm_chain::tokens::ExecutorTokenCountExt

pub trait ExecutorTokenCountExt: Executor {
    // Provided method
    fn split_to_fit(
        &self,
        step: &Step,
        doc: &Parameters,
        base_parameters: &Parameters,
        chunk_overlap: Option<usize>
    ) -> Result<Vec<Parameters>, PromptTokensError> { ... }
}

An extension trait for the Executor trait that provides additional methods for working with token counts.

Provided Methods

fn split_to_fit( &self, step: &Step, doc: &Parameters, base_parameters: &Parameters, chunk_overlap: Option<usize> ) -> Result<Vec<Parameters>, PromptTokensError>

Splits a Parameters object into multiple smaller Parameters objects that fit within the context window size supported by the given model.

Arguments

• step - The step that will process the Parameters. Has impact on tokenizer & text splitter used
• doc - The parameter object to split into multiple, smaller, parameters
• chunk_overlap - The amount of tokens each split part should overlap with previous & next chunk

Errors

Returns a PromptTokensError if there is an issue computing the tokens.

Implementors

impl<E: Executor> ExecutorTokenCountExt for E

Blanket implementation of ExecutorTokenCountExt for all Executors