Trait llm_chain::traits::Executor

source · 
pub trait Executor: Sized {
    type StepTokenizer<'a>: Tokenizer
       where Self: 'a;

    // Required methods
    fn new_with_options(options: Options) -> Result<Self, ExecutorCreationError>;
    fn execute<'life0, 'life1, 'life2, 'async_trait>(
        &'life0 self,
        options: &'life1 Options,
        prompt: &'life2 Prompt
    ) -> Pin<Box<dyn Future<Output = Result<Output, ExecutorError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait;
    fn tokens_used(
        &self,
        options: &Options,
        prompt: &Prompt
    ) -> Result<TokenCount, PromptTokensError>;
    fn max_tokens_allowed(&self, options: &Options) -> i32;
    fn answer_prefix(&self, prompt: &Prompt) -> Option<String>;
    fn get_tokenizer(
        &self,
        options: &Options
    ) -> Result<Self::StepTokenizer<'_>, TokenizerError>;

    // Provided method
    fn new() -> Result<Self, ExecutorCreationError> { ... }
}
Expand description

The Executor trait represents an executor that performs a single step in a chain. It takes a step, executes it, and returns the output.

Required Associated Types§

source

type StepTokenizer<'a>: Tokenizer where Self: 'a

Required Methods§

source

fn new_with_options(options: Options) -> Result<Self, ExecutorCreationError>

Create a new executor with the given options. If you don’t need to set any options, you can use the new method instead.

Parameters
  • executor_options: The options to set.
source

fn execute<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, options: &'life1 Options, prompt: &'life2 Prompt ) -> Pin<Box<dyn Future<Output = Result<Output, ExecutorError>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

source

fn tokens_used( &self, options: &Options, prompt: &Prompt ) -> Result<TokenCount, PromptTokensError>

Calculates the number of tokens used by the step given a set of parameters.

The step and the parameters together are used to form full prompt, which is then tokenized and the token count is returned.

Parameters
  • step: The step to calculate token usage for.
  • parameters: The parameters to plug into the step.
Returns

A Result containing the token count, or an error if there was a problem.

source

fn max_tokens_allowed(&self, options: &Options) -> i32

Returns the maximum number of input tokens allowed by the model used.

Parameters
  • options: The per-invocation options that affect the token allowance.
Returns

The max token count for the step

source

fn answer_prefix(&self, prompt: &Prompt) -> Option<String>

Returns a possible answer prefix inserted by the model, during a certain prompt mode

Parameters
  • prompt: The prompt passed into step
Returns

A Option containing a String if prefix exists, or none if there is no prefix

source

fn get_tokenizer( &self, options: &Options ) -> Result<Self::StepTokenizer<'_>, TokenizerError>

Creates a tokenizer, depending on the model used by step.

Parameters
  • step: The step to get an associated tokenizer for.
Returns

A Result containing a tokenizer, or an error if there was a problem.

Provided Methods§

source

fn new() -> Result<Self, ExecutorCreationError>

Implementors§