Trait LLMService

pub trait LLMService<M: ModelInfo> {
    // Required method
    fn generate_next_message<'life0, 'life1, 'async_trait>(
        &'life0 self,
        chat: &'life1 Chat,
    ) -> Pin<Box<dyn Future<Output = Result<Message>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
}

This is anything that can generate the next message.

LLMService is responsible for generating the next message in a conversation. It replaces the previous SingleRequestExecutor pattern with a more flexible trait-based approach, allowing for different implementations (HTTP, local, etc).

Required Methods

fn generate_next_message<'life0, 'life1, 'async_trait>( &'life0 self, chat: &'life1 Chat, ) -> Pin<Box<dyn Future<Output = Result<Message>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generates the next message in the conversation.

Takes a Chat instance and returns a Result containing the next message.

Implementors

impl<M: ModelInfo> LLMService<M> for HTTPLlmService<M>