Struct SimpleApi 

pub struct SimpleApi<RT>
where
    RT: TaskInterface + HttpClientInterface,
{ /* private fields */ }

Simple API.

Implementations

impl<RT> SimpleApi<RT>

where RT: TaskInterface + HttpClientInterface,

pub fn new( runtime: RT, baseuri: impl Into<String>, port: u16, model: Option<String>, ) -> Result<Self>

Create a new SimpleApi object that will query a LLM end-point at baseuri (e.g. http://localhost) om the given port (e.g. 8080), using the optional model.

Trait Implementations

impl<RT> Debug for SimpleApi<RT>

where RT: TaskInterface + HttpClientInterface + Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<RT> LargeLanguageModel for SimpleApi<RT>

where RT: TaskInterface + HttpClientInterface,

fn chat_stream( &self, prompt: ChatPrompt, ) -> Result<impl Future<Output = Result<StringStream>> + Send>

Chat with a model, returning a stream

fn generate_stream( &self, prompt: GenerationPrompt, ) -> Result<impl Future<Output = Result<StringStream>> + Send>

Run inference on a model, returning a stream

fn chat( &self, prompt: ChatPrompt, ) -> Result<impl Future<Output = Result<String>> + Send>

Run chat on a model, return once the complete answer has been computed. The default implementation call chat_stream until completion of the stream.

fn generate( &self, prompt: GenerationPrompt, ) -> Result<impl Future<Output = Result<String>> + Send>

Run inference on a model, return once the complete answer has been computed. The default implementation call infer_stream until completion of the stream.