Struct GenericOpenAI

pub struct GenericOpenAI<C: Config + Default = OpenAIConfig> {
    pub stream_full: bool,
    /* private fields */
}

Available on crate feature openai only.

Generic client for OpenAI APIs.

Fields

stream_full: bool

Convenience option to stream the full response. Defaults to true, because nobody has time to reconstruct the delta. Disabling this will make the streamed content only return the delta, for when performance matters. This only has effect when streaming is enabled.

Implementations

impl GenericOpenAI<DashscopeConfig>

pub fn builder() -> DashscopeBuilder

Available on crate feature dashscope only.

impl GenericOpenAI<GeminiConfig>

pub fn builder() -> GeminiBuilder

Available on crate feature gemini only.

impl GenericOpenAI<GroqConfig>

pub fn builder() -> GroqBuilder

Available on crate feature groq only.

impl GenericOpenAI<OllamaConfig>

pub fn builder() -> OllamaBuilder

Available on crate feature ollama only.

Build a new Ollama instance

impl GenericOpenAI<OpenRouterConfig>

pub fn builder() -> OpenRouterBuilder

Available on crate feature open-router only.

Creates a new OpenRouterBuilder for constructing OpenRouter instances.

impl GenericOpenAI

pub fn builder() -> OpenAIBuilder

Creates a new OpenAIBuilder for constructing OpenAI instances.

impl<C: Config + Default> GenericOpenAI<C>

pub async fn estimate_tokens(&self, value: impl Estimatable) -> Result<usize>

Available on crate feature tiktoken only.

Estimates the number of tokens for implementors of the Estimatable trait.

I.e. String, ChatMessage etc

Errors

Errors if tokinization fails in any way

pub fn with_default_prompt_model( &mut self, model: impl Into<String>, ) -> &mut Self

pub fn with_default_embed_model( &mut self, model: impl Into<String>, ) -> &mut Self

pub fn client(&self) -> &Arc<Client<C>>

Retrieve a reference to the inner OpenAI client.

pub fn options(&self) -> &Options

Retrieve a reference to the default options for the OpenAI instance.

pub fn options_mut(&mut self) -> &mut Options

Retrieve a mutable reference to the default options for the OpenAI instance.

Trait Implementations

impl<C: Config + Default + Sync + Send + Debug + Clone> ChatCompletion for GenericOpenAI<C>

fn complete<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatCompletionRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatCompletionResponse, LanguageModelError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

fn complete_stream<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatCompletionRequest, ) -> Pin<Box<dyn Future<Output = ChatCompletionStream> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stream the completion response. If it’s not supported, it will return a single response

impl<C: Clone + Config + Default> Clone for GenericOpenAI<C>

fn clone(&self) -> GenericOpenAI<C>

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<C: Debug + Config + Default> Debug for GenericOpenAI<C>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<C: Config + Default + Sync + Send + Debug + Clone> EmbeddingModel for GenericOpenAI<C>

fn embed<'life0, 'async_trait>( &'life0 self, input: Vec<String>, ) -> Pin<Box<dyn Future<Output = Result<Embeddings, LanguageModelError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn name(&self) -> &'static str

impl<C: Config + Default + Sync + Send + Debug + Clone> SimplePrompt for GenericOpenAI<C>

The SimplePrompt trait defines a method for sending a prompt to an AI model and receiving a response.

fn prompt<'life0, 'async_trait>( &'life0 self, prompt: Prompt, ) -> Pin<Box<dyn Future<Output = Result<String, LanguageModelError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Sends a prompt to the OpenAI API and returns the response content.

Parameters

• prompt: A string slice that holds the prompt to be sent to the OpenAI API.

Returns

• Result<String>: On success, returns the content of the response as a String. On failure, returns an error wrapped in a Result.

Errors

• Returns an error if the model is not set in the default options.
• Returns an error if the request to the OpenAI API fails.
• Returns an error if the response does not contain the expected content.

fn name(&self) -> &'static str