Struct OpenAICompletionStream

pub struct OpenAICompletionStream { /* private fields */ }

OpenAICompletionStream

This structs wraps the EventSource and parses returned messages into prompt messages on demand.

Implementations

impl OpenAICompletionStream

pub fn new(event_source: EventSource) -> Self

OpenAICompletionStream::new

This struct just wraps the EventSource when from the context of streaming chat completions.

Trait Implementations

impl ChatCompletionStream for OpenAICompletionStream

async fn next(&mut self) -> Option<Result<Self::Item, Self::ErrorType>>

ChatCompletionStream::next

Method to iterate over the completion stream. Note it blocks until the next message is received. At which point it will parse the response into a CompletionStreamValue.

Examples

use rag_toolchain::clients::*;

async fn stream_chat_completions(client: OpenAIChatCompletionClient) {
    let user_message: PromptMessage = PromptMessage::HumanMessage("Please ask me a question".into());
    let mut stream: OpenAICompletionStream = client.invoke_stream(vec![user_message]).await.unwrap();
    while let Some(response) = stream.next().await {
        match response {
           Ok(CompletionStreamValue::Connecting) => {},
           Ok(CompletionStreamValue::Message(msg)) => {
                println!("{:?}", msg.content());
           }
           Err(e) => {
                println!("{:?}", e);
                break;
           }
        }
    }
}

Errors

• OpenAIError::ErrorReadingStream - if there was an error reading from the stream.
• OpenAIError::ErrorDeserializingResponseBody - if there was an error deserializing the response body.

Returns

• Option<Result<CompletionStreamValue, OpenAIError>> - the response from the chat client. None represents the stream is finished..

type ErrorType = OpenAIError

type Item = CompletionStreamValue