Struct Orchestrator

pub struct Orchestrator { /* private fields */ }

The central interface for openai_orch. The Orchestrator is responsible for managing the concurrency of requests and their responses.

Using the Orchestrator is simple:

1. Create an Orchestrator with the desired policies and keys.
2. Create a request type that implements OrchRequest (optional).
3. Call add_request on the Orchestrator with the request handler.
4. Call get_response on the Orchestrator with the request ID returned by add_request.

The Orchestrator will handle the concurrency of requests and responses automatically.

To use the Orchestrator in multiple parts of your application, you can clone it. The Orchestrator is backed by an Arc, so cloning it is cheap.

use openai_orch::{
  chat::siso::{ChatSisoRequest, ChatSisoResponse},
  keys::Keys,
  policies::Policies,
  Orchestrator,
};
 
#[tokio::main]
async fn main() {
  let policies = Policies::default();
  let keys = Keys::from_env().unwrap();
  let orchestrator = Orchestrator::new(policies, keys);
 
  let request = ChatSisoRequest::new(
    "You are a helpful assistant.".to_string(),
    "What are you?".to_string(),
    Default::default(),
  );
  let request_id = orchestrator.add_request(request).await;
 
  let response = orchestrator
    .get_response::<ChatSisoResponse>(request_id)
    .await;
  println!("{}", response.unwrap());
}

Implementations

impl Orchestrator

pub fn new(policies: Policies, keys: Keys) -> Self

Create a new Orchestrator with the given policies and keys.

pub async fn add_request<R, Req>(&self, request: Req) -> RequestID<R>

where Req: OrchRequest<Res = R> + Send + Sync + 'static, R: ResponseType,

Add a request to the Orchestrator. Returns a request ID that can be used to get the response.

Behind the scenes the Orchestrator will create a task for the request using the OrchRequest’s send method when the concurrency policy allows it. The result will be sent back to the Orchestrator using a channel which is mapped to the request ID.

pub async fn get_response<R: ResponseType>( &self, request_id: RequestID<R>, ) -> Result<R>

Get the response for a given request ID.

This will block until the response is received.

Behind the scenes, this listens on a channel for a task to send the response back to the Orchestrator. Once the response is received, it is returned.

Trait Implementations

impl Clone for Orchestrator

fn clone(&self) -> Orchestrator

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.