Struct Client 

pub struct Client { /* private fields */ }

A connection to the System One API. One shared pool; call ask from as many threads as Config::pool_size allows.

Implementations

impl Client

pub fn new(api_key: &str, cfg: Config) -> Result<Self, Error>

A client that sends api_key as a bearer token to Config::base_url over HTTPS. Whitespace around the key, such as the newline at the end of a key file, is dropped.

Nothing is sent until the first ask: a key the API rejects is reported then, as Error::Auth.

Errors

What no request could be sent with is reported here rather than retried later: Error::Auth for a blank key or one that cannot go in a header, and Error::InvalidConfig for a Config::base_url that is not an absolute http or https URL or a Config::user_agent that is not a valid header value.

pub fn from_env(cfg: Config) -> Result<Self, Error>

A client that reads its API key from the API_KEY_VAR environment variable.

Errors

Error::Auth if the variable is unset or blank; otherwise as Client::new.

Examples found in repository

examples/ask.rs (line 21)

fn main() -> Result<(), Error> {
    let client = Client::from_env(Config::default())?;

    let ticket = Ticket {
        customer_plan: "business",
        message: "Hi, I've been trying to connect my Stripe account for 3 days and the integration keeps failing. \
                  I'm losing sales. Please help ASAP.",
    };
    let questions = Questions::new()
        .with(
            "department",
            Choice::new(
                "Which team should handle the `message`?",
                [
                    ("billing", "Payment or subscription issues"),
                    ("technical", "Bugs or integration problems"),
                    ("sales", "Pricing or account questions"),
                ],
            ),
        )
        .with(
            "frustration",
            Score::new(
                "How frustrated the customer appears",
                ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"],
            ),
        )
        .with("is_urgent", Noul::new("The message conveys urgency or time-sensitivity"));

    let response = match client.ask(&ticket, &questions) {
        Ok(response) => response,
        // Too much state for one request: a real caller would split it and ask again.
        Err(Error::TokenLimit(message)) => {
            eprintln!("request too large: {message}");
            return Ok(());
        }
        Err(other) => return Err(other),
    };

    if let Some(department) = response.choice("department") {
        println!("department: {} (confidence {:.2})", department.choice, department.confidence);
        for (option, probability) in &department.probabilities {
            println!("  {option}: {probability:.2}");
        }
    }
    if let Some(frustration) = response.score("frustration") {
        println!("frustration: {:.2} of {} (confidence {:.2})", frustration.score, frustration.legend.len() - 1, frustration.confidence);
    }
    if let Some(urgent) = response.noul("is_urgent") {
        println!("urgent: p={:.2}", urgent.noul);
    }

    let usage = client.usage();
    println!(
        "{}: {} request(s), {} retries, {} input tokens, ~${:.6}",
        response.model,
        usage.requests(),
        usage.retries(),
        usage.input_tokens(),
        usage.cost_usd()
    );
    Ok(())
}

pub fn with_transport(transport: impl Transport + 'static, cfg: Config) -> Self

A client over any Transport, for tests and local fakes. Config::base_url, Config::user_agent and the timeouts are the transport’s business and are ignored.

pub fn set_debug_reporter( &mut self, reporter: impl Fn(&str) + Send + Sync + 'static, )

Receives one line per failed attempt (the status or connection error, and any Retry-After) before the client retries. Nothing is reported without a reporter, and the reporter changes neither the retry policy nor the request.

pub fn model(&self) -> &str

The model name sent in every request.

pub fn usage(&self) -> &Usage

Requests, retries and tokens so far.

Examples found in repository

examples/ask.rs (line 72)

fn main() -> Result<(), Error> {
    let client = Client::from_env(Config::default())?;

    let ticket = Ticket {
        customer_plan: "business",
        message: "Hi, I've been trying to connect my Stripe account for 3 days and the integration keeps failing. \
                  I'm losing sales. Please help ASAP.",
    };
    let questions = Questions::new()
        .with(
            "department",
            Choice::new(
                "Which team should handle the `message`?",
                [
                    ("billing", "Payment or subscription issues"),
                    ("technical", "Bugs or integration problems"),
                    ("sales", "Pricing or account questions"),
                ],
            ),
        )
        .with(
            "frustration",
            Score::new(
                "How frustrated the customer appears",
                ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"],
            ),
        )
        .with("is_urgent", Noul::new("The message conveys urgency or time-sensitivity"));

    let response = match client.ask(&ticket, &questions) {
        Ok(response) => response,
        // Too much state for one request: a real caller would split it and ask again.
        Err(Error::TokenLimit(message)) => {
            eprintln!("request too large: {message}");
            return Ok(());
        }
        Err(other) => return Err(other),
    };

    if let Some(department) = response.choice("department") {
        println!("department: {} (confidence {:.2})", department.choice, department.confidence);
        for (option, probability) in &department.probabilities {
            println!("  {option}: {probability:.2}");
        }
    }
    if let Some(frustration) = response.score("frustration") {
        println!("frustration: {:.2} of {} (confidence {:.2})", frustration.score, frustration.legend.len() - 1, frustration.confidence);
    }
    if let Some(urgent) = response.noul("is_urgent") {
        println!("urgent: p={:.2}", urgent.noul);
    }

    let usage = client.usage();
    println!(
        "{}: {} request(s), {} retries, {} input tokens, ~${:.6}",
        response.model,
        usage.requests(),
        usage.retries(),
        usage.input_tokens(),
        usage.cost_usd()
    );
    Ok(())
}

pub fn limiter(&self) -> &AdaptiveLimiter

The concurrency gate, for inspection.

pub fn ask<S: Serialize + ?Sized>( &self, state: &S, questions: &Questions, ) -> Result<Response, Error>

Evaluates questions against state and returns the Response: one Answer per question, under the id the question had.

state is what the questions are about, and anything that serializes to a JSON string, object or array: a &str, a serde_json::Value, or a type of your own that derives Serialize. Jev reads text only. A question can name a field of the state in backticks.

Transient failures are retried with exponential backoff up to Config::max_retries times, honouring Retry-After (capped at 30 seconds) and the shared AdaptiveLimiter. A Retry-After given as a date rather than as seconds is ignored in favour of the backoff. The call blocks the calling thread; run it from a thread pool for concurrency.

Errors

Error::Auth for HTTP 401 and 403, Error::TokenLimit for a request larger than the model’s context, and Error::InvalidRequest for a state that does not serialize or a body the API refuses as malformed (HTTP 422), all without a retry. Error::Api for any other status that is not worth retrying, for a reply that is not the documented JSON, and once the retries are spent; the message then ends with the last failure.

Examples found in repository

examples/ask.rs (line 49)

fn main() -> Result<(), Error> {
    let client = Client::from_env(Config::default())?;

    let ticket = Ticket {
        customer_plan: "business",
        message: "Hi, I've been trying to connect my Stripe account for 3 days and the integration keeps failing. \
                  I'm losing sales. Please help ASAP.",
    };
    let questions = Questions::new()
        .with(
            "department",
            Choice::new(
                "Which team should handle the `message`?",
                [
                    ("billing", "Payment or subscription issues"),
                    ("technical", "Bugs or integration problems"),
                    ("sales", "Pricing or account questions"),
                ],
            ),
        )
        .with(
            "frustration",
            Score::new(
                "How frustrated the customer appears",
                ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"],
            ),
        )
        .with("is_urgent", Noul::new("The message conveys urgency or time-sensitivity"));

    let response = match client.ask(&ticket, &questions) {
        Ok(response) => response,
        // Too much state for one request: a real caller would split it and ask again.
        Err(Error::TokenLimit(message)) => {
            eprintln!("request too large: {message}");
            return Ok(());
        }
        Err(other) => return Err(other),
    };

    if let Some(department) = response.choice("department") {
        println!("department: {} (confidence {:.2})", department.choice, department.confidence);
        for (option, probability) in &department.probabilities {
            println!("  {option}: {probability:.2}");
        }
    }
    if let Some(frustration) = response.score("frustration") {
        println!("frustration: {:.2} of {} (confidence {:.2})", frustration.score, frustration.legend.len() - 1, frustration.confidence);
    }
    if let Some(urgent) = response.noul("is_urgent") {
        println!("urgent: p={:.2}", urgent.noul);
    }

    let usage = client.usage();
    println!(
        "{}: {} request(s), {} retries, {} input tokens, ~${:.6}",
        response.model,
        usage.requests(),
        usage.retries(),
        usage.input_tokens(),
        usage.cost_usd()
    );
    Ok(())
}

Trait Implementations

impl Debug for Client

The model, the retry policy and the counters. The transport is left out: it holds the key.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.