Struct Client 

pub struct Client { /* private fields */ }

A connection to the System One API. One shared pool; call ask from as many threads as Config::pool_size allows.

Implementations

impl Client

pub fn new(api_key: &str, cfg: Config) -> Result<Self, Error>

A client that sends api_key as a bearer token to Config::base_url over HTTPS. Whitespace around the key, such as the newline at the end of a key file, is dropped.

Nothing is sent until the first ask: a key the API rejects is reported then, as Error::Auth.

Errors

What no request could be sent with is reported here rather than retried later: Error::Auth for a blank key or one that cannot go in a header, and Error::InvalidConfig for a Config::base_url that is not an absolute http or https URL or a Config::user_agent that is not a valid header value.

pub fn from_env(cfg: Config) -> Result<Self, Error>

A client that reads its API key from the API_KEY_VAR environment variable.

Errors

Error::Auth if the variable is unset or blank; otherwise as Client::new.

Examples found in repository

examples/ask.rs (line 13)

fn main() -> Result<(), Error> {
    let client = Client::from_env(Config::default())?;

    let state = json!({
        "code": "1| import os\n2| def cwd():\n3|     return os.getcwd()",
        "query": "where is the working directory read",
    });
    let mut questions = Map::new();
    questions.insert(
        "line3".into(),
        json!({
            "type": "noul",
            "instructions": "Does line 3 of the code directly answer the query?",
        }),
    );
    questions.insert(
        "relevance".into(),
        json!({
            "type": "score",
            "instructions": "How relevant is the code to the query?",
            "criteria": ["unrelated", "tangential", "relevant", "exactly what was asked"],
        }),
    );

    match client.ask(&state, &questions) {
        Ok(answers) => {
            for (id, answer) in &answers {
                println!("{id}: {answer}");
            }
        }
        // Too much state for one request: a real caller would split it and ask again.
        Err(Error::TokenLimit(message)) => eprintln!("request too large: {message}"),
        Err(other) => return Err(other),
    }

    let usage = client.usage();
    println!(
        "{} request(s), {} retries, {} input tokens, ~${:.6}",
        usage.requests(),
        usage.retries(),
        usage.input_tokens(),
        usage.cost_usd()
    );
    Ok(())
}

pub fn with_transport(transport: impl Transport + 'static, cfg: Config) -> Self

A client over any Transport, for tests and local fakes. Config::base_url, Config::user_agent and the timeouts are the transport’s business and are ignored.

pub fn set_debug_reporter( &mut self, reporter: impl Fn(&str) + Send + Sync + 'static, )

Receives one line per failed attempt (the status or connection error, and any Retry-After) before the client retries. Nothing is reported without a reporter, and the reporter changes neither the retry policy nor the request.

pub fn model(&self) -> &str

The model name sent in every request.

pub fn usage(&self) -> &Usage

Requests, retries and tokens so far.

Examples found in repository

examples/ask.rs (line 47)

fn main() -> Result<(), Error> {
    let client = Client::from_env(Config::default())?;

    let state = json!({
        "code": "1| import os\n2| def cwd():\n3|     return os.getcwd()",
        "query": "where is the working directory read",
    });
    let mut questions = Map::new();
    questions.insert(
        "line3".into(),
        json!({
            "type": "noul",
            "instructions": "Does line 3 of the code directly answer the query?",
        }),
    );
    questions.insert(
        "relevance".into(),
        json!({
            "type": "score",
            "instructions": "How relevant is the code to the query?",
            "criteria": ["unrelated", "tangential", "relevant", "exactly what was asked"],
        }),
    );

    match client.ask(&state, &questions) {
        Ok(answers) => {
            for (id, answer) in &answers {
                println!("{id}: {answer}");
            }
        }
        // Too much state for one request: a real caller would split it and ask again.
        Err(Error::TokenLimit(message)) => eprintln!("request too large: {message}"),
        Err(other) => return Err(other),
    }

    let usage = client.usage();
    println!(
        "{} request(s), {} retries, {} input tokens, ~${:.6}",
        usage.requests(),
        usage.retries(),
        usage.input_tokens(),
        usage.cost_usd()
    );
    Ok(())
}

pub fn limiter(&self) -> &AdaptiveLimiter

The concurrency gate, for inspection.

pub fn ask( &self, state: &Value, questions: &Map<String, Value>, ) -> Result<Map<String, Value>, Error>

Evaluates questions against state and returns the answers map, keyed like the questions.

state is any JSON the questions refer to. Each question is an object with a type (noul or score) and instructions; a score question also lists its criteria. Each answer is an object with the same type and the model’s judgment: a noul probability in [0, 1], or a score with its confidence and probabilities. The crate documentation shows both shapes.

Transient failures are retried with exponential backoff up to Config::max_retries times, honouring Retry-After (capped at 30 seconds) and the shared AdaptiveLimiter. A Retry-After given as a date rather than as seconds is ignored in favour of the backoff. The call blocks the calling thread; run it from a thread pool for concurrency.

Errors

Error::Auth for HTTP 401 and 403, and Error::TokenLimit for a request larger than the model’s context, both without a retry. Error::Api for any other status that is not worth retrying, for a reply that is not the documented JSON, and once the retries are spent; the message then ends with the last failure.

Examples found in repository

examples/ask.rs (line 36)

fn main() -> Result<(), Error> {
    let client = Client::from_env(Config::default())?;

    let state = json!({
        "code": "1| import os\n2| def cwd():\n3|     return os.getcwd()",
        "query": "where is the working directory read",
    });
    let mut questions = Map::new();
    questions.insert(
        "line3".into(),
        json!({
            "type": "noul",
            "instructions": "Does line 3 of the code directly answer the query?",
        }),
    );
    questions.insert(
        "relevance".into(),
        json!({
            "type": "score",
            "instructions": "How relevant is the code to the query?",
            "criteria": ["unrelated", "tangential", "relevant", "exactly what was asked"],
        }),
    );

    match client.ask(&state, &questions) {
        Ok(answers) => {
            for (id, answer) in &answers {
                println!("{id}: {answer}");
            }
        }
        // Too much state for one request: a real caller would split it and ask again.
        Err(Error::TokenLimit(message)) => eprintln!("request too large: {message}"),
        Err(other) => return Err(other),
    }

    let usage = client.usage();
    println!(
        "{} request(s), {} retries, {} input tokens, ~${:.6}",
        usage.requests(),
        usage.retries(),
        usage.input_tokens(),
        usage.cost_usd()
    );
    Ok(())
}

Trait Implementations

impl Debug for Client

The model, the retry policy and the counters. The transport is left out: it holds the key.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.