TypeSafe AI Rust SDK

An async Rust client for the TypeSafe AI API: the System One endpoint (POST /v1/systemone), which answers named questions about a state, and the model listing (GET /v1/models). It is a port of the official Python SDK, typesafe-sdk-python 0.7.1 (commit 0ffd094); the places where it behaves differently on purpose are listed under Deviations from the Python SDK.

The package is published as typesafe-sdk-rust, because the name typesafe-sdk is already taken on crates.io. The library it builds is typesafe_sdk, so code writes use typesafe_sdk::....

Install

[dependencies]
typesafe-sdk-rust = "0.1.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

The minimum supported Rust version is 1.98, and the crate uses edition 2024.

Runtime requirement

Every network operation is async and runs on the caller's Tokio runtime, which must have its time driver enabled: each attempt has a deadline, retries sleep between attempts, and HTTP/2 keep-alive pings run on a timer. #[tokio::main] and Builder::enable_all() / enable_time() enable it; Tokio panics when a timer is created on a runtime without it.

Quickstart

A call asks a set of named questions about a state. The set is validated and serialized once by Questions::prepare, and the resulting PreparedQuestions is reused by every call that asks it.

use typesafe_sdk::{Choice, Client, Noul, Questions, Score};

#[tokio::main]
async fn main() -> Result<(), typesafe_sdk::Error> {
    // Reads TYPESAFE_API_KEY (and optionally TYPESAFE_BASE_URL, TYPESAFE_DEFAULT_MODEL).
    let client = Client::from_env()?;

    let questions = Questions::new()
        .noul("billing", Noul::new().instructions("Is this about billing?"))
        .choice("tone", Choice::new(["calm", "angry"]).instructions("What is the tone?"))
        .score("urgency", Score::new(["can wait", "this week", "today"]))
        .prepare()?;

    let state = "I was charged twice for one order.";
    let response = client.system_one(state, &questions).model("jev-latest").send().await?;

    let answers = response.answers();
    if let Some(billing) = answers.noul("billing") {
        println!("billing: {}", billing.noul());
    }
    if let Some(tone) = answers.choice("tone") {
        println!("tone: {} ({})", tone.choice(), tone.confidence());
    }
    if let Some(urgency) = answers.score("urgency") {
        println!("urgency: {} ({})", urgency.score(), urgency.confidence());
    }
    println!("request id: {:?}", response.meta().request_id());
    Ok(())
}

state is anything that implements serde::Serialize and encodes as a JSON string, object or array; it is encoded straight into the request body when the request is sent. A request is configured with .model(), .timeout() / .no_timeout(), .header(), .retry() and .extra_body(name, &value) (a top-level body member the SDK does not model; a later member of the same name replaces an earlier one, and one named state, model or questions replaces the built-in member). Nothing is checked or encoded before .send().

A question that the builders do not model is written with RawQuestion, which sends its fields unread and applies only the checks the Python SDK applies to a raw dictionary (type is a non-empty string, a choice or score has criteria, a score's criteria is not empty):

use typesafe_sdk::{Questions, RawQuestion};

fn main() -> Result<(), typesafe_sdk::Error> {
    let questions = Questions::new()
        .raw("spam", RawQuestion::new("noul").field("instructions", "Is this spam?"))
        .prepare()?;
    assert_eq!(questions.names().collect::<Vec<_>>(), ["spam"]);
    Ok(())
}

The answers of a response are looked up by question name (answers().noul(name), .choice(name), .score(name), .get(name)), iterated in wire order (iter()), or filtered by kind (nouls(), choices(), scores(), which copy nothing). An answer of a type this version does not know is skipped with a WARN event; the received bytes stay available through response.meta().raw_body(), together with the status and the headers.

Responses and answers implement Serialize, and the answer types implement Deserialize, so they can be stored and read back. They serialize with any serde format, but reading them back is supported through JSON codecs (sonic-rs, serde_json): through a non-JSON serde format, a score whose legend holds text fails, because a level's description is read back as JSON text.

Typed answers

With the macros feature, a struct with one field per question is both the question set and the answer type. The questions JSON is generated at compile time, and the response decodes straight into the fields, without a map in between.

use typesafe_sdk::{ChoiceAnswer, Client, NoulAnswer, QuestionSet, ScoreAnswer};

#[derive(QuestionSet)]
struct Ticket {
    #[noul(instructions = "Is this about billing?", yes = "payments or invoices")]
    billing: NoulAnswer,
    #[choice(instructions = "What is the tone?", options("calm" = "neutral or polite", "angry"))]
    tone: ChoiceAnswer,
    #[score(instructions = "How urgent?", levels("can wait", "this week", "today"))]
    urgency: ScoreAnswer,
}

#[tokio::main]
async fn main() -> Result<(), typesafe_sdk::Error> {
    let client = Client::from_env()?;
    let response = client.ask::<Ticket>("I was charged twice for one order.").send().await?;
    let ticket = response.answers();
    println!("billing: {}", ticket.billing.noul());
    println!("tone: {}", ticket.tone.choice());
    println!("urgency: {}", ticket.urgency.score());
    Ok(())
}

client.ask::<Ticket>(&state) is client.system_one(&state, Ticket::prepared()).typed::<Ticket>(); the second spelling is the one to use where the builder's type has to be named. The derive accepts text content only; object or array content stays a feature of the runtime Questions builder. It refuses at compile time a duplicate wire name, option or level, a score without levels, a field whose type does not match its question kind, and an Option<...Answer> field. #[question(name = "...")] sends a key that is not a Rust identifier, and #[question_set(crate = ...)] points the generated code at a renamed dependency.

Models

use typesafe_sdk::{Client, Error};

async fn print_models(client: &Client) -> Result<(), Error> {
    let listed = client.models().list().send().await?;
    for model in listed.models() {
        println!("{} ({}): {}", model.name(), model.release_date(), model.description());
    }
    Ok(())
}

Errors

Every fallible operation returns typesafe_sdk::Error, one pointer wide. Error::kind() says what failed:

The enum is #[non_exhaustive], so a match needs a catch-all arm.

ApiError carries status(), kind() (an ApiErrorKind by status: 400 BadRequest, 401 Authentication, 403 PermissionDenied, 404 NotFound, 422 UnprocessableEntity, 429 RateLimit, 500 and above InternalServer, anything else Other), headers(), endpoint(), message(), error_type(), body(), body_text(), body_json::<T>(), request_id() (from x-typesafe-request-id) and retry_after() (from retry-after-ms, else Retry-After as seconds or an HTTP date, truncated to whole milliseconds). The live API answers a request without a key with 403 and error_type() == Some("authentication_error"), not the documented 401.

An error's Display is the sentence to show a user; for an API error it reads POST https://api.typesafe.ai/v1/systemone: 429 Too many requests (request_id=req_123).

A response body that is not valid UTF-8 is not JSON (RFC 8259 requires UTF-8), and every body is checked before the parser reads it. With a 2xx status it is a ResponseValidation error: its decode_error() is a Syntax error at the first bad byte (line and column, the column counted in bytes), its field_path() is empty, and like any body that does not decode it is not retried for its status. With any other status the caller still gets the ApiError, with its status, headers, retry_after() and every byte of the body in body(); its message() is the whole body as lossy text (each bad sequence becomes U+FFFD), escaped and cut at 200 characters like any body that is not JSON, and error_type() is None.

use typesafe_sdk::{ApiErrorKind, Client, ErrorKind, PreparedQuestions};

async fn ask(client: &Client, questions: &PreparedQuestions) {
    match client.system_one("hello", questions).send().await {
        Ok(response) => println!("{} answers", response.answers().len()),
        Err(error) => match error.kind() {
            ErrorKind::Api(api) if api.kind() == ApiErrorKind::RateLimit => {
                eprintln!("rate limited, retry after {:?}: {error}", api.retry_after());
            }
            ErrorKind::ResponseValidation(invalid) => {
                eprintln!("unexpected response at {}: {error}", invalid.field_path());
            }
            _ => eprintln!("{error}"),
        },
    }
}

Retries

A RetryPolicy decides which failed attempts are repeated. The defaults are the Python SDK's: 2 retries; a backoff starting at 500 ms, doubling up to 5 s, with up to a quarter of each delay randomly taken off; the statuses 408, 429 and 500-599; connection failures and timeouts retried; retry-after-ms / Retry-After obeyed, however long they ask for; and a budget of 30 s for the whole call.

use std::time::Duration;

use typesafe_sdk::{Client, Error, PreparedQuestions, RetryPolicy, StatusSet};

async fn ask_twice(questions: &PreparedQuestions) -> Result<(), Error> {
    let client = Client::builder()
        .retry(
            RetryPolicy::default()
                .max_retries(3)
                .http_statuses([429, 502, 503, 504].into_iter().collect::<StatusSet>())
                .timeout(Duration::from_secs(10))?,
        )
        .build()?;

    // Retried as the client's policy says.
    client.system_one("hello", questions).send().await?;
    // One attempt only; the client and its other calls keep their policy.
    client.system_one("hello", questions).retry(RetryPolicy::default().max_retries(0)).send().await?;
    Ok(())
}

• A policy given to a call with .retry() replaces the client's for that call only.
• Budget rule: before each retry, if the time the call has already taken plus the next delay reaches the budget, retrying stops and the call fails with the last attempt's error, unchanged. RetryPolicy::no_timeout() removes the budget. The budget is separate from the per-attempt deadline set with the client's or the call's timeout. Without a budget a server's Retry-After is obeyed however long it is; keep a budget, or turn respect_retry_after off, when the server is not trusted.
• predicate(|error| ...) adds failures of the caller's choosing; it sees every failure, including a response that did not decode or was over the size limit.
• A 2xx status in the status set retries nothing: a success response whose body does not decode is a ResponseValidation error and is never retried for its status, since the same body would come back and every retry is a billed call. A predicate that accepts the error is the way to retry it.
• The body is encoded once; every attempt sends the same bytes. Retries carry X-TypeSafe-Retry-Count. Dropping the call's future cancels a pending retry.

Configuration

Client::from_env() is Client::builder().build(). Every builder setting is optional; the first three fall back to the environment, then to a default:

These three are the only environment variables the SDK reads, and only for a setting the caller left unset: an explicit value always wins. A value from the environment is trimmed (with Python's str.strip() rules), a blank one counts as unset, and one that is not UTF-8 is a Config error naming the variable. An explicit key or default model that is blank is refused instead of sent. Trailing slashes come off the base URL, and a path prefix is kept (https://example.test/prefix/// sends to https://example.test/prefix/v1/systemone). The per-attempt deadline runs from the first byte sent to the last byte received: a multi-megabyte state on a slow link can exceed 10 s and be retried, so raise the deadline for large states.

The API key is sent as Authorization: Bearer <key> and never printed: the Debug output of the builder, the client and a request show neither the key nor any header value. Headers are merged as client defaults < per-call headers < the SDK's own (Authorization, Accept, User-Agent, X-TypeSafe-SDK, X-TypeSafe-Runtime, and Content-Type on a request with a body), and a caller's X-TypeSafe-Retry-Count and framing or connection headers are dropped (see Security notes). A base URL's path appears in Debug and in error messages, so do not put a credential there.

Every request names the SDK in User-Agent and X-TypeSafe-SDK (typesafe-sdk-rust/<version>) and its platform in X-TypeSafe-Runtime (rust (<os>; <arch>)). Two settings, and nothing else, change that. user_agent_product("my-app/1.2.0") puts the application's product in front of the SDK's in User-Agent (my-app/1.2.0 typesafe-sdk-rust/<version>); X-TypeSafe-SDK still names the SDK alone. The product must be name/version, both parts RFC 9110 tokens, at most 64 bytes; anything else is a Config error from build(). send_runtime_header(false) leaves X-TypeSafe-Runtime out of every request.

Custom transport

ClientBuilder::build_with_service(service) sends every request through any tower_service::Service that takes an http::Request<typesafe_sdk::Body> and answers with an http::Response of any http_body::Body: a proxy, a recorder, a middleware stack, or, as below, an in-memory answer for a test. The service owns its connections and their timeouts; the SDK still applies its own per-attempt deadline and response size limit. The settings only the default transport has (add_root_certificate, http_version, connect_timeout) are a Config error with build_with_service, never silently ignored.

use std::{
    convert::Infallible,
    future::{Ready, ready},
    task::{Context, Poll},
};

use http::{Request, Response};
use typesafe_sdk::{Body, Client, Noul, Questions};

/// Answers every request with the same JSON body.
#[derive(Clone)]
struct Canned(&'static str);

impl tower_service::Service<Request<Body>> for Canned {
    type Response = Response<Body>;
    type Error = Infallible;
    type Future = Ready<Result<Response<Body>, Infallible>>;

    fn poll_ready(&mut self, _: &mut Context<'_>) -> Poll<Result<(), Infallible>> {
        Poll::Ready(Ok(()))
    }

    fn call(&mut self, _: Request<Body>) -> Self::Future {
        ready(Ok(Response::new(Body::from(bytes::Bytes::from_static(self.0.as_bytes())))))
    }
}

#[tokio::main]
async fn main() -> Result<(), typesafe_sdk::Error> {
    let answer = r#"{"model":"jev-latest","usage":{},"answers":{"spam":{"type":"noul","noul":0.98}}}"#;
    let client = Client::builder().api_key("test-key").build_with_service(Canned(answer))?;
    let questions = Questions::new().noul("spam", Noul::new().instructions("Spam?")).prepare()?;
    let response = client.system_one("Buy now!", &questions).send().await?;
    assert_eq!(response.answers().noul("spam").map(|answer| answer.noul()), Some(0.98));
    Ok(())
}

When a custom service fails, its error's text becomes the Connection error's message (escaped and cut at 200 characters): a service that prints a request header into its error puts that header's value into the message.

Connections and concurrency

A Client is cheap to clone (one reference count) and every clone shares one connection pool; a clone per task is the intended use. The default transport is hyper over rustls, with the operating system's trust store through rustls-platform-verifier (plus any add_root_certificate roots), TCP_NODELAY, idle connections kept 90 s and HTTP/2 keep-alive pings every 30 s.

• For an https base URL the default is HttpVersion::Http2Only: all requests of a client share one multiplexed HTTP/2 connection, including requests started together on a client that has no connection yet (64 concurrent cold calls open exactly 1 connection in the test suite). HttpVersion::Auto lets ALPN choose HTTP/1.1 or HTTP/2, for a proxy that speaks HTTP/1.1 only; a cold client under Auto may open one connection per request started at the same time. An http base URL uses Auto unless told otherwise.
• client.warm_up().await lists the models once and drops the answer. It checks the API key and leaves an open connection in the pool, so call it before a fan-out: the first requests then pay no TCP or TLS handshake, and a bad key fails once instead of once per request.
• examples/concurrency.rs shows a bounded fan-out over several states.

Logging

With the tracing feature (on by default) the SDK emits tracing events and never installs a subscriber: where the events go is the application's choice. Every event has the target typesafe_sdk, so one filter directive selects them all, for example with tracing-subscriber's EnvFilter:

RUST_LOG=typesafe_sdk=info

A header value is printed as *** when its name is authorization, proxy-authorization, x-api-key, api-key, cookie or set-cookie, when its name contains token or secret, or when the value is flagged sensitive (the SDK flags Authorization). A state may carry personal data, so bodies appear only at TRACE.

Security notes

• The API key is held as a secrecy::SecretString until it becomes the Authorization header value, which is flagged sensitive; no Debug or Display of this crate prints it, and no error message repeats it. Leading and trailing whitespace is stripped from it, as the Python SDK strips it; a key that is then empty, or holds whitespace, a control or a non-ASCII character, is refused when the client is built. When a transport fails with an error that prints the request's credentials, their values are replaced by *** first, as the Python SDK does since 0.7.1: the key and every secret or sensitive header value, in the forms the documentation of ClientBuilder::build_with_service lists. An http:// base URL sends the key unencrypted, so use one only for a local proxy or a test server.
• Server text is escaped and cut. Every message read from a response body (whichever member it came from, or the body itself when no member holds one) has its control characters and text-hiding format characters written as Rust escapes (\n, \u{1b}, \u{202e}) and is cut at 200 characters plus U+2026. The request id, the error type, and the question name and answer type of the skipped-answer WARN are shown escaped and cut at 128; a field path's names at 128 each and the whole path at 320; a connection error's chain (at most 8 links) at 200. The raw data stays reachable through body(), body_text(), body_json(), request_id() and error_type().
• An API error can echo your state. A 422 may repeat part of the request in its message, so the Display of an API error can carry up to 200 escaped characters of state. A caller whose state is sensitive should log error.kind() or the status rather than the whole error. Debug of an ApiError shows the headers as a count and the body as a length.
• {:?} of an Error and chain-walking reporters print the transport's full text. source() keeps the transport's error whole on purpose, and Debug prints it, as do reporters such as anyhow and eyre that walk the chain; that text can be long (an HTTP/2 GOAWAY carries up to 16 KiB of debug data) and is not text this SDK wrote. Log the Display form when that matters.
• Framing and connection headers belong to the transport. content-length, transfer-encoding, connection, keep-alive, proxy-connection, te, trailer and upgrade set as a client default or on a call are dropped without an error, on every protocol, as the SDK's own headers are: a caller's content-length that disagrees with the body would fail an HTTP/2 stream and leave an HTTP/1.1 call waiting until its deadline. host is sent as given, on every protocol; over HTTP/2 the request's :authority still comes from the base URL. Over HTTP/2 a host that differs from the base URL's authority is outside RFC 9113 (section 8.3.1), and a conforming server may refuse the request as malformed. A caller that needs another Host routes by the base URL instead, or speaks HTTP/1.1: HttpVersion::Auto does on an http base URL, and on an https one only when the server picks HTTP/1.1 through ALPN; the default transport cannot insist on HTTP/1.1 over TLS.
• Responses are bounded. A body is read under a 16 MiB cap, and a JSON document nested deeper than 16 levels is refused before it reaches the parser.

Performance notes

The numbers below are measured, not estimated; the method, the machines and every run are in docs/perf/ledger.md. Allocation counts are dhat block counts on the second identical call, 64-bit targets.

• Allocations. Encoding a request body is 1 allocation (2 when the body is kept for a retry). Decoding the three-answer fixture into Answers is 7 blocks (578 bytes), 6 into a derived struct. A whole call's own allocations, beyond what the transport allocates, are 12 blocks (11 with max_retries(0)).

• Encode scratch. The body is encoded into a per-thread scratch buffer whose capacity is kept between calls. A scratch that grew past 8 MiB is dropped after its call, so a state whose encoding needs more (a string of about 1.33 MiB or more) pays a first call's allocations on every call.

• Debug builds and the default transport. Tokio boxes a future larger than 2,048 bytes when it is spawned or blocked on in a debug build (16,384 in release). A System One call over the default transport is a 2,344-byte future, so a debug build that spawns calls pays one more allocation per call; a release build does not, and neither does a call over a custom transport (2,040 bytes).

• Release profile. A library cannot set the profile it is built with, so these belong in the application's Cargo.toml. They are the usual settings for a smaller, faster binary; this repository's numbers were taken with Cargo's default profiles, so their effect on this SDK is not measured here.

  [profile.release]
  lto = "fat"
  codegen-units = 1
  

• -C target-cpu on x86_64. The JSON codec, sonic-rs, selects its SIMD code at compile time. Without a target-cpu, an x86_64 build gets the SSE2 baseline only. An application that knows its hardware can build with, for example, RUSTFLAGS="-C target-cpu=x86-64-v3" (AVX2) or -C target-cpu=native; the binary then does not run on CPUs without those features. Every number in the ledger was taken without such a flag, which is what a default build gets.

Testing

cargo nextest run and cargo test, without --workspace or -p, run the default members: the SDK, crates/macros and crates/test-support. Their tests run against local servers and need neither a key nor the network.

crates/live-tests holds the tests against the live API. It is a workspace member, so clippy --workspace compiles it, but not a default member. Its tests make real, billed calls on the key's account when both TYPESAFE_LIVE_TESTS=1 and TYPESAFE_API_KEY are set and a command reaches them: cargo test --workspace, cargo nextest run --workspace, or anything naming -p typesafe-sdk-rust-live-tests. Without either variable they fail, never skip, before any request is made, so a key exported for other work does not make --workspace bill anyone; it makes those tests fail instead. Run them only on purpose:

TYPESAFE_LIVE_TESTS=1 TYPESAFE_API_KEY=... cargo nextest run -p typesafe-sdk-rust-live-tests

• Fuzzing. fuzz/ holds libFuzzer targets for the response decoders and the Retry-After parser, in a workspace of its own that needs the nightly toolchain and cargo-fuzz; see fuzz/README.md.
• Coverage. CI holds the crate at 85% line coverage; docs/uncovered-lines.md records the measured total and why each uncovered line is not reached.
• The port. docs/port-test-matrix.md maps every test of the Python SDK to the Rust tests that cover it, the deviation that explains why none does, or the reason it was left out; python3 .github/scripts/port-test-matrix.py checks that every Rust test and deviation it names exists (CI runs it; --upstream <checkout> also checks it against a checkout of the Python SDK).

Deviations from the Python SDK

What is not measured

A connect timeout mapping to ErrorKind::Timeout, the HTTP/2 keep-alive ping (30 s), the idle pool timeout (90 s) and TCP_NODELAY are configured but not observed by any test, and Android is compiled by no CI job. Neither are the derive's compile-error texts under any rustc other than the pinned 1.98.1, the real distribution of the retry jitter (only an injected draw is tested), instruction counts on arm64, nor future sizes on targets other than macOS and Linux.

License

This crate is licensed under the Apache License 2.0; see LICENSE. It is a port of typesafe-sdk-python, which is MIT-licensed; that license text is reproduced, as the upstream repository ships it, in LICENSE-THIRD-PARTY and applies to the material derived from it.