Crate typesafe_jev 

Client for TypeSafe’s System One API and its Jev model.

Jev does not generate text. A request carries a JSON state and a map of typed questions about it; the reply carries one calibrated answer per question. Every question in a request is evaluated against the same state in parallel, so a request with hundreds of questions takes about as long as one with a single question.

use serde_json::{json, Map};
use typesafe_jev::{Client, Config};

let client = Client::from_env(Config::default())?; // reads TYPESAFE_API_KEY

let state = json!({
    "code": "1| import os\n2| def cwd():\n3|     return os.getcwd()",
    "query": "where is the working directory read",
});
let mut questions = Map::new();
questions.insert("line3".into(), json!({
    "type": "noul",
    "instructions": "Does line 3 of the code directly answer the query?",
}));
questions.insert("relevance".into(), json!({
    "type": "score",
    "instructions": "How relevant is the code to the query?",
    "criteria": ["unrelated", "tangential", "relevant", "exactly what was asked"],
}));

let answers = client.ask(&state, &questions)?;
let p = answers["line3"]["noul"].as_f64().unwrap_or(0.0); // probability in [0, 1]
let score = answers["relevance"]["score"].as_f64().unwrap_or(0.0); // position among the criteria
println!("line 3 answers the query with p={p:.2}; relevance {score:.1}");
println!("{} input tokens, ~${:.4}", client.usage().input_tokens(), client.usage().cost_usd());

Questions and answers

Two question types are supported:

• noul asks a yes/no question. The answer is {"type": "noul", "noul": p}, with p the probability in [0, 1] that the answer is yes.
• score asks for a judgment on an ordered scale. The question lists its criteria, one per level, in order; the answer is {"type": "score", "score": s, "confidence": c, "probabilities": {...}, "legend": {...}}, with s a position on that scale counted from the first criterion.

The probabilities are calibrated: a noul of 0.9 is right about nine times in ten. The questions map is passed through as given, so any field the API accepts can be sent.

Retries, concurrency and cost

Client::ask retries connection failures and transient HTTP statuses with exponential backoff and jitter, honouring Retry-After. Rejected credentials come back at once as Error::Auth; a request too large for the model’s context as Error::TokenLimit, which the caller should split and retry. A key or a Config that no request could be sent with is reported by Client::new, as Error::Auth or Error::InvalidConfig, not retried.

Calls are blocking. The client is Send + Sync and shares one connection pool, so call it from as many threads as Config::pool_size. An AdaptiveLimiter caps the concurrency and halves it whenever the API throttles, growing it back as requests succeed.

Usage counts requests, retries and tokens across threads; Usage::cost_usd prices the input tokens at Jev’s list price, USD_PER_INPUT_MTOK.

Testing without the network

Client::with_transport takes any Transport, including a closure from the request body to a Reply:

use serde_json::{json, Map};
use typesafe_jev::{Client, Config, Reply};

let client = Client::with_transport(
    |_body: &[u8]| Ok(Reply { status: 200, retry_after: None, body: r#"{"answers": {"q": {"type": "noul", "noul": 0.5}}}"#.into() }),
    Config::default(),
);
let mut questions = Map::new();
questions.insert("q".into(), json!({"type": "noul", "instructions": "Is the state empty?"}));
let answers = client.ask(&json!({}), &questions).unwrap();
assert_eq!(answers["q"]["noul"], 0.5);

Structs

AdaptiveLimiter

Shared concurrency gate with additive-increase, multiplicative-decrease backoff.

Client

A connection to the System One API. One shared pool; call ask from as many threads as Config::pool_size allows.

Config

How a Client reaches the API and how hard it tries.

Reply

What came back from one HTTP POST, before the client interprets it.

Usage

Thread-safe request and token counters.

Enums

Error

Why a client could not be built, or a request could not be answered.

Constants

API_KEY_VAR

The environment variable Client::from_env reads.

DEFAULT_BASE_URL

TypeSafe’s hosted System One endpoint.

DEFAULT_MODEL

The model name that follows TypeSafe’s latest Jev release.

USD_PER_INPUT_MTOK

Jev’s list price per million input tokens, in dollars. Output tokens are free.

Traits

Transport

The wire: one POST of a JSON request body to the API endpoint.