Crate timvw_rust_runtime

Expand description

§timvw-rust-runtime

The runtime of Rust clients generated by openapi-generator-timvw-rust (-g timvw-rust). A generated client sends every request through a Client: a tower service from http::Request<Body> to http::Response<Body>. That makes everything around sending a tower layer: retries, timeouts, logging, headers, authentication, rate limits. Use the layers this crate ships, any layer from the tower ecosystem, or write your own.

Quick start
Retries
Timeouts
Logging
Middleware: headers, authentication, layers of your own
Providers: the same API at another base URL
Concurrency and rate limits
Choosing the HTTP client, and tests without a network
Your own types, and streams of server-sent events
Uploads and downloads
Credentials
Headers and query parameters for one call
Errors
WebAssembly
Features

§Quick start

A generated client keeps its Client in Configuration.client, which starts as Client::default(): reqwest, with retries of 429, 408, 5xx and failed connections. layer wraps it in more:

use std::time::Duration;
use timvw_rust_runtime::timeout::TimeoutLayer;
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::Client;

let client = Client::default() // reqwest, with retries
    .layer(TraceLayer::new()) // logs each call, and each retry
    .layer(TimeoutLayer::new(Duration::from_secs(120))); // bounds the whole call, retries included

// In a generated client:
// let mut configuration = my_client::apis::configuration::Configuration::new();
// configuration.client = client;

Layers apply from the inside out: the last one added sees a request first. Layers added to Client::default() wrap its retries. To put a layer inside the retries (a timeout or a log line per attempt), or to retry differently, build the stack from the bare service, Client::from_service(ReqwestService::default()):

use std::time::Duration;
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::timeout::TimeoutLayer;
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::{Client, ReqwestService};

let client = Client::from_service(ReqwestService::default()) // reqwest, no retries
    .layer(TimeoutLayer::new(Duration::from_secs(60))) // each attempt
    .layer(TraceLayer::new()) // logs each attempt
    .layer(RetryLayer::new(RetryPolicy::default().retry_timeouts(true))); // retries, a timed-out attempt too

A Client is cheap to clone, and clones share their layers’ state (a rate limit, a token cache).

§Retries

Client::default(), Client::new() and Client::from_reqwest(..) retry with retry::RetryLayer::new(RetryPolicy::default()) (feature retry, on by default). retry::RetryPolicy is a tower::retry::Policy for HTTP APIs:

It retries 429 Too Many Requests, 408 Request Timeout, every 5xx and failures to connect. Any other response or error is returned at once.
It waits as long as the server asks, with retry-after-ms (milliseconds, as OpenAI sends) or Retry-After (seconds, or an HTTP date such as Wed, 21 Oct 2015 07:28:00 GMT), up to max_retry_after (60 s by default). A response that asks for longer is returned at once instead of retried: waiting less than asked would likely fail again, and an hour-long wait is rarely what a caller wants. The wait is read once into a deadline, counted from when the response arrived (a date in the past means now), and the policy sleeps until that deadline: reading a 429’s body for permanent_429 does not make it wait longer than asked.
Otherwise it backs off exponentially: 200 ms, 400 ms, 800 ms, …, capped at max_delay (8 s). Each delay is jittered, picked at random between half and all of it, so that clients that failed together do not retry together; .jitter(false) turns that off. A server’s Retry-After is not jittered, nor capped by max_delay.
It retries at most 3 times by default.
It retries a request only when its body can be sent again: JSON, form and text bodies, bytes, and a file Upload, which is opened again for each attempt (so is a multipart form of such parts). A body that can be read once (a stream, a reader) is not retried.

use std::time::Duration;
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::{Client, ReqwestService};

let policy = RetryPolicy::new(5)
    .base_delay(Duration::from_millis(500))
    .max_delay(Duration::from_secs(30)) // the backoff
    .max_retry_after(Duration::from_secs(120)); // the longest Retry-After to wait for
let client = Client::from_service(ReqwestService::default()).layer(RetryLayer::new(policy));

retry::RetryLayer::new(policy) retries just like tower::retry::RetryLayer::new(policy), which works too.

§`429`s retried only when the server says when

Some APIs send Retry-After with a 429 that passes by waiting, and leave it out when waiting will not help. With RetryPolicy::server_directed_429(true), a 429 without Retry-After or retry-after-ms (or with one that does not parse) is returned at once, and one with them is retried at the time the server gave. 408, 5xx and failed connections still back off without advice. The default retries every 429.

use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::{Client, ReqwestService};

let client = Client::from_service(ReqwestService::default())
    .layer(RetryLayer::new(RetryPolicy::default().server_directed_429(true)));

§Permanent `429`s: an exhausted quota

Some APIs answer 429 both for a rate limit, which passes, and for a quota or credit that is used up, which does not: OpenAI’s insufficient_quota. Only the body tells them apart. RetryLayer::permanent_429 reads the body of each 429 (up to body_limit, 64 KiB by default) and gives it to a function; when that returns true, the response is returned at once, marked with the retry::Permanent extension. The body is put back, so the caller, and a generated client’s error, still read all of it. retry::openai_insufficient_quota is that function for OpenAI (an error.code or error.type of insufficient_quota):

use timvw_rust_runtime::retry::{openai_insufficient_quota, RetryLayer, RetryPolicy};
use timvw_rust_runtime::{Client, ReqwestService};

let client = Client::from_service(ReqwestService::default()).layer(
    RetryLayer::new(RetryPolicy::default())
        .permanent_429(openai_insufficient_quota)
        .body_limit(16 * 1024),
);

// Any API: a function of the body's bytes.
let client = Client::from_service(ReqwestService::default()).layer(
    RetryLayer::default().permanent_429(|body: &[u8]| body.windows(14).any(|w| w == b"quota_exceeded")),
);

Client::default() does not read 429 bodies: what makes a 429 permanent differs between APIs, so the runtime leaves it to the client of each API to turn on. A body longer than the limit, or one that fails to read, is treated as a rate limit. A layer of your own inside the retries can mark any response not to retry: response.extensions_mut().insert(retry::Permanent).

Without retries: Client::from_service(ReqwestService::default()), or build without the retry feature.

The policy retries every method, POST included: for most APIs a 429 or 503 means the request was not processed. If a retried POST could act twice on your API, send an idempotency key the API understands, or write a tower::retry::Policy of your own: the trait has two methods, retry (whether to try again and after which delay) and clone_request. Body::try_clone clones a body held in memory.

§Timeouts

A client has no timeout until you add one. timeout::TimeoutLayer (feature time, on with retry) fails a request whose response (its status and headers) does not arrive in time; reading the body is not part of it. It waits with tokio’s timer natively and the host’s setTimeout on WebAssembly, where fetch has no timeout of its own and tower’s TimeoutLayer (tokio’s timer) does not work. A timeout is an Error whose is_timeout() is true:

use std::time::Duration;
use timvw_rust_runtime::{Client, Method};

async fn list_items(client: &Client) -> Result<(), timvw_rust_runtime::Error> {
    match client.request(Method::GET, "https://api.example.com/items").send().await {
        Err(error) if error.is_timeout() => eprintln!("too slow, try again later"),
        response => println!("{}", response?.status()),
    }
    Ok(())
}

let client = Client::default().layer(timvw_rust_runtime::timeout::TimeoutLayer::new(Duration::from_secs(30)));

Where it sits decides what it bounds (layers apply from the inside out, see Quick start):

Stack	The timeout bounds
`Client::default().layer(TimeoutLayer::new(t))`	the whole call: every attempt and the waits between them
`Client::from_service(service).layer(TimeoutLayer::new(t)).layer(RetryLayer::new(policy))`	each attempt; one that times out ends the call, unless `policy.retry_timeouts(true)` retries it
both, the per-attempt one first	each attempt, and the whole call

RetryPolicy::retry_timeouts(true) retries a request that timed out like one that failed to connect. It is off by default: a request that timed out may have reached the server, so turn it on for requests that are safe to repeat.

use std::time::Duration;
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::timeout::TimeoutLayer;
use timvw_rust_runtime::{Client, ReqwestService};

let client = Client::from_service(ReqwestService::default())
    .layer(TimeoutLayer::new(Duration::from_secs(30))) // each attempt
    .layer(RetryLayer::new(RetryPolicy::default().retry_timeouts(true)))
    .layer(TimeoutLayer::new(Duration::from_secs(120))); // the whole call

tower’s tower::timeout::TimeoutLayer works natively too, and its Elapsed error is a timeout for is_timeout() and retry_timeouts as well.

reqwest’s own timeouts (connect timeout, read timeout) are set on a reqwest::Client, see Choosing the HTTP client. For a response streamed for minutes (server-sent events), a total timeout ends the stream; use a connect timeout instead, or none, and bound the silence between events with Events::idle_timeout.

§Logging

trace::TraceLayer (feature tracing) logs every request with tracing:

a http.request span with http.request.method, url.full (without the query, which can hold an API key) and http.response.status_code;
an event when the response arrives, with status and elapsed_ms: DEBUG below 500, WARN for a 5xx or a failure to send;
with RetryPolicy, a DEBUG event retrying with attempt, max_retries, delay_ms and status, and a DEBUG event not retrying: ... for a Retry-After beyond max_retry_after or a permanent 429.

Headers and bodies are never logged. The target is timvw_rust_runtime::trace:

use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::Client;

// RUST_LOG=timvw_rust_runtime=debug shows every request, RUST_LOG=timvw_rust_runtime=warn only failures.
tracing_subscriber::fmt()
    .with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
    .init();
let client = Client::default().layer(TraceLayer::new());

 WARN http.request{http.request.method=POST url.full=https://api.example.com/v1/items http.response.status_code=503}: timvw_rust_runtime::trace: response status=503 elapsed_ms=41
DEBUG timvw_rust_runtime::trace: retrying attempt=1 max_retries=3 delay_ms=200 status=503
DEBUG http.request{http.request.method=POST url.full=https://api.example.com/v1/items http.response.status_code=200}: timvw_rust_runtime::trace: response status=200 elapsed_ms=380

reqwest’s and hyper’s own DEBUG events (connecting, pooling) are logged inside the http.request span too.

For more (headers, custom spans and fields, OpenTelemetry), use tower-http’s TraceLayer; it fits the same way:

let client = timvw_rust_runtime::Client::default().layer(tower_http::trace::TraceLayer::new_for_http());

§Middleware

Anything that changes a request or a response is a layer. Three ways to write one, from the shortest:

A header on every request: tower::util::MapRequestLayer with a function, or tower-http’s SetRequestHeaderLayer:

use timvw_rust_runtime::http::{self, HeaderName, HeaderValue};
use timvw_rust_runtime::{Body, Client};

let client = Client::default()
    .layer(tower::util::MapRequestLayer::new(|mut request: http::Request<Body>| {
        request.headers_mut().insert("x-team", HeaderValue::from_static("search"));
        request
    }))
    .layer(tower_http::set_header::SetRequestHeaderLayer::if_not_present(
        HeaderName::from_static("x-request-source"),
        HeaderValue::from_static("batch"),
    ));

Several layers at once: tower::ServiceBuilder composes layers top to bottom (the first one sees a request first), the reverse of chaining Client::layer:

use std::time::Duration;
use timvw_rust_runtime::retry::RetryLayer;
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::{Client, ReqwestService};

let client = Client::from_service(ReqwestService::default()).layer(
    tower::ServiceBuilder::new()
        .layer(RetryLayer::default())
        .layer(TraceLayer::new())
        .timeout(Duration::from_secs(60)),
);

A layer of your own, for work that needs state or await, such as fetching and caching an access token. A layer makes the service; the service changes the request and calls the service inside it:

use std::future::Future;
use std::pin::Pin;
use std::sync::Arc;
use std::task::{Context, Poll};
use timvw_rust_runtime::http::{self, header::AUTHORIZATION, HeaderValue};
use timvw_rust_runtime::{Body, Client};
use tower::{Layer, Service};

#[derive(Clone)]
struct AccessToken {
    cached: Arc<tokio::sync::Mutex<Option<String>>>,
}

impl<S> Layer<S> for AccessToken {
    type Service = WithAccessToken<S>;
    fn layer(&self, inner: S) -> Self::Service {
        WithAccessToken { inner, cached: self.cached.clone() }
    }
}

#[derive(Clone)]
struct WithAccessToken<S> {
    inner: S,
    cached: Arc<tokio::sync::Mutex<Option<String>>>,
}

impl<S> Service<http::Request<Body>> for WithAccessToken<S>
where
    S: Service<http::Request<Body>> + Clone + Send + 'static,
    S::Future: Send,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = Pin<Box<dyn Future<Output = Result<S::Response, S::Error>> + Send>>;

    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), S::Error>> {
        self.inner.poll_ready(cx)
    }

    fn call(&mut self, mut request: http::Request<Body>) -> Self::Future {
        // Call the service that was polled ready, and keep a fresh clone for the next request.
        let fresh = self.inner.clone();
        let mut inner = std::mem::replace(&mut self.inner, fresh);
        let cached = self.cached.clone();
        Box::pin(async move {
            let mut cached = cached.lock().await;
            if cached.is_none() {
                *cached = Some(fetch_token().await);
            }
            let value = HeaderValue::from_str(&format!("Bearer {}", cached.as_deref().unwrap())).unwrap();
            drop(cached);
            request.headers_mut().insert(AUTHORIZATION, value);
            inner.call(request).await
        })
    }
}

async fn fetch_token() -> String {
    "token".to_owned() // ask your identity provider
}

let client = Client::default().layer(AccessToken { cached: Arc::default() });

A layer’s service must be Clone + Send; it need not be Sync (on WebAssembly, not Send either: MaybeSend). Its response body can be any http_body::Body.

§Providers

Several services serve one API under different base URLs: OpenAI-compatible providers, a proxy, a local server, a cloud’s deployment of it. ProviderLayer (or provider::ProviderLayer) sends the requests of a generated client to one of them. To each request for the provider, it:

replaces the base URL (scheme, host, port and path prefix) with the provider’s, keeping the rest of the path and the query;
rewrites the rest of the path with map_path, if set (a deployment in the path, say);
sets the headers of header, sensitive_header and headers, each replacing the request’s values of that name (the generated authentication included); sensitive_header marks the value sensitive, for an API key;
sets the query parameters of query (an api-version, say), each replacing the request’s parameters of that name.

A request is the provider’s when its URL is under the provider’s base URL, or under the base declared with rewrite_from: the same scheme, host and port, and a path that starts with the base’s at a segment boundary (/v1 holds /v1/models, not /v10). Any other request (a download from another host) goes on unchanged, without the provider’s headers. A generated Configuration.base_path works with it in either of two ways:

base_path set to the provider: set configuration.base_path to the provider’s base URL and use ProviderLayer::new(provider_base) for headers, query parameters and paths.
base_path left as generated: declare it with .rewrite_from(generated_base), and the layer moves every request under it. This suits a Client shared by several Configurations, or a crate that hands out a client for a provider.

An OpenAI-compatible provider under another base URL, with its own key and headers:

use timvw_rust_runtime::http::header::AUTHORIZATION;
use timvw_rust_runtime::{Client, ExposeSecret, Method, ProviderLayer, SecretString};

let key = SecretString::from("sk-or-123");
let provider = ProviderLayer::new("https://openrouter.example/api/v1")?
    .rewrite_from("https://api.openai.com/v1")? // the generated client's base_path
    .sensitive_header(AUTHORIZATION, format!("Bearer {}", key.expose_secret()))?
    .header("x-title", "my app")?;
let client = Client::default().layer(provider.clone());
// configuration.client = client;

let uri = "https://api.openai.com/v1/chat/completions?stream=true".parse()?;
assert_eq!(
    provider.rewrite_uri(&uri)?.to_string(),
    "https://openrouter.example/api/v1/chat/completions?stream=true"
);

A provider that puts a deployment in the path, takes an api-version query parameter and an api-key header:

use timvw_rust_runtime::{Client, ProviderLayer};

let deployment = "gpt-4o";
let provider = ProviderLayer::new("https://my-resource.example.com/openai")?
    .rewrite_from("https://api.openai.com/v1")?
    .sensitive_header("api-key", "secret")?
    .query(&[("api-version", "2024-10-21")])?
    .map_path(move |path| format!("/deployments/{deployment}{path}"));
let client = Client::default().layer(provider.clone());

let uri = "https://api.openai.com/v1/chat/completions".parse()?;
assert_eq!(
    provider.rewrite_uri(&uri)?.to_string(),
    "https://my-resource.example.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21"
);

map_path gets the path after the base (/chat/completions), percent-encoded, and returns the one to put after the provider’s base. rewrite_uri shows where a URL goes, without sending. Added last, the layer sees each request first, so it rewrites once, before the retries. A call’s RequestOptions win over the layer: it does not replace a header or query parameter the options set (with an api-version in both, the call’s is sent, once), and it does not move a call the options sent to a base URL of their own; if that base is the provider’s, the layer still maps the path and adds its headers and query, any other base gets nothing from it. The layer knows no provider by name: presets for one API belong in a crate for that API.

§Concurrency and rate limits

tower::limit::ConcurrencyLimitLayer caps requests in flight; the clones of a Client share the cap. RateLimit cannot be cloned, so put a Buffer in front of it: the buffer shares one limiter between all clones. A buffer starts a tokio task, so build that client inside a tokio runtime.

use std::time::Duration;
use timvw_rust_runtime::Client;

#[tokio::main]
async fn main() {
    let client = Client::default()
        .layer(tower::limit::ConcurrencyLimitLayer::new(8))
        .layer(tower::ServiceBuilder::new().buffer(256).rate_limit(50, Duration::from_secs(1)));
}

These layers need tower’s limit and buffer features in your Cargo.toml, like timeout for TimeoutLayer: tower = { version = "0.5", features = ["limit", "buffer", "timeout"] }.

§Choosing the HTTP client

Client::from_reqwest sends with a reqwest::Client you built (a proxy, a custom TLS setup, connect and read timeouts, pool settings), with the default retries; Client::from_service(ReqwestService::new(reqwest_client)) without them:

use std::time::Duration;
use timvw_rust_runtime::Client;

let reqwest_client = reqwest::Client::builder()
    .connect_timeout(Duration::from_secs(5))
    .proxy(reqwest::Proxy::all("http://proxy.internal:3128").unwrap())
    .build()
    .unwrap();
let client = Client::from_reqwest(reqwest_client);

Client::from_service sends with any tower service from http::Request<Body> to http::Response<B>, as is, with no retries added: a hyper client, a client for another runtime, or a function that answers without a network, for tests:

use timvw_rust_runtime::{http, Body, BoxError, Client};

let client = Client::from_service(tower::service_fn(|request: http::Request<Body>| async move {
    assert_eq!(request.uri().path(), "/v1/items");
    let page = r#"{"data":[{"id":"1","name":"Lamp"}],"has_more":false}"#;
    Ok::<_, BoxError>(
        http::Response::builder()
            .header("content-type", "application/json")
            .body(Body::from(page))?,
    )
}));

Without the reqwest feature (and rustls or native-tls), the crate does not depend on reqwest, and Client::default() fails every request: set a service with Client::from_service.

§Your own types and server-sent events

A generated function decodes to the spec’s models. To send and decode types of your own (fewer fields, a provider whose replies differ, serde_json::Value), call the client directly: the same layers apply (retries, logging). Response::events() reads a text/event-stream body, and .json::<T>() decodes each event’s data::

use futures_util::StreamExt;
use serde::Deserialize;
use timvw_rust_runtime::{Client, ExposeSecret, Method, SecretString};

#[derive(Deserialize)]
struct Delta {
    id: String,
}

async fn run(client: &Client, api_key: &SecretString) -> Result<(), timvw_rust_runtime::Error> {
    let request = serde_json::json!({"model": "some-model", "stream": true});
    let response = client
        .request(Method::POST, "https://api.example.com/v1/chat/completions")
        .bearer_auth(api_key.expose_secret())
        .json(&request)
        .send()
        .await?;
    let mut deltas = response.events().json::<Delta>();
    while let Some(delta) = deltas.next().await {
        println!("{}", delta?.id);
    }
    Ok(())
}

data: [DONE] or the end of the body ends the stream. A payload that does not decode yields an error whose is_decode() is true, and the stream goes on. events() alone gives each sse::Event undecoded (event:, data:, id:), and sse::Decoder is the parser, for a body read some other way. A generated client’s typed event streams use the same parser.

A stream can stall without closing: the server or a proxy keeps the connection open and sends nothing. events().idle_timeout(duration) ends it with an error (is_timeout() is true) when no bytes arrive for duration; every chunk, a : keep-alive comment too, restarts the clock, and the whole stream has no limit (feature time, part of retry):

use std::time::Duration;
use futures_util::StreamExt;

async fn run(response: timvw_rust_runtime::Response) -> Result<(), timvw_rust_runtime::Error> {
    let mut events = response.events().idle_timeout(Duration::from_secs(120));
    while let Some(event) = events.next().await {
        match event {
            Err(error) if error.is_timeout() => eprintln!("the stream stalled"),
            event => println!("{}", event?.data),
        }
    }
    Ok(())
}

§Uploads and downloads

An Upload is something to send: a request body (.body(upload)) or a multipart part (multipart::Part::upload). A generated client takes one for every file parameter (and a multipart union’s file variant holds one), and a PathBuf, a &Path, a Vec<u8> or Bytes converts into one. Two uploads are equal (PartialEq) when they send the same thing: equal bytes, the same path, or the same stream (an upload and its clones).

Made with	Length	Sent again by a retry
`Upload::bytes(..)`, `Vec<u8>`, `Bytes`, `&'static [u8]`	known	yes, without a copy
`Upload::path(path)`, `Upload::file(path).await`, `PathBuf`, `&Path`	from the file’s metadata	yes: the file is opened again
`Upload::from_fn(\|\| stream)`	`.len(n)`	yes: a fresh stream per attempt
`Upload::reader(reader)`, `Upload::stream(stream)`	`.len(n)`	no, unless `.buffer_for_retry(limit)` holds it

A file is streamed from disk with tokio, in chunks, never read into memory. Its part is named after the file and typed from its extension (photo.png, image/png); .file_name(..) and .mime(..) set others, and a part of an upload without a name is named file.
When the length of the body is known, which for a multipart form means the length of every part, it is sent with a Content-Length, not chunked: some servers refuse chunked uploads. Upload::resolve().await reads a path’s length (Upload::file is Upload::path(..).resolve(), and fails at once on a missing file); generated clients resolve every upload.
.slice(range) sends part of a file (a seek, no copy), of bytes or of a stream: for APIs that take a large file in parts.
.buffer_for_retry(limit) lets resolve read a stream or a reader into memory when it is no longer than limit, so that a retry can send it again; a longer one is sent once.

use timvw_rust_runtime::multipart::{Form, Part};
use timvw_rust_runtime::{Client, Method, Upload};

async fn upload(client: &Client) -> Result<(), timvw_rust_runtime::Error> {
    let form = Form::new()
        .text("purpose", "fine-tune")
        .part("file", Part::upload(Upload::file("train.jsonl").await?));
    client
        .request(Method::POST, "https://api.example.com/v1/files")
        .multipart(form) // with a Content-Length; a retry opens train.jsonl again
        .send()
        .await?;

    // The second 8 MiB of a large file, as the whole body.
    let chunk = Upload::path("video.mp4").slice(8 << 20..16 << 20).resolve().await?;
    client.request(Method::PUT, "https://api.example.com/v1/uploads/1/parts/2").body(chunk).send().await?;
    Ok(())
}

A response body can go to a file, or be read as a tokio AsyncRead. Response::save_to(path) streams it into a temporary file in the directory of path, flushes it to disk and renames it to path: path holds the whole body or is left as it was, never half a download. It returns the number of bytes written; a failure to write is an Error whose is_io() is true:

use timvw_rust_runtime::{Client, Method};

async fn download(client: &Client) -> Result<(), Box<dyn std::error::Error>> {
    let response = client.request(Method::GET, "https://api.example.com/v1/files/1/content").send().await?;
    let written = response.save_to("data/file-1.jsonl").await?;

    let response = client.request(Method::GET, "https://api.example.com/v1/files/2/content").send().await?;
    let mut reader = response.into_async_read();
    let mut file = tokio::fs::File::create("data/file-2.jsonl").await?;
    tokio::io::copy(&mut reader, &mut file).await?; // an error reading the body is an io::Error here
    Ok(())
}

Files, readers and downloads are feature fs (on by default; a generated crate with a file parameter turns it on).

§Credentials

A generated Configuration keeps its tokens, basic-auth password and api key in secrecy’s SecretString, which this crate re-exports (timvw_rust_runtime::secrecy, SecretString and ExposeSecret). A SecretString is made from a String or a &str, its Debug is redacted, and it is zeroed when dropped. bearer_auth, basic_auth and sensitive_header take the exposed value and mark the header sensitive, so the Debug of the request’s headers leaves it out:

use timvw_rust_runtime::{Client, ExposeSecret, Method, SecretString};

let key = SecretString::from("sk-123");
assert_eq!(format!("{key:?}"), "SecretBox<str>([REDACTED])");

let request = Client::default()
    .request(Method::GET, "https://api.example.com/v1/models")
    .sensitive_header("x-api-key", key.expose_secret())
    .build()?;
assert!(request.headers()["x-api-key"].is_sensitive());
assert!(!format!("{:?}", request.headers()).contains("sk-123"));

§Headers and query parameters for one call

A layer changes every request. RequestOptions holds extra headers and query parameters, and another base URL, for one call; every operation of a generated client takes it (.options(&options) on its builder, or its last argument without builders), and RequestBuilder::options applies it to a request of your own:

use timvw_rust_runtime::{Client, Method, RequestOptions};

async fn run(client: &Client) -> Result<(), timvw_rust_runtime::Error> {
    let options = RequestOptions::new()
        .header("x-request-source", "batch")?
        .query(&[("trace", "1")])?;
    // generated: items_api::list_items().configuration(&configuration).options(&options).call().await
    let response = client
        .request(Method::GET, "https://api.example.com/v1/items?limit=10")
        .bearer_auth("secret")
        .options(&options)
        .send()
        .await?; // GET /v1/items?limit=10&trace=1, with x-request-source: batch
    Ok(())
}

A header of the options replaces every value the request has under its name, whatever set it: a generated header parameter, Authorization, Content-Type, Accept. Generated clients apply the options last, after everything else. RequestOptions::header sets one value (a second call for the same name replaces it); headers merges a HeaderMap, keeping several values of one name.
Query parameters are appended after the request’s own: a name the request already has is sent twice.
header and query check their input at once and return an Error whose is_builder() is true for an invalid header name or value, or a query that is not a flat list of pairs.
base_url sends the call to another base: the scheme, host, port and path prefix of the base the request was built on are replaced, and the operation’s path and the query are kept. Generated code declares its Configuration.base_path with RequestBuilder::base, so https://api.example.com/v1/items with base_url("https://eu.example.com/v2") goes to https://eu.example.com/v2/items. On a request of your own without base, the whole path is kept. There is no per-call path: the operation decides it.
The options do not change the body. One value can be cloned, or reused for many calls.
They win over a ProviderLayer.
A header value marked with HeaderValue::set_sensitive stays sensitive (hidden from Debug output).

§Errors

A request that fails to send is an Error:

Method	True when
`is_builder()`	the request could not be built (an invalid URL, header or body)
`is_request()`	the service failed; `is_timeout()` and `is_connect()` narrow it down
`is_body()`	reading the response body failed
`is_decode()`	the response could not be decoded (JSON)
`is_io()`	a file could not be read or written: a missing upload, a failed `save_to`

source() is the error of the service or layer that failed: a reqwest::Error, a timeout, or your own. A generated client wraps it as apis::Error::Http(error). A response with an error status is not an Error here; the generated client turns it into apis::Error::ResponseError.

§WebAssembly

The crate, and the clients generated with it, build for wasm32-unknown-unknown (a browser, a web worker, Node.js, Cloudflare Workers: any JavaScript host). Everything that differs is behind #[cfg(target_family = "wasm")], so the same Cargo.toml works for both:

# Natively and on WebAssembly. On WebAssembly `rustls`/`native-tls` add nothing (the host does TLS), and `fs` adds only
# `Upload::path` and `From<PathBuf>`, which fail when sent.
timvw-rust-runtime = { version = "0.4", default-features = false, features = ["reqwest", "retry"] }

default-features = true builds too. On WebAssembly:

Sending. reqwest sends with the host’s fetch. fetch takes a request body whole, so a streamed body (an Upload::stream, a multipart form with one) is read into memory first; responses still stream as they arrive. Proxies and TLS settings belong to the host, and fetch has no timeout: add timeout::TimeoutLayer, which waits with the host’s setTimeout (tower’s TimeoutLayer needs tokio’s timer and does not work there). A request that times out is dropped, and reqwest aborts its fetch. Any other layer works.
Retries wait as natively: exponential backoff with jitter, Retry-After and retry-after-ms (an HTTP date is read against the host’s clock, Date.now()). The timer is the host’s setTimeout (gloo-timers) and deadlines use performance.now() (web-time), as std::time::Instant::now() panics there. A failure to fetch is not retried: fetch does not tell a failure to connect from one after the request was sent (or a CORS refusal), so Error::is_connect() is always false.
Server-sent events work, Events::idle_timeout with the host’s timer; TraceLayer measures with performance.now().
Send and Sync. The host’s futures and streams cannot leave its thread, so on WebAssembly nothing here needs to be Send or Sync: the bounds of Client::from_service, Client::layer, Body::wrap, Body::wrap_stream, Upload::stream, Upload::from_fn, Events::from_bytes and the layers are MaybeSend and MaybeSync: Send and Sync natively, nothing on WebAssembly. Use them in your own bounds to write code that builds for both. HttpService is a LocalBoxCloneService there (tower’s BoxCloneSyncService natively), and Client, Body, Response and the futures of send are not Send: run them with wasm_bindgen_futures::spawn_local, not on another thread.
No files. There is no file system: Upload::file, Upload::reader, Response::save_to and Response::into_async_read are not there, and an Upload::path (or a PathBuf given to a generated client) fails with an I/O error of kind Unsupported when it is resolved or sent. Send Upload::bytes, Upload::stream or Upload::from_fn; read a response with bytes, bytes_stream or events.

tests/wasm.rs runs in Node.js with wasm-bindgen-test-runner (see the CI workflow).

§Features

Feature	Default	What it adds
`rustls`	yes	reqwest with rustls, as `Client::default()` (on WebAssembly: reqwest, the host does TLS)
`native-tls`	no	reqwest with the platform’s TLS (on WebAssembly: reqwest, the host does TLS)
`reqwest`	via `rustls` or `native-tls`	`ReqwestService`, `Client::from_reqwest`
`retry`	yes	`retry::RetryLayer` and `retry::RetryPolicy` (sleep with the `time` timer, read `Retry-After` dates with `httpdate`), and retries in `Client::default()`; turns on `time`
`time`	via `retry`	`timeout::TimeoutLayer`, `sse::Events::idle_timeout` and the sleeps of retries: tokio’s timer, the host’s `setTimeout` on WebAssembly
`fs`	yes	file and reader uploads (`Upload::path`, `Upload::file`, `Upload::reader`), `Response::save_to` and `Response::into_async_read` (tokio); on WebAssembly only `Upload::path` and `From<PathBuf>`, which fail when sent
`tracing`	no	`trace::TraceLayer`, and a log event per retry

A generated crate forwards native-tls, rustls, retry and tracing under the same names, turns on fs when it has a file parameter and time when it has typed event streams (their idle_timeout), and builds for WebAssembly as it is.

§What the generated code uses

The generated functions build requests with RequestBuilder (query, header, sensitive_header, bearer_auth, basic_auth, json, form, body, multipart, options, build) and read responses with Response (status, headers, text, bytes, bytes_stream): the methods of reqwest’s types of the same names. multipart::Form encodes multipart bodies, and Upload (resolve, multipart::Part::upload) carries file parameters.

§License

MIT OR Apache-2.0.

Re-exports§

pub use provider::ProviderLayer;
pub use http;
pub use secrecy;
pub use tower;

Modules§

header: HTTP header types
multipart: multipart/form-data bodies, with the API of reqwest’s multipart module that generated clients use.
provider: Send a generated client to another provider of the same API: ProviderLayer moves each request to the provider’s base URL and adds the headers, query parameters and path changes the provider needs.
retryretry: Retries for HTTP APIs: RetryPolicy, a tower retry policy, and RetryLayer, which adds to it a look at the body of a 429 to tell a rate limit (retried) from an exhausted quota (not retried).
sse: Server-sent events (text/event-stream): a parser, the events of a response, and their data: decoded as JSON.
timeouttime: A request timeout that works natively and on WebAssembly: TimeoutLayer.
tracetracing: Logs every request with tracing (feature tracing).

Structs§

Body: The body of a request or a response: bytes held in memory, a stream, or a stream that can be made again.
Client: Sends the requests of a generated client through a tower service.
Error: An error building a request, sending it, or reading the response body.
HeaderMap: A specialized multimap for header names and values.
LocalBoxCloneService: A boxed service that can be cloned, like tower’s BoxCloneService, but neither the service nor its future need be Send: HttpService on WebAssembly (target_family = "wasm"), where reqwest’s futures are not Send.
Method: The Request Method (VERB)
RequestBuilder: Builds a request, with the methods of reqwest’s RequestBuilder that generated clients use.
RequestOptions: Extra headers and query parameters, and another base URL, for one call, applied after everything the generated code sets.
ReqwestServicereqwest: A tower service that sends http::Request<Body> with a reqwest::Client.
Response: A response, with the methods of reqwest’s Response that generated clients use.
StatusCode: An HTTP status code (status-code in RFC 9110 et al.).
Upload: Something to upload: a file, bytes, a reader or a stream, with a file name and a media type.

Traits§

ExposeSecret: Expose a reference to an inner secret
MaybeSend: Send natively; any type on WebAssembly (target_family = "wasm"), where futures and streams of the JavaScript host are not Send. Use it in bounds that must hold on both, such as a service given to Client::from_service.
MaybeSync: Sync natively; any type on WebAssembly (target_family = "wasm").

Type Aliases§

BoxError: The error type services and layers return: any error that can cross threads.
HttpServiceNon-target_family=wasm: The service a Client sends requests through, with its type erased: tower’s BoxCloneSyncService.
SecretString: Secret string type.