# timvw-rust-runtime
The runtime of Rust clients generated by
[openapi-generator-timvw-rust](https://github.com/timvw/openapi-generator-timvw-rust) (`-g timvw-rust`, with the
default `httpStack=tower`). A generated client sends every request through a `Client`: a
[tower](https://docs.rs/tower) service from `http::Request<Body>` to `http::Response<Body>`. That makes everything
around sending a tower layer: retries, timeouts, logging, headers, authentication, rate limits. Use the layers this
crate ships, any layer from the tower ecosystem, or write your own.
- [Quick start](#quick-start)
- [Retries](#retries)
- [Timeouts](#timeouts)
- [Logging](#logging)
- [Middleware: headers, authentication, layers of your own](#middleware)
- [Providers: the same API at another base URL](#providers)
- [Concurrency and rate limits](#concurrency-and-rate-limits)
- [Choosing the HTTP client, and tests without a network](#choosing-the-http-client)
- [Your own types, and streams of server-sent events](#your-own-types-and-server-sent-events)
- [Uploads and downloads](#uploads-and-downloads)
- [Credentials](#credentials)
- [Headers and query parameters for one call](#headers-and-query-parameters-for-one-call)
- [Errors](#errors)
- [WebAssembly](#webassembly)
- [Features](#features)
## Quick start
A generated client keeps its `Client` in `Configuration.client`, which starts as `Client::default()`: reqwest, with
[retries](#retries) of `429`, `408`, `5xx` and failed connections. `layer` wraps it in more:
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::timeout::TimeoutLayer;
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::Client;
let client = Client::default() // reqwest, with retries
.layer(TraceLayer::new()) // logs each call, and each retry
.layer(TimeoutLayer::new(Duration::from_secs(120))); // bounds the whole call, retries included
// In a generated client:
// let mut configuration = my_client::apis::configuration::Configuration::new();
// configuration.client = client;
```
**Layers apply from the inside out: the last one added sees a request first.** Layers added to `Client::default()`
wrap its retries. To put a layer inside the retries (a timeout or a log line per attempt), or to retry differently,
build the stack from the bare service, `Client::from_service(ReqwestService::default())`:
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::timeout::TimeoutLayer;
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::{Client, ReqwestService};
let client = Client::from_service(ReqwestService::default()) // reqwest, no retries
.layer(TimeoutLayer::new(Duration::from_secs(60))) // each attempt
.layer(TraceLayer::new()) // logs each attempt
.layer(RetryLayer::new(RetryPolicy::default().retry_timeouts(true))); // retries, a timed-out attempt too
```
A `Client` is cheap to clone, and clones share their layers' state (a rate limit, a token cache).
## Retries
`Client::default()`, `Client::new()` and `Client::from_reqwest(..)` retry with
`retry::RetryLayer::new(RetryPolicy::default())` (feature `retry`, on by default). `retry::RetryPolicy` is a
`tower::retry::Policy` for HTTP APIs:
- It retries `429 Too Many Requests`, `408 Request Timeout`, every `5xx` and failures to connect. Any other response
or error is returned at once.
- It waits as long as the server asks, with `retry-after-ms` (milliseconds, as OpenAI sends) or `Retry-After`
(seconds, or an HTTP date such as `Wed, 21 Oct 2015 07:28:00 GMT`), up to `max_retry_after` (60 s by default). A
response that asks for longer is returned at once instead of retried: waiting less than asked would likely fail
again, and an hour-long wait is rarely what a caller wants. The wait is read once into a deadline, counted from when
the response arrived (a date in the past means now), and the policy sleeps until that deadline: reading a `429`'s
body for `permanent_429` does not make it wait longer than asked.
- Otherwise it backs off exponentially: 200 ms, 400 ms, 800 ms, ..., capped at `max_delay` (8 s). Each delay is
jittered, picked at random between half and all of it, so that clients that failed together do not retry together;
`.jitter(false)` turns that off. A server's `Retry-After` is not jittered, nor capped by `max_delay`.
- It retries at most 3 times by default.
- It retries a request only when its body can be sent again: JSON, form and text bodies, bytes, and a file
[`Upload`](#uploads-and-downloads), which is opened again for each attempt (so is a multipart form of such parts).
A body that can be read once (a stream, a reader) is not retried.
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::{Client, ReqwestService};
let policy = RetryPolicy::new(5)
.base_delay(Duration::from_millis(500))
.max_delay(Duration::from_secs(30)) // the backoff
.max_retry_after(Duration::from_secs(120)); // the longest Retry-After to wait for
let client = Client::from_service(ReqwestService::default()).layer(RetryLayer::new(policy));
```
`retry::RetryLayer::new(policy)` retries just like `tower::retry::RetryLayer::new(policy)`, which works too.
### `429`s retried only when the server says when
Some APIs send `Retry-After` with a `429` that passes by waiting, and leave it out when waiting will not help. With
`RetryPolicy::server_directed_429(true)`, a `429` without `Retry-After` or `retry-after-ms` (or with one that does not
parse) is returned at once, and one with them is retried at the time the server gave. `408`, `5xx` and failed
connections still back off without advice. The default retries every `429`.
```rust,no_run
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::{Client, ReqwestService};
let client = Client::from_service(ReqwestService::default())
.layer(RetryLayer::new(RetryPolicy::default().server_directed_429(true)));
```
### Permanent `429`s: an exhausted quota
Some APIs answer `429` both for a rate limit, which passes, and for a quota or credit that is used up, which does not:
OpenAI's `insufficient_quota`. Only the body tells them apart. `RetryLayer::permanent_429` reads the body of each
`429` (up to `body_limit`, 64 KiB by default) and gives it to a function; when that returns `true`, the response is
returned at once, marked with the `retry::Permanent` extension. The body is put back, so the caller, and a generated
client's error, still read all of it. `retry::openai_insufficient_quota` is that function for OpenAI (an `error.code`
or `error.type` of `insufficient_quota`):
```rust,no_run
use timvw_rust_runtime::retry::{openai_insufficient_quota, RetryLayer, RetryPolicy};
use timvw_rust_runtime::{Client, ReqwestService};
let client = Client::from_service(ReqwestService::default()).layer(
RetryLayer::new(RetryPolicy::default())
.permanent_429(openai_insufficient_quota)
.body_limit(16 * 1024),
);
// Any API: a function of the body's bytes.
let client = Client::from_service(ReqwestService::default()).layer(
RetryLayer::default().permanent_429(|body: &[u8]| body.windows(14).any(|w| w == b"quota_exceeded")),
);
```
`Client::default()` does not read `429` bodies: what makes a `429` permanent differs between APIs, so the runtime
leaves it to the client of each API to turn on. A body longer than the limit, or one that fails to read, is treated as
a rate limit. A layer of your own inside the retries can mark any response not to retry:
`response.extensions_mut().insert(retry::Permanent)`.
Without retries: `Client::from_service(ReqwestService::default())`, or build without the `retry` feature.
The policy retries every method, `POST` included: for most APIs a `429` or `503` means the request was not processed.
If a retried `POST` could act twice on your API, send an idempotency key the API understands, or write a
`tower::retry::Policy` of your own: the trait has two methods, `retry` (whether to try again and after which delay)
and `clone_request`. `Body::try_clone` clones a body held in memory.
## Timeouts
A client has no timeout until you add one. `timeout::TimeoutLayer` (feature `time`, on with `retry`) fails a request
whose response (its status and headers) does not arrive in time; reading the body is not part of it. It waits with
tokio's timer natively and the host's `setTimeout` on [WebAssembly](#webassembly), where `fetch` has no timeout of its
own and tower's `TimeoutLayer` (tokio's timer) does not work. A timeout is an `Error` whose `is_timeout()` is true:
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::{Client, Method};
async fn list_items(client: &Client) -> Result<(), timvw_rust_runtime::Error> {
match client.request(Method::GET, "https://api.example.com/items").send().await {
Err(error) if error.is_timeout() => eprintln!("too slow, try again later"),
response => println!("{}", response?.status()),
}
Ok(())
}
let client = Client::default().layer(timvw_rust_runtime::timeout::TimeoutLayer::new(Duration::from_secs(30)));
```
Where it sits decides what it bounds (layers apply from the inside out, see [Quick start](#quick-start)):
| `Client::default().layer(TimeoutLayer::new(t))` | the whole call: every attempt and the waits between them |
| `Client::from_service(service).layer(TimeoutLayer::new(t)).layer(RetryLayer::new(policy))` | each attempt; one that times out ends the call, unless `policy.retry_timeouts(true)` retries it |
| both, the per-attempt one first | each attempt, and the whole call |
`RetryPolicy::retry_timeouts(true)` retries a request that timed out like one that failed to connect. It is off by
default: a request that timed out may have reached the server, so turn it on for requests that are safe to repeat.
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::retry::{RetryLayer, RetryPolicy};
use timvw_rust_runtime::timeout::TimeoutLayer;
use timvw_rust_runtime::{Client, ReqwestService};
let client = Client::from_service(ReqwestService::default())
.layer(TimeoutLayer::new(Duration::from_secs(30))) // each attempt
.layer(RetryLayer::new(RetryPolicy::default().retry_timeouts(true)))
.layer(TimeoutLayer::new(Duration::from_secs(120))); // the whole call
```
tower's `tower::timeout::TimeoutLayer` works natively too, and its `Elapsed` error is a timeout for `is_timeout()` and
`retry_timeouts` as well.
reqwest's own timeouts (connect timeout, read timeout) are set on a `reqwest::Client`, see
[Choosing the HTTP client](#choosing-the-http-client). For a response streamed for minutes (server-sent events), a
total timeout ends the stream; use a connect timeout instead, or none, and bound the silence between events with
[`Events::idle_timeout`](#your-own-types-and-server-sent-events).
## Logging
`trace::TraceLayer` (feature `tracing`) logs every request with [`tracing`](https://docs.rs/tracing):
- a `http.request` span with `http.request.method`, `url.full` (without the query, which can hold an API key) and
`http.response.status_code`;
- an event when the response arrives, with `status` and `elapsed_ms`: `DEBUG` below `500`, `WARN` for a `5xx` or a
failure to send;
- with `RetryPolicy`, a `DEBUG` event `retrying` with `attempt`, `max_retries`, `delay_ms` and `status`, and a
`DEBUG` event `not retrying: ...` for a `Retry-After` beyond `max_retry_after` or a permanent `429`.
Headers and bodies are never logged. The target is `timvw_rust_runtime::trace`:
```rust,no_run
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::Client;
// RUST_LOG=timvw_rust_runtime=debug shows every request, RUST_LOG=timvw_rust_runtime=warn only failures.
tracing_subscriber::fmt()
.with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
.init();
let client = Client::default().layer(TraceLayer::new());
```
```text
WARN http.request{http.request.method=POST url.full=https://api.example.com/v1/items http.response.status_code=503}: timvw_rust_runtime::trace: response status=503 elapsed_ms=41
DEBUG timvw_rust_runtime::trace: retrying attempt=1 max_retries=3 delay_ms=200 status=503
DEBUG http.request{http.request.method=POST url.full=https://api.example.com/v1/items http.response.status_code=200}: timvw_rust_runtime::trace: response status=200 elapsed_ms=380
```
reqwest's and hyper's own `DEBUG` events (connecting, pooling) are logged inside the `http.request` span too.
For more (headers, custom spans and fields, OpenTelemetry), use [`tower-http`](https://docs.rs/tower-http)'s
`TraceLayer`; it fits the same way:
```rust,no_run
let client = timvw_rust_runtime::Client::default().layer(tower_http::trace::TraceLayer::new_for_http());
```
## Middleware
Anything that changes a request or a response is a layer. Three ways to write one, from the shortest:
**A header on every request**: `tower::util::MapRequestLayer` with a function, or tower-http's
`SetRequestHeaderLayer`:
```rust,no_run
use timvw_rust_runtime::http::{self, HeaderName, HeaderValue};
use timvw_rust_runtime::{Body, Client};
let client = Client::default()
.layer(tower::util::MapRequestLayer::new(|mut request: http::Request<Body>| {
request.headers_mut().insert("x-team", HeaderValue::from_static("search"));
request
}))
.layer(tower_http::set_header::SetRequestHeaderLayer::if_not_present(
HeaderName::from_static("x-request-source"),
HeaderValue::from_static("batch"),
));
```
**Several layers at once**: `tower::ServiceBuilder` composes layers top to bottom (the first one sees a request
first), the reverse of chaining `Client::layer`:
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::retry::RetryLayer;
use timvw_rust_runtime::trace::TraceLayer;
use timvw_rust_runtime::{Client, ReqwestService};
let client = Client::from_service(ReqwestService::default()).layer(
tower::ServiceBuilder::new()
.layer(RetryLayer::default())
.layer(TraceLayer::new())
.timeout(Duration::from_secs(60)),
);
```
**A layer of your own**, for work that needs state or `await`, such as fetching and caching an access token. A layer
makes the service; the service changes the request and calls the service inside it:
```rust,no_run
use std::future::Future;
use std::pin::Pin;
use std::sync::Arc;
use std::task::{Context, Poll};
use timvw_rust_runtime::http::{self, header::AUTHORIZATION, HeaderValue};
use timvw_rust_runtime::{Body, Client};
use tower::{Layer, Service};
#[derive(Clone)]
struct AccessToken {
cached: Arc<tokio::sync::Mutex<Option<String>>>,
}
impl<S> Layer<S> for AccessToken {
type Service = WithAccessToken<S>;
fn layer(&self, inner: S) -> Self::Service {
WithAccessToken { inner, cached: self.cached.clone() }
}
}
#[derive(Clone)]
struct WithAccessToken<S> {
inner: S,
cached: Arc<tokio::sync::Mutex<Option<String>>>,
}
impl<S> Service<http::Request<Body>> for WithAccessToken<S>
where
S: Service<http::Request<Body>> + Clone + Send + 'static,
S::Future: Send,
{
type Response = S::Response;
type Error = S::Error;
type Future = Pin<Box<dyn Future<Output = Result<S::Response, S::Error>> + Send>>;
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), S::Error>> {
self.inner.poll_ready(cx)
}
fn call(&mut self, mut request: http::Request<Body>) -> Self::Future {
// Call the service that was polled ready, and keep a fresh clone for the next request.
let fresh = self.inner.clone();
let mut inner = std::mem::replace(&mut self.inner, fresh);
let cached = self.cached.clone();
Box::pin(async move {
let mut cached = cached.lock().await;
if cached.is_none() {
*cached = Some(fetch_token().await);
}
let value = HeaderValue::from_str(&format!("Bearer {}", cached.as_deref().unwrap())).unwrap();
drop(cached);
request.headers_mut().insert(AUTHORIZATION, value);
inner.call(request).await
})
}
}
async fn fetch_token() -> String {
"token".to_owned() // ask your identity provider
}
let client = Client::default().layer(AccessToken { cached: Arc::default() });
```
A layer's service must be `Clone + Send`; it need not be `Sync` (on [WebAssembly](#webassembly), not `Send` either:
`MaybeSend`). Its response body can be any `http_body::Body`.
## Providers
Several services serve one API under different base URLs: OpenAI-compatible providers, a proxy, a local server, a
cloud's deployment of it. `ProviderLayer` (or `provider::ProviderLayer`) sends the requests of a generated client to
one of them. To each request for the provider, it:
1. replaces the base URL (scheme, host, port and path prefix) with the provider's, keeping the rest of the path and the
query;
2. rewrites the rest of the path with `map_path`, if set (a deployment in the path, say);
3. sets the headers of `header`, `sensitive_header` and `headers`, each replacing the request's values of that name
(the generated authentication included); `sensitive_header` marks the value sensitive, for an API key;
4. sets the query parameters of `query` (an `api-version`, say), each replacing the request's parameters of that name.
A request is the provider's when its URL is under the provider's base URL, or under the base declared with
`rewrite_from`: the same scheme, host and port, and a path that starts with the base's at a segment boundary (`/v1`
holds `/v1/models`, not `/v10`). Any other request (a download from another host) goes on unchanged, without the
provider's headers. A generated `Configuration.base_path` works with it in either of two ways:
- **`base_path` set to the provider**: set `configuration.base_path` to the provider's base URL and use
`ProviderLayer::new(provider_base)` for headers, query parameters and paths.
- **`base_path` left as generated**: declare it with `.rewrite_from(generated_base)`, and the layer moves every request
under it. This suits a `Client` shared by several `Configuration`s, or a crate that hands out a client for a
provider.
An OpenAI-compatible provider under another base URL, with its own key and headers:
```rust
use timvw_rust_runtime::http::header::AUTHORIZATION;
use timvw_rust_runtime::{Client, ExposeSecret, Method, ProviderLayer, SecretString};
let key = SecretString::from("sk-or-123");
let provider = ProviderLayer::new("https://openrouter.example/api/v1")?
.rewrite_from("https://api.openai.com/v1")? // the generated client's base_path
.sensitive_header(AUTHORIZATION, format!("Bearer {}", key.expose_secret()))?
.header("x-title", "my app")?;
let client = Client::default().layer(provider.clone());
// configuration.client = client;
let uri = "https://api.openai.com/v1/chat/completions?stream=true".parse()?;
assert_eq!(
provider.rewrite_uri(&uri)?.to_string(),
"https://openrouter.example/api/v1/chat/completions?stream=true"
);
# Ok::<(), Box<dyn std::error::Error>>(())
```
A provider that puts a deployment in the path, takes an `api-version` query parameter and an `api-key` header:
```rust
use timvw_rust_runtime::{Client, ProviderLayer};
let deployment = "gpt-4o";
let provider = ProviderLayer::new("https://my-resource.example.com/openai")?
.rewrite_from("https://api.openai.com/v1")?
.sensitive_header("api-key", "secret")?
.query(&[("api-version", "2024-10-21")])?
.map_path(move |path| format!("/deployments/{deployment}{path}"));
let client = Client::default().layer(provider.clone());
let uri = "https://api.openai.com/v1/chat/completions".parse()?;
assert_eq!(
provider.rewrite_uri(&uri)?.to_string(),
"https://my-resource.example.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21"
);
# Ok::<(), Box<dyn std::error::Error>>(())
```
`map_path` gets the path after the base (`/chat/completions`), percent-encoded, and returns the one to put after the
provider's base. `rewrite_uri` shows where a URL goes, without sending. Added last, the layer sees each request first,
so it rewrites once, before the retries. The provider's headers and query parameters win over a call's
[`RequestOptions`](#headers-and-query-parameters-for-one-call) of the same name; for another value per call, use
another client. The layer knows no provider by name: presets for one API belong in a crate for that API.
## Concurrency and rate limits
`tower::limit::ConcurrencyLimitLayer` caps requests in flight; the clones of a `Client` share the cap.
`RateLimit` cannot be cloned, so put a `Buffer` in front of it: the buffer shares one limiter between all clones. A
buffer starts a tokio task, so build that client inside a tokio runtime.
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::Client;
#[tokio::main]
async fn main() {
let client = Client::default()
.layer(tower::limit::ConcurrencyLimitLayer::new(8))
.layer(tower::ServiceBuilder::new().buffer(256).rate_limit(50, Duration::from_secs(1)));
}
```
These layers need tower's `limit` and `buffer` features in your `Cargo.toml`, like `timeout` for `TimeoutLayer`:
`tower = { version = "0.5", features = ["limit", "buffer", "timeout"] }`.
## Choosing the HTTP client
`Client::from_reqwest` sends with a `reqwest::Client` you built (a proxy, a custom TLS setup, connect and read timeouts,
pool settings), with the default retries; `Client::from_service(ReqwestService::new(reqwest_client))` without them:
```rust,no_run
use std::time::Duration;
use timvw_rust_runtime::Client;
let reqwest_client = reqwest::Client::builder()
.connect_timeout(Duration::from_secs(5))
.proxy(reqwest::Proxy::all("http://proxy.internal:3128").unwrap())
.build()
.unwrap();
let client = Client::from_reqwest(reqwest_client);
```
`Client::from_service` sends with any tower service from `http::Request<Body>` to `http::Response<B>`, as is, with no
retries added: a hyper client, a client for another runtime, or a function that answers without a network, for tests:
```rust,no_run
use timvw_rust_runtime::{http, Body, BoxError, Client};
let page = r#"{"data":[{"id":"1","name":"Lamp"}],"has_more":false}"#;
Ok::<_, BoxError>(
http::Response::builder()
.header("content-type", "application/json")
.body(Body::from(page))?,
)
}));
```
Without the `reqwest` feature (and `rustls` or `native-tls`), the crate does not depend on reqwest, and
`Client::default()` fails every request: set a service with `Client::from_service`.
## Your own types and server-sent events
A generated function decodes to the spec's models. To send and decode types of your own (fewer fields, a provider
whose replies differ, `serde_json::Value`), call the client directly: the same layers apply (retries, logging).
`Response::events()` reads a `text/event-stream` body, and `.json::<T>()` decodes each event's `data:`:
```rust,no_run
use futures_util::StreamExt;
use serde::Deserialize;
use timvw_rust_runtime::{Client, ExposeSecret, Method, SecretString};
#[derive(Deserialize)]
struct Delta {
id: String,
}
async fn run(client: &Client, api_key: &SecretString) -> Result<(), timvw_rust_runtime::Error> {
let request = serde_json::json!({"model": "some-model", "stream": true});
let response = client
.request(Method::POST, "https://api.example.com/v1/chat/completions")
.bearer_auth(api_key.expose_secret())
.json(&request)
.send()
.await?;
let mut deltas = response.events().json::<Delta>();
while let Some(delta) = deltas.next().await {
println!("{}", delta?.id);
}
Ok(())
}
```
`data: [DONE]` or the end of the body ends the stream. A payload that does not decode yields an error whose
`is_decode()` is true, and the stream goes on. `events()` alone gives each `sse::Event` undecoded (`event:`, `data:`,
`id:`), and `sse::Decoder` is the parser, for a body read some other way. A generated client's typed event streams use
the same parser.
A stream can stall without closing: the server or a proxy keeps the connection open and sends nothing.
`events().idle_timeout(duration)` ends it with an error (`is_timeout()` is true) when no bytes arrive for `duration`;
every chunk, a `: keep-alive` comment too, restarts the clock, and the whole stream has no limit (feature `time`, part
of `retry`):
```rust,no_run
use std::time::Duration;
use futures_util::StreamExt;
async fn run(response: timvw_rust_runtime::Response) -> Result<(), timvw_rust_runtime::Error> {
let mut events = response.events().idle_timeout(Duration::from_secs(120));
while let Some(event) = events.next().await {
match event {
Err(error) if error.is_timeout() => eprintln!("the stream stalled"),
event => println!("{}", event?.data),
}
}
Ok(())
}
```
## Uploads and downloads
An `Upload` is something to send: a request body (`.body(upload)`) or a multipart part (`multipart::Part::upload`).
A generated client takes one for every file parameter (and a multipart union's file variant holds one), and a
`PathBuf`, a `&Path`, a `Vec<u8>` or `Bytes` converts into one. Two uploads are equal (`PartialEq`) when they send the
same thing: equal bytes, the same path, or the same stream (an upload and its clones).
| `Upload::bytes(..)`, `Vec<u8>`, `Bytes`, `&'static [u8]` | known | yes, without a copy |
| `Upload::path(path)`, `Upload::file(path).await`, `PathBuf`, `&Path` | from the file's metadata | yes: the file is opened again |
| `Upload::from_fn(\|\| stream)` | `.len(n)` | yes: a fresh stream per attempt |
| `Upload::reader(reader)`, `Upload::stream(stream)` | `.len(n)` | no, unless `.buffer_for_retry(limit)` holds it |
- A file is streamed from disk with tokio, in chunks, never read into memory. Its part is named after the file and
typed from its extension (`photo.png`, `image/png`); `.file_name(..)` and `.mime(..)` set others, and a part of an
upload without a name is named `file`.
- When the length of the body is known, which for a multipart form means the length of every part, it is sent with a
`Content-Length`, not chunked: some servers refuse chunked uploads. `Upload::resolve().await` reads a path's length
(`Upload::file` is `Upload::path(..).resolve()`, and fails at once on a missing file); generated clients resolve
every upload.
- `.slice(range)` sends part of a file (a seek, no copy), of bytes or of a stream: for APIs that take a large file in
parts.
- `.buffer_for_retry(limit)` lets `resolve` read a stream or a reader into memory when it is no longer than `limit`,
so that a retry can send it again; a longer one is sent once.
```rust,no_run
use timvw_rust_runtime::multipart::{Form, Part};
use timvw_rust_runtime::{Client, Method, Upload};
async fn upload(client: &Client) -> Result<(), timvw_rust_runtime::Error> {
let form = Form::new()
.text("purpose", "fine-tune")
.part("file", Part::upload(Upload::file("train.jsonl").await?));
client
.request(Method::POST, "https://api.example.com/v1/files")
.multipart(form) // with a Content-Length; a retry opens train.jsonl again
.send()
.await?;
// The second 8 MiB of a large file, as the whole body.
let chunk = Upload::path("video.mp4").slice(8 << 20..16 << 20).resolve().await?;
client.request(Method::PUT, "https://api.example.com/v1/uploads/1/parts/2").body(chunk).send().await?;
Ok(())
}
```
A response body can go to a file, or be read as a tokio `AsyncRead`. `Response::save_to(path)` streams it into a
temporary file in the directory of `path`, flushes it to disk and renames it to `path`: `path` holds the whole body or
is left as it was, never half a download. It returns the number of bytes written; a failure to write is an `Error`
whose `is_io()` is true:
```rust,no_run
use timvw_rust_runtime::{Client, Method};
async fn download(client: &Client) -> Result<(), Box<dyn std::error::Error>> {
let response = client.request(Method::GET, "https://api.example.com/v1/files/1/content").send().await?;
let written = response.save_to("data/file-1.jsonl").await?;
let response = client.request(Method::GET, "https://api.example.com/v1/files/2/content").send().await?;
let mut reader = response.into_async_read();
let mut file = tokio::fs::File::create("data/file-2.jsonl").await?;
tokio::io::copy(&mut reader, &mut file).await?; // an error reading the body is an io::Error here
Ok(())
}
```
Files, readers and downloads are feature `fs` (on by default; a generated crate with a file parameter turns it on).
## Credentials
A generated `Configuration` keeps its tokens, basic-auth password and api key in
[`secrecy`](https://docs.rs/secrecy)'s `SecretString`, which this crate re-exports (`timvw_rust_runtime::secrecy`,
`SecretString` and `ExposeSecret`). A `SecretString` is made from a `String` or a `&str`, its `Debug` is redacted,
and it is zeroed when dropped. `bearer_auth`, `basic_auth` and `sensitive_header` take the exposed value and mark the
header sensitive, so the `Debug` of the request's headers leaves it out:
```rust
use timvw_rust_runtime::{Client, ExposeSecret, Method, SecretString};
let key = SecretString::from("sk-123");
assert_eq!(format!("{key:?}"), "SecretBox<str>([REDACTED])");
let request = Client::default()
.request(Method::GET, "https://api.example.com/v1/models")
.sensitive_header("x-api-key", key.expose_secret())
.build()?;
assert!(request.headers()["x-api-key"].is_sensitive());
assert!(!format!("{:?}", request.headers()).contains("sk-123"));
# Ok::<(), timvw_rust_runtime::Error>(())
```
## Headers and query parameters for one call
A layer changes every request. `RequestOptions` holds extra headers and query parameters for one call; every
operation of a generated client takes it, `.options(&options)` on its builder, and `RequestBuilder::options` applies it
to a request of your own:
```rust,no_run
use timvw_rust_runtime::{Client, Method, RequestOptions};
async fn run(client: &Client) -> Result<(), timvw_rust_runtime::Error> {
let options = RequestOptions::new()
.header("x-request-source", "batch")?
.query(&[("trace", "1")])?;
// generated: items_api::list_items().configuration(&configuration).options(&options).call().await
let response = client
.request(Method::GET, "https://api.example.com/v1/items?limit=10")
.bearer_auth("secret")
.options(&options)
.send()
.await?; // GET /v1/items?limit=10&trace=1, with x-request-source: batch
Ok(())
}
```
- A header of the options replaces every value the request has under its name, whatever set it: a generated header
parameter, `Authorization`, `Content-Type`, `Accept`. Generated clients apply the options last, after everything
else. `RequestOptions::header` sets one value (a second call for the same name replaces it); `headers` merges a
`HeaderMap`, keeping several values of one name.
- Query parameters are appended after the request's own: a name the request already has is sent twice.
- `header` and `query` check their input at once and return an `Error` whose `is_builder()` is true for an invalid
header name or value, or a query that is not a flat list of pairs.
- The options do not change the path or the body. One value can be cloned, or reused for many calls.
- A header value marked with `HeaderValue::set_sensitive` stays sensitive (hidden from `Debug` output).
## Errors
A request that fails to send is an `Error`:
| `is_builder()` | the request could not be built (an invalid URL, header or body) |
| `is_request()` | the service failed; `is_timeout()` and `is_connect()` narrow it down |
| `is_body()` | reading the response body failed |
| `is_decode()` | the response could not be decoded (JSON) |
| `is_io()` | a file could not be read or written: a missing [upload](#uploads-and-downloads), a failed `save_to` |
`source()` is the error of the service or layer that failed: a `reqwest::Error`, a timeout, or your own. A
generated client wraps it as `apis::Error::Http(error)`. A response with an error status is not an `Error` here; the
generated client turns it into `apis::Error::ResponseError`.
## WebAssembly
The crate, and the clients generated with it, build for `wasm32-unknown-unknown` (a browser, a web worker, Node.js,
Cloudflare Workers: any JavaScript host). Everything that differs is behind `#[cfg(target_family = "wasm")]`, so the
same `Cargo.toml` works for both:
```toml
# Natively and on WebAssembly. On WebAssembly `rustls`/`native-tls` add nothing (the host does TLS), and `fs` adds only
# `Upload::path` and `From<PathBuf>`, which fail when sent.
timvw-rust-runtime = { version = "0.3", default-features = false, features = ["reqwest", "retry"] }
```
`default-features = true` builds too. On WebAssembly:
- **Sending.** reqwest sends with the host's `fetch`. `fetch` takes a request body whole, so a streamed body (an
`Upload::stream`, a multipart form with one) is read into memory first; responses still stream as they arrive.
Proxies and TLS settings belong to the host, and `fetch` has no timeout: add
[`timeout::TimeoutLayer`](#timeouts), which waits with the host's `setTimeout` (tower's `TimeoutLayer` needs tokio's
timer and does not work there). A request that times out is dropped, and reqwest aborts its `fetch`. Any other layer
works.
- **Retries** wait as natively: exponential backoff with jitter, `Retry-After` and `retry-after-ms` (an HTTP date is
read against the host's clock, `Date.now()`). The timer is the host's `setTimeout` (`gloo-timers`) and deadlines use
`performance.now()` (`web-time`), as `std::time::Instant::now()` panics there. A failure to `fetch` is not retried:
`fetch` does not tell a failure to connect from one after the request was sent (or a CORS refusal), so
`Error::is_connect()` is always `false`.
- **Server-sent events** work, `Events::idle_timeout` with the host's timer; `TraceLayer` measures with
`performance.now()`.
- **`Send` and `Sync`.** The host's futures and streams cannot leave its thread, so on WebAssembly nothing here needs
to be `Send` or `Sync`: the bounds of `Client::from_service`, `Client::layer`, `Body::wrap`, `Body::wrap_stream`,
`Upload::stream`, `Upload::from_fn`, `Events::from_bytes` and the layers are [`MaybeSend`] and [`MaybeSync`]:
`Send` and `Sync` natively, nothing on WebAssembly. Use them in your own bounds to write code that builds for both.
`HttpService` is a `LocalBoxCloneService` there (tower's `BoxCloneSyncService` natively), and `Client`, `Body`,
`Response` and the futures of `send` are not `Send`: run them with `wasm_bindgen_futures::spawn_local`, not on
another thread.
- **No files.** There is no file system: `Upload::file`, `Upload::reader`, `Response::save_to` and
`Response::into_async_read` are not there, and an `Upload::path` (or a `PathBuf` given to a generated client) fails
with an I/O error of kind `Unsupported` when it is resolved or sent. Send `Upload::bytes`, `Upload::stream` or
`Upload::from_fn`; read a response with `bytes`, `bytes_stream` or `events`.
`tests/wasm.rs` runs in Node.js with `wasm-bindgen-test-runner` (see the CI workflow).
[`MaybeSend`]: https://docs.rs/timvw-rust-runtime/latest/timvw_rust_runtime/trait.MaybeSend.html
[`MaybeSync`]: https://docs.rs/timvw-rust-runtime/latest/timvw_rust_runtime/trait.MaybeSync.html
## Features
| `rustls` | yes | reqwest with rustls, as `Client::default()` (on WebAssembly: reqwest, the host does TLS) |
| `native-tls` | no | reqwest with the platform's TLS (on WebAssembly: reqwest, the host does TLS) |
| `reqwest` | via `rustls` or `native-tls` | `ReqwestService`, `Client::from_reqwest` |
| `retry` | yes | `retry::RetryLayer` and `retry::RetryPolicy` (sleep with the `time` timer, read `Retry-After` dates with `httpdate`), and retries in `Client::default()`; turns on `time` |
| `time` | via `retry` | [`timeout::TimeoutLayer`](#timeouts), `sse::Events::idle_timeout` and the sleeps of retries: tokio's timer, the host's `setTimeout` on [WebAssembly](#webassembly) |
| `fs` | yes | file and reader [uploads](#uploads-and-downloads) (`Upload::path`, `Upload::file`, `Upload::reader`), `Response::save_to` and `Response::into_async_read` (tokio); on WebAssembly only `Upload::path` and `From<PathBuf>`, which fail when sent |
| `tracing` | no | `trace::TraceLayer`, and a log event per retry |
A generated crate forwards `native-tls`, `rustls`, `retry` and `tracing` under the same names, turns on `fs` when it
has a file parameter and `time` when it has typed event streams (their `idle_timeout`), and builds for WebAssembly as
it is.
## What the generated code uses
The generated functions build requests with `RequestBuilder` (`query`, `header`, `sensitive_header`, `bearer_auth`,
`basic_auth`, `json`, `form`, `body`, `multipart`, `options`, `build`) and read responses with `Response` (`status`, `headers`, `text`, `bytes`,
`bytes_stream`): the methods of reqwest's types of the same names. `multipart::Form` encodes multipart bodies, and
`Upload` (`resolve`, `multipart::Part::upload`) carries file parameters.
## License
MIT OR Apache-2.0.