better-fetch

Typed HTTP client layer on top of reqwest, inspired by @better-fetch/fetch. Independent Rust implementation.

Installation

Pick one crate name (same library):

[dependencies]
better-fetch = "0.4"
serde = { version = "1", features = ["derive"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
tokio-util = "0.7"

Aliases on crates.io: typed-fetch, api-fetch — pub use better_fetch::*.

Optional features (defaults: json, rustls-tls):

better-fetch = { version = "0.4", features = ["tower", "validate", "multipart"] }

Minimal build (pick one TLS feature — do not enable rustls-tls and native-tls together):

better-fetch = { version = "0.4", default-features = false, features = ["json", "rustls-tls"] }

Quick start

Flexible requests with .get() — string paths, typed JSON response:

use better_fetch::{Client, Result};
use serde::Deserialize;

#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
struct Todo {
    user_id: u64,
    id: u64,
    title: String,
    completed: bool,
}

#[tokio::main]
async fn main() -> Result<()> {
    let client = Client::new("https://jsonplaceholder.typicode.com")?;

    // send() returns Response (any status); json() fails on non-2xx
    let todo: Todo = client
        .get("/todos/:id")
        .param("id", 1)
        .send()
        .await?
        .json()
        .await?;

    // Or in one step (response type from the variable or turbofish):
    let todo: Todo = client.get("/todos/:id").param("id", 1).send_json().await?;

    println!("{todo:#?}");
    Ok(())
}

For compile-time route definitions (method, path, params, query, response), see Typed endpoint below.

Migrating from 0.3.x to 0.4.0

• Typed .query(...) — returns Result; add ? before .send_json(): .query(MyQuery { ... })?.
• ClientConfig::hooks — field is no longer public; use effective_hooks() for the merged hook chain used at runtime.
• on_request body — setting RequestContext::body in a hook now updates the outgoing request (previously ignored).
• throw_on_error + send_stream — non-2xx errors include a peeked body in Error::Http, matching buffered send().
• EndpointRequestBuilder — no DerefMut; use typed .query(MyQuery)? instead of stringly .query("key", "value") on Ready.
• http_service / http_service_boxed — Tower layers apply to buffered send() only; use transport_stack for send_stream().
• schema-validate strict — query/params wire values are coerced to JSON numbers/bools when possible before JSON Schema checks.
• Manual impl Endpoint — add type Body = (); type Headers = (); when missing.

See CHANGELOG.md for the full list.

When to use better-fetch vs reqwest alone

Use better-fetch	Use reqwest directly
Typed routes (Endpoint), plugins, hooks, retries, throw_on_error	One-off raw HTTP with minimal deps
Shared client config (base URL, auth, default headers, retry policy)	You already have a custom stack and only need transport
Testing via HttpBackend / RecordingBackend	Full control of every reqwest option with no abstraction

better-fetch is reqwest under the hood for the default backend; you can pass a custom reqwest::Client with Client::with_http_client (or ClientBuilder::reqwest_client on the builder).

Concurrency: ClientBuilder::max_in_flight limits concurrent requests (including retries) inside the client. With feature = "tower", ConcurrencyLimitLayer caps the transport separately — avoid stacking both at the same numeric limit unless intentional. See examples/tower_vs_streaming.rs.

See docs/testing.md (testing) and docs/observability.md (tracing / OTel / miette).

Highlights

• Builder API — Client::builder(), per-request .timeout(), .retry(), .auth(), headers, JSON body.
• Retries — linear, exponential, or count; Retry-After, jitter, custom should_retry; default retry on 408/429/502/503/504.
• Hooks & plugins — compose client and plugin hooks; optional LoggerPlugin (requires a tracing subscriber in your app).
• Errors — Result + ?; TransportKind on transport failures; Error::api_json() to parse JSON error bodies from APIs; Error::hook() from on_request / on_response hooks.
• Typed endpoints — Endpoint trait + client.call::<E>() with typed params/query structs and send_json() (path/body enforced by type-state when applicable; query typed but optional).
• Testing — inject ClientBuilder::backend(Arc<dyn HttpBackend>).
• Cancellation — CancellationToken per request; cooperative abort during requests and retry backoff.
• Throw mode — throw_on_error(true) makes send() return Err on non-2xx (like upstream throw: true).
• Form & multipart — .form([...]) for url-encoded bodies; .multipart(form) with feature multipart.

Request options

On RequestBuilder (client.get(...) / .post(...)):

Method	Description
.param / .params / .params_iter	Path template :id substitution
.query / .queries	Query string (stable insertion order via IndexMap)
.query_json	Serialize a value into a query param (feature json)
.json / .body	Request body
.form	application/x-www-form-urlencoded body
.multipart	Multipart form (feature multipart)
.timeout / .retry	Per-request overrides
.auth / .bearer_token	Per-request auth
.cancellation_token	Cancel in-flight request + retry sleeps
.throw_on_error	send() returns Err on non-2xx when true
.send / .send_json	Execute request (send_json::<T>() for typed JSON)
.json_parser	Custom Bytes → Value parser (feature json; see below)

On EndpointRequestBuilder (client.call::<E>()):

Method	Description
.params(E::Params)	Typed path parameters (required when E::Params is not ())
.query(E::Query)	Typed query struct or IndexMap<String, QueryValue> (optional — call to attach query string)
.header / .bearer_token / .cancellation_token / .throw_on_error	Same as RequestBuilder
.send / .send_json	Execute; send_json() returns E::Response

Custom JSON parsing

By default, JSON responses deserialize in one step (Bytes → T via serde_json::from_slice).

ClientBuilder::json_parser (or per-request .json_parser) uses two steps: your function returns serde_json::Value, then the library maps to T. Use this for BOM stripping or payload normalization:

use better_fetch::{ClientBuilder, Result};
use bytes::Bytes;

let client = ClientBuilder::new()
    .base_url("https://api.example.com")?
    .json_parser(|body: &Bytes| {
        let slice = body.strip_prefix(b"\xef\xbb\xbf").unwrap_or(body);
        serde_json::from_slice(slice).map_err(|e| e.to_string())
    })
    .build()?;

For maximum performance on a single response, skip a global parser and use Response::into_json_with for a direct Bytes → T closure.

Cancellation

use better_fetch::{CancellationToken, Client, Result};
use std::time::Duration;

#[tokio::main]
async fn main() -> Result<()> {
    let token = CancellationToken::new();
    let client = Client::new("https://httpbin.org")?;

    let handle = tokio::spawn({
        let token = token.clone();
        let client = client.clone();
        async move {
            client
                .get("/delay/10")
                .cancellation_token(token)
                .send()
                .await
        }
    });

    tokio::time::sleep(Duration::from_millis(100)).await;
    token.cancel();

    assert!(handle.await.unwrap().unwrap_err().is_cancelled());
    Ok(())
}

Throw on HTTP error

// Default: Ok(Response) even for 404
let response = client.get("/missing").send().await?;

// Like upstream throw: true
let err = client
    .get("/missing")
    .throw_on_error(true)
    .send()
    .await
    .unwrap_err();

Typed endpoint

Typed endpoints bind HTTP method, path template, and response type at compile time. Path parameters use E::Params with .params(); query parameters use E::Query with .query() when you want them on the wire.

What compile-time guarantees apply

Part	Enforced before send?
Path (E::Params)	Yes, when not () — type-state NeedsParams
Body (E::Body on POST, etc.)	Yes, when the macros endpoint marks a required body — NeedsBody
Query (E::Query)	No — typed, but .query() is optional; omit it to send no query string
Headers (E::Headers)	No — typed, optional via .with_headers()
Response (E::Response)	Typed deserialization via .send_json()

E::Query: Default does not mean the library applies a default query on send — you must call .query(...)? to serialize E::Query. For required query params at runtime, use schema-validate strict or tests.

use better_fetch::{Client, Endpoint, Result, define_params};
use http::Method;
use serde::Deserialize;

define_params!(GetTodoParams for "/todos/:id" { id: u64 });

struct GetTodo;
impl Endpoint for GetTodo {
    const METHOD: Method = Method::GET;
    const PATH: &'static str = "/todos/:id";
    type Response = Todo;
    type Params = GetTodoParams;
    type Query = ();
    type Body = ();
    type Headers = ();
}

#[derive(Deserialize)]
struct Todo { id: u64, title: String }

async fn example(client: &Client) -> Result<()> {
    let todo = client
        .call::<GetTodo>()
        .params(GetTodoParams { id: 1 })
        .send_json()
        .await?;
    Ok(())
}

With the macros feature, use #[derive(EndpointParamsDerive)] and #[derive(EndpointQueryDerive)] instead of define_params! / impl_serde_endpoint_query!.

POST + typed body (NeedsBody): when using #[derive(Endpoint)] with POST and a non-unit #[body], you must call .json(), .with_body(), or .body() before .send() / .send_json() (see examples/needs_body.rs).

Schema registry: #[endpoint(register)] on #[derive(Endpoint)] generates YourEndpoint::register(&mut SchemaRegistry).

With the schema-validate feature and a strict registry, runtime JSON Schema checks run when schemas are registered for that route: request body, response body (after send() or StreamingResponse::collect()), query string, and path params. Failures surface as Error::SchemaValidation.

Macros: #[query] MyQuery on #[derive(Endpoint)] implements EndpointQuery for MyQuery (requires Serialize). Use #[query_field] for inline query fields on the endpoint struct instead.

Typed query example:

use better_fetch::{impl_serde_endpoint_query, Endpoint, Client};
use serde::Serialize;

#[derive(Default, Serialize)]
struct ListQuery { user_id: Option<u64> }
impl_serde_endpoint_query!(ListQuery);

// impl Endpoint { type Query = ListQuery; ... }
// client.call::<ListTodos>().query(ListQuery { user_id: Some(1) })?.send_json().await?;

.get() / .post() remain available for ad-hoc requests; only client.call::<E>() uses the typed builder.

Form and multipart

// URL-encoded form
client
    .post("/login")
    .form([("user", "alice"), ("pass", "secret")])
    .send()
    .await?;

// Multipart (feature "multipart")
let form = better_fetch::multipart::Form::new().text("file", "hello");
client.post("/upload").multipart(form).send().await?;

Note: automatic retry is not supported with .multipart() (the body cannot be replayed). Use .form, JSON, or raw bytes if you need retries.

Client builder

ClientBuilder::build() requires .base_url(...) — otherwise Error::MissingBaseUrl.

use better_fetch::{Client, ClientBuilder, Error, RetryPolicy, Result};
use std::time::Duration;

let client = ClientBuilder::new()
    .base_url("https://api.example.com")?
    .retry(RetryPolicy::exponential(3, Duration::from_secs(1), Duration::from_secs(30)))
    .build()?;

// Or pass a custom reqwest client:
let reqwest = reqwest::Client::builder()
    .pool_max_idle_per_host(0)
    .build()
    .map_err(|e| Error::Config(e.to_string()))?;
let client = Client::with_http_client(reqwest, "https://api.example.com")?;

Concurrency limits

ClientBuilder::max_in_flight uses a tokio semaphore in the core client (counts retries as in-flight work). The tower feature’s ConcurrencyLimitLayer is a separate transport-level cap. Use one of these at a given limit unless you intentionally want two stacked caps (e.g. app-wide budget + per-host transport limit).

Tower transport (feature = "tower")

Wire a custom transport with ClientBuilder::http_service, http_service_boxed, or transport_stack. Stack helpers live in better_fetch::tower::stack (build, ConcurrencyLimitLayer, with_buffer, etc.).

ServiceBackend clones the boxed Tower stack per request, so concurrent transport calls can run in parallel. Wrap the inner service with tower::buffer::Buffer only when the inner service is not [Clone] or is expensive to clone (see examples/tower_stack and stack::with_buffer). When you do not need Tower middleware, use the default reqwest backend. Do not stack max_in_flight and ConcurrencyLimitLayer at the same numeric limit without intent.

Response bodies: buffered vs streaming

API	Use when
send() → Response	Typical JSON APIs; full body in memory; hooks and retry predicates can inspect the body
send_stream() → StreamingResponse	Large downloads, chunked bodies, incremental processing
collect() on a stream	Opt back into the buffered Response API after streaming

send() buffers the full body in memory by default (reqwest bytes().await when no size cap is set). With max_response_bytes, send() and send_json() read the body through the streaming transport and stop at the limit. send_stream() uses bytes_stream() and does not buffer until you call collect().

Streaming hooks: on_response_stream and on_success_stream run on the streaming path (status + headers only). Buffered on_response / on_success are not invoked for send_stream.

Retry on streams: Status/header retries work as with send(). Custom RetryPolicy::with_should_retry predicates can peek at up to retry_body_peek_bytes (default 64 KiB, bounded by max_response_bytes when set). Without a custom predicate, the body is not read before retrying.

Tower + streaming: transport_stack wires separate buffered and streaming Tower stacks. Request middleware (e.g. map_request) runs on both send() and send_stream() when you configure both inner services in the closure.

Other notes:

• send_json is not available on streams — use collect() then into_json(), or deserialize from chunks yourself.
• Cancellation is cooperative: the stream wakes on cancel when the inner read is pending, not inside a blocking OS read.
• Without max_response_bytes, send(), send_json(), and collect() can use unbounded memory; set a cap for untrusted payloads.

Optional caps: ClientBuilder::max_response_bytes and per-request .max_response_bytes() apply to buffered and streaming responses and yield Error::BodyTooLarge when exceeded.

See examples/streaming.rs, examples/buffered_vs_streaming.rs, examples/throw_on_error.rs, and examples/cancel.rs.

Plugins

Plugin::init receives PreparedRequest with url, path, method, and headers (after auth, before lifecycle hooks). Use it to rewrite URLs or inspect auth headers.

Logging

LoggerPlugin emits tracing spans (http.request, http.response). Install a subscriber in your app, for example:

tracing_subscriber::fmt::init();

For OpenTelemetry export via tracing-opentelemetry, see docs/observability.md.

Features

Feature	Description
json, rustls-tls (default)	JSON API + TLS via rustls (reqwest is always the default backend)
native-tls	Platform TLS instead of rustls (do not combine with rustls-tls)
blocking, cookies	Passed through to reqwest
multipart	RequestBuilder::multipart + better_fetch::multipart re-export
schema / openapi	schemars registry, strict routes, and OpenAPI 3.0 export
tower / tower-http	Tower Service transport stack (better_fetch::tower)
validate	Request/response validation with garde (json_validated, send_json_validated)
schema-validate	Runtime JSON Schema validation (jsonschema) when registry is strict (request/response body, query, params)
macros	#[derive(Endpoint)], define_params!, EndpointParamsDerive, EndpointQueryDerive
miette	DiagnosticError — Cargo feature only (wrap errors in your app; not a Plugin)
otel	Re-exports opentelemetry, opentelemetry_sdk, tracing_opentelemetry; see docs/observability.md
full	Enables json, rustls-tls, macros, schema, validate, schema-validate, miette, otel, tower, multipart, openapi

Use better_fetch::prelude::* for common imports in application code.

See CHANGELOG.md for release notes.

Examples

cargo run -p better-fetch --example streaming
cargo run -p better-fetch --example buffered_vs_streaming --features json
cargo run -p better-fetch --example throw_on_error --features json
cargo run -p better-fetch --example cancel
cargo run -p better-fetch --example basic
cargo run -p better-fetch --example typed_endpoint --features json
cargo run -p better-fetch --example tower_stack --features tower,json
cargo run -p better-fetch --example tower_vs_streaming --features tower,json
cargo run -p better-fetch --example multipart --features multipart
cargo run -p better-fetch --example retry
cargo run -p better-fetch --example openapi_export --features openapi
cargo run -p better-fetch --example upload_stream --features json
cargo run -p better-fetch --example validated_request --features validate
cargo test -p better-fetch
cargo test -p better-fetch --features default,validate,tower,multipart,macros

Crates in this repository

crates.io	Role
better-fetch	Main library
typed-fetch	Re-export alias
api-fetch	Re-export alias
better-fetch-macros	Proc-macros for typed endpoint params/query

Publishing

Push a semver tag matching version in the workspace Cargo.toml:

git tag v0.5.0
git push origin v0.5.0

That runs .github/workflows/release.yml: CI, crates.io trusted publishing (crates better-fetch-macros, better-fetch, typed-fetch, api-fetch), then a GitHub Release with the matching section from CHANGELOG.md.

Trusted publishing (one-time per crate): on crates.io → crate → Settings → Trusted Publishing — GitHub owner sebasxsala, repository better-fetch-rs, workflow release.yml, environment release. Create the release environment on GitHub if you use approval gates. Each crate must have been published manually at least once before trusted publishing works.

Manual order (if needed): cargo publish -p better-fetch-macros, then better-fetch, then typed-fetch, then api-fetch.

License

MIT — see LICENSE. Upstream inspiration: THIRD_PARTY_NOTICES.md.