ugi
A runtime-agnostic Rust HTTP client with full protocol coverage and a uniform builder API.
let client = builder.build?;
let resp = client.get.await?;
let user: User = resp.json.await?;
Features
| Feature | Detail |
|---|---|
| Protocols | HTTP/1.1, HTTP/2, HTTP/3, H2C upgrade, WebSocket, SSE, gRPC |
| Compression | Auto-negotiate gzip / brotli / deflate / zstd; manual override |
| Connection pooling | Per-host keep-alive, configurable pool limits |
| TLS | async-tls (rustls) by default; optional native-tls; cert pinning |
| Redirects | Follow with configurable hop limit; strip Authorization on cross-origin |
| Cookies | CookieJar middleware with domain/path/expiry semantics |
| Retry | Per-request RetryPolicy with exponential back-off |
| Progress | Upload and download progress callbacks |
| Proxy | HTTP and SOCKS5 proxies with optional auth |
| File download | Parallel byte-range chunks, integrity hashing, rate limiting |
| Rate limiting | Token-bucket RateLimiter; per-request or shared across requests |
| Runtime | Works with Tokio, smol, or bare block_on — no runtime lock-in |
Implementation
These are the optimizations that separate a production HTTP client from a toy — ugi implements all of them.
| Optimization | What ugi does |
|---|---|
| TCP_NODELAY | Set on every connection; disables Nagle's algorithm for request/response workloads where coalescing small packets adds latency rather than saving bandwidth |
| Happy Eyeballs (RFC 6555) | Races IPv4 and IPv6 connections in parallel with a 250 ms IPv6 head-start; uses whichever wins, eliminating the long fallback delay on dual-stack hosts |
| TTL-aware DNS cache | Resolved addresses are cached per their DNS TTL and reused across requests; avoids redundant lookups on every call |
| Connection pooling | Per-host idle connection pool (max_idle_per_host = 8, idle_timeout = 90 s); TCP and TLS handshakes are paid once |
| Stale connection detection | Pooled connections that have been silently closed by the server are detected and the request is transparently retried on a fresh connection |
| TLS session resumption | rustls maintains an in-memory session cache by default; TLS 1.3 resumption eliminates the handshake round-trip on reconnect |
| HTTP/2 multiplexing | Multiple concurrent requests share one TCP+TLS connection; eliminates head-of-line blocking and connection setup overhead |
| H2 PING keepalive | Periodic PING frames keep HTTP/2 connections alive through NAT and idle-kill firewalls; configurable interval and ACK timeout |
| H2C prior-knowledge upgrade | Speaks HTTP/2 in cleartext directly (RFC 7540 §3.4) without the round-trip cost of an HTTP/1.1 Upgrade handshake |
| ALPN negotiation | Advertises h2 during TLS handshake so the server can confirm HTTP/2 without an extra round-trip |
| QUIC 0-RTT | HTTP/3 requests resume with 0-RTT early data on cached QUIC session tickets, eliminating the TLS handshake round-trip on reconnect |
| Automatic compression | Sends Accept-Encoding: gzip, deflate, br, zstd and transparently decodes the response; reduces transfer size without any user code |
| Idempotent-safe retry | Retries on transient errors only for safe methods (GET, HEAD, OPTIONS, TRACE); never retries mutations to avoid duplicate side-effects; exponential back-off (100 ms → 5 s cap) between attempts |
| Progress throttling | Progress callbacks are gated by a minimum time interval and a minimum byte delta; prevents callback overhead from dominating fast transfers |
| Token-bucket rate limiting | Per-request or shared RateLimiter caps outbound bandwidth without busy-looping; works across concurrent requests |
| WebSocket kernel flush | Every WebSocket frame is flushed to the kernel after write_all; ensures frames are not held in the OS socket buffer when the TLS layer is involved |
| Alt-Svc / HTTP/3 upgrade | Parses Alt-Svc response headers (including draft versions h3-*) and opportunistically upgrades subsequent Auto-policy requests to HTTP/3 for origins that advertise it; entries respect the ma max-age and clear directive |
Performance
Benchmarked on loopback (Apple M-series, cargo bench):
| Scenario | ugi | competitor | Delta |
|---|---|---|---|
| HTTP/1.1 keep-alive (2000 × GET) | ~40,000 req/s | reqwest ~34,000 req/s | ugi ~18% faster |
| HTTP/2 keep-alive (50 × GET) | ~12,000 req/s | reqwest ~10,000 req/s | ugi ~20% faster |
| gRPC unary JSON (200 × call) | ~17,000 rps | raw h2 crate ~21,000 rps | 1.3× over bare transport |
| HTTP/3 QUIC unary GET (100 × GET) | ~13,000 rps | raw quiche ~17,000 rps | 1.3× over bare transport |
| WebSocket echo (500 × msg) | ~21,000 ops/s | tokio-tungstenite ~55,000 ops/s | tungstenite ~2.6× faster |
The WebSocket gap is an async-runtime ecosystem difference (async-io vs tokio reactor),
not a protocol correctness issue. The gRPC baseline is the raw h2 crate with no
middleware, builder, or JSON encode/decode — measuring ugi's builder + codec overhead.
Full methodology and optimization notes in docs/performance.md.
Quick start
Add to Cargo.toml:
[]
= "0.2.1"
The default feature set is full-featured (rustls + h2 + brotli + zstd). Opt out of heavy codecs or switch TLS back-ends as needed:
# System TLS instead of rustls, no brotli/zstd
= { = "0.2.1", = false, = ["native-tls", "h2"] }
Cargo features
| Feature | Default | Description |
|---|---|---|
rustls |
yes | TLS via async-tls / rustls (no OpenSSL dependency) |
native-tls |
no | TLS via OS certificate store (mutually exclusive with rustls) |
btls-backend |
no | High-fidelity BoringSSL-family TLS backend without preset emulation |
boring-backend |
no | Compatibility alias for btls-backend |
emulation |
no | Browser/network emulation presets, btls TLS fingerprinting, H2 fingerprinting, and strict no-downgrade behavior |
h2 |
yes | HTTP/2 support |
h3 |
no | HTTP/3 / QUIC via quiche (vendored BoringSSL, slow to compile) |
brotli |
yes | Brotli response decompression |
zstd |
yes | Zstandard response decompression |
gzip and deflate are always available (lightweight via flate2). WebSocket, SSE, gRPC, and file download are always compiled in.
Browser/network emulation
Enable the feature explicitly:
= { = "0.2.1", = ["emulation"] }
Simple preset path:
let client = builder
.emulation
.build?;
Custom profile path:
let profile = builder
.default_header?
.build;
let client = builder
.emulation
.build?;
.emulation(..) is the preferred public entrypoint for both presets and custom
profiles. Compatibility aliases exist, but the library is intentionally
centered on a single emulation switch.
Example binaries:
GraphQL with other crates
ugi stays at the transport layer for GraphQL use cases. The intended pattern
is to build the GraphQL envelope with plain serde types or a dedicated query
crate, then send it with RequestBuilder::json(...).
Repository examples:
graphql_jsonshows the minimal plain-JSON envelope pattern.graphql_graphql_clientshowsgraphql_client::QueryBodyoverugitransport without changingugi's public API.
Manual online smoke tests for emulation
The repository includes an ignored online smoke suite for the current browser
presets. It defaults to tls.peet.ws, but can be
pointed at a different probe endpoint with UGI_EMULATION_SMOKE_URL.
UGI_RUN_ONLINE_SMOKE=1
This suite is intentionally opt-in:
- the tests are
#[ignore] UGI_RUN_ONLINE_SMOKE=1must be set explicitly- the target URL can be overridden for nightly or internal probe infrastructure
Runtime compatibility
// Tokio
async
// smol
// No runtime dependency — futures-lite block_on as a single-threaded executor.
// The async block only handles I/O; the result surfaces to synchronous code.
API overview
All builders share the same pattern: chain options, then .await (or call .save / .connect / .probe).
Client construction
let client = builder
.base_url?
.bearer_auth?
.timeout
.retry
.build?;
ClientBuilder — connection and defaults
| Method | Description |
|---|---|
base_url(url) |
Prefix for relative request URLs |
header(name, value) / headers(iter) |
Default headers sent on every request |
cookie(name, value) / cookies(iter) |
Static cookies sent on every request |
user_agent(value) |
Set user-agent header |
bearer_auth(token) |
Set authorization: Bearer … |
basic_auth(user, pass) |
Set authorization: Basic … |
timeout(dur) |
Total request timeout |
connect_timeout(dur) |
TCP / TLS handshake timeout |
read_timeout(dur) |
Per-read idle timeout |
write_timeout(dur) |
Per-write idle timeout |
redirect(RedirectPolicy) |
None or Limit(n) (default: Limit(10)) |
retry(RetryPolicy) |
None or Limit(n) for idempotent requests |
proxy(proxy) |
HTTP or SOCKS5 proxy |
local_addr(addr) |
Bind outgoing connections to this address |
dual_stack(bool) |
Interleave IPv6 / IPv4 candidates (Happy Eyeballs) |
dns_cache_ttl(dur) |
Override default DNS TTL |
dns_prefetch(hosts) |
Resolve hosts at build time |
on_progress(fn) |
Default progress callback (upload + download) |
progress_config(cfg) |
Throttle callback frequency (ProgressConfig::new(interval, min_bytes)) |
middleware(m) |
Append a Middleware to the chain |
cookie_store() |
Enable automatic Set-Cookie / Cookie handling |
cookie_jar(jar) |
Supply a pre-existing CookieJar |
max_idle_per_host(n) |
Connection pool size per host (default: 8) |
idle_timeout(dur) |
Evict pooled connections after this idle time (default: 90 s) |
pool_config(cfg) |
Supply a complete PoolConfig |
h2_keepalive(idle, ack) |
HTTP/2 PING keepalive timers |
disable_h2_keepalive() |
Turn off HTTP/2 PING keepalive |
compression_mode(mode) |
Auto (default) or Manual |
emulation(profile) |
Apply a preset or custom emulation profile (emulation feature) |
ClientBuilder — protocol selection (applies to all requests from this client)
| Method | Description |
|---|---|
prefer_http2() |
Try HTTP/2, fall back to HTTP/1.1 |
prefer_http3() |
Try HTTP/3 (QUIC), fall back to HTTP/1.1 |
http1_only() |
Force HTTP/1.1 |
http2_only() |
Force HTTP/2; error if unavailable |
http3_only() |
Force HTTP/3 (QUIC); error if unavailable |
prior_knowledge_h2c(bool) |
Skip HTTP/1.1 upgrade and speak h2c directly |
protocol_policy(ProtocolPolicy) |
Set any ProtocolPolicy variant directly |
ClientBuilder — TLS
| Method | Description |
|---|---|
tls_config(cfg) |
Replace the entire TlsConfig |
tls_backend(TlsBackend) |
Rustls (default) or NativeTls |
root_store(RootStore) |
System, WebPki, PemFile(path), or Pem(str) |
pin_certificate(domain, fingerprint) |
Pin a SHA-256 hex fingerprint for a domain |
danger_accept_invalid_certs(bool) |
Skip certificate verification (tests only) |
alpn_protocols(iter) |
Override ALPN list |
disable_alpn() |
Send no ALPN extension |
Requests
Every HTTP method on Client returns a RequestBuilder that implements IntoFuture.
// top-level convenience (uses a shared default client)
get.await?;
// per-client
client.get.query.await?;
client.post.json?.await?;
client.put.text?.await?;
client.delete.await?;
RequestBuilder — body
| Method | Description |
|---|---|
body(impl Into<Body>) |
Raw bytes body |
body_stream(BodyStream) |
Streaming / chunked body |
text(str) |
Body + content-type: text/plain; charset=utf-8 |
json(&T) |
Serialize to JSON + content-type: application/json |
form(iter) |
URL-encoded form + content-type: application/x-www-form-urlencoded |
multipart_form(iter<MultipartPart>) |
RFC 2046 multipart/form-data (text fields and file uploads) |
RequestBuilder — common options (override the client default per-request)
| Method | Description |
|---|---|
base_url(url) |
Override the client base URL for this request |
header(name, value) / headers(iter) |
Append headers |
cookie(name, value) / cookies(iter) |
Append cookies |
query(iter) |
Append query pairs |
bearer_auth(token) / basic_auth(user, pass) |
Auth header |
timeout(dur) / connect_timeout(dur) / read_timeout(dur) / write_timeout(dur) |
Timeouts |
redirect(RedirectPolicy) |
Override redirect policy |
retry(RetryPolicy) |
Override retry policy |
proxy(proxy) |
Override proxy |
on_progress(fn) / progress_config(cfg) |
Progress reporting |
rate_limit(bytes_per_sec) |
Throttle this response body |
rate_limit_shared(RateLimiter) |
Share a rate budget with other requests |
compression_mode(mode) |
Override compression negotiation |
prefer_http2() / prefer_http3() |
Prefer a protocol version |
http1_only() / http2_only() / http3_only() |
Force a protocol version |
prior_knowledge_h2c(bool) |
h2c without upgrade |
h2_keepalive(idle, ack) / disable_h2_keepalive() |
HTTP/2 PING |
last_event_id(id) |
Set last-event-id header (SSE reconnect) |
tls_config(cfg) / tls_backend(b) / root_store(s) |
TLS overrides |
pin_certificate(domain, fp) / danger_accept_invalid_certs(bool) |
Cert pinning / skip verify |
alpn_protocols(iter) / disable_alpn() |
ALPN overrides |
emulation(profile) |
Replace the effective emulation profile for this request (emulation feature) |
Responses
let resp = client.get.await?;
resp.status // StatusCode (.as_u16(), .is_success(), …)
resp.version // Version::Http11 | Http2 | Http3
resp.url // &Url — final URL after any redirects
resp.headers // &HeaderMap
resp.headers_mut // &mut HeaderMap
resp.cookies // Vec<&str> of raw Set-Cookie values
resp.trailers // Option<&HeaderMap> — HTTP/2 trailer headers
resp.metrics // &Metrics (timing, bytes transferred)
resp.error_for_status // Err if 4xx / 5xx
// consume the body (pick one)
resp.text.await? // String
resp.bytes.await? // Bytes
resp..await? // serde-deserialized T
resp.bytes_stream // Pin<Box<dyn Stream<Item=Result<Bytes>>>>
resp.bytes_and_trailers.await? // (Bytes, Option<HeaderMap>)
resp.sse? // SseStream — use only for text/event-stream responses
Metrics — returned by resp.metrics():
| Method | Type | Description |
|---|---|---|
dns_duration() |
Option<Duration> |
Time spent on DNS resolution |
connect_duration() |
Option<Duration> |
Time to establish the TCP connection |
tls_duration() |
Option<Duration> |
Time for TLS handshake (None for plain HTTP) |
request_write_duration() |
Option<Duration> |
Time to write the full request |
ttfb() |
Option<Duration> |
Time to first byte of the response |
download_duration() |
Option<Duration> |
Time to download the full response body |
reused_connection() |
bool |
Whether the connection came from the pool |
protocol() |
Option<Version> |
Negotiated protocol (Http11, Http2, Http3) |
WebSocket
let mut ws = client.ws
.header?
.protocol // request a sub-protocol
.origin
.timeout
.connect
.await?;
ws.send.await?;
ws.send.await?;
let msg = ws.recv.await?;
ws.close.await?;
WebSocketBuilder options:
| Method | Description |
|---|---|
header(name, value) / headers(iter) |
Append handshake request headers |
cookie(name, value) / cookies(iter) |
Append cookies to the handshake request |
bearer_auth(token) / basic_auth(user, pass) |
Auth header |
protocol(name) / protocols(iter) |
Sec-WebSocket-Protocol values |
origin(value) |
Origin header |
timeout(dur) / connect_timeout(dur) / read_timeout(dur) / write_timeout(dur) |
Timeouts |
tls_config(TlsConfig) |
Full TLS override |
tls_backend(TlsBackend) |
Switch between rustls / native-tls |
danger_accept_invalid_certs(bool) |
Disable TLS certificate validation |
alpn_protocols(iter) |
Custom ALPN protocol list |
local_addr(SocketAddr) |
Bind the outgoing socket to a specific local address |
Server-Sent Events
let resp = client.get
.last_event_id // reconnect hint
.await?;
let mut stream = resp.sse?;
while let Some = stream.next.await
SseEvent fields
| Field | Type | Description |
|---|---|---|
event |
String |
Event type (defaults to "message") |
data |
String |
Event payload |
id |
Option<String> |
Last-event-id set by the server |
retry |
Option<Duration> |
Reconnection delay hint from the server |
gRPC
// unary
let resp = client.grpc
.metadata?
.message?
.await?;
let reply: MyReply = resp.message.await?;
// server-streaming
let mut stream = client.grpc
.message?
.send_streaming
.await?;
while let Some = stream..await?
// bidirectional
let mut call = client.grpc
.open_duplex
.await?;
call.send.await?;
let reply: ChatMessage = call.recv.await?;
GrpcRequestBuilder options:
| Method | Description |
|---|---|
metadata(name, value) |
Append a single ASCII request metadata header |
metadata_map(iter) |
Append multiple ASCII metadata headers at once |
metadata_bin(name, bytes) |
Append a binary metadata header (base64-encoded, name must end in -bin) |
message(&T) |
Serialize a single Protobuf/JSON message as the request payload |
messages(stream) |
Stream multiple messages (client-streaming / bidirectional) |
message_bytes(bytes) |
Raw pre-encoded message bytes |
messages_bytes(stream) |
Stream of raw pre-encoded message bytes |
codec(GrpcCodec) |
GrpcCodec::Json (default) or GrpcCodec::Protobuf |
compression(algo) |
Request-body compression algorithm (e.g. "gzip") |
timeout(dur) / connect_timeout(dur) / read_timeout(dur) / write_timeout(dur) |
Timeouts |
prefer_http2() / prefer_http3() / http2_only() / http3_only() |
Protocol preference |
prior_knowledge_h2c(bool) |
Use HTTP/2 cleartext without upgrade handshake |
tls_config(TlsConfig) |
Full TLS override |
tls_backend(TlsBackend) |
Switch between rustls / native-tls |
danger_accept_invalid_certs(bool) |
Disable TLS certificate validation |
.await |
Execute as unary call; returns GrpcResponse |
send_streaming() |
Execute; returns GrpcStreamingResponse for server-streaming calls |
open_duplex() |
Open a bidirectional stream; returns GrpcDuplexCall |
File download
use ;
// simple download
client.download
.save
.await?;
// parallel chunks + integrity check
client.download
.chunks // parallel range requests
.verify // fail on mismatch
.save
.await?;
// hash only, no expected value
client.download
.hash
.save
.await?;
// result.digest contains the computed hex string
// probe server without downloading
let caps = client.download.probe.await?;
println!;
DownloadBuilder options
| Method | Description |
|---|---|
chunks(n) |
Concurrent byte-range requests (default: 1; falls back if server rejects ranges) |
hash(HashAlgorithm) |
Compute digest while downloading; available in DownloadResult.digest |
verify(HashAlgorithm, expected_hex) |
Same as hash, but fails with ErrorKind::Decode on mismatch |
rate_limit(bytes_per_sec) |
Throttle this download |
rate_limit_shared(RateLimiter) |
Share a rate budget across concurrent downloads |
probe() |
HEAD the URL; returns DownloadCapabilities without downloading |
save(path) |
Execute the download; returns DownloadResult |
DownloadResult fields:
| Field | Type | Description |
|---|---|---|
total_bytes |
u64 |
Total bytes written to disk |
chunks_used |
usize |
Number of parallel byte-range chunks actually used |
capabilities |
DownloadCapabilities |
Server capabilities probed before download (supports_range, content_length) |
digest |
Option<DownloadDigest> |
Computed hash if hash() or verify() was called; contains algorithm and hex fields |
Rate limiting
// per-request
client.get
.rate_limit // 512 KB/s cap on this response body
.await?;
// shared budget across concurrent requests or downloads
let limiter = new; // 1 MB/s total
let r1 = client.get.rate_limit_shared;
let r2 = client.get.rate_limit_shared;
// drive r1 and r2 concurrently — they share the same token bucket
// adjust at runtime
limiter.set_rate.await; // raise to 2 MB/s
Cloning a RateLimiter shares its underlying bucket — all clones draw from the same budget.
TLS and certificate pinning
// pin a SHA-256 fingerprint (colon-separated hex or base64, "sha256/" prefix optional)
let client = builder
.pin_certificate?
.build?;
// custom root CA from a PEM file
let client = builder
.root_store
.build?;
// accept self-signed in tests
let client = builder
.danger_accept_invalid_certs
.build?;
// accept self-signed in tests
let client = builder
.danger_accept_invalid_certs
.build?;
Proxy
use Proxy;
// HTTP proxy
let client = builder
.proxy
.build?;
// HTTP proxy with authentication
let client = builder
.proxy
.build?;
// SOCKS5 proxy
let client = builder
.proxy
.build?;
// SOCKS5 with authentication
let client = builder
.proxy
.build?;
| Constructor | Description |
|---|---|
Proxy::http(addr) |
Plain HTTP CONNECT proxy |
Proxy::http_with_auth(addr, user, pass) |
HTTP CONNECT proxy with Basic auth |
Proxy::socks5(addr) |
SOCKS5 proxy (no auth) |
Proxy::socks5_with_auth(addr, user, pass) |
SOCKS5 proxy with username/password |
Middleware
use ;
;
let client = builder
.middleware
.build?;
Middlewares run in registration order. Next::run dispatches to the next middleware or the transport.
Progress reporting
use ;
use Duration;
// throttle: fire at most once per 250 ms or 128 KB, whichever comes first
let config = new;
let client = builder
.on_progress
.build?;
// override per-request
client.get.on_progress.await?;
Type reference
ProtocolPolicy
| Variant | Description |
|---|---|
Auto |
Use Alt-Svc hints; prefer HTTP/2 over HTTP/1.1; upgrade to HTTP/3 when advertised |
PreferHttp2 |
Prefer HTTP/2, fall back to HTTP/1.1 |
Http2Only |
Require HTTP/2; error if not negotiated |
PreferHttp3 |
Prefer HTTP/3 (QUIC), fall back to HTTP/1.1 |
Http3Only |
Require HTTP/3; error if not negotiated |
Http1Only |
Force HTTP/1.1 always |
RetryPolicy
| Variant | Description |
|---|---|
None |
No retries (default) |
Limit(n) |
Retry up to n times on transient errors for idempotent methods; exponential back-off (100 ms → 5 s cap) |
CompressionMode
| Variant | Description |
|---|---|
Auto |
Send Accept-Encoding and decompress automatically (default) |
Manual |
Send no Accept-Encoding; body bytes are returned as-is |
Disabled |
Explicitly opt out of compression negotiation |
PoolConfig
| Field | Type | Default | Description |
|---|---|---|---|
max_idle_per_host |
usize |
8 | Maximum idle connections per host |
idle_timeout |
Option<Duration> |
90 s | Evict connections idle longer than this |
RootStore
| Variant | Description |
|---|---|
RootStore::system() |
Use the OS / system certificate store |
RootStore::webpki() |
Use the built-in Mozilla WebPKI root bundle |
RootStore::pem_file(path) |
Load PEM certificates from a file path |
RootStore::pem(string) |
Load PEM certificates from a string |
Examples
| Example | Description |
|---|---|
get.rs |
Simple GET and JSON decode |
json_post.rs |
POST JSON body |
client_configured.rs |
Full client configuration |
download.rs |
Parallel download with verification |
ws_echo.rs |
WebSocket echo |
grpc_unary_json.rs |
gRPC unary call |
grpc_server_stream_json.rs |
gRPC server streaming |
response_stream.rs |
SSE / streaming body |
tls_pinning.rs |
Certificate pinning |
http2_get.rs |
Force HTTP/2 |
http3_get.rs |
HTTP/3 (QUIC) |
tokio_get.rs |
Tokio runtime |
smol_get.rs |
smol runtime |
Run any example:
License
MIT OR Apache-2.0