eggfetch

eggfetch is a Rust-native HTTP client engine with Python bindings and a CLI tool. The core is async-first: a Rust engine built on tokio and hyper provides connection pooling, phase-aware timeouts, TLS configuration, streaming, and response decompression. The Python bindings expose both sync and async APIs; the sync API blocks on the async engine while releasing the GIL, and the async API integrates with asyncio. There is exactly one networking implementation, living entirely in Rust.

Features

• HTTP/1.1, HTTP/2, HTTP/3 -- ALPN negotiation, multiplexed connections, experimental QUIC transport (bounded per-origin cache, shared connect budget with address fallback, phase-correct timeouts, keepalive-derived idle, authenticated Alt-Svc discovery with suppression/safe fallback/draining; retained experimental — see docs/architecture/core-tls-proxy-protocols.md § "Production Graduation Decision")
• Streaming -- request and response bodies stream without eager buffering; bytes_stream() and text_lines() for incremental reads
• HTTP trailers -- H1 chunked trailers, H2 trailing HEADERS, and H3 trailing headers captured without buffering (Response::trailers() after body EOF; H1 duplicate same-name trailers collapse upstream in hyper and are documented)
• Response decompression -- gzip, brotli, zstd, deflate via feature-gated streaming decoders
• Connection pooling -- semaphore-based logical in-flight request concurrency (max_in_flight_requests*, aliases max_connections* for pre-1.0) with per-origin limits, pool metrics, and separate transport observability counters
• Transport observability -- connector/DNS/TLS attempt counters, H3 creation/eviction counts, Alt-Svc learned/expired/cleared/rejected, H3 attempted/suppressed/fallback/drain/close/reconnect, and 101 upgrade counts where observable; HTTP/3 builds also expose bounded Quinn snapshots (remote address, RTT, path counters, route generation, and sanitized close codes); Hyper socket-reuse counts intentionally absent
• Phase-aware timeouts -- pool, connect, write, read, and total timeout phases with cancellation safety
• TLS -- rustls with custom CA bundles, client certificates (mTLS), version policy, and verification toggle
• Proxy -- HTTP forwarding, HTTPS CONNECT tunneling, proxy auth, per-request override, NO_PROXY bypass
• Cookies -- RFC 6265 cookie jar with domain/path matching, cross-origin stripping
• Authentication -- Basic and Bearer auth with credential redaction in all output paths
• Multipart -- streaming multipart/form-data with known-length optimization
• Retries -- policy-driven retries with exponential backoff and Retry-After support
• Python API -- requests/HTTPX-compatible sync and async interfaces, GIL-releasing blocking I/O
• HTTPX compatibility facade -- compatible asyncio surface targeting HTTPX 0.28.1 (eggfetch.compat.httpx)
• Network stream exposure -- 101 Switching Protocols responses expose an owned upgraded stream through extensions["network_stream"]; direct-connector upgrades carry real local/remote addrs and TLS version/cipher/ALPN, UDS upgrades report Unix without IPs, standard opaque upgrades remain explicitly unavailable; start_tls uses the same safe TLS translation as the default client
• Python/FFI trailer policy -- core retains trailers; Python native, HTTPX facade, FFI, and Node defer trailer exposure in this milestone (facade unchanged; HTTPX 0.28.1 has no trailers surface to compare against)
• CLI -- full-featured HTTP client with streaming output, machine-readable formats, and shell completions
• Node.js (experimental prototype) -- N-API binding with narrow guarantees (UTF-8 string bodies, buffered responses, unstructured errors, stub declarations); see docs/architecture/ffi-and-node.md

HTTP/3 remains experimental. See docs/architecture/core-tls-proxy-protocols.md § "Production Graduation Decision" for the graduation gate and current status.

Installation

Python:

pip install eggfetch

Rust:

[dependencies]
eggfetch-core = { version = "0.1", features = ["http1", "tls-rustls"] }

CLI:

cargo install eggfetch-cli

Usage -- Python

Quick requests

import eggfetch

r = eggfetch.get("https://httpbin.org/get")
print(r.status_code)
print(r.text)

Using a client

import eggfetch

with eggfetch.Client(headers={"User-Agent": "my-app/1.0"}) as client:
    # Buffered response
    r = client.get("https://httpbin.org/get")
    print(r.json())

    # POST with JSON body
    r = client.post("https://httpbin.org/post", json={"key": "value"})
    print(r.status_code)

    # Streaming response
    with client.stream("GET", "https://httpbin.org/stream-bytes/10000") as r:
        for chunk in r.iter_bytes():
            print(f"chunk: {len(chunk)} bytes")

Async client

import asyncio
import eggfetch

async def main():
    async with eggfetch.AsyncClient() as client:
        r = await client.get("https://httpbin.org/get")
        print(r.status_code)

        # Concurrent requests
        responses = await asyncio.gather(
            client.get("https://httpbin.org/get"),
            client.get("https://httpbin.org/ip"),
        )
        for resp in responses:
            print(resp.json())

asyncio.run(main())

Configuration

import eggfetch

client = eggfetch.Client(
    timeout=10.0,
    headers={"User-Agent": "my-app/1.0"},
    limits=eggfetch.Limits(max_connections=100),
    verify="/path/to/ca-bundle.pem",        # custom CA bundle
    cert=("/path/to/cert.pem", "/path/to/key.pem"),  # mTLS
    proxy="http://proxy:8080",
    http2=True,
)

HTTPX-compatible facade

from eggfetch.compat.httpx import Client, AsyncClient

# HTTPX 0.28.1 asyncio-compatible facade
client = Client()
response = client.get("https://example.com")

See docs/python/guide.md for the full Python API reference.

Usage -- Rust

Basic requests

use eggfetch_core::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new();

    // GET request
    let resp = client.get("https://httpbin.org/get").send().await?;
    println!("Status: {}", resp.status());
    println!("Body: {}", resp.text().await?);

    // POST with JSON
    let resp = client
        .post("https://httpbin.org/post")
        .header("Content-Type", "application/json")
        .body(r#"{"key": "value"}"#)
        .send()
        .await?;
    println!("Status: {}", resp.status());

    Ok(())
}

Builder pattern

use eggfetch_core::{Client, Timeout};

let client = Client::builder()
    .timeout(Timeout::from_secs(30))
    .follow_redirects(true)
    .max_redirects(5)
    .user_agent("my-app/1.0")
    .automatic_decompression(true)
    .build();

let resp = client
    .get("https://httpbin.org/get")?
    .header("accept", "application/json")
    .query("page", "1")
    .send()
    .await?;

Streaming

use eggfetch_core::Client;
use futures_util::StreamExt;

let mut resp = client.get("https://httpbin.org/stream/3").send().await?;
let mut stream = resp.bytes_stream()?;

while let Some(chunk) = stream.next().await {
    let chunk = chunk?;
    println!("chunk: {} bytes", chunk.len());
}

Feature flags

[dependencies]
eggfetch-core = { version = "0.1", features = [
    "http1",          # HTTP/1.1 (default)
    "http2",          # HTTP/2 via ALPN
    "tls-rustls",     # TLS via rustls (default)
    "cookies",        # RFC 6265 cookie jar
    "proxy",          # HTTP proxy and CONNECT tunneling
    "compression-gzip",
    "compression-brotli",
    "compression-zstd",
    "compression-deflate",
    "multipart",      # streaming multipart/form-data
] }

See docs/rust/guide.md for the full Rust API reference.

Usage -- CLI

# GET request
eggfetch https://httpbin.org/get

# POST JSON
eggfetch -X POST https://httpbin.org/post --json '{"key": "value"}'

# With authentication
eggfetch --auth user:pass https://httpbin.org/basic-auth/user/pass

# Streaming download
eggfetch --output file.bin https://httpbin.org/stream-bytes/10000

# Machine-readable output
eggfetch --json-output https://httpbin.org/get

See docs/cli/guide.md for the full CLI reference.

HTTPX Compatibility

eggfetch provides two versioned, independent compatibility facades over the single Rust engine. They coexist and never mutate each other.

HTTPX 0.28.1 (eggfetch.compat.httpx)

An asyncio-compatible facade targeting HTTPX 0.28.1. See compat/httpx/0.28.1/profile.toml for the pinned compatibility profile.

Key differences from HTTPX:

• Trio/AnyIO not supported (asyncio only, tokio-based)
• Python 3.8/3.9 not supported (requires 3.10+)
• ssl_context and proxy ssl_context are translated through the safe rustls boundary when representable; contexts with unrepresentable cipher, ALPN, TLS-version, or client-certificate provenance fail closed with TypeError
• HTTPX timeout values map to connect, read, write, and pool only; the facade does not synthesize EggFetch's native total deadline
• HTTPX Timeout preserves omitted versus explicitly disabled (None) phase values; Timeout() without a scalar or all four phases raises as in HTTPX
• Core proxy configuration is explicit; the HTTPX compatibility facade honors scheme-specific HTTP_PROXY/HTTPS_PROXY with ALL_PROXY fallback, lowercase forms, NO_PROXY, and trust_env=False
• Redirects with buffered retained bodies replay correctly; arbitrary one-shot body iterators are rejected before the next hop
• Request-local cookies and explicit Cookie headers are preserved within the facade jar model
• Response streaming is asyncio-compatible and supports incremental text decoding and chunk-size control
• Proxy(headers=...) is forwarded on the proxy leg only and is never sent through a CONNECT tunnel or to the origin; sensitive values are redacted in diagnostic representations
• HTTP/2-only works for direct TLS, cleartext prior knowledge, the SNI override route, the SOCKS HTTPS route, and the specialized direct/UDS paths. HTTP CONNECT proxy origin framing remains HTTP/1.1, and HTTP/2 stream_id remains metadata-only and unavailable
• Sync trace callbacks work on both Client and AsyncClient; coroutine trace callbacks are rejected with TypeError before dispatch because the core TraceObserver is synchronous

Remaining differences are documented in compat/httpx/0.28.1/allowed-differences.toml; the compatibility claim is limited to the pinned HTTPX 0.28.1 profile and the supported asyncio surface.

HTTPX2 2.12.0 (eggfetch.compat.httpx2)

A sibling facade for the maintained httpx2==2.12.0 line, pinned in compat/httpx2/2.12.0/ with its own reference manifest and allowed-difference ledger. It reuses the same Rust engine and shared compatibility helpers where semantics are identical; HTTPX2-specific semantics live behind explicit profile boundaries and never leak into eggfetch.compat.httpx.

from eggfetch.compat.httpx2 import Client, AsyncClient

client = Client()  # httpx2 2.12.0 surface: FunctionAuth, Origin, QUERY, SSE, optional WS

New surface vs 0.28.1: FunctionAuth, Origin + URL.origin, QUERY (query top-level + Client.query/AsyncClient.query), Headers merge operators (|/|=), alias_httpx() (explicit opt-in only), truststore OS-trust default, RFC 9110 status renames, plus SSE (EventSource over streamed responses) and optional WebSocket (wsproto framing over the existing 101 network_stream — no second socket/TLS stack, pip install httpx2[ws] for WS support). Python 3.10–3.13 distribution scope.

HTTPX 1.0 preview (no compatibility promise)

compat/httpx/1.0-preview/ tracks the HTTPX 1.0 redesign as reconnaissance only. No dev release is a supported contract; see that directory for the observed version, delta notes, and entry criteria.

101 Switching Protocols and network_stream

When a request upgrades (e.g. WebSocket via Connection: Upgrade), the response carries a live owned NetworkStream exposed through response.extensions["network_stream"]. The wrapper type follows the caller's API mode: sync Client.stream() and Client.request() 101 responses expose the sync NetworkStream; async AsyncClient.stream() and AsyncClient.request() 101 responses expose the async wrapper. For ordinary pooled HTTP/1.1 and HTTP/2 connections, network_stream is None; the connection is returned to the pool and not user-writable. Internal HTTPS proxy CONNECT tunnels are also classified as None — the canonical access path is the body iterator, not the upgraded stream.

The NetworkStream object supports read, write, close, is_upgraded, get_extra_info, and start_tls(ssl_context=..., server_hostname=..., timeout=...). start_tls uses the same safe TLS policy as the default client; it is rejected for Hyper-opaque adapter streams and for streams that are already TLS-wrapped. Leading bytes written immediately after the 101 headers are returned by the first reads from the upgraded stream.

See docs/reference/compatibility.md for the full feature matrix.

Documentation

Security

eggfetch follows a security-hardening program covering dependencies, TLS, redirects, auth, cookies, proxies, decompression, multipart, retries, and protocol handling.

• Dependency auditing: cargo-deny configured in deny.toml
• Secret redaction: All Debug/Display/error output redacts credentials, cookies, bearer tokens, and proxy passwords
• Threat model: See docs/architecture/threat-model.md
• Vulnerability reporting: See SECURITY.md

License

eggfetch is licensed under the MIT License.

MSRV

The minimum supported Rust version is 1.80, declared in workspace.package.rust-version and checked in extended validation. rust-toolchain.toml pins the stable channel for development.