# eggfetch
[](https://github.com/eggstack/eggfetch/actions/workflows/ci.yml)
[](https://crates.io/crates/eggfetch-core)
[](https://crates.io/crates/eggfetch-core)
[](https://pypi.org/project/eggfetch/)
[](https://pepy.tech/projects/eggfetch)
[](LICENSE-MIT)
eggfetch is a Rust-native HTTP client engine with Python bindings and a CLI tool. The core is async-first: a Rust engine built on tokio and hyper provides connection pooling, phase-aware timeouts, TLS configuration, streaming, and response decompression. The Python bindings expose both sync and async APIs; the sync API blocks on the async engine while releasing the GIL, and the async API integrates with asyncio. There is exactly one networking implementation, living entirely in Rust.
## Features
- **HTTP/1.1, HTTP/2, HTTP/3** -- ALPN negotiation, multiplexed connections, experimental QUIC transport (bounded per-origin cache, shared connect budget with address fallback, phase-correct timeouts, keepalive-derived idle, authenticated Alt-Svc discovery with suppression/safe fallback/draining; retained experimental — see [`docs/architecture/core-tls-proxy-protocols.md`](docs/architecture/core-tls-proxy-protocols.md) § "Production Graduation Decision")
- **Streaming** -- request and response bodies stream without eager buffering; `bytes_stream()` and `text_lines()` for incremental reads
- **HTTP trailers** -- H1 chunked trailers, H2 trailing HEADERS, and H3 trailing headers captured without buffering (`Response::trailers()` after body EOF; H1 duplicate same-name trailers collapse upstream in hyper and are documented)
- **Response decompression** -- gzip, brotli, zstd, deflate via feature-gated streaming decoders
- **Connection pooling** -- semaphore-based logical in-flight request concurrency (`max_in_flight_requests*`, aliases `max_connections*` for pre-1.0) with per-origin limits, pool metrics, and separate transport observability counters
- **Transport observability** -- connector/DNS/TLS attempt counters, H3 creation/eviction counts, Alt-Svc learned/expired/cleared/rejected, H3 attempted/suppressed/fallback/drain/close/reconnect, and 101 upgrade counts where observable; HTTP/3 builds also expose bounded Quinn snapshots (remote address, RTT, path counters, route generation, and sanitized close codes); Hyper socket-reuse counts intentionally absent
- **Phase-aware timeouts** -- pool, connect, write, read, and total timeout phases with cancellation safety
- **TLS** -- rustls with custom CA bundles, client certificates (mTLS), version policy, and verification toggle
- **Proxy** -- HTTP forwarding, HTTPS CONNECT tunneling, proxy auth, per-request override, `NO_PROXY` bypass
- **Cookies** -- RFC 6265 cookie jar with domain/path matching, cross-origin stripping
- **Authentication** -- Basic and Bearer auth with credential redaction in all output paths
- **Multipart** -- streaming multipart/form-data with known-length optimization
- **Retries** -- policy-driven retries with exponential backoff and `Retry-After` support
- **Python API** -- requests/HTTPX-compatible sync and async interfaces, GIL-releasing blocking I/O
- **HTTPX compatibility facade** -- compatible asyncio surface targeting HTTPX 0.28.1 (`eggfetch.compat.httpx`)
- **Network stream exposure** -- 101 Switching Protocols responses expose an owned upgraded stream through `extensions["network_stream"]`; direct-connector upgrades carry real local/remote addrs and TLS version/cipher/ALPN, UDS upgrades report `Unix` without IPs, standard opaque upgrades remain explicitly unavailable; `start_tls` uses the same safe TLS translation as the default client
- **Python/FFI trailer policy** -- core retains trailers; Python native, HTTPX facade, FFI, and Node defer trailer exposure in this milestone (facade unchanged; HTTPX 0.28.1 has no `trailers` surface to compare against)
- **CLI** -- full-featured HTTP client with streaming output, machine-readable formats, and shell completions
- **Node.js (experimental prototype)** -- N-API binding with narrow guarantees (UTF-8 string bodies, buffered responses, unstructured errors, stub declarations); see [`docs/architecture/ffi-and-node.md`](docs/architecture/ffi-and-node.md)
HTTP/3 remains experimental. See [`docs/architecture/core-tls-proxy-protocols.md`](docs/architecture/core-tls-proxy-protocols.md) § "Production Graduation Decision" for the graduation gate and current status.
## Installation
**Python:**
```bash
pip install eggfetch
```
**Rust:**
```toml
[dependencies]
eggfetch-core = { version = "0.1", features = ["http1", "tls-rustls"] }
```
**CLI:**
```bash
cargo install eggfetch-cli
```
## Usage -- Python
### Quick requests
```python
import eggfetch
r = eggfetch.get("https://httpbin.org/get")
print(r.status_code)
print(r.text)
```
### Using a client
```python
import eggfetch
with eggfetch.Client(headers={"User-Agent": "my-app/1.0"}) as client:
# Buffered response
r = client.get("https://httpbin.org/get")
print(r.json())
# POST with JSON body
r = client.post("https://httpbin.org/post", json={"key": "value"})
print(r.status_code)
# Streaming response
with client.stream("GET", "https://httpbin.org/stream-bytes/10000") as r:
for chunk in r.iter_bytes():
print(f"chunk: {len(chunk)} bytes")
```
### Async client
```python
import asyncio
import eggfetch
async def main():
async with eggfetch.AsyncClient() as client:
r = await client.get("https://httpbin.org/get")
print(r.status_code)
# Concurrent requests
responses = await asyncio.gather(
client.get("https://httpbin.org/get"),
client.get("https://httpbin.org/ip"),
)
for resp in responses:
print(resp.json())
asyncio.run(main())
```
### Configuration
```python
import eggfetch
client = eggfetch.Client(
timeout=10.0,
headers={"User-Agent": "my-app/1.0"},
limits=eggfetch.Limits(max_connections=100),
verify="/path/to/ca-bundle.pem", # custom CA bundle
cert=("/path/to/cert.pem", "/path/to/key.pem"), # mTLS
proxy="http://proxy:8080",
http2=True,
)
```
### HTTPX-compatible facade
```python
from eggfetch.compat.httpx import Client, AsyncClient
# HTTPX 0.28.1 asyncio-compatible facade
client = Client()
response = client.get("https://example.com")
```
See [`docs/python/guide.md`](docs/python/guide.md) for the full Python API reference.
## Usage -- Rust
### Basic requests
```rust
use eggfetch_core::Client;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let client = Client::new();
// GET request
let resp = client.get("https://httpbin.org/get").send().await?;
println!("Status: {}", resp.status());
println!("Body: {}", resp.text().await?);
// POST with JSON
let resp = client
.post("https://httpbin.org/post")
.header("Content-Type", "application/json")
.body(r#"{"key": "value"}"#)
.send()
.await?;
println!("Status: {}", resp.status());
Ok(())
}
```
### Builder pattern
```rust
use eggfetch_core::{Client, Timeout};
let client = Client::builder()
.timeout(Timeout::from_secs(30))
.follow_redirects(true)
.max_redirects(5)
.user_agent("my-app/1.0")
.automatic_decompression(true)
.build();
let resp = client
.get("https://httpbin.org/get")?
.header("accept", "application/json")
.query("page", "1")
.send()
.await?;
```
### Streaming
```rust
use eggfetch_core::Client;
use futures_util::StreamExt;
let mut resp = client.get("https://httpbin.org/stream/3").send().await?;
let mut stream = resp.bytes_stream()?;
while let Some(chunk) = stream.next().await {
let chunk = chunk?;
println!("chunk: {} bytes", chunk.len());
}
```
### Feature flags
```toml
[dependencies]
eggfetch-core = { version = "0.1", features = [
"http1", # HTTP/1.1 (default)
"http2", # HTTP/2 via ALPN
"tls-rustls", # TLS via rustls (default)
"cookies", # RFC 6265 cookie jar
"proxy", # HTTP proxy and CONNECT tunneling
"compression-gzip",
"compression-brotli",
"compression-zstd",
"compression-deflate",
"multipart", # streaming multipart/form-data
] }
```
See [`docs/rust/guide.md`](docs/rust/guide.md) for the full Rust API reference.
## Usage -- CLI
```bash
# GET request
eggfetch https://httpbin.org/get
# POST JSON
eggfetch -X POST https://httpbin.org/post --json '{"key": "value"}'
# With authentication
eggfetch --auth user:pass https://httpbin.org/basic-auth/user/pass
# Streaming download
eggfetch --output file.bin https://httpbin.org/stream-bytes/10000
# Machine-readable output
eggfetch --json-output https://httpbin.org/get
```
See [`docs/cli/guide.md`](docs/cli/guide.md) for the full CLI reference.
## HTTPX Compatibility
eggfetch provides two versioned, independent compatibility facades over the
single Rust engine. They coexist and never mutate each other.
### HTTPX 0.28.1 (`eggfetch.compat.httpx`)
An asyncio-compatible facade targeting HTTPX 0.28.1. See
`compat/httpx/0.28.1/profile.toml` for the pinned compatibility profile.
Key differences from HTTPX:
- Trio/AnyIO not supported (asyncio only, tokio-based)
- Python 3.8/3.9 not supported (requires 3.10+)
- `ssl_context` and proxy `ssl_context` are translated through the safe rustls
boundary when representable; contexts with unrepresentable cipher, ALPN,
TLS-version, or client-certificate provenance fail closed with `TypeError`
- HTTPX timeout values map to `connect`, `read`, `write`, and `pool` only;
the facade does not synthesize EggFetch's native `total` deadline
- HTTPX `Timeout` preserves omitted versus explicitly disabled (`None`) phase
values; `Timeout()` without a scalar or all four phases raises as in HTTPX
- Core proxy configuration is explicit; the HTTPX compatibility facade honors
scheme-specific `HTTP_PROXY`/`HTTPS_PROXY` with `ALL_PROXY` fallback,
lowercase forms, `NO_PROXY`, and `trust_env=False`
- Redirects with buffered retained bodies replay correctly; arbitrary one-shot body iterators are rejected before the next hop
- Request-local cookies and explicit Cookie headers are preserved within the facade jar model
- Response streaming is asyncio-compatible and supports incremental text decoding and chunk-size control
- `Proxy(headers=...)` is forwarded on the proxy leg only and is never sent
through a CONNECT tunnel or to the origin; sensitive values are redacted in
diagnostic representations
- HTTP/2-only works for direct TLS, cleartext prior knowledge, the SNI override
route, the SOCKS HTTPS route, and the specialized direct/UDS paths. HTTP
CONNECT proxy origin framing remains HTTP/1.1, and HTTP/2 `stream_id` remains
metadata-only and unavailable
- Sync trace callbacks work on both `Client` and `AsyncClient`; coroutine
trace callbacks are rejected with `TypeError` before dispatch because
the core `TraceObserver` is synchronous
Remaining differences are documented in `compat/httpx/0.28.1/allowed-differences.toml`; the compatibility claim is limited to the pinned HTTPX 0.28.1 profile and the supported asyncio surface.
### HTTPX2 2.12.0 (`eggfetch.compat.httpx2`)
A sibling facade for the maintained `httpx2==2.12.0` line, pinned in
`compat/httpx2/2.12.0/` with its own reference manifest and
allowed-difference ledger. It reuses the same Rust engine and shared
compatibility helpers where semantics are identical; HTTPX2-specific
semantics live behind explicit profile boundaries and never
leak into `eggfetch.compat.httpx`.
```python
from eggfetch.compat.httpx2 import Client, AsyncClient
client = Client() # httpx2 2.12.0 surface: FunctionAuth, Origin, QUERY, SSE, optional WS
```
New surface vs 0.28.1: `FunctionAuth`, `Origin` + `URL.origin`,
`QUERY` (`query` top-level + `Client.query`/`AsyncClient.query`),
`Headers` merge operators (`|`/`|=`), `alias_httpx()` (explicit
opt-in only), truststore OS-trust default, RFC 9110 status renames,
plus SSE (`EventSource` over streamed responses) and optional WebSocket
(wsproto framing over the existing 101 `network_stream` — no second
socket/TLS stack, `pip install httpx2[ws]` for WS support).
Python 3.10–3.13 distribution scope.
### HTTPX 1.0 preview (no compatibility promise)
`compat/httpx/1.0-preview/` tracks the HTTPX 1.0 redesign as
reconnaissance only. No dev release is a supported contract; see that
directory for the observed version, delta notes, and entry criteria.
### 101 Switching Protocols and `network_stream`
When a request upgrades (e.g. WebSocket via `Connection: Upgrade`), the
response carries a live owned `NetworkStream` exposed through
`response.extensions["network_stream"]`. The wrapper type follows the
caller's API mode: sync `Client.stream()` and `Client.request()` 101
responses expose the sync `NetworkStream`; async `AsyncClient.stream()`
and `AsyncClient.request()` 101 responses expose the async wrapper.
For ordinary pooled HTTP/1.1 and HTTP/2 connections, `network_stream` is
`None`; the connection is returned to the pool and not user-writable.
Internal HTTPS proxy CONNECT tunnels are also classified as `None` — the
canonical access path is the body iterator, not the upgraded stream.
The `NetworkStream` object supports `read`, `write`, `close`, `is_upgraded`,
`get_extra_info`, and `start_tls(ssl_context=..., server_hostname=...,
timeout=...)`. `start_tls` uses the same safe TLS policy as the default
client; it is rejected for Hyper-opaque adapter streams and for streams that
are already TLS-wrapped. Leading bytes written immediately after the 101
headers are returned by the first reads from the upgraded stream.
See [`docs/reference/compatibility.md`](docs/reference/compatibility.md) for the full feature matrix.
## Documentation
| [getting-started/](docs/getting-started/) | Installation and quickstart guide |
| [concepts/](docs/concepts/) | Architecture, lifecycle, timeouts, streaming, cookies, auth, proxy, TLS |
| [rust/guide.md](docs/rust/guide.md) | Rust API guide with examples |
| [python/guide.md](docs/python/guide.md) | Python sync/async API guide |
| [cli/guide.md](docs/cli/guide.md) | CLI reference and usage guide |
| [migration/](docs/migration/) | Migration guides from requests and HTTPX |
| [cookbook/](docs/cookbook/) | Practical runnable examples |
| [reference/](docs/reference/) | Compatibility matrix, feature matrix, error reference |
| [security/](docs/security/) | Security guidelines and troubleshooting |
| [architecture/](docs/architecture/) | Internal architecture documentation |
| [ffi/](docs/ffi/) | C ABI and FFI binding guide |
## Security
eggfetch follows a security-hardening program covering dependencies, TLS, redirects, auth, cookies, proxies, decompression, multipart, retries, and protocol handling.
- **Dependency auditing:** `cargo-deny` configured in `deny.toml`
- **Secret redaction:** All `Debug`/`Display`/error output redacts credentials, cookies, bearer tokens, and proxy passwords
- **Threat model:** See [docs/architecture/threat-model.md](docs/architecture/threat-model.md)
- **Vulnerability reporting:** See [SECURITY.md](SECURITY.md)
## License
eggfetch is licensed under the [MIT License](LICENSE-MIT).
## MSRV
The minimum supported Rust version is **1.80**, declared in `workspace.package.rust-version` and checked in extended validation. `rust-toolchain.toml` pins the stable channel for development.