hpx 0.1.3

High Performance HTTP Client
Documentation

hpx

An ergonomic all-in-one HTTP client for browser emulation with TLS, JA3/JA4, and HTTP/2 fingerprints.

Additional learning resources include:

Emulation

The emulation module provides a way to simulate various browser TLS/HTTP2 fingerprints.

use hpx_util::Emulation;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Use the API you're already familiar with
    let resp = hpx::get("https://tls.peet.ws/api/all")
        .emulation(Emulation::Firefox136)
        .send()
        .await?;
    println!("{}", resp.text().await?);

    Ok(())
}

Websocket

The websocket module provides a way to upgrade a connection to a websocket.

# #[cfg(feature = "ws")]
# #[tokio::main]
# async fn main() -> hpx::Result<()> {
use futures_util::{SinkExt, StreamExt, TryStreamExt};
use hpx::{header, ws::message::Message};

// Use the API you're already familiar with
let websocket = hpx::websocket("wss://echo.websocket.org")
    .header(header::USER_AGENT, env!("CARGO_PKG_NAME"))
    .send()
    .await?;

assert_eq!(websocket.version(), http::Version::HTTP_11);

let (mut tx, mut rx) = websocket.into_websocket().await?.split();

tokio::spawn(async move {
    for i in 1..11 {
        if let Err(err) = tx.send(Message::text(format!("Hello, World! {i}"))).await {
            eprintln!("failed to send message: {err}");
        }
    }
});

while let Some(message) = rx.try_next().await? {
    if let Message::Text(text) = message {
        println!("received: {text}");
    }
}

Ok(())
# }
# #[cfg(not(feature = "ws"))]
# fn main() {}

Making a GET request

Making a GET request is simple.

# async fn run() -> hpx::Result<()> {
let body = hpx::get("https://www.rust-lang.org")
    .send()
    .await?
    .text()
    .await?;

println!("body = {:?}", body);
# Ok(())
# }

NOTE: If you plan to perform multiple requests, it is best to create a Client and reuse it, taking advantage of keep-alive connection pooling.

Making POST requests (or setting request bodies)

There are several ways you can set the body of a request. The basic one is by using the body() method of a RequestBuilder. This lets you set the exact raw bytes of what the body should be. It accepts various types, including String and Vec<u8>. If you wish to pass a custom type, you can use the hpx::Body constructors.

# use hpx::Error;
#
# async fn run() -> Result<(), Error> {
let client = hpx::Client::new();
let res = client
    .post("http://httpbin.org/post")
    .body("the exact body that is sent")
    .send()
    .await?;
# Ok(())
# }

Forms

It's very common to want to send form data in a request body. This can be done with any type that can be serialized into form data.

This can be an array of tuples, or a HashMap, or a custom type that implements Serialize.

The feature form is required.

# use hpx::Error;
# #[cfg(feature = "form")]
# async fn run() -> Result<(), Error> {
// This will POST a body of `foo=bar&baz=quux`
let params = [("foo", "bar"), ("baz", "quux")];
let client = hpx::Client::new();
let res = client
    .post("http://httpbin.org/post")
    .form(&params)
    .send()
    .await?;
# Ok(())
# }

JSON

There is also a json method helper on the RequestBuilder that works in a similar fashion the form method. It can take any value that can be serialized into JSON. The feature json is required.

# use hpx::Error;
# use std::collections::HashMap;
#
# #[cfg(feature = "json")]
# async fn run() -> Result<(), Error> {
// This will POST a body of `{"lang":"rust","body":"json"}`
let mut map = HashMap::new();
map.insert("lang", "rust");
map.insert("body", "json");

let client = hpx::Client::new();
let res = client
    .post("http://httpbin.org/post")
    .json(&map)
    .send()
    .await?;
# Ok(())
# }

Redirect Policies

By default, the client does not handle HTTP redirects. To customize this behavior, you can use redirect::Policy with ClientBuilder.

Cookies

The automatic storing and sending of session cookies can be enabled with the [cookie_store][ClientBuilder::cookie_store] method on ClientBuilder.

Proxies

NOTE: System proxies are enabled by default.

System proxies look in environment variables to set HTTP or HTTPS proxies.

HTTP_PROXY or http_proxy provide HTTP proxies for HTTP connections while HTTPS_PROXY or https_proxy provide HTTPS proxies for HTTPS connections. ALL_PROXY or all_proxy provide proxies for both HTTP and HTTPS connections. If both the all proxy and HTTP or HTTPS proxy variables are set the more specific HTTP or HTTPS proxies take precedence.

These can be overwritten by adding a [Proxy] to ClientBuilder i.e. let proxy = hpx::Proxy::http("https://secure.example")?; or disabled by calling ClientBuilder::no_proxy().

socks feature is required if you have configured socks proxy like this:

export https_proxy=socks5://127.0.0.1:1086
  • http:// is the scheme for http proxy
  • https:// is the scheme for https proxy
  • socks4:// is the scheme for socks4 proxy
  • socks4a:// is the scheme for socks4a proxy
  • socks5:// is the scheme for socks5 proxy
  • socks5h:// is the scheme for socks5h proxy

TLS

By default, clients will utilize BoringSSL transport layer security to connect to HTTPS targets.

  • Various parts of TLS can also be configured or even disabled on the ClientBuilder.

Certificate Store

By default, hpx uses Mozilla's root certificates through the webpki-roots crate. This static root certificate bundle is not automatically updated and ignores any root certificates installed on the host. You can disable default-features to use the system's default certificate path. Additionally, hpx provides a certificate store for users to customize and update certificates.

Custom Certificate Store verification supports Root CA certificates, peer certificates, and self-signed certificate SSL pinning.

Optional Features

The following are a list of Cargo features that can be enabled or disabled:

  • cookies: Provides cookie session support.
  • gzip: Provides response body gzip decompression.
  • brotli: Provides response body brotli decompression.
  • zstd: Provides response body zstd decompression.
  • deflate: Provides response body deflate decompression.
  • query: Provides query parameter serialization.
  • form: Provides form data serialization.
  • json: Provides serialization and deserialization for JSON bodies.
  • multipart: Provides functionality for multipart forms.
  • charset: Improved support for decoding text.
  • stream: Adds support for futures::Stream.
  • socks: Provides SOCKS5 and SOCKS4 proxy support.
  • ws: Provides websocket support.
  • hickory-dns: Enables a hickory-dns async resolver instead of default threadpool using getaddrinfo.
  • webpki-roots (enabled by default): Use the webpki-roots crate for root certificates.
  • system-proxy: Enable system proxy support.
  • tracing: Enable tracing logging support.