proxy_http 0.1.6

High-performance, asynchronous HTTP proxy server in Rust, routing traffic through dynamic upstream proxies. / 一个使用 Rust 构建的高性能、异步 HTTP 代理服务器，可通过动态上游代理池路由流量。
Documentation

English | 中文

English

Introduction

proxy_http is a lightweight, high-performance, asynchronous HTTP proxy server built with Rust. Its primary function is to route outgoing traffic through a dynamic pool of upstream proxy servers, fetched from external subscription links. It is designed for scenarios that require robust IP rotation, enhanced user anonymity, or bypassing network restrictions. The server also includes basic authentication to ensure controlled access. It can be used as a library in your Rust projects or run as a standalone binary.

Tech Stack

This project leverages modern, efficient, and battle-tested Rust libraries to achieve high performance and reliability:

  • Tokio: An asynchronous runtime for writing fast and reliable network applications.
  • Hyper: A fast, safe, and correct HTTP implementation for Rust, serving as the foundation of the proxy server.
  • proxy-fetch: A utility to fetch and manage lists of upstream proxy providers from subscription URLs.
  • Reqwest: An ergonomic, higher-level HTTP client used for integration testing.
  • ThisError: A library for creating descriptive, structured error types, simplifying error handling.

Installation and Configuration

You can run this project as a standalone application.

Installation

Install the binary using cargo:

cargo install proxy_http

Configuration

The application is configured via environment variables.

  • PROXY_SUBSCRITION_URL: Required. A semicolon-separated list of subscription URLs for fetching upstream proxy servers.
  • PROXY_USER: Required. The username for clients to authenticate with this proxy server.
  • PROXY_PASSWORD: Required. The password for clients to authenticate.
  • PORT: Optional. The port for the proxy server to listen on. Defaults to 15080.

Running

Once installed and configured, simply run the binary:

proxy_http

The server will start and listen on the configured port (e.g., 0.0.0.0:15080).

Library Usage

The following examples, adapted from the integration tests in tests/main.rs, demonstrate how to use proxy_http as a library.

Example 1: Standard HTTP Proxy Request

This test initializes the server, sends a request with incorrect credentials to verify authentication, and then sends a successful request.

#[tokio::test]
async fn test_proxy() -> Void {
  // Load upstream proxies from an environment variable
  let fetch = proxy_fetch::load(PROXY_SUBSCRITION_URL.split(";")).await?;

  let user = "test";
  let password = "pwd";
  let port = 32342;
  let addr = SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), port));

  // Run the proxy server in a background task
  tokio::spawn(async move {
    xerr::log!(proxy_http::run(fetch, addr, user, &password).await);
  });
  tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;

  let url = "http://ifconfig.me/ip";

  // 1. Test with incorrect credentials
  let client_fail = reqwest::Client::builder()
    .proxy(reqwest::Proxy::http(format!("http://test:err@127.0.0.1:{port}"))?)
    .build()?;
  let res_fail = client_fail.get(url).send().await?;
  assert_eq!(res_fail.status(), reqwest::StatusCode::PROXY_AUTHENTICATION_REQUIRED);
  info!("✅ Proxy Authentication Required");

  // 2. Test with correct credentials
  let client_ok = reqwest::Client::builder()
    .proxy(reqwest::Proxy::http(format!("http://{user}:{password}@127.0.0.1:{port}"))?)
    .build()?;
  let res_ok = client_ok.get(url).send().await?;
  let ip = res_ok.text().await?;
  info!("IP via proxy: {ip}");
  assert!(!ip.is_empty());

  OK
}

Example 2: HTTP Tunnel (CONNECT)

This test establishes an HTTP tunnel for a CONNECT request, which is typically used for HTTPS traffic.

#[tokio::test]
async fn test_tunnel_proxy() -> Void {
  // Server setup is similar to the above...
  let fetch = proxy_fetch::load(PROXY_SUBSCRITION_URL.split(";")).await?;
  let user = "test";
  let password = "pwd";
  let port = 32343; // Use a different port
  let addr = SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), port));
  tokio::spawn(async move {
    xerr::log!(proxy_http::run(fetch, addr, user, &password).await);
  });
  tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;

  let target_host = "ifconfig.me";
  let target_port = 80;

  // 1. Connect directly to the proxy server
  let mut stream = TcpStream::connect(format!("127.0.0.1:{}", port)).await?;

  // 2. Send a CONNECT request to establish the tunnel
  let connect_request = format!(
    "CONNECT {}:{} HTTP/1.1\\r\\n\
     Host: {}: {}\r\n\
     Proxy-Authorization: Basic {}\r\n\
     \r\n",
    target_host, target_port,
    target_host, target_port,
    base64::general_purpose::STANDARD.encode(format!("{}:{}", user, password))
  );
  stream.write_all(connect_request.as_bytes()).await?;

  // 3. Read the response to confirm the tunnel is established (HTTP/1.1 200)
  let mut response = vec![0u8; 1024];
  stream.read(&mut response).await?;
  assert!(String::from_utf8_lossy(&response).contains("200"));
  info!("✅ Tunnel established successfully");

  // 4. Send an actual HTTP GET request through the established tunnel
  let http_request = "GET /ip HTTP/1.1\\r\\nHost: ifconfig.me\\r\\n\\r\\n";
  stream.write_all(http_request.as_bytes()).await?;

  // 5. Read the final response from the target server
  let mut final_response = Vec::new();
  stream.read_to_end(&mut final_response).await?;
  info!("Response via tunnel: {}", String::from_utf8_lossy(&final_response));

  OK
}

Design Philosophy

The design of proxy_http adheres to several core principles: Asynchronous, Modular, and Minimalist. The request handling process clearly illustrates this design.

Request Flow

The server handles two main types of requests: standard HTTP proxy requests and CONNECT tunnel requests.

  1. Connection Acceptance (run.rs): The run function binds a TcpListener to the specified address. It enters a loop, accepting incoming TCP connections. For each connection, it spawns a new asynchronous Tokio task to handle it concurrently.

  2. Request Handling (handle.rs): Inside the task, hyper serves the connection. For each incoming HTTP request, the central handle function is called.

  3. Authentication (is_authorized.rs): The handle function first calls is_authorized to check the Proxy-Authorization header. If authentication fails, it immediately returns a 407 Proxy Authentication Required response.

  4. Request Routing (handle.rs):

    • For CONNECT requests: The handle function identifies the CONNECT method. It initiates a protocol upgrade on the connection via hyper::upgrade::on. The resulting upgraded TCP stream (a tunnel) is passed to the upgrade::upgrade function.
    • For standard HTTP requests: The handle function passes the request directly to the proxy::proxy function.

  5. Forwarding Logic:

    • Inside a Tunnel (upgrade.rs): The upgrade function serves the established tunnel. It reads subsequent requests from the client within the tunnel and uses the proxy::proxy function to forward them to their destination through an upstream proxy.
    • Standard Forwarding (proxy.rs): The proxy function is the core forwarding engine. It takes the request, cleans up proxy-specific headers, and uses the proxy_fetch instance to execute the request against a chosen upstream proxy. It then reconstructs the response and sends it back to the client.

This modular flow ensures a clear separation of concerns: run manages connections, handle authenticates and routes, upgrade manages tunnels, and proxy forwards the traffic.

File Structure

The project is organized to promote clarity and separation of concerns:

  • Cargo.toml: Defines project metadata, dependencies, and build settings.
  • src/main.rs: The main entry point for the executable, responsible for initializing and running the proxy server.
  • src/lib.rs: The library's root, which declares and exposes the server's core modules.
  • src/error.rs: Defines custom, structured error types for the application using thiserror.
  • src/handle.rs: The main request handler. It performs authentication and routes requests to either the standard proxy logic (proxy.rs) or the CONNECT tunnel logic (upgrade.rs).
  • src/is_authorized.rs: Implements the Proxy-Authorization header check.
  • src/proxy.rs: Contains the core request forwarding logic. It prepares and sends outgoing requests to an upstream proxy provider, and is used by both the standard proxy and the CONNECT tunnel handlers.
  • src/run.rs: Contains the main run function that binds the server to a socket and spawns tasks for each incoming connection.
  • src/upgrade.rs: Handles CONNECT requests by setting up a TCP tunnel to the destination and forwarding traffic.
  • tests/main.rs: Contains integration tests that verify the end-to-end functionality of the proxy server.

A Little History

The concept of proxy servers is nearly as old as the web itself. The first proxies were developed at CERN in the early 1990s, around the same time Tim Berners-Lee was creating the World Wide Web. Initially, they were used as "gateways" to handle different protocols, but their role quickly evolved into caching, which was crucial for reducing traffic on the slow and expensive international networks of the time. This simple yet powerful idea of an intermediary has since become a fundamental building block of modern network architecture, enabling everything from security and content filtering to the very distributed systems this project relies on.

中文

项目简介

proxy_http 是一个使用 Rust 构建的轻量级、高性能、异步 HTTP 代理服务器。其核心功能是通过从外部订阅链接获取的动态上游代理服务器池来路由出站流量。它专为需要 IP 轮换、增强用户匿名性或绕过网络限制的场景而设计。服务器还包含基本身份验证功能，以确保访问受控。它既可以作为库在您的 Rust 项目中使用，也可以作为独立的可执行文件运行。

技术栈

本项目利用了现代化、高效且经过实战检验的 Rust 库，以实现卓越的性能和可靠性：

  • Tokio: 一个用于编写快速、可靠网络应用的异步运行时。
  • Hyper: 一个快速、安全且正确的 Rust HTTP 实现，是本代理服务器的基石。
  • proxy-fetch: 一个用于从订阅 URL 获取和管理上游代理提供者列表的工具库。
  • Reqwest: 一个符合人体工程学的高级 HTTP 客户端，用于集成测试。
  • ThisError: 一个用于创建描述性、结构化错误类型的库，简化了错误处理。

安装与配置

您可以将此项目作为独立的应用程序运行。

安装

使用 cargo 安装二进制文件：

cargo install proxy_http

配置

该应用程序通过环境变量进行配置。

  • PROXY_SUBSCRITION_URL: 必需。用于获取上游代理服务器的订阅 URL，多个 URL 之间用分号分隔。
  • PROXY_USER: 必需。供客户端连接此代理服务器时进行身份验证的用户名。
  • PROXY_PASSWORD: 必需。供客户端进行身份验证的密码。
  • PORT: 可选。代理服务器监听的端口。默认为 15080。

运行

安装和配置完成后，直接运行二进制文件即可：

proxy_http

服务器将启动并监听配置的端口（例如 0.0.0.0:15080）。

作为库使用

以下示例改编自 tests/main.rs 中的集成测试，演示了如何将 proxy_http 作为库来使用。

示例一：标准 HTTP 代理请求

此测试初始化服务器，首先用错误的凭据发送请求以验证身份验证功能，然后发送一个成功的请求。

#[tokio::test]
async fn test_proxy() -> Void {
  // 从环境变量加载上游代理
  let fetch = proxy_fetch::load(PROXY_SUBSCRITION_URL.split(";")).await?;

  let user = "test";
  let password = "pwd";
  let port = 32342;
  let addr = SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), port));

  // 在后台任务中运行代理服务器
  tokio::spawn(async move {
    xerr::log!(proxy_http::run(fetch, addr, user, &password).await);
  });
  tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;

  let url = "http://ifconfig.me/ip";

  // 1. 使用错误凭据进行测试
  let client_fail = reqwest::Client::builder()
    .proxy(reqwest::Proxy::http(format!("http://test:err@127.0.0.1:{port}"))?)
    .build()?;
  let res_fail = client_fail.get(url).send().await?;
  assert_eq!(res_fail.status(), reqwest::StatusCode::PROXY_AUTHENTICATION_REQUIRED);
  info!("✅ 需要代理身份验证");

  // 2. 使用正确凭据进行测试
  let client_ok = reqwest::Client::builder()
    .proxy(reqwest::Proxy::http(format!("http://{user}:{password}@127.0.0.1:{port}"))?)
    .build()?;
  let res_ok = client_ok.get(url).send().await?;
  let ip = res_ok.text().await?;
  info!("通过代理获取的 IP: {ip}");
  assert!(!ip.is_empty());

  OK
}

示例二：HTTP 隧道 (CONNECT)

此测试为 CONNECT 请求建立 HTTP 隧道，该请求通常用于 HTTPS 流量。

#[tokio::test]
async fn test_tunnel_proxy() -> Void {
  // 服务器设置与上一个示例类似...
  let fetch = proxy_fetch::load(PROXY_SUBSCRITION_URL.split(";")).await?;
  let user = "test";
  let password = "pwd";
  let port = 32343; // 使用不同端口
  let addr = SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), port));
  tokio::spawn(async move {
    xerr::log!(proxy_http::run(fetch, addr, user, &password).await);
  });
  tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;

  let target_host = "ifconfig.me";
  let target_port = 80;

  // 1. 直接连接到代理服务器
  let mut stream = TcpStream::connect(format!("127.0.0.1:{}", port)).await?;

  // 2. 发送 CONNECT 请求以建立隧道
  let connect_request = format!(
    "CONNECT {}:{} HTTP/1.1\\r\\n\
     Host: {}: {}\r\n\
     Proxy-Authorization: Basic {}\r\n\
     \r\n",
    target_host, target_port,
    target_host, target_port,
    base64::general_purpose::STANDARD.encode(format!("{}:{}", user, password))
  );
  stream.write_all(connect_request.as_bytes()).await?;

  // 3. 读取响应以确认隧道已建立 (HTTP/1.1 200)
  let mut response = vec![0u8; 1024];
  stream.read(&mut response).await?;
  assert!(String::from_utf8_lossy(&response).contains("200"));
  info!("✅ 隧道建立成功");

  // 4. 通过已建立的隧道发送实际的 HTTP GET 请求
  let http_request = "GET /ip HTTP/1.1\\r\\nHost: ifconfig.me\\r\\n\\r\\n";
  stream.write_all(http_request.as_bytes()).await?;

  // 5. 从目标服务器读取最终响应
  let mut final_response = Vec::new();
  stream.read_to_end(&mut final_response).await?;
  info!("通过隧道得到的响应: {}", String::from_utf8_lossy(&final_response));

  OK
}

设计思路

proxy_http 的设计遵循几个核心原则：异步、模块化和极简主义。请求处理流程清晰地展示了这一设计思想。

请求流程

服务器主要处理两种类型的请求：标准 HTTP 代理请求和 CONNECT 隧道请求。

  1. 连接接收 (run.rs): run 函数将一个 TcpListener 绑定到指定地址。它进入一个循环，接收传入的 TCP 连接。对于每个连接，它会生成一个新的异步 Tokio 任务来并发处理。

  2. 请求处理 (handle.rs): 在任务内部，hyper 负责处理该连接。对于每个传入的 HTTP 请求，都会调用中央的 handle 函数。

  3. 身份验证 (is_authorized.rs): handle 函数首先调用 is_authorized 来检查 Proxy-Authorization 头。如果身份验证失败，它会立即返回 407 Proxy Authentication Required 响应。

  4. 请求路由 (handle.rs):

    • 对于 CONNECT 请求: handle 函数识别 CONNECT 方法。它通过 hyper::upgrade::on 在连接上启动协议升级。由此产生的升级后的 TCP 流（一个隧道）被传递给 upgrade::upgrade 函数。
    • 对于标准 HTTP 请求: handle 函数将请求直接传递给 proxy::proxy 函数。

  5. 转发逻辑:

    • 在隧道内部 (upgrade.rs): upgrade 函数负责处理已建立的隧道。它从客户端读取隧道内的后续请求，并使用 proxy::proxy 函数通过上游代理将它们转发到目的地。
    • 标准转发 (proxy.rs): proxy 函数是核心的转发引擎。它接收请求，清理代理特定的头部，并使用 proxy_fetch 实例通过选定的上游代理执行请求。然后，它重新构建响应并将其发送回客户端。

这种模块化的流程确保了清晰的关注点分离：run 管理连接，handle 进行身份验证和路由，upgrade 管理隧道，而 proxy 负责转发流量。

文件结构

项目结构清晰，旨在促进关注点分离：

  • Cargo.toml: 定义项目元数据、依赖项和构建配置。
  • src/main.rs: 可执行文件的主入口点，负责初始化和运行代理服务器。
  • src/lib.rs: 库的根模块，声明并导出服务器的核心模块。
  • src/error.rs: 使用 thiserror 为应用程序定义自定义的结构化错误类型。
  • src/handle.rs: 主请求处理器。它执行身份验证，并将请求路由到标准代理逻辑（proxy.rs）或 CONNECT 隧道逻辑（upgrade.rs）。
  • src/is_authorized.rs: 实现 Proxy-Authorization 头的检查逻辑。
  • src/proxy.rs: 包含核心的请求转发逻辑。它负责准备请求并将其发送到上游代理，供标准代理和 CONNECT 隧道处理器共同使用。
  • src/run.rs: 包含主 run 函数，负责将服务器绑定到套接字并为每个传入连接创建任务。
  • src/upgrade.rs: 处理 CONNECT 请求，通过建立到目标的 TCP 隧道来转发流量。
  • tests/main.rs: 包含集成测试，用于验证代理服务器的端到端功能。

相关历史

代理服务器的概念几乎与万维网本身一样古老。最早的代理是 1990 年代初在欧洲核子研究中心（CERN）开发的，大约在蒂姆·伯纳斯-李（Tim Berners-Lee）创建万维网的同一时期。最初，它们被用作处理不同协议的“网关”，但其角色很快演变为缓存，这对于在当时缓慢而昂贵的国际网络上减少流量至关重要。这个简单而强大的中介理念，后来成为现代网络架构的基本构建块，支撑着从安全、内容过滤到本项目所依赖的分布式系统等各种技术。

About

This project is an open-source component of i18n.site ⋅ Internationalization Solution.

关于

本项目为 i18n.site ⋅ 国际化解决方案 的开源组件。