proxy_http 0.1.3

[English](#english) | [中文](#chinese)

<a id="english"></a>

## English

- [Introduction](#introduction-en)
- [Tech Stack](#tech-stack-en)
- [Usage](#usage-en)
- [Design Philosophy](#design-philosophy-en)
- [File Structure](#file-structure-en)
- [A Little History](#a-little-history-en)

<a id="introduction-en"></a>
### Introduction

`proxy_http` is a lightweight, high-performance, asynchronous HTTP proxy server built with Rust. Its primary function is to route outgoing traffic through a dynamic pool of upstream proxy servers. It is designed for scenarios that require robust IP rotation, enhanced user anonymity, or bypassing network restrictions. The server also includes basic authentication to ensure controlled access.

<a id="tech-stack-en"></a>
### Tech Stack

This project leverages modern, efficient, and battle-tested Rust libraries to achieve high performance and reliability:

- **[Tokio](https://tokio.rs/)**: An asynchronous runtime for writing fast and reliable network applications.
- **[Hyper](https://hyper.rs/)**: A fast, safe, and correct HTTP implementation for Rust, serving as the foundation of the proxy server.
- **[Reqwest](https://github.com/seanmonstar/reqwest)**: An ergonomic, higher-level HTTP client used for testing the proxy's functionality.
- **[ThisError](https://github.com/dtolnay/thiserror)**: A library for creating descriptive, structured error types, simplifying error handling.

<a id="usage-en"></a>
### Usage

The following examples, inspired by the integration tests, demonstrate how to start the proxy server and route requests through it.

#### Example 1: Standard HTTP Proxy Request

First, initialize and run the proxy server. It fetches a list of upstream proxies from a specified subscription URL.

```rust
use std::net::{Ipv4Addr, SocketAddr, SocketAddrV4};
use aok::{OK, Void};
use log::info;

// Assume SS_SUBSCRIPTION_URL is set in the environment
genv::s!(SS_SUBSCRIPTION_URL: String);

async fn run_proxy_server() -> Void {
  // Fetch upstream proxy providers from a subscription URL
  let fetch = proxy_fetch::load(SS_SUBSCRIPTION_URL.split(";")).await?;

  let user = "test";
  let password = "pwd";
  let port = 32342;
  let addr = SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), port));

  // Run the proxy server in a separate task
  tokio::spawn(async move {
    xerr::log!(proxy_http::run(fetch, addr, user, &password).await);
  });

  tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;
  OK
}
```

Next, use a client like `reqwest` to make a request through the proxy with basic authentication.

```rust
async fn client_request() -> Void {
  let user = "test";
  let password = "pwd";
  let port = 32342;
  let url = "http://ifconfig.me/ip";

  // Configure the client to use the proxy
  let proxy = reqwest::Proxy::http(format!("http://{{user}}:{{password}}@127.0.0.1:{port}"))?;
  let client = reqwest::Client::builder().proxy(proxy).build()?;

  // Send the request
  let res = client.get(url).send().await?;
  let ip = res.text().await?;
  info!("IP via proxy: {{ip}}");
  assert!(!ip.is_empty());

  OK
}
```

#### Example 2: HTTP Tunnel (CONNECT)

The server also supports establishing an HTTP tunnel for protocols like HTTPS. The client sends a `CONNECT` request, and the server establishes a direct TCP connection to the target.

```rust
async fn test_tunnel_proxy() -> Void {
  // ... (server setup is the same)

  let target_host = "ifconfig.me";
  let target_port = 80;

  // Connect to the proxy server
  let mut stream = TcpStream::connect("127.0.0.1:32343").await?;

  // Send a CONNECT request to establish the tunnel
  let connect_request = format!(
    "CONNECT {{target_host}}:{{target_port}} HTTP/1.1\r\n\
     Host: {{target_host}}:{{target_port}}\r\n\
     Proxy-Authorization: Basic {{}}
\n\
     \r\n",
    target_host,
    target_port,
    target_host,
    target_port,
    base64::general_purpose::STANDARD.encode("test:pwd")
  );

  stream.write_all(connect_request.as_bytes()).await?;

  // Read the response to confirm the tunnel is established
  let mut response = vec![0u8; 1024];
  stream.read(&mut response).await?;
  assert!(String::from_utf8_lossy(&response).contains("200"));

  // Now, send an HTTP GET request through the established tunnel
  let http_request = "GET /ip HTTP/1.1\r\nHost: ifconfig.me\r\n\r\n";
  stream.write_all(http_request.as_bytes()).await?;

  // Read the final response from the target server
  let mut final_response = Vec::new();
  stream.read_to_end(&mut final_response).await?;
  info!("Response via tunnel: {{}}", String::from_utf8_lossy(&final_response));

  OK
}
```

<a id="design-philosophy-en"></a>
### Design Philosophy

The design of `proxy_http` adheres to several core principles:

- **Asynchronous First**: By using `tokio` and `hyper`, all operations are non-blocking, allowing the server to handle thousands of concurrent connections with minimal resource overhead.
- **Modularity and Reusability**: The core logic is encapsulated in a library (`lib.rs`), separating it from the executable entry point (`main.rs`). This makes the proxy functionality easy to integrate into other applications.
- **Dynamic Configuration**: Upstream proxies are not hardcoded. They are fetched from an external source at runtime, providing flexibility and enabling dynamic proxy rotation without server restarts.
- **Minimalism**: The project focuses on providing a core HTTP proxy service without unnecessary features, keeping the codebase small, maintainable, and efficient.

<a id="file-structure-en"></a>
### File Structure

The project is organized to promote clarity and separation of concerns:

- `Cargo.toml`: Defines project metadata, dependencies, and build settings.
- `src/main.rs`: The main entry point for the executable, responsible for initializing and running the proxy server.
- `src/lib.rs`: The library core, containing the primary proxy logic, including request handling, authentication, and forwarding.
- `src/error.rs`: Defines custom, structured error types for the application using `thiserror`.
- `src/fetch.rs`: Handles the logic for fetching and managing the list of upstream proxy providers.
- `src/handle.rs`: Contains the core logic for handling incoming client requests.
- `src/is_authorized.rs`: Implements the basic authentication check.
- `src/run.rs`: Contains the main server loop that accepts and processes connections.
- `src/upgrade.rs`: Manages the TCP stream upgrade for `CONNECT` requests (tunnelling).
- `tests/main.rs`: Contains integration tests that verify the end-to-end functionality of the proxy server.

<a id="a-little-history-en"></a>
### A Little History

The concept of proxy servers is nearly as old as the web itself. The first proxies were developed at CERN in the early 1990s, around the same time Tim Berners-Lee was creating the World Wide Web. Initially, they were used as "gateways" to handle different protocols, but their role quickly evolved into caching, which was crucial for reducing traffic on the slow and expensive international networks of the time. This simple yet powerful idea of an intermediary has since become a fundamental building block of modern network architecture, enabling everything from security and content filtering to the very distributed systems this project relies on.

---

<a id="chinese"></a>

## 中文

- [项目简介](#项目简介-zh)
- [技术栈](#技术栈-zh)
- [使用演示](#使用演示-zh)
- [设计思路](#设计思路-zh)
- [文件结构](#文件结构-zh)
- [相关历史](#相关历史-zh)

<a id="项目简介-zh"></a>
### 项目简介

`proxy_http` 是一个使用 Rust 构建的轻量级、高性能、异步 HTTP 代理服务器。其核心功能是将出站流量通过一个动态的上游代理服务器池进行路由。它专为需要 IP 轮换、增强用户匿名性或绕过网络限制的场景而设计。服务器还包含基本身份验证功能，以确保访问受控。

<a id="技术栈-zh"></a>
### 技术栈

本项目利用了现代化、高效且经过实战检验的 Rust 库，以实现卓越的性能和可靠性：

- **[Tokio](https://tokio.rs/)**: 一个用于编写快速、可靠网络应用的异步运行时。
- **[Hyper](https://hyper.rs/)**: 一个快速、安全且正确的 Rust HTTP 实现，是本代理服务器的基石。
- **[Reqwest](https://github.com/seanmonstar/reqwest)**: 一个符合人体工程学的高级 HTTP 客户端，用于测试代理的功能。
- **[ThisError](https://github.com/dtolnay/thiserror)**: 一个用于创建描述性、结构化错误类型的库，简化了错误处理。

<a id="使用演示-zh"></a>
### 使用演示

以下示例源于项目的集成测试，演示了如何启动代理服务器并通过它路由请求。

#### 示例一：标准 HTTP 代理请求

首先，初始化并运行代理服务器。它会从指定的订阅 URL 获取上游代理列表。

```rust
use std::net::{Ipv4Addr, SocketAddr, SocketAddrV4};
use aok::{OK, Void};
use log::info;

// 假设环境变量中设置了 SS_SUBSCRIPTION_URL
genv::s!(SS_SUBSCRIPTION_URL: String);

async fn run_proxy_server() -> Void {
  // 从订阅 URL 获取上游代理提供者
  let fetch = proxy_fetch::load(SS_SUBSCRIPTION_URL.split(";")).await?;

  let user = "test";
  let password = "pwd";
  let port = 32342;
  let addr = SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), port));

  // 在一个独立的任务中运行代理服务器
  tokio::spawn(async move {
    xerr::log!(proxy_http::run(fetch, addr, user, &password).await);
  });

  tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;
  OK
}
```

接着，使用像 `reqwest` 这样的客户端，通过基本身份验证向代理服务器发出请求。

```rust
async fn client_request() -> Void {
  let user = "test";
  let password = "pwd";
  let port = 32342;
  let url = "http://ifconfig.me/ip";

  // 配置客户端以使用代理
  let proxy = reqwest::Proxy::http(format!("http://{{user}}:{{password}}@127.0.0.1:{port}"))?;
  let client = reqwest::Client::builder().proxy(proxy).build()?;

  // 发送请求
  let res = client.get(url).send().await?;
  let ip = res.text().await?;
  info!("通过代理获取的 IP: {{ip}}");
  assert!(!ip.is_empty());

  OK
}
```

#### 示例二：HTTP 隧道 (CONNECT)

服务器还支持为 HTTPS 等协议建立 HTTP 隧道。客户端发送 `CONNECT` 请求，服务器则与目标建立直接的 TCP 连接。

```rust
async fn test_tunnel_proxy() -> Void {
  // ... (服务器设置与之前相同)

  let target_host = "ifconfig.me";
  let target_port = 80;

  // 连接到代理服务器
  let mut stream = TcpStream::connect("127.0.0.1:32343").await?;

  // 发送 CONNECT 请求以建立隧道
  let connect_request = format!(
    "CONNECT {{target_host}}:{{target_port}} HTTP/1.1\r\n\
     Host: {{target_host}}:{{target_port}}\r\n\
     Proxy-Authorization: Basic {{}}
\n\
     \r\n",
    target_host,
    target_port,
    target_host,
    target_port,
    base64::general_purpose::STANDARD.encode("test:pwd")
  );

  stream.write_all(connect_request.as_bytes()).await?;

  // 读取响应以确认隧道已建立
  let mut response = vec![0u8; 1024];
  stream.read(&mut response).await?;
  assert!(String::from_utf8_lossy(&response).contains("200"));

  // 现在，通过已建立的隧道发送 HTTP GET 请求
  let http_request = "GET /ip HTTP/1.1\r\nHost: ifconfig.me\r\n\r\n";
  stream.write_all(http_request.as_bytes()).await?;

  // 从目标服务器读取最终响应
  let mut final_response = Vec::new();
  stream.read_to_end(&mut final_response).await?;
  info!("通过隧道得到的响应: {{}}", String::from_utf8_lossy(&final_response));

  OK
}
```

<a id="设计思路-zh"></a>
### 设计思路

`proxy_http` 的设计遵循几个核心原则：

- **异步优先**: 通过使用 `tokio` 和 `hyper`，所有操作都是非阻塞的，使服务器能够以最小的资源开销处理数千个并发连接。
- **模块化与可重用性**: 核心逻辑被封装在一个库（`lib.rs`）中，并与可执行文件入口点（`main.rs`）分离。这使得代理功能可以轻松集成到其他应用程序中。
- **动态配置**: 上游代理不是硬编码的。它们在运行时从外部源获取，提供了灵活性，并支持在不重启服务器的情况下动态轮换代理。
- **极简主义**: 项目专注于提供核心的 HTTP 代理服务，没有不必要的功能，从而保持代码库小巧、易于维护和高效。

<a id="文件结构-zh"></a>
### 文件结构

项目结构清晰，旨在促进关注点分离：

- `Cargo.toml`: 定义项目元数据、依赖项和构建配置。
- `src/main.rs`: 可执行文件的主入口点，负责初始化和运行代理服务器。
- `src/lib.rs`: 库的核心，包含主要的代理逻辑，如请求处理、身份验证和转发。
- `src/error.rs`: 使用 `thiserror` 为应用程序定义自定义的结构化错误类型。
- `src/fetch.rs`: 处理获取和管理上游代理提供者列表的逻辑。
- `src/handle.rs`: 包含处理传入客户端请求的核心逻辑。
- `src/is_authorized.rs`: 实现基本身份验证检查。
- `src/run.rs`: 包含接受和处理连接的主服务器循环。
- `src/upgrade.rs`: 管理 `CONNECT` 请求（隧道）的 TCP 流升级。
- `tests/main.rs`: 包含集成测试，用于验证代理服务器的端到端功能。

<a id="相关历史-zh"></a>
### 相关历史

代理服务器的概念几乎与万维网本身一样古老。最早的代理是 1990 年代初在欧洲核子研究中心（CERN）开发的，大约在蒂姆·伯纳斯-李（Tim Berners-Lee）创建万维网的同一时期。最初，它们被用作处理不同协议的“网关”，但其角色很快演变为缓存，这对于在当时缓慢而昂贵的国际网络上减少流量至关重要。这个简单而强大的中介理念，后来成为现代网络架构的基本构建块，支撑着从安全、内容过滤到本项目所依赖的分布式系统等各种技术。

---

<+ ../about.md >