reqwest-proxy-pool

Proxy pool middleware implementation for reqwest-middleware.

Features

✨ Comprehensive Proxy Support

• Automatic parsing of free SOCKS5/SOCKS5H proxies from multiple sources
• Built-in health checking with customizable timeout and test URL

⚡ Intelligent Proxy Management

• Multiple proxy selection strategies (FastestResponse, RoundRobin, Random)
• Per-proxy rate limiting to avoid bans
• Automatic retry mechanism for failed requests
• Custom response classifier for business-level proxy health (anti-bot/captcha detection)

🔧 Easy Configuration

• Simple builder pattern for configuration
• Seamless integration with reqwest middleware stack

Quickstart

Installation

Add to your Cargo.toml:

[dependencies]
reqwest = "0.13"
reqwest-proxy-pool = "0.2"
reqwest-middleware = "0.5"
tokio = { version = "1", features = ["full"] }

Usage

use reqwest_middleware::ClientBuilder;
use reqwest_proxy_pool::{
    ProxyPoolConfig, ProxyPoolMiddleware, ProxyResponseVerdict, ProxySelectionStrategy,
    ResponseClassifier,
};
use std::time::Duration;

struct CaptchaDetector;

impl ResponseClassifier for CaptchaDetector {
    fn classify(&self, response: &reqwest::Response) -> ProxyResponseVerdict {
        match response.status().as_u16() {
            403 | 429 => ProxyResponseVerdict::ProxyBlocked,
            500..=599 => ProxyResponseVerdict::Passthrough,
            _ => ProxyResponseVerdict::Success,
        }
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ProxyPoolConfig::builder()
        .sources(vec![
            "https://cdn.jsdelivr.net/gh/dpangestuw/Free-Proxy@main/socks5_proxies.txt",
            "https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/protocols/socks5/data.txt",
        ])
        .health_check_timeout(Duration::from_secs(5))
        .health_check_url("https://httpbin.org/ip")
        .retry_count(2)
        .selection_strategy(ProxySelectionStrategy::FastestResponse)
        .max_requests_per_second(3.0)
        .response_classifier(CaptchaDetector)
        .danger_accept_invalid_certs(true)
        .build();

    let proxy_pool = ProxyPoolMiddleware::new(config).await?;

    let client = ClientBuilder::new(reqwest::Client::new())
        .with(proxy_pool)
        .build();

    let response = client.get("https://httpbin.org/ip").send().await?;
    println!("Status: {}", response.status());
    println!("Response: {}", response.text().await?);

    Ok(())
}

Configuration Options

License