Module fetcher 

Proxy list fetching — port trait and free-list HTTP adapter.

ProxyFetcher is the port trait. Implement it to pull proxies from any source (remote HTTP list, database, commercial API, etc.) and integrate with ProxyManager via load_from_fetcher.

The built-in FreeListFetcher downloads plain-text host:port proxy lists from public URLs (e.g. the TheSpeedX/PROXY-List feeds on GitHub) and parses them into Proxy records. It is suitable for development, testing, and low-stakes scraping where proxy quality is less critical.

Example — load from a free list and populate the pool

use std::sync::Arc;
use stygian_proxy::{
    ProxyManager,
    storage::MemoryProxyStore,
    fetcher::{FreeListFetcher, ProxyFetcher, FreeListSource},
};

let fetcher = FreeListFetcher::new(vec![
    FreeListSource::TheSpeedXHttp,
]);

let manager = ProxyManager::builder()
    .storage(Arc::new(MemoryProxyStore::default()))
    .build()?;
let loaded = stygian_proxy::fetcher::load_from_fetcher(&manager, &fetcher).await?;
println!("Loaded {loaded} proxies");

Structs

FreeListFetcher

Fetches plain-text host:port proxy lists from one or more public URLs.

Enums

FreeListSource

A well-known free/public proxy list feed.

Traits

ProxyFetcher

A source that can produce a list of Proxy records asynchronously.

Functions

load_from_fetcher

Fetch proxies from fetcher and add them all to manager.