Crate stealthscraper_rs 

Stealthy Rust web scraping that defeats modern bot protection (Cloudflare, Akamai, DataDome) on two fronts at once:

• JavaScript / CDP probing — a real headless Chrome instance is driven via the Chrome DevTools Protocol, with stealth scripts masking navigator, WebGL, Canvas, and Audio fingerprints.
• Network (JA3/JA4) fingerprinting — a local MITM proxy (TlsSpoofingProxy) re-emits the browser’s traffic through wreq so the TLS ClientHello and HTTP/2 settings match the impersonated browser.

Capabilities

• Challenge handling (challenge) — classify a page with detect and choose a retry/rotate Action via MitigationPolicy.
• Proxy rotation (proxy_pool) — a rotatable ProxyPool; the MITM upstream client is hot-swapped so the egress IP changes without relaunching the browser.
• Geo/locale consistency (geo) — derive Accept-Language, navigator.languages, and the timezone from the egress proxy’s country so the IP and locale never contradict each other.
• Profile rotation — relaunch under a fresh BrowserProfile when the fingerprint identity itself is burned (the browser feature’s CloudScraper).
• Session state (state) — per-domain outcomes and cooldowns behind a StateStore; durable with the persistence feature.
• Observability (events) — a ScraperEvent / EventSink stream.

Feature flags

• browser (off by default) — the headless-Chrome API (CloudScraper, solve_challenge, profile rotation, human-behavior helpers). Required for the quick start below.
• persistence (off by default) — the durable, redb-backed state store.

With no features enabled the crate builds only the pure, dependency-light core (challenge, proxy_pool, geo, the state model, events) for embedding into your own pipeline.

Quick start

use stealthscraper_rs::{BrowserProfile, CloudScraper};

// The builder spins up the MITM proxy and launches a stealth browser whose
// JA4 fingerprint matches the chosen profile.
let scraper = CloudScraper::builder()
    .profile(BrowserProfile::random())
    .build()
    .await?;

let tab = scraper.new_stealth_tab()?;
tab.navigate_to("https://protected.example.com").expect("navigate");
tab.wait_until_navigated().expect("wait");

// Detect and wait out / solve any bot-protection challenge on the page.
let signal = scraper.solve_challenge(&tab)?;
println!("page cleared (challenge: {:?})", signal.kind);

solve_challenge is synchronous and blocking; on an async runtime call it from tokio::task::spawn_blocking and run the proxy on a multi-threaded runtime.

Re-exports

pub use challenge::Action;

pub use challenge::ChallengeKind;

pub use challenge::ChallengeSignal;

pub use challenge::Confidence;

pub use challenge::DetectionInput;

pub use challenge::MitigationPolicy;

pub use challenge::detect;

pub use error::Error;

pub use events::EventSink;

pub use events::LogEventSink;

pub use events::NoopEventSink;

pub use events::ScraperEvent;

pub use geo::CountryCode;

pub use geo::GeoResolver;

pub use geo::Locale;

pub use profile::BrowserProfile;

pub use proxy::TlsSpoofingProxy;

pub use proxy_pool::ProxyPool;

pub use proxy_pool::RotationStrategy;

pub use scraper::CloudScraper;

pub use scraper::CloudScraperBuilder;

pub use solver::GenericSolver;

pub use state::DomainState;

pub use state::InMemoryStateStore;

pub use state::Outcome;

pub use state::StateStore;

Modules

behavior

Emulation of human-like interaction patterns (typing delays, mouse curves). Human-interaction emulation: Bézier-curve mouse paths and human-like keystroke timing, used to make CDP-driven input look organic.

challenge

Pure detection and mitigation policy for bot-protection challenges. Bot-protection challenge detection and mitigation policy.

error

Strong typed Error enums for the scraper and underlying HTTP proxy. The crate’s error type, covering the browser, proxy, challenge, and state layers.

events

Observability events and sinks emitted during a scrape. Observability events emitted during a scrape, and sinks that consume them.

geo

Geo/locale consistency: country codes, locale table, and a resolver port. Geo/locale consistency: country codes, a curated country -> locale table, and a resolver port for discovering a proxy’s exit country.

profile

Management of browser fingerprints, user agents, and localized hardware characteristics. Browser fingerprint profiles: the user agent, platform, hardware characteristics, WebGL strings, viewport, and locale that define a synthetic-yet-realistic browser identity.

proxy

Local MITM TLS spoofing proxy using Hyper and Rustls. Local MITM TLS-spoofing proxy (TlsSpoofingProxy) that terminates the browser’s TLS, then re-emits each request through wreq with a forged JA4 ClientHello and HTTP/2 fingerprint. The upstream client is hot-swappable so the egress proxy can rotate without relaunching the browser.

proxy_pool

Rotatable pool of upstream proxies with selection strategy (pure domain logic). Pure upstream-proxy pool and rotation strategy.

scraper

Core headless Chrome browser lifecycle and orchestration.

solver

Automated solvers for bypassing common JavaScript challenges. Interactive challenge solving — e.g. locating and clicking a Cloudflare Turnstile widget with human-like mouse movement.

state

Per-domain session state: model, store port, and adapters. Per-domain session state: model, the StateStore port, and adapters.

stealth

Injection scripts to mask navigator and WebGL hooks. Stealth JavaScript injected via CDP to mask headless-browser signatures (navigator, WebGL, Canvas, AudioContext, WebRTC) so the rendered page matches the active BrowserProfile.