Enum HttpClient 

pub enum HttpClient {
    Reqwest(Client, RetryConfig),
    Fantoccini(Client, RetryConfig),
    Chromium(Arc<Browser>, ChromiumWaitConfig, RetryConfig, Option<Arc<TempDir>>),
}

Enum for different HTTP client implementations

Variants

Reqwest(Client, RetryConfig)

Fantoccini(Client, RetryConfig)

Chromium(Arc<Browser>, ChromiumWaitConfig, RetryConfig, Option<Arc<TempDir>>)

Headless Chrome via CDP. new_page() waits for the load event, which covers synchronous JS execution. Additional wait strategies in ChromiumWaitConfig can be used to handle SPAs that fetch data asynchronously after load.

Implementations

impl HttpClient

pub fn new_reqwest(timeout: f64) -> Result<Self, String>

Create a new reqwest-based HTTP client optimized for single-domain crawling

pub fn new_reqwest_multi_domain( timeout: f64, max_idle_per_host: usize, ) -> Result<Self, String>

Create a new reqwest-based HTTP client optimized for multi-domain crawling

pub fn new_reqwest_with_options( timeout: f64, max_idle_per_host: usize, retry_config: RetryConfig, headers: HeaderMap, auth: Option<AuthConfig>, ) -> Result<Self, String>

Create a reqwest-based HTTP client with retry, default headers/cookies, and auth.

auth, if set, overrides any Authorization entry already in headers.

pub async fn new_chromium( chrome_path: Option<PathBuf>, wait_config: ChromiumWaitConfig, retry_config: RetryConfig, ) -> Result<Self, String>

Create a headless Chrome client that launches Chrome/Chromium automatically. No external WebDriver server is required — only Chrome/Chromium must be installed. If chrome_path is None, the system Chrome is auto-detected.

Pages are fetched after the browser’s load event fires, which includes synchronous JavaScript execution. Additional wait strategies in ChromiumWaitConfig (network-idle, CSS selector polling, fixed delay) can be layered on top for SPAs that fetch data asynchronously after load.

pub async fn fetch(&self, url: Url) -> Result<String, String>

Fetch content from a URL, retrying transient failures with exponential backoff according to this client’s RetryConfig.

Trait Implementations

impl Clone for HttpClient

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HttpClient

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for HttpClient

fn default() -> Self

Returns the “default value” for a type.