Struct ScraperConfig 

pub struct ScraperConfig {
    pub headless: bool,
    pub max_scroll_iterations: usize,
    pub enrich: bool,
    pub between_query_delay: Duration,
    pub place_panel_delay: Duration,
    pub place_panel_jitter: Duration,
    pub max_places: Option<usize>,
    pub nav_timeout: Duration,
    pub proxy: Option<String>,
    pub user_agent: Option<String>,
    pub browserless_url: Option<String>,
}

Scraper configuration.

Fields

headless: bool

Run Chrome in headless mode (default: true). Use false for debugging.

max_scroll_iterations: usize

How many scrollTop = scrollHeight iterations to perform per search before assuming the feed is exhausted. Default: 30.

enrich: bool

If false, skip clicking each result and only return surface data (name, maps_url) — much faster but no website / phone. Default: true.

between_query_delay: Duration

Delay between consecutive HTTP requests inside Chrome.

place_panel_delay: Duration

Per-place delay after clicking (gives the panel time to render). The effective wait is this plus a random 0..=place_panel_jitter — see ScraperConfig::place_panel_jitter.

place_panel_jitter: Duration

Upper bound on the random extra delay added on top of ScraperConfig::place_panel_delay before each place visit. A fresh value in 0..=place_panel_jitter is drawn per place to de-regularise the request cadence (mild bot-detection mitigation, not a guarantee). The jitter is purely additive, so place_panel_delay is always the minimum; set this to Duration::ZERO to disable it. Default: 750 ms.

max_places: Option<usize>

Maximum number of unique places to return per query.

None (default) means unlimited — a dense query (e.g. a feed of several hundred results) will then perform one Chrome navigation per place with no upper bound, so a single call can run for many minutes. Set this when you need to bound run time. Note the cap counts added (deduplicated) places, so in enrich mode a few extra navigations may still occur for duplicates before the cap is reached (in non-enrich mode dedup is pre-navigation, so no navigations are wasted).

nav_timeout: Duration

Timeout for each page navigation / JS evaluation step. Default: 30s.

proxy: Option<String>

Optional proxy passed to Chrome as --proxy-server=<value>. If None, falls back to the PROXY_URL environment variable. The value must not contain whitespace (rejected at launch).

user_agent: Option<String>

Optional User-Agent override.

None (default) leaves Chrome to report its own, always-current UA, which stays consistent with the real TLS/HTTP2 fingerprint. Set this only if you need to pin a specific UA string.

browserless_url: Option<String>

Optional WebSocket URL of a remote Chrome (e.g. a Browserless instance) to connect to instead of launching a local browser. If None, falls back to the BROWSERLESS_URL environment variable.

When set, the browser is reached via Browser::connect, so local launch arguments (headless, proxy, window size, user agent) are controlled by the remote endpoint and ignored here.

Trait Implementations

impl Clone for ScraperConfig

fn clone(&self) -> ScraperConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ScraperConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ScraperConfig

fn default() -> Self

Returns the “default value” for a type.