pub struct RendererConfig {Show 25 fields
pub mode: RendererMode,
pub page_timeout_ms: u64,
pub http_timeout_ms: Option<u64>,
pub lightpanda_timeout_ms: Option<u64>,
pub chrome_timeout_ms: Option<u64>,
pub pool_size: usize,
pub render_js_default: Option<bool>,
pub lightpanda: Option<CdpEndpoint>,
pub playwright: Option<CdpEndpoint>,
pub chrome: Option<CdpEndpoint>,
pub chrome_proxy: Option<CdpEndpoint>,
pub chrome_proxy_timeout_ms: Option<u64>,
pub chrome_intercept_resources: bool,
pub chrome_intercept_stylesheets: bool,
pub chrome_host_intercept_disable: Vec<String>,
pub chrome_nav_budget_ms: u64,
pub chrome_context_pool_enabled: bool,
pub chrome_pool: ChromePoolConfig,
pub chrome_backend: ChromeBackend,
pub use_predictor: bool,
pub escalation: EscalationConfig,
pub antibot: AntibotConfig,
pub proxy_base_user: Option<String>,
pub proxy_base_pass: Option<String>,
pub proxy_default_country: Option<String>,
}Fields§
§mode: RendererMode§page_timeout_ms: u64Generic per-page navigation timeout. Used as the fallback when no per-tier override is configured. Kept for backward compatibility — the per-tier knobs below are preferred for new deployments.
http_timeout_ms: Option<u64>Override for the HTTP-only fetcher request timeout. Falls back to
page_timeout_ms when unset. HTTP responses arrive quickly when they
arrive at all, so 15s is generous and keeps slow upstreams from
hogging the request budget that should be spent on JS retries.
lightpanda_timeout_ms: Option<u64>Override for the LightPanda CDP renderer. LightPanda completes most renders in <10s; if it stalls past 20s it almost always means an adversarial page that Chrome will render anyway, so failing fast and escalating beats waiting it out.
chrome_timeout_ms: Option<u64>Override for the full-Chromium tier. Chrome is the slow path
(gov/legal SPAs need 30–40s for networkidle); the larger budget here
recovers ~6 URLs per fc-wins iteration without affecting the fast path.
pool_size: usize§render_js_default: Option<bool>If set, applies to every request that doesn’t specify renderJs explicitly.
Some(true) = force JS rendering; Some(false) = skip JS; None = auto-detect.
Accepts the force_js alias for backward compatibility.
lightpanda: Option<CdpEndpoint>§playwright: Option<CdpEndpoint>§chrome: Option<CdpEndpoint>§chrome_proxy: Option<CdpEndpoint>Residential-proxy Chrome tier (opt-in 4th renderer). Same Chromium
browser as chrome, but egress routed through a forwarder that adds
upstream proxy auth (e.g. DataImpulse). Tried after Chrome fails —
covers IP-blocked targets where the browser fingerprint is fine but
the VPS egress IP is flagged.
chrome_proxy_timeout_ms: Option<u64>Per-tier nav timeout override for chrome_proxy. When unset, defaults
to chrome_timeout() + 15_000 — the proxy hop adds latency, so the
fallback tier needs more headroom than direct Chrome.
chrome_intercept_resources: boolEnable Chrome resource interception (Fetch.enable blocking of media,
fonts, trackers). Default false; flipped after the CDP-fake suite
validates pump + cleanup behaviour. See plan Phase 2.
chrome_intercept_stylesheets: boolAdditionally block stylesheet requests when interception is enabled.
Default false — kept off in v1 because some extractors depend on
CSS-driven visibility / lazy-content triggers.
chrome_host_intercept_disable: Vec<String>Per-host opt-out for chrome interception. Hosts in this list run with
interception disabled even when chrome_intercept_resources = true.
Hard chrome-tier navigation budget in ms. Wraps wait_for_page_ready
in an inner race; on budget hit the renderer snapshots whatever DOM is
present and returns truncated = true. Calibrated as
p90(successful chrome renders) clamped to [8_000, 12_000].
chrome_context_pool_enabled: boolEnable the bounded browser-context pool. Default false; v1 ships
RECYCLE_AFTER_NAV = 1 (recreate every release) before optimising to
reuse-with-clearing. See plan Phase 4. Gated off when
chrome_backend = "browserless" — browserless v2’s
Target.createBrowserContext semantics with long-lived sessions are
unproven; lib.rs forces this to false with a WARN log in that case.
chrome_pool: ChromePoolConfigPer-knob pool configuration. Read only when
chrome_context_pool_enabled = true AND backend is Vanilla.
chrome_backend: ChromeBackendWhich Chrome backend the WS URL points at. Explicit — never sniff
from URL substrings (k8s svc names, port-forwards, custom routes break
substring detection per plan §C2). Default Vanilla.
use_predictor: boolEnable the success-ratio renderer predictor in HostPreferences.
Default false; flipped after the predictor replay harness gates
on the 1k bench (false-skip < 2 %, false-escalate < 5 %, churn < 3 / 1k).
escalation: EscalationConfigEngine escalation policy (firecrawl-shaped: race + on-error). When disabled (default), the renderer keeps its current ladder unchanged.
antibot: AntibotConfigAnti-bot detection policy (crawl4ai 3-tier classifier).
proxy_base_user: Option<String>DataImpulse residential-proxy base username (without __cr.<cc>
country suffix). When set alongside [proxy_base_pass], the engine
drives Chrome’s proxy auth via CDP Fetch.authRequired and composes
the country-suffixed username per request. Read only by the
chrome_proxy tier. None = no upstream proxy auth (chrome_proxy
tier still functional only if a no-auth or pre-authed proxy is in
front of Chrome).
proxy_base_pass: Option<String>DataImpulse base password — see [proxy_base_user].
proxy_default_country: Option<String>Fallback country code used when a request omits country. Lowercased
2-letter ISO 3166-1 alpha-2 (e.g. “us”). None = global pool (no suffix).
Implementations§
Source§impl RendererConfig
impl RendererConfig
Sourcepub fn http_timeout(&self) -> u64
pub fn http_timeout(&self) -> u64
Resolved per-tier nav timeout in milliseconds. Resolution rules:
- If the explicit per-tier field is set, use it verbatim.
- Otherwise fall back to
page_timeout_ms(which itself defaults to 30s for backward compatibility with pre-multi-tier configs).
New deployments are encouraged to set the per-tier knobs to 15/20/45s (see config.docker.toml) — these match the bench-tuned values that recover slow gov sites in the chrome tier without giving the http tier permission to hog the request budget.
pub fn lightpanda_timeout(&self) -> u64
pub fn chrome_timeout(&self) -> u64
pub fn chrome_proxy_timeout(&self) -> u64
Sourcepub fn effective_proxy_credentials(
&self,
country: Option<&str>,
) -> Option<(String, String)>
pub fn effective_proxy_credentials( &self, country: Option<&str>, ) -> Option<(String, String)>
Compose the DataImpulse-style proxy credentials for a single request.
Resolution order for the country suffix:
countryargument (per-request override)self.proxy_default_country(server default)- No suffix → DataImpulse global pool
Returns None when no base credentials are configured — caller treats
this as “no auth required”. An invalid country code (wrong length,
non-alphabetic) silently falls through to the default; that keeps a
malformed ?country= query from creating an unauthenticated request
while still letting through a well-known default.
Sourcepub fn cdp_tier_count(&self) -> usize
pub fn cdp_tier_count(&self) -> usize
Number of active CDP tiers (lightpanda + playwright + chrome) under
the current mode. Mirrors the predicate used at runtime in
crw-renderer/src/lib.rs when constructing the renderer ladder:
want(mode) && config.<tier>.is_some().
Returns 0 when the binary is built without the cdp feature — in
that case no JS renderer can be constructed regardless of the config,
so the deadline auto-extension policy must collapse to HTTP-only.
Sourcepub fn min_deadline_for_full_ladder_ms(&self) -> u64
pub fn min_deadline_for_full_ladder_ms(&self) -> u64
Minimum request deadline budget (ms) required so that every configured
tier can use its full allowance when fallback exhausts the chain.
Sums the per-tier timeouts and adds CDP_TIER_OVERHEAD_MS for each
active CDP tier, matching the runtime ladder built in
crw-renderer/src/lib.rs.
Trait Implementations§
Source§impl Clone for RendererConfig
impl Clone for RendererConfig
Source§fn clone(&self) -> RendererConfig
fn clone(&self) -> RendererConfig
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more