pub struct AppConfig {
pub server: ServerConfig,
pub renderer: RendererConfig,
pub crawler: CrawlerConfig,
pub extraction: ExtractionConfig,
pub auth: AuthConfig,
pub request: RequestConfig,
pub search: SearchConfig,
pub map: MapConfig,
pub client: ClientConfig,
}Fields§
§server: ServerConfig§renderer: RendererConfig§crawler: CrawlerConfig§extraction: ExtractionConfig§auth: AuthConfig§request: RequestConfig§search: SearchConfig§map: MapConfig§client: ClientConfig[client] — settings for the local CLI/MCP when it proxies to the
hosted SaaS. Written by crw setup into the user-config file.
Implementations§
Source§impl AppConfig
impl AppConfig
Sourcepub fn load() -> Result<Self, ConfigError>
pub fn load() -> Result<Self, ConfigError>
Load config from config.default.toml + per-user config + environment variable overrides.
Precedence (highest wins):
CRW_*env vars (CI/Docker)$CRW_CONFIGfile (orconfig.local.tomlin cwd)~/.config/crw/config.toml(written bycrw setup)config.default.toml(bundled defaults)
Env stays on top so a one-off CRW_FOO=bar crw … always wins over
whatever the user has saved, matching how every other shell tool works.
Sourcepub fn effective_deadline_ms(
&self,
requested_deadline_ms: Option<u64>,
wait_for_ms: Option<u64>,
) -> u64
pub fn effective_deadline_ms( &self, requested_deadline_ms: Option<u64>, wait_for_ms: Option<u64>, ) -> u64
Compute the effective end-to-end request deadline (ms). Implements the issue-#35 auto-extension policy:
- If the caller supplied an explicit
requested_deadline_ms, return it verbatim — operators trust the request budget over our heuristic. - Otherwise, when
request.auto_extend_deadline_for_ladderis on, returnmax(deadline_ms_default, ladder_min + wait_for_extra).ladder_mincovers the configured tier ladder;wait_for_extracompensates for callers that bumpedwait_for_msabove the default SPA budget (8s) — without it, a longwait_forwould silently re-clamp inside CDP. - When the policy is disabled, return
deadline_ms_defaultunchanged.
wait_for_ms is the per-request override (ScrapeRequest::wait_for /
CrawlRequest::wait_for); pass None for sub-fetches that don’t
surface a wait_for to the caller (search/map enrichment).
Sourcepub fn effective_request_timeout_secs(&self) -> u64
pub fn effective_request_timeout_secs(&self) -> u64
Tower middleware outer timeout (seconds). Must accommodate the longest legitimate handler runtime so a healthy request isn’t cancelled by the outer layer before the inner deadline fires.
Covers the three route envelopes:
/scrape,/mcp— auto-extended scrape deadline./search— SearXNG fetch + bounded enrichment fan-out (ceil(max_limit / max_concurrency)batches × scrape_ms)./crawl/jobs/:id,/map— handler-side caps up to 300s.
When auto-extend is disabled, returns the operator-configured baseline unchanged.