Struct ScrapeSpec 

pub struct ScrapeSpec {

    pub engine: Option<String>,
    pub impersonate: Option<String>,
    pub render_wait: Option<String>,
    pub proxy: String,
    pub respect_robots: Option<bool>,
    pub extract: Vec<String>,
    pub adaptive: Option<bool>,
    pub similarity_floor: Option<f64>,
    pub follow: String,
    pub max_depth: Option<i64>,
    pub max_pages: Option<i64>,
    pub concurrency: Option<i64>,
    pub politeness: String,
    pub checkpoint: String,
    pub loc: Loc,
}

v2.52.0 — the closed-catalog scrape configuration block (scrape: { … }) on a web-acquisition tool. Every field is optional/defaulted so a minimal scrape: {} is legal; the fields that apply depend on the tool’s provider: (the type-checker cross- validates provider ↔ field applicability, axon-T905). The whole struct is deliberately flat + serializable-friendly (Option/Vec/ scalar), mirroring the v2.39.0 technician-field discipline, so the IR stays byte-stable and the runtime classifies identically.

Fields

engine: Option<String>

The acquisition engine: impersonate (HTTP-fingerprint stealth, the GA tier) | browser (headless-render sidecar, the gray tier). None ⇒ impersonate. Closed catalog (axon-T905). Applies to scrape_http / scrape_crawl.

impersonate: Option<String>

The browser-fingerprint impersonation PROFILE name (chrome, firefox, safari — closed catalog). Only meaningful with engine: impersonate. The concrete JA3/JA4 + HTTP/2 profile is resolved by the enterprise engine (v2.52.0); OSS records the declared intent. None ⇒ the engine’s default profile.

render_wait: Option<String>

The post-navigation settle wait for engine: browser (a Duration, e.g. 2s) — how long to let JS render before snapshotting. Bounded . Ignored by the impersonate engine (no JS runtime).

proxy: String

The per-tenant proxy-pool config KEY (a dotted key, resolved via the same SecretResolver secret:/tool.base_url use — the design decision), never a proxy URL literal. Empty ⇒ direct connection.

respect_robots: Option<bool>

Whether robots.txt is honored (default TRUE, the design decision). Setting respect_robots: false is the audited, scrape.aggressive-gated override (enforced enterprise-side, v2.52.0); in OSS it is recorded.

extract: Vec<String>

scrape_dom extraction spec: an ordered list of name=selector FieldSpecs (["title=h1", "price=.amount"]). A closed, bracketed string list (reuses the v2.38.0/v2.39.0 list helper). Each entry must be a single name=selector pair (axon-T906).

adaptive: Option<bool>

scrape_dom adaptive relocation: when a declared selector misses, the engine attempts a HEURISTIC relocation above similarity_floor (the design decision — a heuristic, NOT a proof). None/false ⇒ strict selectors only. Enabling it makes the tool carry <storage> (the per-tenant selector-memory, v2.52.0).

similarity_floor: Option<f64>

The similarity threshold ∈ [0,1] governing adaptive relocation (axon-T907). Only meaningful with adaptive: true.

follow: String

scrape_crawl link-follow selector/pattern: which links to enqueue from each fetched page. Empty ⇒ no expansion (single-page crawl).

max_depth: Option<i64>

scrape_crawl maximum link depth from the seed (bounded, the design decision).

max_pages: Option<i64>

scrape_crawl maximum total pages fetched (bounded, the design decision). A hostile/infinite site can never exhaust the crawler (axon-T908).

concurrency: Option<i64>

scrape_crawl fetch concurrency (bounded, ≥ 1).

politeness: String

scrape_crawl politeness/rate reference: a declared budget (budget{rate:/max:}, v2.28.0) governing per-host request pacing (D98 reuse of the budget kernel). Empty ⇒ engine default pacing.

checkpoint: String

scrape_crawl checkpoint store reference: a declared axonstore the crawler persists frontier/visited state into for resumable, at-least-once crawling. Empty ⇒ in-memory (non-resumable).

loc: Loc

Trait Implementations

impl Debug for ScrapeSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for ScrapeSpec

fn default() -> ScrapeSpec

Returns the “default value” for a type.