Skip to main content

batch_scrape_http

Function batch_scrape_http 

Source
pub async fn batch_scrape_http(
    urls: &[String],
    robots: RobotsPolicy,
    opts: &ScrapeOpts,
    concurrency: usize,
) -> Result<Value, CliError>
Expand description

Batch scrape N URLs (HTTP engine, sequential by default).