spider-1.7.17
- spider 1.7.17
- Docs.rs crate page
- MIT
- Links
- Repository
- crates.io
- Source
- Owners
- madeindjs
- j-mendez
- Dependencies
- hashbrown ^0.12 normal
  lazy_static ^1.4.0 normal
  log ^0.4.16 normal
  num_cpus ^1.13.1 normal
  rayon ^1.5.2 normal
  regex ^1.5.0 normal
  reqwest ^0.11.10 normal
  robotparser-fork ^0.10.5 normal
  scraper ^0.13 normal
  tokio ^1.17.0 normal
  url ^2.2.2 normal
- Versions
- 94.44% of the crate is documented
Go to latest version
Platform
Feature flags

docs.rs

Rust

Crate spider

Version 1.7.17
All Items

Modules

Crate spider

source · [−]

Modules

Black list checking url exist.

Configuration structure for Website.

A page scraped.

Application utils.

A website to crawl.