spider-2.20.0
- spider 2.20.0
- Docs.rs crate page
- MIT
- Links
- Repository
- crates.io
- Source
- Owners
- madeindjs
- j-mendez
- Dependencies
- ahash ^0.8 normal
  aho-corasick ^1 normal optional
  async-openai ^0.26 normal optional
  async-trait ^0.1 normal optional
  async_job ^0.1 normal optional
  auto_encoder ^0.1 normal
  base64 ^0.22 normal optional
  bytes ^1 normal
  case_insensitive_string ^0.2 normal
  chrono ^0.4 normal optional
  const_format ^0.2 normal optional
  cron ^0.12 normal optional
  cssparser ^0.31.2 normal
  ego-tree ^0.6.2 normal
  fast_html5ever ^0.26.6 normal
  fastrand ^2 normal optional
  flexbuffers ^2 normal optional
  hashbrown ^0.15 normal
  http ^1 normal optional
  http-cache ^0.20 normal optional
  http-cache-reqwest ^0.15 normal optional
  http-cache-semantics ^2 normal optional
  httpdate ^1 normal optional
  itertools ^0.13 normal optional
  lazy_static ^1 normal
  log ^0.4 normal
  lol_html ^2 normal
  moka ^0.12 normal optional
  num_cpus ^1 normal
  percent-encoding ^2 normal
  phf ^0.11 normal
  phf_codegen ^0.11 normal
  quick-xml ^0.37 normal
  rand ^0.8 normal optional
  regex ^1 normal
  reqwest-middleware ^0.4 normal optional
  selectors ^0.25.0 normal
  serde ^1 normal optional
  serde_json ^1 normal optional
  serde_regex ^1 normal optional
  sitemap ^0.4 normal optional
  smallvec ^1 normal
  spider_chrome ^2 normal optional
  statrs ^0.17 normal optional
  string-interner ^0.18 normal optional
  string_concat ^0.0.1 normal
  strum ^0.26 normal
  tendril ^0.4.3 normal
  tiktoken-rs ^0.5 normal optional
  tokio-stream ^0.1 normal
  tracing ^0.1 normal optional
  ua_generator ^0.5 normal optional
  url ^2 normal
  tikv-jemallocator ^0.6 normal optional
  fastrand ^2 normal optional
  reqwest ^0.12 normal
  tokio ^1 normal
  fastrand ^2 normal optional
  reqwest ^0.12 normal
  tokio ^1 normal
- Versions
- 100% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Files

spider/packages/
mod.rs

1
2
3

/// robot parser
pub mod robotparser;
pub mod scraper;

spider/packages/mod.rs

spider/packages/
mod.rs