spider 1.50.22

The fastest web crawler written in Rust.
Documentation

spider

There is very little structured metadata to build this page from currently. You should check the main library docs, readme, or Cargo.toml in case the author documented the features in them.

This version has 29 feature flags, 2 of them enabled by default.

default

  • sync
  • reqwest/native-tls-alpn

sync

  • tokio/sync

cron

  • dep:async_job
  • dep:chrono
  • dep:cron
  • dep:async-trait

serde

  • dep:serde
  • hashbrown/serde
  • compact_str/serde

glob

  • dep:regex
  • dep:itertools

napi_rustls_tls

  • napi
  • reqwest/rustls-tls

decentralized

  • serde
  • flexbuffers

chrome_cpu

  • chrome

jemalloc

  • tikv-jemallocator

fs

  • tokio/fs

regex

  • dep:regex

js

  • dep:jsdom

sitemap

  • dep:sitemap

chrome_screenshot

  • chrome

cookies

  • reqwest/cookies

tikv-jemallocator

  • dep:tikv-jemallocator

flexbuffers

  • dep:flexbuffers

napi

  • dep:napi

reqwest_json

  • reqwest/json

chrome_stealth

  • chrome

chrome_headed

  • chrome

chrome

  • dep:chromiumoxide

http3

  • reqwest/http3

ua_generator

  • dep:ua_generator

socks

  • reqwest/socks

full_resources

    This feature flag does not enable additional features.

budget

    This feature flag does not enable additional features.

control

    This feature flag does not enable additional features.

time

    This feature flag does not enable additional features.