scrapling-spider 0.1.1

Concurrent web crawler framework for scrapling

Coverage
100%
162 out of 162 items documented0 out of 101 items with examples
Size
Source code size: 189.04 kB This is the summed size of all the files inside the crates.io package for this release.
Documentation size: 3.48 MB This is the summed size of all files generated by rustdoc for all configured targets
Ø build duration
this release: 4m 27s Average build duration of successful builds.
all releases: 4m 11s Average build duration of successful builds in releases after 2024-10-23.
Links
chrisabruce/scrapling-rs
1 0 0
crates.io
Dependencies
Versions
Owners

scrapling-spider

Concurrent web crawler framework for scrapling-rs.

Concurrent crawling with configurable parallelism
Request deduplication via SHA-1 fingerprinting
Robots.txt compliance with crawl-delay support
Checkpoint/resume for long-running crawls
Development mode with response caching
Built on scrapling-fetch for TLS impersonation and scrapling-browser for JS-rendered pages

MIT