spider_cli-1.27.3 is not a library.
Spider CLI
A fast command line spider or crawler.
Dependencies
On Linux
- OpenSSL 1.0.1, 1.0.2, 1.1.0, or 1.1.1
Usage
The CLI is a binary so do not add it to your Cargo.toml
file.
Cli
The following can also be ran via command line to run the crawler.
If you need loging pass in the -v
flag.
Crawl and output all links visited to a file.
)
All features are available except the Website struct on_link_find_callback
configuration option.