spider_cli-1.8.2 is not a library.
Spider CLI
Is a command line tool to utilize the spider.
Dependencies
On Linux
- OpenSSL 1.0.1, 1.0.2, 1.1.0, or 1.1.1
Usage
The CLI is a binary so do not add it to your Cargo.toml
file.
Cli
The following can also be ran via command line to run the crawler.
Website args are optional except domain
. If you need verbose output pass in the -v
flag.
Crawl and output all links visited on finished to a file.
All website options are available except website.on_link_find_callback
.