Maman
Maman is a Rust Web Crawler saving pages on Redis.
Pages are send to list <MAMAN_ENV>:queue:maman
using
Sidekiq job format
Dependencies
Installation
With cargo
cargo install maman
With make
PREFIX=~/.local make install
Usage
maman URL [LIMIT]
LIMIT
must be an integer or 0
is the default, meaning no limit.
Environment variables
Defaults
- MAMAN_ENV=development
- REDIS_URL="redis://127.0.0.1/"
Others
- RUST_LOG=maman=info
LICENSE
The MIT License
Copyright (c) 2016-2017 Laurent Arnoud laurent@spkdev.net