Dyer is designed for reliable, flexible and fast web-crawling, providing some high-level, comprehensive features without compromising speed.
By means of event-driven, non-blocking I/O tokio and powerful, reliable Rust programming
language, inspired by scrapy, Dyer provides some high-level features.
- asynchronous, concurrent streamimg and I/O, make the best of thread pool, network, and system resource.
- event-driven, once you set the initials and recursive generator of
Task,Dyerwill handle the rest of it. - user-friendly, considering the philosophy of rust programming language, more source code,
proper framework may contribute the awkwardness when efficiency and learning cost are taken
into consideration.
Dyeroffers high-level and flexible wrappers and API what does a lot for you.