url-crawler 0.1.1

Configurable parallel web crawler, designed to crawl for content
Documentation 
# 0.1.1

- Remove dependency on the failure crates
- Add the `content_type` field to `UrlEntry::File`

# 0.1.0

- Initial release