url-crawler 0.3.0

Configurable parallel web crawler, designed to crawl for content
Documentation