vil_crawler 0.4.0

VIL Web Crawler — async concurrent BFS crawling with robots.txt support (I01)
Documentation