argus-crawler 0.1.0

A production-ready web crawler capable of handling billions of URLs