halldyll-robots 0.1.0

robots.txt parser and compliance for halldyll scraper
Documentation