Function parse_robots_txt 

pub fn parse_robots_txt(origin: Origin, input: &str) -> ParseResult<RobotsTxt>

Parses the text of the robots.txt file located in the specified origin.