pub fn get_crawl_delay(policy: &RobotsPolicy, user_agent: &str) -> Option<f64>Expand description
Get crawl delay for a user-agent (convenience function)
pub fn get_crawl_delay(policy: &RobotsPolicy, user_agent: &str) -> Option<f64>Get crawl delay for a user-agent (convenience function)