get_crawl_delay

Function get_crawl_delay 

Source
pub fn get_crawl_delay(policy: &RobotsPolicy, user_agent: &str) -> Option<f64>
Expand description

Get crawl delay for a user-agent (convenience function)