List of all items

Structs

• http::reqwest::RobotsTxtResponse
• model::Error
• model::FetchedRobotsTxt
• model::RequestRate
• model::RobotsTxt
• parser::ParseResult
• parser::ParseWarning

Enums

• model::ErrorKind
• parser::WarningReason

Traits

• http::RobotsTxtClient
• service::RobotsTxtService

Functions

• parser::parse_fetched_robots_txt
• parser::parse_robots_txt

Constants

• http::DEFAULT_USER_AGENT