Module robotstxt::matcher [−][src]
A matcher module.
Structs
LongestMatchRobotsMatchStrategy | Implements the default robots.txt matching strategy. The maximum number of characters matched by a pattern is returned as its match priority. |
RobotsMatcher | RobotsMatcher - matches robots.txt against URLs. |
Traits
RobotsMatchStrategy | Create a RobotsMatcher with the default matching strategy. |