Module robotstxt_with_cache::matcher [−][src]
Structs
| CachingRobotsMatcher | |
| LongestMatchRobotsMatchStrategy | Implements the default robots.txt matching strategy. The maximum number of characters matched by a pattern is returned as its match priority. |
| Match | Instead of just maintaining a Boolean indicating whether a given line has matched, we maintain a count of the maximum number of characters matched by that pattern. |
| RobotsMatcher | RobotsMatcher - matches robots.txt against URLs. |
Traits
| RobotsMatchStrategy | Create a RobotsMatcher with the default matching strategy. |