Module robotstxt::matcher [−][src]
A matcher module.
Structs
| LongestMatchRobotsMatchStrategy | Implements the default robots.txt matching strategy. The maximum number of characters matched by a pattern is returned as its match priority. |
| RobotsMatcher | RobotsMatcher - matches robots.txt against URLs. |
Traits
| RobotsMatchStrategy | Create a RobotsMatcher with the default matching strategy. |