susbot 2.2.0

robots.txt checker for the command line and CI (sus.bot). Installs the same `susbot` program as the susbot-cli crate.

Size
Source code size: 27.51 kB This is the summed size of all the files inside the crates.io package for this release.
Links
Homepage
sitefig/robots
0 0 0
crates.io
Dependencies
- susbot-cli ^2.2.0 normal
Versions
Owners

susbot-2.2.0 is not a library.

susbot

The command-line version of sus.bot, a robots.txt checker: which search engines and AI crawlers a site lets in, what is wrong with the file, and which paths it reveals.

cargo install susbot
susbot https://example.com

This crate installs the same susbot program as susbot-cli, where the usage is documented. Source and GitHub Action: github.com/sitefig/robots.

The same tool elsewhere: susbot on PyPI, @sitefig/susbot on npm

Licence

PolyForm Noncommercial 1.0.0: free for noncommercial use. Commercial use needs a licence from Sitefig.