About
This tool can be used to determine the adoption of the security.txt standard among several domains. You feed it a list of domains and it will tell you which of them implement the standard already.
The idea was shamelessly stolen from inspired by haksecuritytxt.
So why did I recreate a tool that already exists?
Admittedly, the main motivation was to play around with Rust's new async/await syntax and learn something new.
Besides, I wanted to enforce stricter checks for the standard, i.e., the server must answer with the correct Content-Type header, which leads to more accurate results.
Usage
Moz maintains a list of popular websites.
Running ./scripts/checktop500 downloads that list, and runs sectxt against it.
As a result, you will see which of these websites deploy a security.txt file.
For the shell script to run you need to install xsv, which is another convenient utility by BurntSushi.