Expand description
One crawled URL and the fields pulled out of it.
Structs§
Enums§
- Fetch
Failure - Why a fetch produced no response.
- Indexability
- Json
LdStatus
Functions§
- indexability
- Whether search engines would index this URL. Status 0 (no response) counts as a server error.
- is_
nofollow nofollowornonein the meta robots tag or theX-Robots-Tagheader.- is_
noindex noindexornonein the meta robots tag or theX-Robots-Tagheader.