Expand description
Keeps the cloud crawler away from private and internal addresses.
Two layers, because reqwest never calls the resolver for IP-literal hosts:
check_url rejects blocked IP literals before a request is made, and
GuardedResolver drops blocked addresses from DNS answers. The fetcher
re-runs check_url on every redirect hop.
Structs§
- Guarded
Resolver - A reqwest resolver that only returns addresses allowed by
is_blocked. - System
Lookup - The operating system’s resolver.
Enums§
Traits§
- Lookup
- Turns a host name into IP addresses.
Functions§
- check_
url - Rejects URLs whose host is a blocked IP literal. Domain names are checked
later, by
GuardedResolver. - is_
blocked - True for addresses the cloud crawler must never connect to.