Struct twistrs::enrich::DomainMetadata

pub struct DomainMetadata {
    pub fqdn: String,
    pub ips: Option<Vec<IpAddr>>,
    pub smtp: Option<SmtpMetadata>,
    pub http_banner: Option<String>,
    pub geo_ip_lookups: Option<Vec<(IpAddr, String)>>,
    pub who_is_lookup: Option<String>,
}

Container to store interesting FQDN metadata on domains that we resolve.

Whenever any domain enrichment occurs, the following struct is return to indicate the information that was derived.

N.B—there will be cases where a single domain can have multiple DomainMetadata instancees associated with it.

Fields

fqdn: String

The domain that is being enriched.

ips: Option<Vec<IpAddr>>

Any IPv4 and IPv6 ips that were discovered during domain resolution.

smtp: Option<SmtpMetadata>

Any SMTP message data (if any) that was returned by an SMTP server.

http_banner: Option<String>

HTTP server banner data extracted.

geo_ip_lookups: Option<Vec<(IpAddr, String)>>

IP addresses resolved through GeoIP lookup to City, Country, Continent.

who_is_lookup: Option<String>

Block of text returned by the WhoIs registrar.

Implementations

impl DomainMetadata

pub fn new(fqdn: String) -> DomainMetadata

Create a new empty state for a particular FQDN.

pub async fn dns_resolvable<'_>(&'_ self) -> Result<DomainMetadata>

Asynchronous DNS resolution on a DomainMetadata instance.

Returns Ok(DomainMetadata) is the domain was resolved, otherwise returns Err(EnrichmentError).

N.B—also host lookups are done over port 80.

pub async fn mx_check<'_>(&'_ self) -> Result<DomainMetadata>

Asynchronous SMTP check. Attempts to establish an SMTP connection to the FQDN on port 25 and send a pre-defi ned email.

Currently returns Ok(DomainMetadata) always, which internally contains Option<SmtpMetadata>. To check if the SMTP relay worked, check that DomainMetadata.smtp is Some(v).

pub async fn http_banner<'_>(&'_ self) -> Result<DomainMetadata>

Asynchronous HTTP Banner fetch. Searches and parses server header from an HTTP request to gather the HTTP banner.

Note that a HEAD request is issued to minimise bandwidth. Also note that the internal HttpConnector sets the response buffer window to 1024 bytes, the CONNECT timeout to 5s and enforces HTTP scheme.

use twistrs::enrich::DomainMetadata;
 
#[tokio::main]
async fn main() {
    let domain_metadata = DomainMetadata::new(String::from("www.phishdeck.com"));
    println!("{:?}", domain_metadata.http_banner().await);
}

Panics

Currently panics if the HTTP server header value is not parseable. For more information please refer to the Hyper implementation.

pub async fn geoip_lookup<'_, '_>(
    &'_ self,
    geoip: &'_ Reader<Vec<u8>>
) -> Result<DomainMetadata>

Asynchronous cached GeoIP lookup. Interface deviates from the usual enrichment interfaces and requires the callee to pass a maxminddb::Reader to perform the lookup through. Internally, the maxminddb call is blocking and may result in performance drops, however the lookups are in-memory.

The only reason you would want to do this, is to be able to get back a DomainMetadata to then process as you would with other enrichment methods. Internally the lookup will try to stitch together the City, Country & Continent that the IpAddr resolves to.

use maxminddb::Reader;    
use twistrs::enrich::DomainMetadata;
 
#[tokio::main]
async fn main() {
    let reader = maxminddb::Reader::open_readfile("./data/MaxMind-DB/test-data/GeoIP2-City-Test.mmdb").unwrap();
    let domain_metadata = DomainMetadata::new(String::from("www.phishdeck.com"));
    println!("{:?}", domain_metadata.geoip_lookup(&reader).await);
}

Panics

Currently assumes that if a City/Country/Continent is found, that the English ("en") result is available.

Features

This function requires the geoip_lookup feature toggled.

pub async fn whois_lookup<'_>(&'_ self) -> Result<DomainMetadata>

Asyncrhonous WhoIs lookup using cached WhoIs server config. Note that the internal lookups are not async and so this should be considered a heavy/slow call.

use twistrs::enrich::DomainMetadata;
 
#[tokio::main]
async fn main() {
    let domain_metadata = DomainMetadata::new(String::from("www.phishdeck.com"));
    println!("{:?}", domain_metadata.whois_lookup().await);
}

Features

This function requires the whois_lookup feature toggled.

pub async fn all<'_>(&'_ self) -> Result<Vec<DomainMetadata>>

Performs all FQDN enrichment methods on a given FQDN. This is the only function that returns a Vec<DomainMetadata>.

N.B -- this is currently very slow, and serializes the operations rather than running them concurrently. It should only be used for testing or debugging purposes.

Panics

Currently panics if any of the internal enrichment methods returns an Err.

Trait Implementations

impl Clone for DomainMetadata

fn clone(&self) -> DomainMetadata

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DomainMetadata

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DomainMetadata

fn default() -> DomainMetadata

Returns the "default value" for a type.