1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
//! Domain extraction and normalization utilities.
//!
//! This module provides functions to extract and normalize domain names from URLs
//! using the Public Suffix List (PSL) to correctly identify registrable domains.
//!
//! Key functions:
//! - `extract_domain()` - Extracts the registrable domain from a URL
//! - `normalize_domain()` - Normalizes domain names (lowercase, removes www)
//! - `root_domain()` - Registrable domain (eTLD+1) from a bare hostname/FQDN
//!
//! ## Strict hostname validation
//!
//! For URL-based input, PSL and `url` parsing are sufficient. If we add features that
//! accept raw hostnames or subdomains (e.g. config-driven allowlists or subdomain
//! enumeration), consider using a crate such as [`addr`](https://crates.io/crates/addr)
//! for strict hostname validation (invalid labels, leading/trailing dots, character
//! allowlists) alongside PSL.
use ;
/// Extracts the registrable domain from a URL using PSL.
///
/// # Arguments
///
/// * `url` - The URL to extract the domain from
///
/// # Returns
///
/// The registrable domain (e.g., "example.com" from "<https://www.example.com/path>").
/// For IP-literal hosts, returns the IP string (used as the UPSERT/domain key).
/// Production SSRF checks still block private/loopback IPs before fetch.
///
/// # Errors
///
/// Returns an error if the URL cannot be parsed, has no host, or PSL extraction fails
/// for a non-IP hostname.
///
/// Uses `psl` to correctly extract the registrable domain, handling
/// both simple TLDs (e.g., "example.com") and multi-part TLDs (e.g., "example.co.uk").
/// Normalizes a domain name: ASCII-lowercase and strip a leading `www.`.
///
/// Does not apply PSL rules — use [`extract_domain`] or [`root_domain`] for
/// registrable-domain extraction.
///
/// Kept as a shared helper even when no production call site needs it yet;
/// unit tests cover the documented contract.
/// Returns the PSL registrable domain (eTLD+1) for a given FQDN/hostname.
///
/// Uses `psl::domain_str()` which correctly implements the Public Suffix List
/// algorithm. For most domains this gives the expected organizational domain
/// (e.g., `www.facebook.com` -> `facebook.com`).
///
/// For wildcard PSL entries like `*.cloudfront.net`, the registrable domain
/// equals the full FQDN (e.g., `d123.cloudfront.net`) because each subdomain
/// under a wildcard suffix is independently registered.
///
/// Bare public suffixes (e.g. `googleapis.com`) have no registrable domain in
/// PSL terms; we fall back to the FQDN itself when `psl::suffix_str` recognizes
/// it, so callers still get a stable organizational key.