Crate public_suffix

Expand description

About

public-suffix provides a public suffix list based on data from https://publicsuffix.org/.

A public suffix is one under which Internet users can directly register names. It is related to, but different from, a TLD (top level domain).

“com” is a TLD (top level domain). Top level means it has no dots.

“com” is also a public suffix. Amazon and Google have registered different siblings under that domain: “amazon.com” and “google.com”.

“au” is another TLD, again because it has no dots. But it’s not “amazon.au”. Instead, it’s “amazon.com.au”.

“com.au” isn’t an actual TLD, because it’s not at the top level (it has dots). But it is an eTLD (effective TLD), because that’s the branching point for domain name registrars.

Another name for “an eTLD” is “a public suffix”. Often, what’s more of interest is the eTLD+1, or one more label than the public suffix. For example, browsers partition read/write access to HTTP cookies according to the eTLD+1. Web pages served from “amazon.com.au” can’t read cookies from “google.com.au”, but web pages served from “maps.google.com” can share cookies from “www.google.com”, so you don’t have to sign into Google Maps separately from signing into Google Web Search. Note that all four of those domains have 3 labels and 2 dots. The first two domains are each an eTLD+1, the last two are not (but share the same eTLD+1: “google.com”).

All of these domains have the same eTLD+1:

“www.books.amazon.co.uk”
“books.amazon.co.uk”
“amazon.co.uk” Specifically, the eTLD+1 is “amazon.co.uk”, because the eTLD is “co.uk”.

use public_suffix::{DEFAULT_PROVIDER, EffectiveTLDProvider, Error};

assert_eq!(
    DEFAULT_PROVIDER.effective_tld_plus_one("www.books.amazon.com.au"),
    Ok("amazon.com.au")
);
assert_eq!(
    DEFAULT_PROVIDER.effective_tld_plus_one("books.amazon.com.au"),
    Ok("amazon.com.au")
);
assert_eq!(
    DEFAULT_PROVIDER.effective_tld_plus_one("amazon.com.au"),
    Ok("amazon.com.au")
);
assert_eq!(
    DEFAULT_PROVIDER.effective_tld_plus_one("com.au"),
    Err(Error::CannotDeriveETldPlus1)
);
assert_eq!(
    DEFAULT_PROVIDER.effective_tld_plus_one("au"),
    Err(Error::CannotDeriveETldPlus1)
);

There is no closed form algorithm to calculate the eTLD of a domain. Instead, the calculation is data driven. This package provides a pre-compiled snapshot of Mozilla’s PSL (Public Suffix List) data at https://publicsuffix.org/

`default_provider` Feature and Custom TLD Lists

This crate comes with a version of the Mozilla Public Suffix List built in. This is controlled by a crate feature called default_provider which is enabled by default. Disabling this feature removes the provided TLD list from the compiled binary, potentially saving some size, and allows the user to provide their own. See the documentation for ListProvider and Table for more details.

Updating to the latest version of the Public Suffix List:

Make sure you have golang installed.
Make the public-suffix crate the current working directory.
wget https://publicsuffix.org/list/public_suffix_list.dat, which will overwrite the old version of this file.
Run ./gen.sh to regenerate the list from the updated public_suffix_list.dat. The first time you run this, you’ll need network connectivity to go get the dependencies.
Commit the changed generated source code and the updated public_suffix_list.dat.

We intentionally do not try to download the latest version of the public suffix list during the build to keep the build deterministic and networking-free.

We’d like to avoid checking in the Rust source code generated from public_suffix_list.dat, but we don’t want the build to depend on the Go compiler.

Structs

ListProvider
ListProvider is a generic struct that provides results based on a standard eTLD list generated by the included Golang program. To override the list included with this crate, disable the default_provider crate feature and create a ListProvider with your own implmentation of the Table trait, generated from your own custom list.

Enums

Error
Error types returned from ListProvider::effective_tld_plus_one

Constants

DEFAULT_PROVIDER
DEFAULT_PROVIDER provides a default instance of ListProvider that provides results based on the standard Mozilla Public Suffix List.

Traits

EffectiveTLDProvider
The EffectiveTLDProvider trait allows other crates in passkey-rs to use a custom domain TLD provider instead of using the DEFAULT_PROVIDER from this crate.
Table
Implementation to allow the use of custom tables.

Type Aliases

PublicSuffixList
This type is provided as part of the default_provider feature as a concrete instantiation of ListProvider using this crate’s default TLD list.