pub fn sniff(
bytes: &[u8],
transport: Option<&'static Encoding>,
fallback: &'static Encoding,
) -> (&'static Encoding, Confidence)Expand description
The HTML Standard’s encoding sniffing algorithm, for a parser that has
every byte: the byte order mark, then transport, then the prescan of
the first 1,024 bytes, then autodetection, then fallback.
The spec lets a browser guess from the bytes before falling back, and
browsers do: Firefox’s chardetng and Chrome’s CED both recognise
UTF-8. Parsec’s guess is that one only, and deterministic: bytes with
something beyond ASCII that are all valid UTF-8 are UTF-8, tentative,
so a later <meta> can still change it. Anything else falls back.
use parsec_encoding::{Confidence, sniff};
let page = b"<meta charset=windows-1251><p>\xcf\xf0\xe8\xe2\xe5\xf2";
let found = sniff(page, None, encoding_rs::WINDOWS_1252);
assert_eq!(found, (encoding_rs::WINDOWS_1251, Confidence::Tentative));