Module charset 

Decoding response bodies that are not UTF-8.

Most of the web is UTF-8 and takes the fast path here. The rest is decoded through encoding_rs, the same implementation Firefox uses, which covers the whole WHATWG Encoding Standard: the single-byte Western family (windows-1252, ISO-8859-*), the CJK multi-byte encodings (Shift_JIS, GBK, GB18030, Big5, EUC-KR, EUC-JP, ISO-2022-JP), KOI8, and UTF-16.

An earlier version hand-rolled a windows-1252 table and reported everything else as undecodable, on the grounds that conversion tables would bloat a binary that advertises being small. Measured, the tables cost about 0.2 MB — against handing back mojibake for every Japanese, Chinese and Korean page, which is most of the non-Latin web.

Label lookup follows the WHATWG rules, so the aliases real pages use (latin1, sjis, x-gbk, ms949, …) all resolve.

Enums

Charset

What decoding a body needs, given its declared charset.

Functions

classify

Classify a charset label.

decode

Decode a body according to its declared charset.

from_content_type

Pull a charset= value out of a Content-Type header.

sniff_meta

Find <meta charset=…> in the head of a body whose header declared nothing.