Skip to main content

Module charset

Module charset 

Source
Expand description

Decoding response bodies that are not UTF-8.

Most of the web is UTF-8, and String::from_utf8_lossy handles it. The common exception by a wide margin is the single-byte Western European family — windows-1252, and ISO-8859-1/latin1, which browsers treat as windows-1252 anyway. Those are a 128-entry table, decoded here directly.

Multi-byte legacy encodings (Shift_JIS, GBK, Big5, EUC-KR) need real tables, and pulling in a full encoding library would add roughly a megabyte to a binary whose whole pitch is being small. Those are decoded lossily and the declared charset is reported on the result instead, so a caller can see why the text looks wrong rather than guessing.

Enums§

Charset
What decoding a body needs, given its declared charset.

Functions§

classify
Classify a charset label.
decode
Decode a body according to its declared charset.
decode_cp1252
Decode windows-1252. Every byte maps to exactly one character, so this cannot fail.
from_content_type
Pull a charset= value out of a Content-Type header.