Skip to main content

Module encoding

Module encoding 

Source
Expand description

Encoding detection and transcoding to UTF-8.

Tier 1 support: UTF-8, US-ASCII, ISO-8859-1, Windows-1252. These cover the large majority of legacy Western XML documents. Other encodings (UTF-16, Shift-JIS, etc.) are reported as Encoding::Other and produce a clear error if you try to transcode them — Tier 2/3 work to follow.

§Why composable

Detection and transcoding live behind small, separate functions so callers can use them however suits their pipeline:

// Auto-detect + transcode in one call, then parse the resulting UTF-8.
let utf8 = transcode_to_utf8(bytes)?;
let doc  = parse_bytes(&utf8, &ParseOptions::default())?;

For UTF-8 inputs the transcode step is zero-copy (returns Cow::Borrowed) and only adds ~100 bytes of detection work to the parse path.

Enums§

Encoding
A character encoding the parser may encounter.

Constants§

IBM037_TO_UNICODE
IBM037 (EBCDIC US/Canada Latin) to Unicode mapping, one entry per byte.
IBM500_TO_UNICODE
IBM500 (International EBCDIC) to Unicode mapping.
IBM1047_TO_UNICODE
IBM1047 (EBCDIC Open Systems / z/OS Unix Services Latin-1) to Unicode.
IBM1140_TO_UNICODE
IBM1140 (EBCDIC US/Canada Latin with Euro update) to Unicode mapping.

Functions§

declared_encoding_name
Return the encoding name as written in the document’s <?xml ... encoding="X"?> declaration, if present. Works on the raw bytes before any transcoding, so it reports the name a consumer-supplied converter should key on (e.g. "EUC-JP"), unnormalized.
detect
Sniff the encoding of an XML document from its first bytes.
encoding_from_name
Map an encoding name (case-insensitive, with common aliases) onto an Encoding variant. Unknown names become Encoding::Other.
transcode_to_utf8
Detect the encoding of bytes and transcode them into UTF-8.
transcode_to_utf8_as
Transcode bytes into UTF-8 assuming they are in the given encoding.
transcode_to_utf8_strict
Like transcode_to_utf8 but also verifies that the inner XML declaration’s encoding="X" attribute agrees with the encoding detected from the byte stream. Catches malformed documents like a UTF-8 BOM paired with encoding="ISO-8859-1" or a UTF-16 BOM paired with encoding="UTF-8".