Module encoding 

Character encoding handling (§22, §85 Phase 4).

Encoding detection, XML declaration encoding, BOM behavior, UTF-8/UTF-16 validity, legacy encodings, conversion errors, output conversion, serializer fallback, custom encoding handlers.

Architecture

ABI exports (exports_xml2.rs)  ←  pub(crate) functions in this module
                                          ↕
                          Encoding handler registry (global RwLock)
                                          ↕
             Built-in handlers: UTF-8, UTF-16LE, UTF-16BE, Latin-1, ASCII

The internal encoding is always UTF-8. All conversions go to/from UTF-8. The handler registry stores _xmlCharEncodingHandler structs that contain function pointers for input (→UTF-8) and output (UTF-8→) conversion.

Upstream contract

Mirrors upstream encoding.c / encoding.h (SRC-LIBXML2-2.15.0-ENCODING-C, parity target libxml2 2.15.3 oracle). ABI surface: xmlLookupCharEncodingHandler, xmlGetCharEncodingHandler, xmlOpenCharEncodingHandler, xmlCreateCharEncodingHandler, xmlCharEncInput/xmlCharEncOutput and the _xmlCharEncodingHandler C-layout struct (R-000129 fixed the Rust mirror from 48 to the upstream 56 bytes).

Conceptual behavior

Detection runs BOM first, then the XML declaration, then registry lookup. The registry mirrors upstream defaultHandlers[32] plus the extra-handler table (globalHandlers, encoding.c): named handlers are registered under their canonical lowercased alias and found by xmlFindCharEncodingHandler via find_encoding_handler.

Ownership & safety invariants

Handlers are allocated with xmlMalloc and owned by the registry; xmlFree releases them at teardown. Registry access is serialized by an RwLock; that serialization is exactly what makes the raw HandlerPtr Send+Sync SAFETY sound (documented on the wrapper). Names from xmlGetCharEncodingName/alias tables are borrowed statics — the caller never frees them.

Historical quirks & epochs

R-000157 (OPEN, UNRESOLVED): the crate ships no iconv/ICU backend, so the iconv/ICU-only encodings (UCS-4LE/BE, EBCDIC, UCS-2, ISO-8859-2..16, ISO-2022-JP, Shift_JIS, EUC-JP, windows-1252) report XML_ERR_UNSUPPORTED_ENCODING (32) where the 2.15.3 oracle (built with Iconv+ICU enabled) returns a converter, while the native set (UTF-8, UTF-16LE/BE, UTF-16, ISO-8859-1, US-ASCII) and all error paths are byte-identical. This is a REAL current executed-platform difference, so the residual is UNRESOLVED (11.1-Z.1) — closure requires implementing an iconv/ICU backend, a future implementation work item, not a waiver. Upstream itself removed the libiconv dependence where possible in the 2.10+ era (HISTORY.md §1.8), which is the epoch this module targets.

Deliberate oddities

The bounded native set is a deliberate divergence, not a stub: the missing encodings are absent because no converter exists, and every error path matches the oracle. xmlLookupCharEncodingHandler returns XML_ERR_OK with a NULL handler for UTF-8/NONE exactly like upstream encoding.c (/* Return NULL handler for UTF-8 */). R-000157 is tracked UNRESOLVED: adding an iconv/ICU backend would close the gap for the encodings the executed oracle serves.

Proving courts

ENCODING-001 (courts/suites/data-abi/encoding-family-probe.c) compiles one C probe against the oracle DSO and the candidate and requires byte-identical stdout across the native set and all error paths.

Tempting simplifications that would break parity

Do not collapse the registry to a fixed match statement: custom handlers added through xmlAddCharEncodingHandler must stay discoverable by later lookups. Do not fabricate handlers for the iconv-only encodings — that would fake a converter that does not exist and break the R-000157 UNRESOLVED record (the honest path is a real iconv/ICU backend). Do not touch the struct layout: R-000129 proved a 48-byte mirror breaks the C ABI.