Crate oxixml_encoding 

oxixml-encoding — Character-encoding decoders and encoders for XML processing.

Part of the OxiXML workspace — the COOLJAPAN Pure-Rust foundation for all XML and RDF processing.

This crate provides the character-encoding subset that an XML processor actually needs: it plays the role that encoding_rs plays in other Rust XML stacks, but is written from scratch against std alone, with no external dependencies.

What’s here

• Encoding — the core set of encodings an XML parser must recognize (UTF-8, UTF-16LE, UTF-16BE, ISO-8859-1, windows-1252, US-ASCII), extended with the rest of the WHATWG Encoding Standard’s legacy single-byte encodings and its three legacy Japanese multi-byte encodings (Shift_JIS, EUC-JP, ISO-2022-JP) — see Encoding::ALL for the complete, up-to-date list — plus Encoding::for_label to resolve an IANA/WHATWG charset label (as found in an encoding="..." declaration or a Content-Type header) to one of them.
• Decoder — an incremental, chunk-safe decoder: bytes can be fed in arbitrarily small pieces (even one byte at a time) and a partial multi-byte sequence at a chunk boundary is carried over correctly. Both a strict mode (which reports a DecodeError with a byte offset) and a lossy mode (which substitutes U+FFFD REPLACEMENT CHARACTER) are provided.
• encode_from_utf8 — the inverse direction, used when writing XML in a legacy encoding; reports the first unmappable character and its position.
• detect_bom — recognizes the three byte-order marks relevant to XML.
• sniff_xml_encoding — implements the heuristic described in XML 1.0 Appendix F: inspect the first few bytes of an entity, then (if the bytes look ASCII-compatible or UTF-16) scan the prologue for an encoding="..." pseudo-attribute.

Example

use oxixml_encoding::{sniff_xml_encoding, Encoding};

// A UTF-8 document that self-declares its own encoding.
let xml = b"<?xml version=\"1.0\" encoding=\"UTF-8\"?><root/>";
let sniffed = sniff_xml_encoding(xml);
assert_eq!(sniffed.declared_encoding, Some(Encoding::Utf8));

// Decode a chunk of UTF-8 incrementally, one byte at a time, and get back the original text.
let mut decoder = Encoding::Utf8.new_decoder();
let mut out = String::new();
for (i, byte) in "héllo".as_bytes().iter().enumerate() {
    let last = i + 1 == "héllo".len();
    decoder
        .decode_to_string(std::slice::from_ref(byte), &mut out, last)
        .expect("valid UTF-8 fed one byte at a time");
}
assert_eq!(out, "héllo");

Structs

DecodeError

An error reported by Decoder::decode_to_string.

Decoder

An incremental decoder for one Encoding.

EncodeError

An error reported by encode_from_utf8: character has no representation in the requested target encoding.

SniffResult

The result of sniff_xml_encoding: everything the Appendix F heuristic could determine about an entity’s encoding from its leading bytes and (if reachable) its XML declaration.

Enums

ByteOrderGuess

The coarse encoding family implied purely by the first few bytes of an XML entity, before any attempt to read an encoding="..." declaration.

DecodeErrorKind

The kind of problem a DecodeError reports.

Encoding

A character encoding recognized by this crate.

Functions

detect_bom

Detect a byte-order mark (BOM) at the start of bytes, one of the three signatures relevant to the encodings this crate supports (see XML 1.0 Appendix F.1):

encode_from_utf8

Encode input into encoding, appending the resulting bytes to dst.

sniff_xml_encoding

Sniff the encoding of an XML entity from its leading bytes, per XML 1.0 Appendix F.