Enum Encoding 

pub enum Encoding {
    Utf8,
    Utf16Le,
    Utf16Be,
    Latin1,
    Windows1252,
    Ascii,
    Iso8859_2,
    Iso8859_3,
    Iso8859_4,
    Iso8859_5,
    Iso8859_6,
    Iso8859_7,
    Iso8859_8,
    Iso8859_9,
    Iso8859_10,
    Iso8859_13,
    Iso8859_14,
    Iso8859_15,
    Iso8859_16,
    Windows1250,
    Windows1251,
    Windows1253,
    Windows1254,
    Windows1255,
    Windows1256,
    Windows1257,
    Windows1258,
    Koi8R,
    Koi8U,
    Ibm866,
    Macintosh,
    XMacCyrillic,
    ShiftJis,
    EucJp,
    Iso2022Jp,
    Gbk,
    Gb18030,
    Big5,
    EucKr,
}

A character encoding recognized by this crate.

The original core set is exactly what an XML processor needs to deal with in practice: UTF-8 and UTF-16 (in both byte orders) are the two encodings XML 1.0 requires every conformant processor to support; Encoding::Latin1 and Encoding::Windows1252 cover the overwhelming majority of legacy (encoding="...") documents seen in the wild, and Encoding::Ascii is useful as a strict validation target.

Beyond that core, this crate also implements the rest of the WHATWG Encoding Standard’s legacy single-byte encodings (the full ISO-8859-*, windows-125x, KOI8-*, IBM866, macintosh and x-mac-cyrillic families), its three legacy Japanese multi-byte encodings (Shift_JIS, EUC-JP, and ISO-2022-JP), and its legacy CJK multi-byte encodings (GBK, gb18030, Big5, EUC-KR), prioritized for COOLJAPAN. See Encoding::ALL for the complete list.

Unlike the WHATWG Encoding Standard, which folds ISO-8859-1 into windows-1252 for web compatibility, this crate keeps them distinct: XML tooling (libxml2, Xerces, .NET, the JDK, …) has always treated ISO-8859-1 as the “pure” Latin-1 repertoire (bytes 0x00..=0xFF map to U+0000..=U+00FF unchanged) and windows-1252 as the Microsoft superset that additionally assigns the 0x80..=0x9F range to punctuation such as smart quotes and the euro sign. The same reasoning applies to Encoding::Iso8859_9 versus Encoding::Windows1254: the WHATWG standard folds the "iso-8859-9"/"latin5" labels into windows-1254 too, but this crate keeps pure ISO-8859-9 (ISO/IEC 8859-9:1999, a.k.a. Latin-5) as its own encoding for the same reason.

Variants

Utf8

UTF-8: the ASCII-compatible variable-width encoding of Unicode, and the default assumed by XML 1.0 in the absence of a byte-order mark or encoding declaration.

Utf16Le

UTF-16, little-endian byte order.

Utf16Be

UTF-16, big-endian byte order.

Latin1

ISO-8859-1 (Latin-1): each byte b decodes to the scalar value U+00xx with xx == b.

Windows1252

windows-1252 (a.k.a. cp1252): ISO-8859-1 with the C1 control range 0x80..=0x9F reassigned to printable characters (the five positions Microsoft left unassigned keep their Latin-1 fallback, per the WHATWG index table).

Ascii

US-ASCII: the 7-bit repertoire U+0000..=U+007F. Bytes 0x80..=0xFF are not valid.

Iso8859_2

ISO-8859-2 (Latin-2): Central/Eastern European.

Iso8859_3

ISO-8859-3 (Latin-3): South European (Maltese, Esperanto).

Iso8859_4

ISO-8859-4 (Latin-4): North European (Baltic languages, pre-ISO-8859-13).

Iso8859_5

ISO-8859-5: Cyrillic.

Iso8859_6

ISO-8859-6: Arabic.

Iso8859_7

ISO-8859-7: Greek.

Iso8859_8

ISO-8859-8: Hebrew (visual order; this crate makes no distinction from the “logical order” ISO-8859-8-I variant, since directionality is outside this crate’s scope and both share the same byte/code-point repertoire).

Iso8859_9

ISO-8859-9 (Latin-5): identical to ISO-8859-1 except six positions replaced with the Turkish letters Ğ/ğ, İ/ı, Ş/ş. See the type-level docs for why this is kept distinct from Encoding::Windows1254.

Iso8859_10

ISO-8859-10 (Latin-6): Nordic (an alternative to -4 with full Icelandic coverage).

Iso8859_13

ISO-8859-13 (Latin-7): Baltic.

Iso8859_14

ISO-8859-14 (Latin-8): Celtic.

Iso8859_15

ISO-8859-15 (Latin-9): Latin-1 with the euro sign and a handful of French/Finnish letters.

Iso8859_16

ISO-8859-16 (Latin-10): South-Eastern European.

Windows1250

windows-1250: Central/Eastern European.

Windows1251

windows-1251: Cyrillic.

Windows1253

windows-1253: Greek.

Windows1254

windows-1254 (Turkish): the Microsoft superset of Encoding::Iso8859_9.

Windows1255

windows-1255: Hebrew.

Windows1256

windows-1256: Arabic.

Windows1257

windows-1257: Baltic.

Windows1258

windows-1258: Vietnamese.

Koi8R

KOI8-R: Russian Cyrillic.

Koi8U

KOI8-U: Ukrainian Cyrillic (a KOI8-R variant with a handful of Ukrainian letters).

Ibm866

IBM866 (code page 866): Cyrillic, MS-DOS.

Macintosh

macintosh (a.k.a. “Mac OS Roman”): the classic Mac OS Western European encoding.

XMacCyrillic

x-mac-cyrillic (a.k.a. “Mac OS Cyrillic”): the classic Mac OS Cyrillic encoding.

ShiftJis

Shift_JIS (a.k.a. MS932/Windows-31J): the JIS X 0201 + JIS X 0208 encoding most commonly used on Japanese Windows and in Japanese web content.

EucJp

EUC-JP: the Unix-family Japanese encoding covering JIS X 0201, JIS X 0208, and (via its 0x8F-prefixed SS3 sequences) JIS X 0212.

Iso2022Jp

ISO-2022-JP: the 7-bit, escape-sequence-switched Japanese encoding historically used for Japanese email (Content-Transfer-Encoding: 7bit-safe).

Gbk

GBK: the 2-byte-only Chinese (Simplified) encoding, a.k.a. CP936. Decodes identically to Encoding::Gb18030 (per the WHATWG Encoding Standard, “GBK’s decoder is gb18030’s decoder”); its encoder is the more conservative one, refusing to emit the 4-byte sequences Encoding::Gb18030’s encoder falls back to for code points outside GBK’s repertoire.

Gb18030

gb18030: the Chinese national standard encoding. A superset of Encoding::Gbk with 1/2/4-byte sequences whose encoder is total over Unicode – every scalar value except U+E5E5 has a representation (see crate::encode_from_utf8). This does not quite mean every scalar value round-trips through encode-then-decode: per the WHATWG gb18030 encoder algorithm, 18 Private Use Area code points are deliberately encoded to a fixed legacy byte pair (for compatibility with the GB18030-2005 standard) that decodes back to a different code point under the current index – see crate::cjk::gb18030::encode’s own docs for the full list.

Big5

Big5: the Traditional Chinese encoding historically used in Taiwan and Hong Kong, including the Hong Kong Supplementary Character Set (HKSCS) extensions per the WHATWG index.

EucKr

EUC-KR: the Korean encoding covering KS X 1001 plus the Unified Hangul Code (a.k.a. Windows-949) extension, per the WHATWG index.

Implementations

impl Encoding

pub const ALL: &'static [Encoding]

Every encoding this crate supports, in declaration order. Useful for exhaustively testing or listing supported encodings without having to enumerate the (growing) variant list by hand.

Examples

use oxixml_encoding::Encoding;

assert!(Encoding::ALL.contains(&Encoding::Utf8));
assert!(Encoding::ALL.contains(&Encoding::ShiftJis));
// Every entry's canonical name resolves back to itself.
for &encoding in Encoding::ALL {
    assert_eq!(Encoding::for_label(encoding.name()), Some(encoding));
}

pub fn for_label(label: &str) -> Option<Encoding>

Resolve an IANA/WHATWG charset label to an Encoding, case-insensitively and after trimming ASCII whitespace, mirroring the label tables in the WHATWG Encoding Standard — with the Encoding::Iso8859_9-versus-Encoding::Windows1254 exception documented on Encoding itself.

Returns None for labels this crate does not resolve to a supported encoding (for example "utf-32", "windows-874" or "ebcdic-cp-us").

Examples

use oxixml_encoding::Encoding;

assert_eq!(Encoding::for_label("UTF-8"), Some(Encoding::Utf8));
assert_eq!(Encoding::for_label(" utf8 \t"), Some(Encoding::Utf8));
assert_eq!(Encoding::for_label("Windows-1252"), Some(Encoding::Windows1252));
assert_eq!(Encoding::for_label("iso-8859-1"), Some(Encoding::Latin1));
assert_eq!(Encoding::for_label("Shift_JIS"), Some(Encoding::ShiftJis));
assert_eq!(Encoding::for_label("EUC-JP"), Some(Encoding::EucJp));
assert_eq!(Encoding::for_label("gb18030"), Some(Encoding::Gb18030));
assert_eq!(Encoding::for_label("windows-874"), None);

pub const fn name(&self) -> &'static str

The canonical name of this encoding, as it should appear in generated encoding="..." declarations or diagnostics.

Examples

use oxixml_encoding::Encoding;

assert_eq!(Encoding::Utf8.name(), "UTF-8");
assert_eq!(Encoding::Windows1252.name(), "windows-1252");
assert_eq!(Encoding::ShiftJis.name(), "Shift_JIS");

pub const fn is_ascii_compatible_single_byte(&self) -> bool

Whether this encoding is ASCII-compatible in the sense relevant to XML prologue scanning: every byte in 0x00..=0x7F decodes to the identically-valued scalar, so an ASCII-only substring (such as an <?xml ... ?> declaration) can be located by scanning raw bytes without first performing a full decode.

This holds for every single-byte encoding this crate supports (all of them agree with ASCII below 0x80, by construction — see Encoding::ALL), not only the original four. It does not hold for the seven multi-byte encodings (the three legacy Japanese ones and the four legacy CJK ones): even though each of them also encodes 0x00..=0x7F as ASCII at the start of the document (which is enough to make crate::sniff_xml_encoding’s prologue scan work, since an <?xml ...?> declaration is required to be the first thing in the entity and is itself pure ASCII), a byte elsewhere in the document is not reliably ASCII-interpretable without tracking decoder state (ISO-2022-JP’s shift state, or – for the other six – whether the byte is a lead or trail byte of a multi-byte sequence), so this method intentionally answers false for all seven.

pub const fn new_decoder(self) -> Decoder

Create a new incremental Decoder for this encoding.

Examples

use oxixml_encoding::Encoding;

let mut decoder = Encoding::Utf8.new_decoder();
let mut out = String::new();
decoder.decode_to_string(b"hello", &mut out, true).expect("valid utf-8");
assert_eq!(out, "hello");

Trait Implementations

impl Clone for Encoding

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Encoding

impl Debug for Encoding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Encoding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for Encoding

impl Hash for Encoding

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for Encoding

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Encoding