Expand description
Bytes into Parsec. parse_bytes finds a page’s encoding the way the
HTML Standard says a browser does, decodes it and parses it.
ByteParser does the same with the bytes as they arrive.
The decoding is encoding_rs, Firefox’s implementation of the WHATWG
Encoding Standard, re-exported here. It has every encoding the web
uses, decoding and encoding, whole or streamed:
use parsec_encoding::Encoding;
let sjis = Encoding::for_label(b"sjis").unwrap();
assert_eq!(sjis.name(), "Shift_JIS");
let (text, _, had_errors) = sjis.decode(b"\x93\xfa\x96\x7b");
assert_eq!((&*text, had_errors), ("日本", false));
let (bytes, _, _) = sjis.encode("日本");
assert_eq!(&*bytes, b"\x93\xfa\x96\x7b");Re-exports§
pub use encoding_rs;
Structs§
- Byte
Parser - Parses a page given as bytes a piece at a time, as a browser does while
it downloads one. You get what
parse_bytesgives for all the pieces joined: the same document, encoding and confidence, however the bytes were cut. - Encoding
- An encoding as defined in the Encoding Standard.
- Offsets
- Turns offsets into a decoded page into offsets into its bytes, and back, for the spans and parse errors of a document parsed from bytes.
- Options
- How to turn bytes into a document.
- Parsed
- A document parsed from bytes, and how its encoding was decided.
Enums§
- Confidence
- How sure the parser is of an encoding, in the HTML Standard’s terms.
Functions§
- locale_
default - The fallback encoding a browser uses for a user whose language is
locale(a BCP 47 tag such asja,ru-RUorzh-Hant), from the HTML Standard’s table of suggested defaults. Pass it asOptions::fallbackwhen you know your users’ language, or a site’s. - parse_
bytes - Parses a page given as bytes, as a browser would: sniffs its encoding
(
sniff), decodes it and parses it. - prescan
- The encoding a
<meta>(or an XML declaration) inbytesdeclares, as the HTML Standard’s prescan finds it, readingbytesand nothing past them.crate::sniffgives it the first 1,024 bytes, as the spec advises; pass more or fewer to scan further or less. - sniff
- The HTML Standard’s encoding sniffing algorithm, for a parser that has
every byte: the byte order mark, then
transport, then the prescan of the first 1,024 bytes, then autodetection, thenfallback.