pub struct Offsets { /* private fields */ }Expand description
Turns offsets into a decoded page into offsets into its bytes, and back, for the spans and parse errors of a document parsed from bytes.
use parsec_encoding::{Offsets, Options, parse_bytes};
let page = b"<meta charset=shift_jis><p>\x93\xfa\x96\x7b<b>!</b>";
let parsed = parse_bytes(page, Options {
html: parsec::Options { spans: true, ..Default::default() },
..Options::default()
}).unwrap();
let offsets = Offsets::new(page, parsed.encoding);
let doc = &parsed.document;
let b = (0..doc.len() as u32).find(|&n| doc.lower_name(n) == Some("b")).unwrap();
let span = doc.span(b).unwrap();
// 日本 is six bytes decoded, four in Shift_JIS.
assert_eq!(span, 33..36);
let (start, end) = (offsets.to_source(span.start), offsets.to_source(span.end));
assert_eq!(&page[start..end], b"<b>");It’s built by decoding the bytes again, noting where each character’s bytes started (UTF-8 and the single-byte encodings in one pass, the others a byte at a time, which takes two or three times as long as the parse), and keeps only the places where the two stop moving together: after the byte order mark, at each character whose length differs between the two (each Shift_JIS kanji, say), and after each run of bytes the decoding replaced with U+FFFD. ASCII costs nothing; text that’s all non-ASCII costs 8 bytes a character.
Implementations§
Source§impl Offsets
impl Offsets
Sourcepub fn new(bytes: &[u8], encoding: &'static Encoding) -> Offsets
pub fn new(bytes: &[u8], encoding: &'static Encoding) -> Offsets
The offsets for bytes decoded as parse_bytes
and ByteParser decode them in encoding:
with the encoding’s own byte order mark removed.
Sourcepub fn is_identity(&self) -> bool
pub fn is_identity(&self) -> bool
Whether every offset is the same in both: ASCII, or UTF-8 with no byte order mark and nothing replaced.
Sourcepub fn to_source(&self, decoded: usize) -> usize
pub fn to_source(&self, decoded: usize) -> usize
The offset into the bytes of offset decoded into the decoded text.
Sourcepub fn to_decoded(&self, source: usize) -> usize
pub fn to_decoded(&self, source: usize) -> usize
The offset into the decoded text of offset source into the bytes.
One inside a character’s bytes gives one inside its decoding, never
past it.