Expand description
Lazy, source-agnostic reading of a document: the header, then only the bytes asked for. See container::Document.
Lazy access to a VSF document: the header first, then exactly the bytes a caller asks for — and, when the document has been appended to, its generations.
crate::VsfHeader::decode and crate::VsfSection::parse want the whole document in memory and copy every value they meet, which is right for a 2 KB message and wrong for a 50-megapixel plane or a 4 GB recording. A Document opens a ByteSource — a slice, a Vec, a File — by reading its header (and nothing past it), hands out the table of contents, and fetches a section’s bytes only when asked. For the bulk values, the bit-packed planes and wrapped payloads, Document::packed and Document::wrapped locate the payload without copying it, and Document::packed_rows_u16 reads just the rows a viewer needs.
A document edited by appending (see append) ends in a trailer naming its latest header; open follows it, and Document::generations walks the chain back to the original. Offsets in a table of contents are measured from the start of the file, so a section is found by its entry wherever it was written, never by scanning.
Re-exports§
pub use append::Appender;pub use append::TRAILER_LEN;pub use append::append_to_file;pub use append::truncate_file;pub use skip::FieldSpan;pub use skip::ValueKind;pub use skip::ValueSpan;pub use source::ByteSource;pub use source::SourceError;
Modules§
- append
- Append-only generations: edit a document by adding to its end, never by rewriting a byte of it.
- skip
- Walk a section’s bytes recording WHERE each value is, without materialising the big ones. The bulk types — bit-packed planes
pand wrapped payloadsv— are located from their headers alone; everything else is small and goes through the ordinary parser, which doubles as the skip. - source
- Where a document’s bytes come from. A
Documentnever asks for more than it needs — the header, then the entries a caller touches — so the same code reads a 40-byte ping in memory and a 4 GB recording on disk. In-memory sources also lend slices, so acrate::types::BitPackedViewover them copies nothing.
Structs§
- Document
- A document open on a source. Holds the decoded header of its latest generation; everything else is fetched on demand.
- Generation
- One generation of a document: where its bytes are and what wrote it.
- Packed
Span - A located bit-packed tensor: header decoded, samples still in the source at
data_offset. - Wrapped
Span - A located wrapped payload (
v): its code and the bytes, still in the source.