Expand description
Pure-Rust implementation of the Riegeli/records file format.
Riegeli is a high-performance, seekable, compressed record store used in machine learning and data pipelines. This crate provides byte-level interoperability with the C++ reference implementation.
Re-exports§
pub use proto_generated::RecordsMetadata;
Modules§
- proto
- Protobuf field-level streaming I/O.
- proto_
generated - varint
- Variable-length integer encoding/decoding (LEB128, protobuf-compatible).
Structs§
- Compress
Options - Compression tuning options passed to low-level compression functions.
- Field
- A proto field path from the root message, represented as a sequence of field numbers.
- Field
Projection - A set of proto field paths used to prune columns at read time.
- Reader
Options - Options for configuring a
RecordReader. - Record
Position - A logical position within a Riegeli file identifying a specific record.
- Record
Reader - A reader that parses a Riegeli file record by record.
- Record
Writer - A writer that produces a valid Riegeli file.
- Skipped
Region - A region of invalid file content skipped by the recovery mechanism.
- Writer
Options - Options for configuring a
RecordWriter.
Enums§
- Compression
Type - The compression algorithm applied to the record data inside a chunk.
- Riegeli
Error - A decoded error from chunk operations.