Expand description
Columnar segment format and memtable for NodeDB analytical storage.
Provides the shared segment format used by all columnar profiles (plain, timeseries, spatial). Segments are self-contained files with per-column compressed blocks, block statistics, and a CRC-validated footer.
§Segment Layout
[SegmentHeader: magic "NDBS" + version + endianness]
[Column 0 blocks][Column 1 blocks]...[Column N blocks]
[SegmentFooter: schema_hash, column metadata, block stats, CRC32C]Re-exports§
pub use compaction::compact_segments;pub use delete_bitmap::DeleteBitmap;pub use error::ColumnarError;pub use filter::bitmask_all;pub use filter::bitmask_and;pub use filter::decoded_dict_eval_contains;pub use filter::decoded_dict_eval_eq;pub use filter::decoded_dict_eval_like;pub use filter::decoded_dict_eval_ne;pub use filter::dict_eval_contains;pub use filter::dict_eval_eq;pub use filter::dict_eval_like;pub use filter::dict_eval_ne;pub use filter::words_for;pub use format::BLOCK_SIZE;pub use format::BlockStats;pub use format::ColumnMeta;pub use format::MAGIC;pub use format::SegmentHeader;pub use format::VERSION_MAJOR;pub use format::VERSION_MINOR;pub use memtable::ColumnarMemtable;pub use memtable::IngestValue;pub use memtable::MemtableRowIter;pub use mutation::MutationEngine;pub use pk_index::PkIndex;pub use predicate::BLOOM_BYTES;pub use predicate::PredicateOp;pub use predicate::PredicateValue;pub use predicate::ScanPredicate;pub use predicate::bloom_insert;pub use predicate::bloom_may_contain;pub use predicate::build_bloom;pub use reader::SegmentReader;pub use writer::SegmentWriter;
Modules§
- compaction
- Segment compaction: merge segments, drop deleted rows, re-encode.
- delete_
bitmap - Per-segment delete bitmap for columnar UPDATE/DELETE.
- error
- Error types for columnar segment operations.
- filter
- Vectorized predicate evaluation on dictionary-encoded columns.
- format
- Columnar segment binary format definitions.
- memtable
- Columnar memtable: in-memory row buffer with typed column vectors.
- mutation
- Columnar mutation engine: coordinates PK index, delete bitmaps, memtable, and WAL records for full INSERT/UPDATE/DELETE.
- pk_
index - Primary key index for columnar collections.
- predicate
- Scan predicates for block-level predicate pushdown.
- reader
- Segment reader: decode compressed columns from a segment into typed vectors or Arrow arrays, with column projection and block predicate pushdown.
- wal_
record - WAL record types for columnar operations.
- writer
- Segment writer: drains a memtable into a compressed columnar segment.