Expand description
Segment reader — reads .csx segment files.
The reader validates the file integrity (magic bytes, CRC32c checksum),
parses header/footer/metadata, and provides methods to read individual
columns or entire row groups as Apache Arrow arrays.
§I/O strategy — memory-mapped prefetch
The entire segment file is memory-mapped via memmap2::Mmap. No
per-column seek()/read() system calls are issued; instead, column data
is accessed by slicing the mapped byte range, and the OS page cache
transparently pages data in from disk on first access.
Two advisory hints accelerate sequential scans:
madvise(MADV_SEQUENTIAL)— set once at open time, tells the kernel the file will be read front-to-back so it can perform aggressive read-ahead (typically 128–256 KiB on Linux).madvise(MADV_WILLNEED)— issued per-column inSegmentReader::read_columns: while decoding column N, the reader hints the kernel to begin paging in column N+1 (and the same column in the next row group for multi-row-group scans). This overlaps I/O with CPU decoding.
Because the Mmap holds the entire file mapping, there is no separate
read-ahead buffer or prefetch thread; the OS page cache is the buffer.
This is the most efficient approach for columnar segment files that fit in
the page cache — it avoids double-copying into user-space buffers and lets
the kernel coalesce I/O for adjacent columns.
§Predicate pushdown
The reader supports multi-level predicate pushdown to minimise the amount of data that is read and decoded:
| Level | Mechanism | Method |
|---|---|---|
| Segment | Catalog-level time-range + tag-index + bloom filter | Chronix::prune_segments() |
| Row group — time | min_ts/max_ts from timestamp column stats | read_projected_filtered |
| Row group — tags | value_count == 0 with a non-zero null_count (a genuinely all-null block → impossible match) | read_projected_filtered_with_predicates |
| Column | projection pushdown (only requested columns decoded) | read_projected |
Structs§
- Field
Predicate - A field predicate that can be evaluated against per-row-group
ColumnStatsmin/max to skip row groups that cannot possibly match. - Segment
Reader - Reads data from a
.csxsegment file.
Enums§
- StatsOp
- A numeric comparison operator for zone-map row-group pruning.