Module writer 

Segment writer — builds .csx segment files.

The writer accepts rows (as Points), sorts them by series key and timestamp, splits them into row groups, encodes each column with chronix-encoding, optionally compresses with LZ4, and writes the segment file using streaming I/O with incremental CRC-32c.

Streaming Write Architecture

Row groups are encoded one at a time and flushed directly to a temp file via BufWriter. An incremental CRC-32c checksum (crc32c_append) is updated as each chunk is written. Peak memory is bounded by O(row_group_size) — typically 1-10 MiB — rather than O(segment_size).

The write path uses temp-file → fsync → rename → dir-fsync for crash-safe atomic visibility.

Zstd dictionary compression

When zstd_dict_training is enabled and the compression codec is Zstd, the writer collects encoded column blocks from the first row group as training samples, trains a dictionary via zstd::dict::from_samples, and compresses subsequent row groups with the trained dictionary. The dictionary is embedded in the segment metadata for the reader.

This improves compression ratio by 20–40% on small blocks (< 64 KiB) where column values have repeating structure (e.g. string tags, enum fields).

Usage

let config = SegmentWriterConfig::default();
let mut writer = SegmentWriter::new("/tmp/segment.csx", config).unwrap();
// writer.write_rows(&points).unwrap();
// let meta = writer.finalize().unwrap();

Structs

SegmentMeta

Metadata returned after a segment is finalized.

SegmentWriter

Builds a .csx segment file from time-series data points.

SegmentWriterConfig

Configuration for the segment writer.

Constants

DEFAULT_ROW_GROUP_SIZE

Default maximum rows per row group.

DEFAULT_TARGET_SEGMENT_SIZE_BYTES

Default target segment file size (256 MiB).