Skip to main content

external_sort_delimited

Function external_sort_delimited 

Source
pub fn external_sort_delimited(
    reader: impl Read,
    from_delim: u8,
    writer: impl Write,
    to_delim: u8,
    transforms: &Transforms,
    sort: &SortSpec,
    chunk_rows: usize,
) -> Result<(), Box<dyn Error>>
Expand description

Sorts a delimited stream using bounded memory via external merge sort.

Reads chunk_rows rows at a time, sorts each chunk, spills it to a temp file, then k-way merges the temp files into the output. Peak memory is bounded by chunk_rows * row_size, independent of input size.

§Errors

Returns an error if any I/O step fails or the sort spec references a missing column.

§Examples

use sheeter_engine::{external_sort_delimited, SortSpec, Transforms};

let input = b"n\n3\n1\n2\n";
let mut out = Vec::new();
external_sort_delimited(
    input.as_slice(), b',', &mut out, b',',
    &Transforms::default(),
    &SortSpec::parse("n").unwrap(),
    10,
).unwrap();
assert_eq!(out, b"n\n1\n2\n3\n");