Struct ExternalSorterBuilder

pub struct ExternalSorterBuilder { /* private fields */ }

Exposes external sorting (i.e. on disk sorting) capability on arbitrarily sized iterator, even if the generated content of the iterator doesn’t fit in memory.

Implementations

impl ExternalSorterBuilder

pub fn new() -> Self

pub fn with_chunk_size(self, size: usize) -> Self

Sets the maximum size of each segment in number of sorted items.

This number of items needs to fit in memory. While sorting, a in-memory buffer is used to collect the items to be sorted. Once it reaches the maximum size, it is sorted and then written to disk.

Using a higher segment size makes sorting faster by leveraging faster in-memory operations.

pub fn with_tmp_dir<P: AsRef<Path>>(self, path: P) -> Self

Sets directory in which sorted segments will be written (if it doesn’t fit in memory).

pub fn with_compression(self, level: u32) -> Self

pub fn num_threads(self, num_threads: usize) -> Self

Uses Rayon to sort the in-memory buffer.

This may not be needed if the buffer isn’t big enough for parallelism to be gainful over the overhead of multithreading.

pub fn build(self) -> Result<ExternalSorter>