Struct extsort::sorter::ExternalSorter

pub struct ExternalSorter { /* fields omitted */ }

Exposes external sorting (i.e. on disk sorting) capability on arbitrarily sized iterator, even if the generated content of the iterator doesn't fit in memory.

It uses an in-memory buffer sorted and flushed to disk in segment files when full. Once sorted, it returns a new sorted iterator with all items. In order to remain efficient for all implementations, the crate doesn't handle serialization, but leaves that to the user.

Implementations

impl ExternalSorter

pub fn new() -> ExternalSorter

pub fn with_segment_size(mut self: Self, size: usize) -> Self

Sets the maximum size of each segment in number of sorted items.

This number of items needs to fit in memory. While sorting, a in-memory buffer is used to collect the items to be sorted. Once it reaches the maximum size, it is sorted and then written to disk.

Using a higher segment size makes sorting faster by leveraging faster in-memory operations.

pub fn with_sort_dir(mut self: Self, path: PathBuf) -> Self

Sets directory in which sorted segments will be written (if it doesn't fit in memory).

pub fn with_parallel_sort(mut self: Self) -> Self

Uses Rayon to sort the in-memory buffer.

This may not be needed if the buffer isn't big enough for parallelism to be gainful over the overhead of multithreading.

pub fn sort<T, I>(
    &self,
    iterator: I
) -> Result<SortedIterator<T, impl Fn(&T, &T) -> Ordering + Send + Sync>, Error> where
    T: Sortable + Ord,
    I: Iterator<Item = T>, 

Sorts a given iterator, returning a new iterator with items

pub fn sort_by<T, I, F>(
    &self,
    iterator: I,
    cmp: F
) -> Result<SortedIterator<T, F>, Error> where
    T: Sortable,
    I: Iterator<Item = T>,
    F: Fn(&T, &T) -> Ordering + Send + Sync, 

Sorts a given iterator with a comparator function, returning a new iterator with items

pub fn sort_by_key<T, I, F, K>(
    &self,
    iterator: I,
    f: F
) -> Result<SortedIterator<T, impl Fn(&T, &T) -> Ordering + Send + Sync>, Error> where
    T: Sortable,
    I: Iterator<Item = T>,
    F: Fn(&T) -> K + Send + Sync,
    K: Ord, 

Sorts a given iterator with a key extraction function, returning a new iterator with items

Trait Implementations

impl Default for ExternalSorter

fn default() -> Self

Returns the "default value" for a type.