Struct grenad::Sorter

pub struct Sorter<MF, CC: ChunkCreator = DefaultChunkCreator> { /* fields omitted */ }

A struct you can use to automatically sort and merge duplicate entries.

You can insert key-value pairs in arbitrary order, it will use the ChunkCreator and you the generated chunks to buffer when the dump_threashold setting is reached.

Implementations

impl<MF> Sorter<MF, DefaultChunkCreator>

pub fn builder(merge: MF) -> SorterBuilder<MF, DefaultChunkCreator>

Creates a SorterBuilder from a merge function, it can be used to configure your Sorter to better fit your needs.

pub fn new(merge: MF) -> Sorter<MF, DefaultChunkCreator>

Creates a Sorter from a merge function, with the default parameters.

pub fn estimated_dumped_memory_usage(&self) -> u64

A rough estimate of how much memory usage it will take on the disk once dumped to disk.

This is a very bad estimate in the sense that it does not calculate the amount of duplicate entries that are in the dumped chunks and neither the fact that the in-memory buffer will likely be compressed once written to disk. This estimate will always be greater than the actual end space usage on disk.

impl<MF, CC, U> Sorter<MF, CC> where
    MF: for<'a> Fn(&[u8], &[Cow<'a, [u8]>]) -> Result<Cow<'a, [u8]>, U>,
    CC: ChunkCreator, 

pub fn insert<K, V>(&mut self, key: K, val: V) -> Result<(), Error<U>> where
    K: AsRef<[u8]>,
    V: AsRef<[u8]>, 

Insert an entry into the Sorter making sure that conflicts are resolved by the provided merge function.

pub fn write_into_stream_writer<W: Write>(
    self,
    writer: &mut Writer<W>
) -> Result<(), Error<U>>

Consumes this Sorter and streams the entries to the Writer given in parameter.

pub fn into_stream_merger_iter(
    self
) -> Result<MergerIter<CC::Chunk, MF>, Error<U>>

Consumes this Sorter and outputs a stream of the merged entries in key-order.