Struct grenad::Sorter

pub struct Sorter<MF, CC: ChunkCreator = DefaultChunkCreator> { /* private fields */ }

A struct you can use to automatically sort and merge duplicate entries.

You can insert key-value pairs in arbitrary order, it will use the ChunkCreator and you the generated chunks to buffer when the dump_threashold setting is reached.

Implementations

impl<MF> Sorter<MF, DefaultChunkCreator>

pub fn builder(merge: MF) -> SorterBuilder<MF, DefaultChunkCreator>

Creates a SorterBuilder from a merge function, it can be used to configure your Sorter to better fit your needs.

pub fn new(merge: MF) -> Sorter<MF, DefaultChunkCreator>

Creates a Sorter from a merge function, with the default parameters.

pub fn estimated_dumped_memory_usage(&self) -> u64

A rough estimate of how much memory usage it will take on the disk once dumped to disk.

This is a very bad estimate in the sense that it does not calculate the amount of duplicate entries that are in the dumped chunks and neither the fact that the in-memory buffer will likely be compressed once written to disk. This estimate will always be greater than the actual end space usage on disk.

impl<MF, CC, U> Sorter<MF, CC>

where MF: for<'a> Fn(&[u8], &[Cow<'a, [u8]>]) -> Result<Cow<'a, [u8]>, U>, CC: ChunkCreator,

pub fn insert<K, V>(&mut self, key: K, val: V) -> Result<(), Error<U>>

where K: AsRef<[u8]>, V: AsRef<[u8]>,

Insert an entry into the Sorter making sure that conflicts are resolved by the provided merge function.

pub fn write_into_stream_writer<W: Write>( self, writer: &mut Writer<W> ) -> Result<(), Error<U>>

Consumes this Sorter and streams the entries to the Writer given in parameter.

pub fn into_stream_merger_iter( self ) -> Result<MergerIter<CC::Chunk, MF>, Error<U>>

Consumes this Sorter and outputs a stream of the merged entries in key-order.

pub fn into_reader_cursors( self ) -> Result<Vec<ReaderCursor<CC::Chunk>>, Error<U>>

Consumes this Sorter and outputs the list of reader cursors.