Struct shardio::ShardReader [−][src]
pub struct ShardReader<T, S = DefaultSort> where
S: SortKey<T>, { /* fields omitted */ }
Expand description
Read from a collection of shardio files. The input data is merged to give
a single sorted view of the combined dataset. The input files must
be created with the same sort order S
as they are read with.
Implementations
impl<T, S> ShardReader<T, S> where
T: DeserializeOwned,
<S as SortKey<T>>::Key: Clone + Ord + DeserializeOwned,
S: SortKey<T>,
impl<T, S> ShardReader<T, S> where
T: DeserializeOwned,
<S as SortKey<T>>::Key: Clone + Ord + DeserializeOwned,
S: SortKey<T>,
Open a single shard files into reader
Open a set of shard files into an aggregated reader
Read data from the given range
into data
buffer. The data
buffer is cleared before adding items.
pub fn iter_range<'a>(
&'a self,
range: &Range<<S as SortKey<T>>::Key>
) -> Result<MergeIterator<'a, T, S>, Error>
pub fn iter_range<'a>(
&'a self,
range: &Range<<S as SortKey<T>>::Key>
) -> Result<MergeIterator<'a, T, S>, Error>
Iterate over items in the given range
Iterate over all items
Generate num_chunks
ranges covering the give range
, each with a roughly equal numbers of elements.
The ranges can be fed to iter_range