pub struct ClusterCsr { /* private fields */ }Expand description
Compressed-sparse-row representation of the duplicate-cluster
membership. Built once, in GtAccumulator::finish, from every
(master, record, is_identical) triple belonging to a duplicated master.
Avoids one String allocation per record_id and one per distinct
master_id (the dominant RAM cost of a string-keyed cluster map at 200M+
records): master_id/record_id are packed into u64s
(pipeline::pack_master_parts / the global record index) and stored in
two flat, contiguous buffers instead of a hash map of per-cluster Vecs.
master_id itself is never needed downstream as a string — the only
consumer (graph_gen::push_dup_clusters) only ever used it to get a
deterministic cluster iteration order, and numeric comparison of the
packed key preserves the lexicographic order of the original string
(both segments are zero-padded to a fixed width).
Implementations§
Source§impl ClusterCsr
impl ClusterCsr
Sourcepub fn groups(&self) -> impl Iterator<Item = (&[u64], BitspanRef<'_>)>
pub fn groups(&self) -> impl Iterator<Item = (&[u64], BitspanRef<'_>)>
Iterates clusters in ascending packed-master-key order, yielding each
cluster’s (record_indices, is_identical) pair. A cluster’s
record_indices are already ascending, so callers
(push_dup_clusters) don’t need to sort members themselves.
Auto Trait Implementations§
impl Freeze for ClusterCsr
impl RefUnwindSafe for ClusterCsr
impl Send for ClusterCsr
impl Sync for ClusterCsr
impl Unpin for ClusterCsr
impl UnsafeUnpin for ClusterCsr
impl UnwindSafe for ClusterCsr
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more