Module differential_dataflow::operators::group

Group records by a key, and apply a reduction function.

The group operators act on data that can be viewed as pairs (key, val). They group records with the same key, and apply user supplied functions to the key and a list of values, which are expected to populate a list of output values.

Several variants of group exist which allow more precise control over how grouping is done. For example, the _by suffixed variants take arbitrary data, but require a key-value selector to be applied to each record. The _u suffixed variants use unsigned integers as keys, and will use a dense array rather than a HashMap to store their keys.

The list of values are presented as an iterator which internally merges sorted lists of values. This ordering can be exploited in several cases to avoid computation when only the first few elements are required.

Examples

This example groups a stream of (key,val) pairs by key, and yields only the most frequently occurring value for each key.

stream.group(|key, vals, output| {
    let (mut max_val, mut max_wgt) = vals.next().unwrap();
    for (val, wgt) in vals {
        if wgt > max_wgt {
            max_wgt = wgt;
            max_val = val;
        }
    }
    output.push((max_val.clone(), max_wgt));
})

Traits

Count	Counts the number of occurrences of each element.
Group	Extension trait for the group differential dataflow method
GroupArranged	Extension trait for the group operator on Arrange<_> data.