Skip to main content

Module streaming

Module streaming 

Source
Expand description

Streaming graph partitioning for large-scale graphs.

When a graph is too large to fit in memory or arrives as a stream of edges, streaming partitioners assign nodes to partitions in a single pass. This module provides:

  • LDG (Linear Deterministic Greedy): assigns each arriving vertex to the partition that maximizes the number of neighbors already present, penalized by partition size to maintain balance.
  • Hash partitioning: simple baseline that assigns nodes by hash.

Structs§

StreamingPartitionConfig
Configuration for the stateful streaming partitioner.
StreamingPartitioner
A stateful online streaming partitioner.

Enums§

StreamingPartitionAlgorithm
Algorithm variant for the stateful streaming partitioner.

Functions§

evaluate_partition
Evaluate a partition against an adjacency matrix.
hash_partition
Hash-based streaming partition (baseline).
streaming_partition
Linear Deterministic Greedy (LDG) streaming partitioner.