Struct PartitionedState 

pub struct PartitionedState<V> { /* private fields */ }

Partitioned accumulator state for spillable aggregation.

Manages aggregate state across multiple partitions, with the ability to spill cold partitions to disk under memory pressure.

Implementations

impl<V: Clone + Send + Sync + 'static> PartitionedState<V>

pub fn new<S, D>( manager: Arc<SpillManager>, num_partitions: usize, value_serializer: S, value_deserializer: D, ) -> Self

where S: Fn(&V, &mut dyn Write) -> Result<()> + Send + Sync + 'static, D: Fn(&mut dyn Read) -> Result<V> + Send + Sync + 'static,

Creates a new partitioned state with custom serialization.

pub fn partition_for(&self, key: &[Value]) -> usize

Returns the partition index for a key.

pub fn is_in_memory(&self, partition_idx: usize) -> bool

Returns whether a partition is in memory.

pub fn partition_size(&self, partition_idx: usize) -> usize

Returns the number of groups in a partition.

pub fn total_size(&self) -> usize

Returns the total number of groups across all partitions.

pub fn in_memory_count(&self) -> usize

Returns the number of in-memory partitions.

pub fn spilled_count(&self) -> usize

Returns the number of spilled partitions.

pub fn spill_partition(&mut self, partition_idx: usize) -> Result<usize>

Spills a specific partition to disk.

Errors

Returns an error if writing to disk fails.

pub fn spill_largest(&mut self) -> Result<usize>

Spills the largest in-memory partition.

Returns the number of bytes spilled, or 0 if no partition to spill.

Errors

Returns an error if writing to disk fails.

pub fn spill_lru(&mut self) -> Result<usize>

Spills the least recently used in-memory partition.

Returns the number of bytes spilled, or 0 if no partition to spill.

Errors

Returns an error if writing to disk fails.

pub fn insert(&mut self, key: Vec<Value>, value: V) -> Result<Option<V>>

Inserts or updates a value for a key.

Errors

Returns an error if loading from disk fails.

pub fn get(&mut self, key: &[Value]) -> Result<Option<&V>>

Gets a value for a key.

Errors

Returns an error if loading from disk fails.

pub fn get_or_insert_with<F>( &mut self, key: Vec<Value>, default: F, ) -> Result<&mut V>

where F: FnOnce() -> V,

Gets a mutable value for a key, or inserts a default.

Errors

Returns an error if loading from disk fails.

pub fn drain_all(&mut self) -> Result<Vec<(Vec<Value>, V)>>

Drains all entries from all partitions.

Loads spilled partitions as needed.

Errors

Returns an error if loading from disk fails.

pub fn iter_all(&mut self) -> Result<Vec<(Vec<Value>, V)>>

Iterates over all entries without draining.

Loads spilled partitions as needed.

Errors

Returns an error if loading from disk fails.

pub fn cleanup(&mut self)

Cleans up all spill files.

Trait Implementations

impl<V> Drop for PartitionedState<V>

fn drop(&mut self)

Executes the destructor for this type.