Enum BatchingPolicy 

#[non_exhaustive]
pub enum BatchingPolicy {
    Immediate,
    Size,
    Duration(Duration, OnFull),
    Balanced {
        min_size_hint: usize,
    },
}

A policy controlling when batches get processed.

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Immediate

Immediately process the batch if possible.

When concurrency and resources are available, new items will be processed immediately (with a batch size of one).

When resources are not immediately available, then the batch will remain open while acquiring resources to allow more items to be added, up to the maximum batch size.

In this way, we try to prioritise larger batch sizes, while still keeping latency low.

When concurrency is maximised, new items will added to the next batch (up to the maximum batch size). As soon as a batch finishes the next batch will start. When concurrency is limited to 1, it will run batches serially.

Prioritises low latency.

Size

Process the batch when it reaches the maximum size.

Prioritises high batch utilisation.

Duration(Duration, OnFull)

Process the batch a given duration after it was created.

If using OnFull::Process, then process the batch when either the duration elapses or the batch becomes full, whichever happens first.

Prioritises regularity.

Balanced

Balance between resource efficiency and latency based on system load.

When no batches are processing, the first item processes immediately.

When batches are already processing:

• If batch size < min_size_hint: Wait for either the batch to reach min_size_hint or any batch to complete
• If batch size >= min_size_hint: Start acquiring resources and process immediately

The min_size_hint must be <= max_batch_size.

Prioritises efficient resource usage while maintaining reasonable latency.

Fields

min_size_hint: usize

The minimum batch size to prefer before using additional concurrency.

Trait Implementations

impl Clone for BatchingPolicy

fn clone(&self) -> BatchingPolicy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BatchingPolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for BatchingPolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.