Rust library of batching algorithm implementations.
Batching works by accumulating items and later automatically flushing them all together when the batch has reached a limit. All items collected in the single batch are available at once for further processing (e.g. batch insert into a database).
These implementations will construct batches based on:
- limit of the number of items collected in a batch,
- limit of time duration since the first item appended to the batch,
- calling one of the batch consuming methods,
- sending flush command between batch items (channel-based implementations).
See documentation of available algorithms.
Example
Collect batches of items from two streams by reaching different individual batch limits and using Flush
command.
use MultiBufBatchChannel;
use *;
use Duration;
use assert_matches;
// Create producer thread with a channel-based, multi-stream batching implementation configured with a maximum size
// of 4 items (for each stream) and a maximum batch duration since the first received item of 200 ms.
let mut batch = with_producer_thread;
// Batches flushed due to individual batch size limit
assert_matches!;
assert_matches!;
// Batches flushed due to duration limit
assert_matches!;
assert_matches!;
// Batches flushed by sending `Flush` command starting from batch with stream key `1`
assert_matches!;
assert_matches!;
// Batches flushed by dropping sender (thread exit)
assert_matches!;
assert_matches!;