Macro batched_fn::batched_fn
source · macro_rules! batched_fn { ( handler = |$batch:ident: $batch_input_type:ty $(, $ctx_arg:ident: &$ctx_arg_ty:ty )*| -> $batch_output_type:ty $fn_body:block ; config = { $( $cfg:ident: $cfg_init:expr ),* $(,)? }; context = { $( $ctx:ident: $ctx_init:expr ),* $(,)? } $(;)? ) => { ... }; }
Expand description
Macro for creating a batched function.
This macro has 3 parameters: handler
, config
, and
context
. It returns an async function that wraps
BatchedFn::evaluate_in_batch
.
§Parameters
§handler
The handler must be in the form of a closure declaration that takes a batch and any number of references to objects in the context as input and returns a different type of batch.
§config
Within the config you can specify the max_batch_size
, max_delay
, and channel_cap
.
The batched function will wait at most max_delay
milliseconds after receiving a single
input to fill a batch of size max_batch_size
. If enough inputs to fill a full batch
are not received within max_delay
milliseconds then the partial batch will be ran as-is.
The channel_cap
option allows you to apply back pressure if too many inputs are waiting for
the handler thread to accept another batch. By default channel_cap
is None
, but if
set to Some(usize)
then
BatchedFn::evaluate_in_batch
will
return Error::Full
if the channel between the calling thread and the handler thread is at this
capacity. You probably want to set this to some multiple of max_batch_size
.
§context
Any additional reference that the handler takes as input must be defined within the context.
§Examples
use batched_fn::{batched_fn, Error};
async fn double(x: i32) -> Result<i32, Error> {
let batched_double = batched_fn! {
handler = |batch: Vec<i32>| -> Vec<i32> {
batch.into_iter().map(|x| x*2).collect()
};
config = {
max_batch_size: 4,
max_delay: 50,
channel_cap: Some(20),
};
context = {};
};
batched_double(x).await
}
You can also provide an arbitrary number of additional arguments to the handler by reference.
All of the objects have to be initialized in the context
:
async fn multiply(x: i32) -> Result<i32, Error> {
let batched_multiply = batched_fn! {
handler = |batch: Vec<i32>, factor: &i32| -> Vec<i32> {
batch.into_iter().map(|x| *factor * x ).collect()
};
config = {
max_batch_size: 4,
max_delay: 50
};
context = {
factor: 3
};
};
batched_multiply(x).await
}