Macro batched_fn::batched_fn

macro_rules! batched_fn {
    (
        handler = |$batch:ident: $batch_input_type:ty $(, $ctx_arg:ident: &$ctx_arg_ty:ty )*| -> $batch_output_type:ty $fn_body:block ;
        config = {
            $( $cfg:ident: $cfg_init:expr ),* $(,)?
        };
        context = {
            $( $ctx:ident: $ctx_init:expr ),* $(,)?
        } $(;)?
    ) => { ... };
}

Macro for creating a batched function.

This macro has 3 parameters: handler, config, and context. It returns an async function that wraps BatchedFn::evaluate_in_batch.

Parameters

handler

The handler must be in the form of a closure declaration that takes a batch and any number of references to objects in the context as input and returns a different type of batch.

config

Within the config you can specify the max_batch_size, max_delay, and channel_cap.

The batched function will wait at most max_delay milliseconds after receiving a single input to fill a batch of size max_batch_size. If enough inputs to fill a full batch are not received within max_delay milliseconds then the partial batch will be ran as-is.

The channel_cap option allows you to apply back pressure if too many inputs are waiting for the handler thread to accept another batch. By default channel_cap is None, but if set to Some(usize) then BatchedFn::evaluate_in_batch will return Error::Full if the channel between the calling thread and the handler thread is at this capacity. You probably want to set this to some multiple of max_batch_size.

context

Any additional reference that the handler takes as input must be defined within the context.

Examples

use batched_fn::{batched_fn, Error};

async fn double(x: i32) -> Result<i32, Error> {
    let batched_double = batched_fn! {
        handler = |batch: Vec<i32>| -> Vec<i32> {
            batch.into_iter().map(|x| x*2).collect()
        };
        config = {
            max_batch_size: 4,
            max_delay: 50,
            channel_cap: Some(20),
        };
        context = {};
    };

    batched_double(x).await
}

You can also provide an arbitrary number of additional arguments to the handler by reference. All of the objects have to be initialized in the context:

async fn multiply(x: i32) -> Result<i32, Error> {
    let batched_multiply = batched_fn! {
        handler = |batch: Vec<i32>, factor: &i32| -> Vec<i32> {
            batch.into_iter().map(|x| *factor * x ).collect()
        };
        config = {
            max_batch_size: 4,
            max_delay: 50
        };
        context = {
            factor: 3
        };
    };

    batched_multiply(x).await
}