Struct ultra_batch::Batcher

pub struct Batcher<F> where
    F: Fetcher,  { /* private fields */ }

Used to batch and cache loads from some datastore. A Batcher can be used with any type that implements Fetcher. Batchers are asynchronous, and designed to be passed and shared between threads or tasks. Cloning a Batcher is shallow and can be used to use the same Fetcher across multiple threads or tasks.

A Batcher is designed primarily around batching database lookups– for example, fetching a user from a user ID, where a signle query to retrieve 50 users by ID is significantly faster than 50 separate queries to look up the same set of users.

A Batcher is designed to be ephemeral. In the context of a web service, this means callers should most likely create a new Batcher for each request, and not a Batcher shared across multiple requests. Batchers have no concept of cache invalidation, so old values are stored indefinitely (which means callers may get stale data or may exhaust memory endlessly).

Batchers introduce a small amount of latency for loads. Each time a Batcher receives a key to fetch that hasn’t been cached (or a set of keys), it will first wait for more keys to build a batch. The load will only trigger after a timeout is reached or once enough keys have been queued in the batch. See BatcherBuilder for options to tweak latency and batch sizes.

Load semantics

If the underlying Fetcher returns an error during the batch request, then all pending load and load_many requests will fail. Subsequent calls to load or load_many with the same keys will retry.

If the underlying Fetcher succeeds but does not return a value for a given key during a batch request, then the Batcher will mark that key as “not found” and an error value of NotFound will be returned to all pending load and load_many requests. The “not found” status will be preserved, so subsequent calls with the same key will fail and will not retry.

Implementations

impl<F> Batcher<F> where
    F: Fetcher + Send + Sync + 'static, 

pub fn build(fetcher: F) -> BatcherBuilder<F>

Create a new Batcher that uses the given Fetcher to retrieve data. Returns a BatcherBuilder, which can be used to customize the Batcher. Call .finish() to create the Batcher.

Examples

Creating a Batcher with default options:

let user_fetcher = UserFetcher::new(db_conn);
let batcher = Batcher::build(user_fetcher).finish();

Creating a Batcher with custom options:

let user_fetcher = UserFetcher::new(db_conn);
let batcher = Batcher::build(user_fetcher)
    .eager_batch_size(Some(50))
    .delay_duration(tokio::time::Duration::from_millis(5))
    .finish();

pub async fn load(&self, key: F::Key) -> Result<F::Value, LoadError>

Load the value with the associated key, either by calling the Fetcher or by loading the cached value. Returns an error if the value could not be loaded or if a value for the given key was not found.

See the type-level docs for Batcher for more detailed loading semantics.

pub async fn load_many(
    &self,
    keys: &[F::Key]
) -> Result<Vec<F::Value>, LoadError>

Load all the values for the given keys, either by calling the Fetcher or by loading cached values. Values are returned in the same order as the input keys. Returns an error if any load fails.

See the type-level docs for Batcher for more detailed loading semantics.

Trait Implementations

impl<F> Clone for Batcher<F> where
    F: Fetcher, 

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.