flaque

A fast, concurrent, lock-free, and sometimes wait-free, pool implementation.

Quick overview

A pool is a limited-sized collection of objects. When an object is fetched from the pool, the number of available objects decreases by one. Once the fetched object is dropped, it goes back into the pool, and the number of available objects increases by one. Being as fast as possible, avoiding blocking, and, whenever possible, avoiding waiting, is the name of the game.

All objects are lazily created with a builder, which can be synchronous or asynchronous, fallible or infallible. Objects are recycled, i.e. when an object has been built and fetched once, it can be mutated, and when it goes back in the pool, it stays as is, untouched.

use flaque::Pool;

// Let's define what the pool will hold.
struct T {
    x: u8,
}

impl Default for T {
    fn default() -> Self {
        Self { x: 0 }
    }
}

// Let's create the pool: capacity of 8, and the builder for the `T`s.
let pool = Pool::new(8, T::default);

// Let's try to get an object from the pool.
let object_0 = pool.try_get().unwrap();

// Or, let's wait to get an object from the pool.
let object_1 = pool.get().await;

The main methods are:

• Pool::new to build a Pool with a synchronous builder,
• Pool::new_with_async_builder to build a Pool with an asynchronous builder,
• Pool::try_get if one wants to synchronously, atomically get an object from the pool,
• Pool::get if one wants to asynchronously get an object from the pool, waiting for one to become available.

An object is represented by the Object<T, _> type, which derefs and deref_muts to T. When it drops, the value returns immediately to the pool.

Features

Lock- and wait-free

This pool implementation is lock-free and sometimes wait-free. It is designed to be fast and consistent.

If you're not familiar with these definitions, please read Non-blocking algorithm.

Get and Close futures

Pool::get returns a Get future. It resolves to Ok(object) once one is available in the pool, otherwise it returns to Err(GetError) if the pool is closed or if the builder failed to build an object.

Pool::close returns a Close future. It resolves once all Objects are back into the pool, i.e. when they are all dropped. Pool::close takes self, i.e. it takes ownership of the pool. It will be returned back once the future resolves. This guarantees that it is impossible to reopen the pool, and to fetch new Objects, while waiting for all Objects to return back into the pool. Thus, this is not unusual to see:

let pool = pool.close().await;

Install

See crates.io/crates/flaque.

Documentation

See docs.rs/flaque.

Testing

This crate is tested with loom and miri to increase trust in the implementation.

Benchmarks

This crate has started as a hobby project, but also with the idea of replacing the deadpool crate inside the Matrix Rust SDK. Indeed, we are experiencing performance issues due to deadpool. Here are the results of the benchmarks comparing flaque with deadpool.

Each benchmark basically runs the following code: pool.get().await; —i.e. get and release many objects— multiple times, on different threads.

The first group (pool size of 16, and 8 readers) represents the no contention scenario: there are ½ readers per object. The second group (pool size of 16, 2048 readers) represents the high contention on a small pool scenario: there are 128 readers per object. The third group (pool size of 512, 1 million readers) represents the high contention on a large pool scenario: there are 2048 readers per object.

On average, flaque is 1.7 times faster than deadpool.

The time is in milliseconds. The throughput is in millions of objects per second.