Crate foyer

Expand description

Tutorial & Document: https://foyer-rs.github.io/foyer

§foyer

GitHub License Crates.io MSRV

foyer aims to be an efficient and user-friendly hybrid cache lib in Rust.

foyer draws inspiration from Facebook/CacheLib, a highly-regarded hybrid cache library written in C++, and ben-manes/caffeine, a popular Java caching library, among other projects.

However, foyer is more than just a rewrite in Rust effort; it introduces a variety of new and optimized features.

For more details, please visit foyer’s website: https://foyer-rs.github.io/foyer 🥰

Website | Tutorial | API Docs | Crate

§Features

Hybrid Cache: Seamlessly integrates both in-memory and disk cache for optimal performance and flexibility.
Plug-and-Play Algorithms: Empowers users with easily replaceable caching algorithms, ensuring adaptability to diverse use cases.
Fearless Concurrency: Built to handle high concurrency with robust thread-safe mechanisms, guaranteeing reliable performance under heavy loads.
Zero-Copy In-Memory Cache Abstraction: Leveraging Rust’s robust type system, the in-memory cache in foyer achieves a better performance with zero-copy abstraction.
User-Friendly Interface: Offers a simple and intuitive API, making cache integration effortless and accessible for developers of all levels.
Out-of-the-Box Observability: Integrate popular observation systems such as Prometheus, Grafana, Opentelemetry, and Jaeger in just ONE line.

§Quick Start

This section only shows briefs. Please visit https://foyer-rs.github.io/foyer for more details.

To use foyer in your project, add this line to the dependencies section of Cargo.toml.

foyer = "0.21"

If your project is using the nightly rust toolchain, the nightly feature needs to be enabled.

foyer = { version = "0.21", features = ["nightly"] }

§Out-of-the-box In-memory Cache

The in-memory cache setup is extremely easy and can be setup in at least 1 line.

use foyer::{Cache, CacheBuilder};

let cache: Cache<String, String> = CacheBuilder::new(16).build();

let entry = cache.insert("hello".to_string(), "world".to_string());
let e = cache.get("hello").unwrap();

assert_eq!(entry.value(), e.value());

§Easy-to-use Hybrid Cache

The setup of a hybrid cache is extremely easy.

use foyer::{BlockEngineBuilder, DeviceBuilder, FsDeviceBuilder, HybridCache, HybridCacheBuilder};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let dir = tempfile::tempdir()?;

    let device = FsDeviceBuilder::new(dir.path())
        .with_capacity(256 * 1024 * 1024)
        .build()?;

    let hybrid: HybridCache<u64, String> = HybridCacheBuilder::new()
        .memory(64 * 1024 * 1024)
        .storage()
        // use block-based disk cache engine with default configuration
        .with_engine_config(BlockEngineBuilder::new(device))
        .build()
        .await?;

    hybrid.insert(42, "The answer to life, the universe, and everything.".to_string());
    assert_eq!(
        hybrid.get(&42).await?.unwrap().value(),
        "The answer to life, the universe, and everything."
    );

    Ok(())
}

§Fully Configured Hybrid Cache

Here is an example of a hybrid cache setup with almost all configurations to show th possibilities of tuning.

use std::{hash::BuildHasherDefault, num::NonZeroUsize};

use chrono::Datelike;
use foyer::{
    BlockEngineBuilder, DeviceBuilder, FifoPicker, FsDeviceBuilder, HybridCache, HybridCacheBuilder, HybridCachePolicy,
    IoEngineBuilder, IopsCounter, LruConfig, PsyncIoEngineBuilder, RecoverMode, RejectAll, RuntimeOptions,
    StorageFilter, Throttle, TokioRuntimeOptions,
};
use tempfile::tempdir;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let dir = tempdir()?;

    let device = FsDeviceBuilder::new(dir.path())
        .with_capacity(64 * 1024 * 1024)
        .with_throttle(
            Throttle::new()
                .with_read_iops(4000)
                .with_write_iops(2000)
                .with_write_throughput(100 * 1024 * 1024)
                .with_read_throughput(800 * 1024 * 1024)
                .with_iops_counter(IopsCounter::PerIoSize(NonZeroUsize::new(128 * 1024).unwrap())),
        )
        .build()?;

    let io_engine = PsyncIoEngineBuilder::new().build().await?;

    let hybrid: HybridCache<u64, String> = HybridCacheBuilder::new()
        .with_name("my-hybrid-cache")
        .with_policy(HybridCachePolicy::WriteOnEviction)
        .memory(1024)
        .with_shards(4)
        .with_eviction_config(LruConfig {
            high_priority_pool_ratio: 0.1,
        })
        .with_hash_builder(BuildHasherDefault::default())
        .with_weighter(|_key, value: &String| value.len())
        .with_filter(|_, _| true)
        .storage()
        .with_io_engine(io_engine)
        .with_engine_config(
            BlockEngineBuilder::new(device)
                .with_block_size(16 * 1024 * 1024)
                .with_indexer_shards(64)
                .with_recover_concurrency(8)
                .with_flushers(2)
                .with_reclaimers(2)
                .with_buffer_pool_size(256 * 1024 * 1024)
                .with_clean_block_threshold(4)
                .with_eviction_pickers(vec![Box::<FifoPicker>::default()])
                .with_admission_filter(StorageFilter::new())
                .with_reinsertion_filter(StorageFilter::new().with_condition(RejectAll))
                .with_tombstone_log(false),
        )
        .with_recover_mode(RecoverMode::Quiet)
        .with_compression(foyer::Compression::Lz4)
        .with_runtime_options(RuntimeOptions::Separated {
            read_runtime_options: TokioRuntimeOptions {
                worker_threads: 4,
                max_blocking_threads: 8,
            },
            write_runtime_options: TokioRuntimeOptions {
                worker_threads: 4,
                max_blocking_threads: 8,
            },
        })
        .build()
        .await?;

    hybrid.insert(42, "The answer to life, the universe, and everything.".to_string());
    assert_eq!(
        hybrid.get(&42).await?.unwrap().value(),
        "The answer to life, the universe, and everything."
    );

    let e = hybrid
        .get_or_fetch(&20230512, || async {
            // Mock fetching data from remote source
            let now = chrono::Utc::now();
            if format!("{}{}{}", now.year(), now.month(), now.day()) == "20230512" {
                return Err(anyhow::anyhow!("Hi, time traveler!"));
            }
            Ok("Hello, foyer.".to_string())
        })
        .await?;
    assert_eq!(e.key(), &20230512);
    assert_eq!(e.value(), "Hello, foyer.");

    hybrid.close().await.unwrap();

    Ok(())
}

§`serde` Support

foyer needs to serialize/deserialize entries between memory and disk with hybrid cache. Cached keys and values need to implement the Code trait when using hybrid cache.

The Code trait has already been implemented for general types, such as:

Numeric types: u8, u16, u32, u64, u128, usize, i8, i16, i32, i64, i128, isize, f32, f64.
Buffer: Vec<u8>.
String: String.
Other general types: bool.

For more complex types, you need to implement the Code trait yourself.

To make things easier, foyer provides support for the serde ecosystem. Types implement serde::Serialize and serde::DeserializeOwned, foyer will automatically implement the Code trait. This feature requires enabling the serde feature for foyer.

foyer = { version = "*", features = ["serde"] }

§Other Examples

More code examples and details can be found here.

Case study of real-world usage in the production system can be found in Document - Case Study - RisingWave

§Architecture

The architecture of foyer is still not mature and is undergoing rapid iteration. Currently, the architectural design can refer to Document - Architecture

§Supported Rust Versions

foyer is built against the recent stable release. The minimum supported version is 1.85.0. The current foyer version is not guaranteed to build on Rust versions earlier than the minimum supported version.

§Supported Platforms

foyer is designed to serve on Linux OS, but can still be built on other OS for development.

However, other components may not support non-Linux OS.

Component	Linux	MacOS	Windows
foyer	✓	✓	✓
foyer-bench	✓	✗	✗

§Development State & Roadmap

Currently, foyer is still under heavy development.

The development state and the roadmap can be found in foyer - Development Roadmap.

§Contributing

Contributions for foyer are warmly welcomed! 🥰

Don’t forget to pass cargo x --fast (which runs most necessary checks and tests) locally before submitting a PR. 🚀

If you want to run a broader range of checks locally, run cargo x. 🙌

Thank you for your contribution~

Structs§

AdmitAll: Admit all entries.
Block: A block is a logical partition of a device. It is used to manage the device’s storage space.
BlockEngineBuilder: Builder for the block-based disk cache engine.
BlockStatistics: Block statistics.
CacheBuilder: In-memory cache builder.
CacheProperties: Entry properties for in-memory only cache.
CombinedDeviceBuilder: Builder for a combined device that wraps multiple devices and allows access to their blocks.
EngineBuildContext: Context for building the disk cache engine.
Error: Error is the error struct returned by all foyer functions.
EstimatedSize: A condition that checks if the estimated size is within a specified range.
EvictionInfo: Eviction related information for eviction picker to make decisions.
FifoConfig: Fifo eviction algorithm config.
FifoPicker: A picker that pick block to eviction with a FIFO behavior.
FileDeviceBuilder: Builder for a file-based device that manages a single file or a raw block device.
FsDeviceBuilder: Builder for a filesystem-based device that manages files in a directory.
HybridCache: Hybrid cache that integrates in-memory cache and disk cache.
HybridCacheBuilder: Hybrid cache builder.
HybridCacheBuilderPhaseMemory: Hybrid cache builder to modify the in-memory cache configurations.
HybridCacheBuilderPhaseStorage: Hybrid cache builder modify the disk cache configurations.
HybridCacheProperties: Entry properties for in-memory only cache.
HybridCacheStorageWriter: Writer for disk cache of a hybrid cache to support more flexible write APIs.
HybridCacheWriter: Writer for hybrid cache to support more flexible write APIs.
HybridGet: Future for HybridCache::get.
HybridGetOrFetch: Future for HybridCache::get_or_fetch.
InvalidRatioPicker: Evict the block with the largest invalid data ratio.
IoHandle: A detached I/O handle that can be polled for completion.
LfuConfig: w-TinyLFU eviction algorithm config.
LruConfig: Lru eviction algorithm config.
NoopDeviceBuilder: Builder for a no-operation mock device.
NoopIoEngine: A mock I/O engine that does nothing.
NoopIoEngineBuilder: Builder for a no-operation mock I/O engine.
PartialDeviceBuilder: Builder for a partial device that wraps another device and allows access to only a subset of capacity.
PsyncIoEngine: The synchronous I/O engine that uses pread(2)/pwrite(2) and tokio thread pool for reading and writing.
PsyncIoEngineBuilder: Builder for synchronous I/O engine with pread(2)/pwrite(2).
RawFile: Raw os file resource.
RejectAll: Reject all entries.
Runtime: Runtime holds the runtime reference and non-cloneable handles to prevent handle usage after runtime shutdown.
S3FifoConfig: S3Fifo eviction algorithm config.
Statistics: The statistics of the device.
StorageFilter: StorageFilter filters entries based on multiple conditions for admission and reinsertion.
Store: The disk cache engine that serves as the storage backend of foyer.
StoreBuilder: The builder of the disk cache.
Throttle: Throttle config for the device.
TokioRuntimeOptions: Tokio runtime configuration.
TracingOptionstracing: Options for tracing.
UringIoEngineLinux: The io_uring based I/O engine.
UringIoEngineBuilderLinux: Builder for io_uring based I/O engine.

Enums§

Age: Entry age in the disk cache. Used by hybrid cache.
Cache: In-memory cache with plug-and-play algorithms.
CacheEntry: A cached entry holder of the in-memory cache.
Compression: The compression algorithm of the disk cache.
ErrorKind: ErrorKind is all kinds of Error of foyer.
Event: Event identifier.
EvictionConfig: Eviction algorithm config.
GetOrFetch: A future that is used to get entry value from the remote storage for the in-memory cache.
Hint: Hint for the cache eviction algorithm to decide the priority of the specific entry if needed.
HybridCachePolicy: Control the cache policy of the hybrid cache.
IopsCounter: Device iops counter.
Load: Load result.
Location: Advice cache location for the cache entry.
RecoverMode: The recover mode of the disk cache.
RuntimeOptions: Options for the dedicated runtime.
Source: Source of the cache entry.
StorageFilterResult: Filter result for admission pickers and reinsertion pickers.

Traits§

BufExt: Extend Buf with get_isize() and get_usize().
BufMutExt: Extend BufMut with put_isize() and put_usize().
Code: Encode/decode trait for key and value.
Device: Device trait.
DeviceBuilder: Device builder trait.
Engine: Disk cache engine trait.
EngineConfig: Disk cache engine builder trait.
EventListener: Trait for the customized event listener.
EvictionPicker: The eviction picker for the disk cache.
Filter: The filter for the in-memory cache.
IoEngine: I/O engine builder trait.
IoEngineBuilder: I/O engine builder trait.
Key: Key trait for the in-memory cache.
OptionExt: Extension for std::option::Option.
RangeBoundsExt: The range extensions.
Scope: Scoped functional programming extensions.
StorageFilterCondition: Condition for StorageFilter.
StorageKey: Key trait for the disk cache.
StorageValue: Value trait for the disk cache.
Value: Value trait for the in-memory cache.
Weighter: The weighter for the in-memory cache.

Type Aliases§

DefaultHasher: The default hasher for foyer.
HybridCacheEntry: A cached entry holder of the hybrid cache.
Result: Result type for foyer.

Crate foyer

Crate foyer Copy item path

§foyer

§Features

§Quick Start

§Out-of-the-box In-memory Cache

§Easy-to-use Hybrid Cache

§Fully Configured Hybrid Cache

§serde Support

§Other Examples

§Architecture

§Supported Rust Versions

§Supported Platforms

§Development State & Roadmap

§Contributing

Structs§

Enums§

Traits§

Type Aliases§

Crate foyer

§`serde` Support