Struct Config

pub struct Config {
    pub path: PathBuf,
    pub zstd_compression_level: Option<i32>,
    pub fsync_each_batch: bool,
    pub target_file_size: usize,
    pub file_compaction_percent: u8,
    pub max_object_size: usize,
    pub small_file_cleanup_threshold: usize,
    pub partition_function: fn(object_id: u64, object_size: usize) -> u8,
    pub min_compaction_files: usize,
}

Configuration for configuring Marble.

Fields

path: PathBuf

Storage files will be kept here.

zstd_compression_level: Option<i32>

The compression level to use when compressing each batch of objects. A value of None disables compression. This is one of the most important parameters to experiment with while finding an appropriate configuration for your system.

fsync_each_batch: bool

Issue fsyncs on each new file and the containing directory when it is created. This corresponds to at least one call to fsync for each call to write_batch.

target_file_size: usize

Garbage collection will try to keep storage files around this size or smaller.

file_compaction_percent: u8

Remaining live percentage of a file before it’s considered rewritabe.

max_object_size: usize

The ceiling on the largest allocation this system will ever attempt to perform in order to read an object off of disk.

small_file_cleanup_threshold: usize

The number of total files (of all sizes) that must exist before “small files” are squished together even if they are above the file_compaction_percent. A “small file” is defined as a file whose uncompressed size times min_compaction_files is below the target_file_size.

partition_function: fn(object_id: u64, object_size: usize) -> u8

A partitioning function for objects based on object ID and object size. You may override this to cause objects to be written into separate files so that garbage collection may take advantage of locality effects for your workload that are correlated to object identifiers or the size of data.

Ideally, you will colocate objects that have similar expected lifespans. Doing so minimizes the costs of copying live data over time during storage file GC.

min_compaction_files: usize

The minimum number of files within a generation to collect if below the live compaction percent.

Implementations

impl Config

pub fn open(&self) -> Result<Marble>

Trait Implementations

impl Clone for Config

fn clone(&self) -> Config

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Config

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Config

fn default() -> Config

Returns the “default value” for a type.