Struct mappum_rocksdb::Options

pub struct Options { /* fields omitted */ }

Database-wide options around performance and behavior.

Please read the official tuning guide, and most importantly, measure performance under realistic workloads with realistic hardware.

Examples

use rocksdb::{Options, DB};
use rocksdb::DBCompactionStyle;

fn badly_tuned_for_somebody_elses_disk() -> DB {
   let path = "path/for/rocksdb/storageX";
   let mut opts = Options::default();
   opts.create_if_missing(true);
   opts.set_max_open_files(10000);
   opts.set_use_fsync(false);
   opts.set_bytes_per_sync(8388608);
   opts.optimize_for_point_lookup(1024);
   opts.set_table_cache_num_shard_bits(6);
   opts.set_max_write_buffer_number(32);
   opts.set_write_buffer_size(536870912);
   opts.set_target_file_size_base(1073741824);
   opts.set_min_write_buffer_number_to_merge(4);
   opts.set_level_zero_stop_writes_trigger(2000);
   opts.set_level_zero_slowdown_writes_trigger(0);
   opts.set_compaction_style(DBCompactionStyle::Universal);
   opts.set_max_background_compactions(4);
   opts.set_max_background_flushes(4);
   opts.set_disable_auto_compactions(true);

   DB::open(&opts, path).unwrap()
}

Methods

impl Options

pub fn increase_parallelism(&mut self, parallelism: i32)

By default, RocksDB uses only one background thread for flush and compaction. Calling this function will set it up such that total of total_threads is used. Good value for total_threads is the number of cores. You almost definitely want to call this function if your system is bottlenecked by RocksDB.

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.increase_parallelism(3);

pub fn optimize_level_style_compaction(&mut self, memtable_memory_budget: usize)

pub fn create_if_missing(&mut self, create_if_missing: bool)

If true, the database will be created if it is missing.

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.create_if_missing(true);

pub fn create_missing_column_families(&mut self, create_missing_cfs: bool)

If true, any column families that didn't exist when opening the database will be created.

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.create_missing_column_families(true);

pub fn set_compression_type(&mut self, t: DBCompressionType)

Sets the compression algorithm that will be used for the bottommost level that contain files. If level-compaction is used, this option will only affect levels after base level.

Default: DBCompressionType::None

Example

use rocksdb::{Options, DBCompressionType};

let mut opts = Options::default();
opts.set_compression_type(DBCompressionType::Snappy);

pub fn set_compression_per_level(&mut self, level_types: &[DBCompressionType])

Different levels can have different compression policies. There are cases where most lower levels would like to use quick compression algorithms while the higher levels (which have more data) use compression algorithms that have better compression but could be slower. This array, if non-empty, should have an entry for each level of the database; these override the value specified in the previous field 'compression'.

Example

use rocksdb::{Options, DBCompressionType};

let mut opts = Options::default();
opts.set_compression_per_level(&[
    DBCompressionType::None,
    DBCompressionType::None,
    DBCompressionType::Snappy,
    DBCompressionType::Snappy,
    DBCompressionType::Snappy
]);

pub fn set_compression_options(
    &mut self,
    w_bits: c_int,
    level: c_int,
    strategy: c_int,
    max_dict_bytes: c_int
)

Maximum size of dictionaries used to prime the compression library. Enabling dictionary can improve compression ratios when there are repetitions across data blocks.

The dictionary is created by sampling the SST file data. If zstd_max_train_bytes is nonzero, the samples are passed through zstd's dictionary generator. Otherwise, the random samples are used directly as the dictionary.

When compression dictionary is disabled, we compress and write each block before buffering data for the next one. When compression dictionary is enabled, we buffer all SST file data in-memory so we can sample it, as data can only be compressed and written after the dictionary has been finalized. So users of this feature may see increased memory usage.

Default: 0

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_compression_options(4, 5, 6, 7);

pub fn set_compaction_readahead_size(
    &mut self,
    compaction_readahead_size: usize
)

If non-zero, we perform bigger reads when doing compaction. If you're running RocksDB on spinning disks, you should set this to at least 2MB. That way RocksDB's compaction is doing sequential instead of random reads.

When non-zero, we also force new_table_reader_for_compaction_inputs to true.

Default: 0

pub fn set_level_compaction_dynamic_level_bytes(&mut self, v: bool)

Allow RocksDB to pick dynamic base of bytes for levels. With this feature turned on, RocksDB will automatically adjust max bytes for each level. The goal of this feature is to have lower bound on size amplification.

Default: false.

pub fn set_merge_operator(
    &mut self,
    name: &str,
    full_merge_fn: MergeFn,
    partial_merge_fn: Option<MergeFn>
)

pub fn add_merge_operator(&mut self, name: &str, merge_fn: MergeFn)

Deprecated since 0.5.0:

add_merge_operator has been renamed to set_merge_operator

pub fn set_compaction_filter<F>(&mut self, name: &str, filter_fn: F) where
    F: CompactionFilterFn + Send + 'static, 

Sets a compaction filter used to determine if entries should be kept, changed, or removed during compaction.

An example use case is to remove entries with an expired TTL.

If you take a snapshot of the database, only values written since the last snapshot will be passed through the compaction filter.

If multi-threaded compaction is used, filter_fn may be called multiple times simultaneously.

pub fn set_comparator(
    &mut self,
    name: &str,
    compare_fn: fn(_: &[u8], _: &[u8]) -> Ordering
)

Sets the comparator used to define the order of keys in the table. Default: a comparator that uses lexicographic byte-wise ordering

The client must ensure that the comparator supplied here has the same name and orders keys exactly the same as the comparator provided to previous open calls on the same DB.

pub fn set_prefix_extractor(&mut self, prefix_extractor: SliceTransform)

pub fn add_comparator(
    &mut self,
    name: &str,
    compare_fn: fn(_: &[u8], _: &[u8]) -> Ordering
)

Deprecated since 0.5.0:

add_comparator has been renamed to set_comparator

pub fn optimize_for_point_lookup(&mut self, cache_size: u64)

pub fn set_optimize_filters_for_hits(&mut self, optimize_for_hits: bool)

Sets the optimize_filters_for_hits flag

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_optimize_filters_for_hits(true);

pub fn set_max_open_files(&mut self, nfiles: c_int)

Sets the number of open files that can be used by the DB. You may need to increase this if your database has a large working set. Value -1 means files opened are always kept open. You can estimate number of files based on target_file_size_base and target_file_size_multiplier for level-based compaction. For universal-style compaction, you can usually set it to -1.

Default: -1

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_open_files(10);

pub fn set_use_fsync(&mut self, useit: bool)

If true, then every store to stable storage will issue a fsync. If false, then every store to stable storage will issue a fdatasync. This parameter should be set to true while storing data to filesystem like ext3 that can lose files after a reboot.

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_use_fsync(true);

pub fn set_bytes_per_sync(&mut self, nbytes: u64)

Allows OS to incrementally sync files to disk while they are being written, asynchronously, in the background. This operation can be used to smooth out write I/Os over time. Users shouldn't rely on it for persistency guarantee. Issue one request for every bytes_per_sync written. 0 turns it off.

Default: 0

You may consider using rate_limiter to regulate write rate to device. When rate limiter is enabled, it automatically enables bytes_per_sync to 1MB.

This option applies to table files

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_bytes_per_sync(1024 * 1024);

pub fn set_allow_concurrent_memtable_write(&mut self, allow: bool)

If true, allow multi-writers to update mem tables in parallel. Only some memtable_factory-s support concurrent writes; currently it is implemented only for SkipListFactory. Concurrent memtable writes are not compatible with inplace_update_support or filter_deletes. It is strongly recommended to set enable_write_thread_adaptive_yield if you are going to use this feature.

Default: true

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_allow_concurrent_memtable_write(false);

pub fn set_use_direct_reads(&mut self, enabled: bool)

Enable direct I/O mode for reading they may or may not improve performance depending on the use case

Files will be opened in "direct I/O" mode which means that data read from the disk will not be cached or buffered. The hardware buffer of the devices may however still be used. Memory mapped files are not impacted by these parameters.

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_use_direct_reads(true);

pub fn set_use_direct_io_for_flush_and_compaction(&mut self, enabled: bool)

Enable direct I/O mode for flush and compaction

Files will be opened in "direct I/O" mode which means that data written to the disk will not be cached or buffered. The hardware buffer of the devices may however still be used. Memory mapped files are not impacted by these parameters. they may or may not improve performance depending on the use case

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_use_direct_io_for_flush_and_compaction(true);

pub fn set_allow_os_buffer(&mut self, is_allow: bool)

Deprecated since 0.7.0:

replaced with set_use_direct_reads/set_use_direct_io_for_flush_and_compaction methods

Hints to the OS that it should not buffer disk I/O. Enabling this parameter may improve performance but increases pressure on the system cache.

The exact behavior of this parameter is platform dependent.

On POSIX systems, after RocksDB reads data from disk it will mark the pages as "unneeded". The operating system may - or may not

• evict these pages from memory, reducing pressure on the system cache. If the disk block is requested again this can result in additional disk I/O.

On WINDOWS systems, files will be opened in "unbuffered I/O" mode which means that data read from the disk will not be cached or bufferized. The hardware buffer of the devices may however still be used. Memory mapped files are not impacted by this parameter.

Default: true

Example

#[allow(deprecated)]
use rocksdb::Options;

let mut opts = Options::default();
opts.set_allow_os_buffer(false);

pub fn set_table_cache_num_shard_bits(&mut self, nbits: c_int)

Sets the number of shards used for table cache.

Default: 6

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_table_cache_num_shard_bits(4);

pub fn set_min_write_buffer_number(&mut self, nbuf: c_int)

Sets the minimum number of write buffers that will be merged together before writing to storage. If set to 1, then all write buffers are flushed to L0 as individual files and this increases read amplification because a get request has to check in all of these files. Also, an in-memory merge may result in writing lesser data to storage if there are duplicate records in each of these individual write buffers.

Default: 1

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_min_write_buffer_number(2);

pub fn set_max_write_buffer_number(&mut self, nbuf: c_int)

Sets the maximum number of write buffers that are built up in memory. The default and the minimum number is 2, so that when 1 write buffer is being flushed to storage, new writes can continue to the other write buffer. If max_write_buffer_number > 3, writing will be slowed down to options.delayed_write_rate if we are writing to the last write buffer allowed.

Default: 2

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_write_buffer_number(4);

pub fn set_write_buffer_size(&mut self, size: usize)

Sets the amount of data to build up in memory (backed by an unsorted log on disk) before converting to a sorted on-disk file.

Larger values increase performance, especially during bulk loads. Up to max_write_buffer_number write buffers may be held in memory at the same time, so you may wish to adjust this parameter to control memory usage. Also, a larger write buffer will result in a longer recovery time the next time the database is opened.

Note that write_buffer_size is enforced per column family. See db_write_buffer_size for sharing memory across column families.

Default: 0x4000000 (64MiB)

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_write_buffer_size(128 * 1024 * 1024);

pub fn set_db_write_buffer_size(&mut self, size: usize)

Amount of data to build up in memtables across all column families before writing to disk.

This is distinct from write_buffer_size, which enforces a limit for a single memtable.

This feature is disabled by default. Specify a non-zero value to enable it.

Default: 0 (disabled)

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_db_write_buffer_size(128 * 1024 * 1024);

pub fn set_max_bytes_for_level_base(&mut self, size: u64)

Control maximum total data size for a level. max_bytes_for_level_base is the max total for level-1. Maximum number of bytes for level L can be calculated as (max_bytes_for_level_base) * (max_bytes_for_level_multiplier ^ (L-1)) For example, if max_bytes_for_level_base is 200MB, and if max_bytes_for_level_multiplier is 10, total data size for level-1 will be 200MB, total file size for level-2 will be 2GB, and total file size for level-3 will be 20GB.

Default: 0x10000000 (256MiB).

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_bytes_for_level_base(512 * 1024 * 1024);

pub fn set_max_bytes_for_level_multiplier(&mut self, mul: f64)

Default: 10

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_bytes_for_level_multiplier(4.0);

pub fn set_max_manifest_file_size(&mut self, size: usize)

The manifest file is rolled over on reaching this limit. The older manifest file be deleted. The default value is MAX_INT so that roll-over does not take place.

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_manifest_file_size(20 * 1024 * 1024);

pub fn set_target_file_size_base(&mut self, size: u64)

Sets the target file size for compaction. target_file_size_base is per-file size for level-1. Target file size for level L can be calculated by target_file_size_base * (target_file_size_multiplier ^ (L-1)) For example, if target_file_size_base is 2MB and target_file_size_multiplier is 10, then each file on level-1 will be 2MB, and each file on level 2 will be 20MB, and each file on level-3 will be 200MB.

Default: 0x4000000 (64MiB)

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_target_file_size_base(128 * 1024 * 1024);

pub fn set_min_write_buffer_number_to_merge(&mut self, to_merge: c_int)

Sets the minimum number of write buffers that will be merged together before writing to storage. If set to 1, then all write buffers are flushed to L0 as individual files and this increases read amplification because a get request has to check in all of these files. Also, an in-memory merge may result in writing lesser data to storage if there are duplicate records in each of these individual write buffers.

Default: 1

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_min_write_buffer_number_to_merge(2);

pub fn set_level_zero_file_num_compaction_trigger(&mut self, n: c_int)

Sets the number of files to trigger level-0 compaction. A value < 0 means that level-0 compaction will not be triggered by number of files at all.

Default: 4

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_level_zero_file_num_compaction_trigger(8);

pub fn set_level_zero_slowdown_writes_trigger(&mut self, n: c_int)

Sets the soft limit on number of level-0 files. We start slowing down writes at this point. A value < 0 means that no writing slow down will be triggered by number of files in level-0.

Default: 20

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_level_zero_slowdown_writes_trigger(10);

pub fn set_level_zero_stop_writes_trigger(&mut self, n: c_int)

Sets the maximum number of level-0 files. We stop writes at this point.

Default: 24

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_level_zero_stop_writes_trigger(48);

pub fn set_compaction_style(&mut self, style: DBCompactionStyle)

Sets the compaction style.

Default: DBCompactionStyle::Level

Example

use rocksdb::{Options, DBCompactionStyle};

let mut opts = Options::default();
opts.set_compaction_style(DBCompactionStyle::Universal);

pub fn set_max_background_compactions(&mut self, n: c_int)

Sets the maximum number of concurrent background compaction jobs, submitted to the default LOW priority thread pool. We first try to schedule compactions based on base_background_compactions. If the compaction cannot catch up , we will increase number of compaction threads up to max_background_compactions.

If you're increasing this, also consider increasing number of threads in LOW priority thread pool. For more information, see Env::SetBackgroundThreads

Default: 1

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_background_compactions(2);

pub fn set_max_background_flushes(&mut self, n: c_int)

Sets the maximum number of concurrent background memtable flush jobs, submitted to the HIGH priority thread pool.

By default, all background jobs (major compaction and memtable flush) go to the LOW priority pool. If this option is set to a positive number, memtable flush jobs will be submitted to the HIGH priority pool. It is important when the same Env is shared by multiple db instances. Without a separate pool, long running major compaction jobs could potentially block memtable flush jobs of other db instances, leading to unnecessary Put stalls.

If you're increasing this, also consider increasing number of threads in HIGH priority thread pool. For more information, see Env::SetBackgroundThreads

Default: 1

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_max_background_flushes(2);

pub fn set_disable_auto_compactions(&mut self, disable: bool)

Disables automatic compactions. Manual compactions can still be issued on this column family

Default: false

Dynamically changeable through SetOptions() API

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_disable_auto_compactions(true);

pub fn set_memtable_factory(&mut self, factory: MemtableFactory)

Defines the underlying memtable implementation. See https://github.com/facebook/rocksdb/wiki/MemTable for more information. Defaults to using a skiplist.

Example

use rocksdb::{Options, MemtableFactory};
let mut opts = Options::default();
let factory = MemtableFactory::HashSkipList {
    bucket_count: 1_000_000,
    height: 4,
    branching_factor: 4,
};

opts.set_allow_concurrent_memtable_write(false);
opts.set_memtable_factory(factory);

pub fn set_block_based_table_factory(&mut self, factory: &BlockBasedOptions)

pub fn set_plain_table_factory(&mut self, options: &PlainTableFactoryOptions)

See https://github.com/facebook/rocksdb/wiki/PlainTable-Format.

use rocksdb::{Options, PlainTableFactoryOptions};

let mut opts = Options::default();
let factory_opts = PlainTableFactoryOptions {
  user_key_length: 0,
  bloom_bits_per_key: 20,
  hash_table_ratio: 0.75,
  index_sparseness: 16,
};

opts.set_plain_table_factory(&factory_opts);

pub fn set_report_bg_io_stats(&mut self, enable: bool)

Measure IO stats in compactions and flushes, if true.

Default: false

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_report_bg_io_stats(true);

pub fn set_max_total_wal_size(&mut self, size: u64)

Once write-ahead logs exceed this size, we will start forcing the flush of column families whose memtables are backed by the oldest live WAL file (i.e. the ones that are causing all the space amplification).

Default: 0

Example

use rocksdb::Options;

let mut opts = Options::default();
// Set max total wal size to 1G.
opts.set_max_total_wal_size(1 << 30);

pub fn set_wal_recovery_mode(&mut self, mode: DBRecoveryMode)

Recovery mode to control the consistency while replaying WAL.

Default: DBRecoveryMode::PointInTime

Example

use rocksdb::{Options, DBRecoveryMode};

let mut opts = Options::default();
opts.set_wal_recovery_mode(DBRecoveryMode::AbsoluteConsistency);

pub fn enable_statistics(&mut self)

pub fn get_statistics(&self) -> Option<String>

pub fn set_stats_dump_period_sec(&mut self, period: c_uint)

If not zero, dump rocksdb.stats to LOG every stats_dump_period_sec.

Default: 600 (10 mins)

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_stats_dump_period_sec(300);

pub fn set_advise_random_on_open(&mut self, advise: bool)

When set to true, reading SST files will opt out of the filesystem's readahead. Setting this to false may improve sequential iteration performance.

Default: true

pub fn set_num_levels(&mut self, n: c_int)

Sets the number of levels for this database.

pub fn set_memtable_prefix_bloom_ratio(&mut self, ratio: f64)

When a prefix_extractor is defined through opts.set_prefix_extractor this creates a prefix bloom filter for each memtable with the size of write_buffer_size * memtable_prefix_bloom_ratio (capped at 0.25).

Default: 0

Example

use rocksdb::{Options, SliceTransform};

let mut opts = Options::default();
let transform = SliceTransform::create_fixed_prefix(10);
opts.set_prefix_extractor(transform);
opts.set_memtable_prefix_bloom_ratio(0.2);

pub fn set_wal_dir<P: AsRef<Path>>(&mut self, path: P)

Specifies the absolute path of the directory the write-ahead log (WAL) should be written to.

Default: same directory as the database

Example

use rocksdb::Options;

let mut opts = Options::default();
opts.set_wal_dir("/path/to/dir");

pub fn set_skip_stats_update_on_db_open(&mut self, skip: bool)

If true, then DB::Open() will not update the statistics used to optimize compaction decision by loading table properties from many files. Turning off this feature will improve DBOpen time especially in disk environment.

Default: false

pub fn set_keep_log_file_num(&mut self, nfiles: usize)

Specify the maximal number of info log files to be kept.

pub fn set_allow_mmap_writes(&mut self, is_enabled: bool)

Allow the OS to mmap file for writing.

Default: false

Example

use rocksdb::Options;

let mut options = Options::default();
options.set_allow_mmap_writes(true);

pub fn set_allow_mmap_reads(&mut self, is_enabled: bool)

Allow the OS to mmap file for reading sst tables.

Default: false

Example

use rocksdb::Options;

let mut options = Options::default();
options.set_allow_mmap_reads(true);

pub fn set_atomic_flush(&mut self, atomic_flush: bool)

Guarantee that all column families are flushed together atomically. This option applies to both manual flushes (db.flush()) and automatic background flushes caused when memtables are filled.

Default: false

Example

use rocksdb::Options;

let mut options = Options::default();
options.set_atomic_flush(true);

pub fn set_ratelimiter(
    &mut self,
    rate_bytes_per_sec: i64,
    refill_period_us: i64,
    fairness: i32
)

Use to control write rate of flush and compaction. Flush has higher priority than compaction. If rate limiter is enabled, bytes_per_sync is set to 1MB by default.

Default: disable

Example

use rocksdb::Options;

let mut options = Options::default();
options.set_ratelimiter(1024 * 1024, 100 * 1000, 10);

Trait Implementations

impl Default for Options

fn default() -> Options

Returns the "default value" for a type.

impl Drop for Options

fn drop(&mut self)

Executes the destructor for this type.

impl Send for Options

impl Sync for Options