Struct StorageConfig 

pub struct StorageConfig {
    pub memtable_size_threshold: u64,
    pub memtable_hard_limit: u64,
    pub compaction: CompactionConfig,
    pub compression: CompressionConfig,
    pub use_mmap: bool,
    pub mmap_min_size_bytes: usize,
    pub disk_access_mode: DiskAccessMode,
    pub direct_io_memory_fraction: f64,
    pub prefetch: PrefetchMode,
    pub direct_io_prefetch_bytes: usize,
}

Storage engine configuration

Fields

memtable_size_threshold: u64

MemTable size threshold for flushing, in bytes (default: 64MB).

This is the AUTHORITATIVE flush trigger for the write path: it is the single value WriteEngineConfig::from_config translates into WriteEngineConfig::memtable_flush_threshold (issue #1697).

The default changed 16MB -> 64MB in #1697: before that fix this field had no production reader — the engine carried its own private 64MB default, so 64MB is the value that always actually ran. Keeping the RUNNING value preserves behaviour; adopting the decorative 16MB would have silently quadrupled everyone’s flush rate.

memtable_hard_limit: u64

MemTable HARD limit in bytes (default: 256MB) — the admission ceiling.

Live knob: the write engine’s check_admission REJECTS a write whose mutation exceeds this on its own, or that would push the memtable over it. Before issue #1697 it existed only as the private WriteEngineConfig::DEFAULT_HARD_LIMIT, so an embedder could be hard-failed by a ceiling they had no way to see or change. The default is unchanged (256MB): this exposes the knob, it does not alter behaviour. Config::validate requires it to be STRICTLY GREATER than Self::memtable_size_threshold, since a ceiling at or below the flush threshold wedges the engine — writes are rejected before a flush can ever relieve the memtable, and with zero headroom an ordinary write does it — and requires BOTH knobs to fit in the target’s usize (see validate; only reachable on 32-bit/wasm32). Note that headroom alone is not a wedge-freedom guarantee: a single mutation larger than the headroom still wedges, which is an admission-side defect tracked as #3404.

compaction: CompactionConfig

Compaction configuration

compression: CompressionConfig

Compression configuration

use_mmap: bool

Legacy promote-only flag: it upgrades an explicit DiskAccessMode::Buffered request to DiskAccessMode::Mmap.

It does not select the backend — Self::disk_access_mode does, and its Auto default already memory-maps most Data.db files (see that field). So false does not mean “buffered I/O”, and true changes nothing unless something explicitly requested Buffered. A mapped file is served from the page cache with no per-block read syscall, as Cassandra’s mmap mode does.

Safety / platform constraints

A memory map aliases the file’s bytes for the reader’s lifetime. Only enable this when the SSTables are immutable local files:

• Mutating, truncating, or deleting a mapped file out from under a live reader is undefined behaviour and can raise SIGBUS, terminating the process. CQLite never rewrites its own mapped inputs, but external tools must not either.
• Network and overlay filesystems (NFS, SMB, FUSE, some container overlays) can fault mid-read after a successful map; prefer buffered I/O there.

Interaction with the write engine (Issue #591)

This setting only affects the read path. Compaction’s input readers force use_mmap = false + explicit Buffered (only CQLITE_USE_MMAP=1 promotes even those); each input is unpublished by removing its TOC.txt before the data components, best-effort. So enabling mmap for queries is safe alongside background compaction: a compaction never holds a mapping over a file it then deletes, and on Windows a data file still pinned by a mapped reader becomes an invisible orphan (reclaimed on the next startup) rather than a failed delete or a source of duplicate rows.

Can also be enabled at runtime by setting CQLITE_USE_MMAP=1.

#[serde(default)] keeps configs serialized before this field existed (which omit it) deserializing successfully, defaulting to no promotion.

mmap_min_size_bytes: usize

Minimum Data.db size (bytes) at which DiskAccessMode::Auto maps. Default 4096.

It gates ONLY Auto, which uses buffered I/O below it (a tiny file does not repay the mapping setup); an explicit Mmap — including a Buffered promoted by Self::use_mmap — is not size-gated, only a zero-length file falls back.

#[serde(default)] for backward compatibility with older payloads.

disk_access_mode: DiskAccessMode

How the SSTable read path accesses Data.db on disk.

Defaults to DiskAccessMode::Auto, which sizes each Data.db file against system RAM and picks the backend automatically:

• files below Self::mmap_min_size_bytes use buffered I/O (mapping a tiny file is not worth the setup cost);
• files up to Self::direct_io_memory_fraction of system memory are memory-mapped, so repeated scans stay resident in the page cache;
• files larger than that fraction use direct I/O (O_DIRECT on Linux, F_NOCACHE on macOS), which bypasses the page cache so a single huge scan does not evict everything else the host has cached.

Set an explicit DiskAccessMode::Buffered, DiskAccessMode::Mmap, or DiskAccessMode::Direct to override the heuristic. The legacy Self::use_mmap flag only PROMOTES an explicit Buffered request to Mmap; it never changes what Auto resolves to.

Can also be set at runtime via CQLITE_DISK_ACCESS_MODE (auto / buffered / mmap / direct).

direct_io_memory_fraction: f64

Fraction of total system memory above which DiskAccessMode::Auto switches a file from memory-mapped to direct I/O. Defaults to 0.5 (half of RAM). Ignored when system memory cannot be determined (in which case Auto never escalates to direct I/O).

The legal range is (0.0, 1.0] and Config::validate REJECTS anything outside it, NaN and the infinities included (issue #1696). It used to be silently clamped instead — a 2.0 or a -1 quietly became the 0.5 default — so the value an operator set was not the value that ran. It is a FRACTION, never a byte count; to always bypass the page cache, ask for DiskAccessMode::Direct.

prefetch: PrefetchMode

Read-ahead / prefetch strategy applied to the chosen backend.

Defaults to PrefetchMode::Auto, which issues no mmap madvise (relying on the kernel’s default read-ahead) and only enables the direct-I/O prefetch window of Self::direct_io_prefetch_bytes. Set PrefetchMode::Off to disable explicit hints (relying only on default kernel read-ahead / single-block direct reads). Can also be set via CQLITE_PREFETCH (off / sequential / willneed / auto).

direct_io_prefetch_bytes: usize

Size in bytes of the read-ahead window used by the direct-I/O backend, and by nothing else: the buffered backend ignores it (open_buffered_sources takes no prefetch bytes; its BufReader::new capacity is tokio’s 8 KiB default). Rounded up to the I/O alignment; 1 MiB default; inert while prefetch is Off.

Implementations

impl StorageConfig

pub fn validated_direct_io_memory_fraction(&self) -> Result<f64>

Self::direct_io_memory_fraction if it is a legal fraction, else a configuration error (issue #1696, AH3).

Why this is a method and not an inline check in validate

It is enforced at EVERY public boundary that can act on the value, and several of them are reachable without a Database: Config::validate, Database::open, StorageEngine::open, StorageEngine::open_with_sstables, SSTableManager::new, SSTableManager::new_from_discovered_paths and SSTableReader::open. That many call sites is precisely why the rule needs ONE definition — restated inline they would drift.

The discovery boundaries matter for a second reason (#1696 roborev r3 F2): discovery treats a per-file reader-open error as best-effort, logging and skipping it, so an unvalidated bad fraction there would fail every reader open and the engine would report SUCCESS with ZERO SSTables — a silent empty result instead of a named config error.

The rule, and why the ends of the range are where they are

The legal range is the documented (0.0, 1.0]. Before this existed the value was live but unvalidated: the reader’s resolve_disk_access_mode silently CLAMPED nonsense — <= 0.0, NaN and the infinities fell back to the 0.5 default, and anything above 1.0 was pinned at 1.0. An operator who wrote 2.0 (meaning “twice RAM”) or -1 got the default and no word about it, which is the same dishonesty as a decorative knob: the value they set was not the value that ran.

• 1.0 is LEGAL — “all of RAM” is a coherent ceiling.
• 0.0 is REJECTED, and is NOT read as “never use direct I/O” — that is the whole reason it cannot be accepted. A zero threshold makes EVERY nonempty file exceed it, so Auto would escalate everything to direct I/O: the value reads as “never” and behaves as “always”. Inferring which one the operator meant would be a guess, and CQLite does not guess (issue #28). “Never use direct I/O” is spelled super::DiskAccessMode::Mmap (or super::DiskAccessMode::Buffered); “always” is spelled super::DiskAccessMode::Direct.
• A subnormal or otherwise tiny positive fraction is LEGAL and is honoured LITERALLY: 1e-300 of RAM rounds to a 0-byte threshold, so every nonempty file uses direct I/O. That is the honest consequence of what was asked for, and unlike 0.0 it is unambiguous — a real, if degenerate, fraction rather than a value whose plain reading contradicts its behaviour. It is not clamped and not second-guessed.
• NaN and both infinities are REJECTED. The test is written as !(fraction > 0.0 && fraction <= 1.0) rather than a chain of </> precisely so NaN — for which every ordered comparison is false — is rejected instead of sailing through.

The reader keeps its internal clamp as defense in depth for any future caller that reaches resolve_disk_access_mode without validating.

Trait Implementations

impl Clone for StorageConfig

fn clone(&self) -> StorageConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StorageConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StorageConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for StorageConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for StorageConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.