Struct RegionWriterConfig 

pub struct RegionWriterConfig {

    pub region_id: Uuid,
    pub region_spec_id: u32,
    pub durable_write: bool,
    pub sync_indexed_write: bool,
    pub max_wal_buffer_size: usize,
    pub max_wal_flush_interval: Option<Duration>,
    pub max_memtable_size: usize,
    pub max_memtable_rows: usize,
    pub max_memtable_batches: usize,
    pub ivf_index_partition_capacity_safety_factor: usize,
    pub manifest_scan_batch_size: usize,
    pub max_unflushed_memtable_bytes: usize,
    pub backpressure_log_interval: Duration,
    pub async_index_buffer_rows: usize,
    pub async_index_interval: Duration,
    pub stats_log_interval: Option<Duration>,
}

Configuration for a region writer.

Fields

region_id: Uuid

Unique identifier for this region (UUID v4).

region_spec_id: u32

Region spec ID this region was created with. A value of 0 indicates a manually-created region not governed by any spec.

durable_write: bool

Whether to wait for WAL flush before returning from writes.

When true (durable writes):

• Each write waits for WAL persistence before returning
• Guarantees no data loss on crash
• Higher latency due to object storage writes

When false (non-durable writes):

• Writes return immediately after buffering in memory
• Potential data loss if process crashes before flush
• Lower latency, batched S3 operations

sync_indexed_write: bool

Whether to update indexes synchronously on each write.

When true:

• Newly written data is immediately searchable via indexes
• Higher latency due to index update overhead

When false:

• Index updates are deferred
• New data may not appear in index-accelerated queries immediately

max_wal_buffer_size: usize

Maximum WAL buffer size in bytes before triggering a flush.

This is a soft threshold - write batches are atomic and won’t be split. WAL flushes when buffer exceeds this size OR when max_wal_flush_interval elapses. Default: 10MB

max_wal_flush_interval: Option<Duration>

Time-based WAL flush interval.

WAL buffer will be flushed after this duration even if size threshold hasn’t been reached. This ensures bounded data loss window in non-durable mode and prevents accumulating too much data before flushing to object storage. Default: 100ms

max_memtable_size: usize

Maximum MemTable size in bytes before triggering a flush to storage.

MemTable size is checked every max_wal_flush_interval (during WAL flush ticks). Default: 256MB

max_memtable_rows: usize

Maximum number of rows in a MemTable.

Used to pre-allocate index storage (e.g., IVF-PQ partition capacity). When a partition reaches capacity, memtable will be flushed. Default: 100,000 rows

max_memtable_batches: usize

Maximum number of batches in a MemTable.

Used to pre-allocate batch storage. When this limit is reached, memtable will be flushed. Sized for typical ML workloads with 1024-dim vectors (~82KB per 20-row batch). Default: 8,000 batches

ivf_index_partition_capacity_safety_factor: usize

Safety factor for IVF-PQ index partition capacity calculation.

Accounts for non-uniform distribution of vectors across partitions. Higher values use more memory but reduce overflow risk. Partition capacity = min((max_rows / num_partitions) * safety_factor, max_rows) Default: 8

manifest_scan_batch_size: usize

Batch size for parallel HEAD requests when scanning for manifest versions.

Higher values scan faster but use more parallel requests. Default: 2

max_unflushed_memtable_bytes: usize

Maximum unflushed bytes before applying backpressure.

When total unflushed data (active memtable + frozen memtables) exceeds this, new writes will block until some data is flushed to storage. This prevents unbounded memory growth during write spikes.

Default: 1GB

backpressure_log_interval: Duration

Interval for logging warnings when writes are blocked by backpressure.

When a write is blocked waiting for WAL flush, memtable flush, or index updates to complete, a warning is logged after this duration. The write will continue waiting indefinitely (it never fails due to backpressure), but warnings are logged at this interval to help diagnose slow flushes.

Default: 30 seconds

async_index_buffer_rows: usize

Maximum rows to buffer before flushing to async indexes.

Only applies when sync_indexed_write is false. Larger values enable better vectorization (especially for IVF-PQ) but increase memory usage and latency before data becomes searchable.

Default: 10,000 rows

async_index_interval: Duration

Maximum time to buffer before flushing to async indexes.

Only applies when sync_indexed_write is false. Ensures bounded latency for data to become searchable even during low write throughput.

Default: 1 second

stats_log_interval: Option<Duration>

Interval for periodic stats logging.

Stats (write throughput, backpressure events, memtable size) are logged at this interval. Set to None to disable periodic stats logging.

Default: 60 seconds

Implementations

impl RegionWriterConfig

pub fn new(region_id: Uuid) -> Self

Create a new configuration with the given region ID.

pub fn with_region_spec_id(self, spec_id: u32) -> Self

Set the region spec ID.

pub fn with_durable_write(self, durable: bool) -> Self

Set durable writes mode.

pub fn with_sync_indexed_write(self, indexed: bool) -> Self

Set indexed writes mode.

pub fn with_max_wal_buffer_size(self, size: usize) -> Self

Set maximum WAL buffer size.

pub fn with_max_wal_flush_interval(self, interval: Duration) -> Self

Set maximum flush interval.

pub fn with_max_memtable_size(self, size: usize) -> Self

Set maximum MemTable size.

pub fn with_max_memtable_rows(self, rows: usize) -> Self

Set maximum MemTable rows for index pre-allocation.

pub fn with_max_memtable_batches(self, batches: usize) -> Self

Set maximum MemTable batches for batch store pre-allocation.

pub fn with_ivf_index_partition_capacity_safety_factor( self, factor: usize, ) -> Self

Set partition capacity safety factor for IVF-PQ indexes.

pub fn with_manifest_scan_batch_size(self, size: usize) -> Self

Set manifest scan batch size.

pub fn with_max_unflushed_memtable_bytes(self, size: usize) -> Self

Set maximum unflushed bytes for backpressure.

pub fn with_backpressure_log_interval(self, interval: Duration) -> Self

Set backpressure log interval.

pub fn with_async_index_buffer_rows(self, rows: usize) -> Self

Set async index buffer rows.

pub fn with_async_index_interval(self, interval: Duration) -> Self

Set async index interval.

pub fn with_stats_log_interval(self, interval: Option<Duration>) -> Self

Set stats logging interval. Use None to disable periodic stats logging.

Trait Implementations

impl Clone for RegionWriterConfig

fn clone(&self) -> RegionWriterConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RegionWriterConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RegionWriterConfig

fn default() -> Self

Returns the “default value” for a type.