Struct VarMap 

pub struct VarMap<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K> = NoHook> { /* private fields */ }

Implementations

impl<K: Key + Send + Sync + Hash + Eq> VarMap<K>

pub fn open(path: impl AsRef<Path>, config: Config) -> DbResult<Self>

Open or create a VarMap at the given path. Recovers the index from existing data files on disk.

Exclusive access

Takes an exclusive lease on the directory (db.lock) and holds it for as long as the returned handle — or anything that outlives it, such as a replication server or client — is alive. A second open of the same directory returns DbError::Locked, whether it comes from another process or from another live handle in this one: two engines over one shard set would run independent in-memory indexes and silently drop each other’s writes.

The kernel releases the lease when the handle is dropped, including on SIGKILL, panic or OOM, so there is no stale lock file to clean up. To reopen in the same process, drop every handle first — shadowing a binding (let tree = ...) does not drop it.

On NFS flock is emulated with POSIX locks: cross-process exclusion still works, but the same-process rejection is lost.

impl<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K>> VarMap<K, H>

pub fn open_hooked( path: impl AsRef<Path>, config: Config, hook: H, ) -> DbResult<Self>

Open or create a VarMap with a write hook for secondary index maintenance.

Exclusive access

Takes an exclusive lease on the directory (db.lock) and holds it for as long as the returned handle — or anything that outlives it, such as a replication server or client — is alive. A second open of the same directory returns DbError::Locked, whether it comes from another process or from another live handle in this one: two engines over one shard set would run independent in-memory indexes and silently drop each other’s writes.

The kernel releases the lease when the handle is dropped, including on SIGKILL, panic or OOM, so there is no stale lock file to clean up. To reopen in the same process, drop every handle first — shadowing a binding (let tree = ...) does not drop it.

On NFS flock is emulated with POSIX locks: cross-process exclusion still works, but the same-process rejection is lost.

pub fn clean_shutdown(&self) -> DbResult<()>

Full clean shutdown: write hint files (if enabled), flush write buffers + fsync.

pub fn close(self) -> DbResult<()>

Graceful shutdown: write hint files (if enabled), flush write buffers + fsync.

pub fn flush_buffers(&self) -> DbResult<()>

Flush all shard write buffers to disk (without fsync).

pub fn config(&self) -> &Config

Get the database configuration.

impl<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K>> VarMap<K, H>

pub fn sweep_expired( &self, now_ms: u64, budget: usize, ) -> DbResult<SweepOutcome>

pub fn set_ttl_sweep(&self, enabled: bool) -> DbResult<()>

pub fn mark_replication_follower(&self)

pub fn rebuild_ttl_index(&self) -> DbResult<()>

impl<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K>> VarMap<K, H>

pub fn compact(&self) -> DbResult<usize>

Trigger a compaction pass across all shards.

pub fn get(&self, key: &K) -> Option<ByteView>

Get a value by key. O(1) average index lookup; checks block cache first, then disk. Takes a brief per-shard mutex to look up the location; the pread on a cache miss runs outside the lock.

pub fn try_get(&self, key: &K) -> DbResult<Option<ByteView>>

Strict read: Ok(None) only when the key is genuinely absent; Err when the key exists but its value cannot be read. The result-returning primitive for VarMap (no node helper — re-locks the shard index per stale-retry).

pub fn get_or_err(&self, key: &K) -> DbResult<ByteView>

Get a value by key, returning Err(KeyNotFound) if absent.

pub fn put(&self, key: &K, value: &[u8]) -> DbResult<bool>

Insert or update a key-value pair.

Returns true if a previous value for key existed (i.e. this was an overwrite), false for a fresh insert. The flag is read directly from the index update and adds no extra I/O or value decoding.

pub fn insert(&self, key: &K, value: &[u8]) -> DbResult<()>

Insert a key-value pair only if the key does not exist. Returns Err(KeyExists) if the key is already present.

pub fn delete(&self, key: &K) -> DbResult<bool>

Delete a key. Returns true if the key existed.

pub fn cas(&self, key: &K, expected: &[u8], new_value: &[u8]) -> DbResult<()>

Compare-and-swap: if current value == expected, replace with new_value. Returns Ok(()) on success, Err(CasMismatch) if current != expected, Err(KeyNotFound) if key doesn’t exist.

Caveat: Holds the shard lock while reading the current value. On a block-cache miss this performs disk I/O under the lock, blocking all writes to the same shard. Pre-warm the cache to avoid latency spikes.

pub fn compare_delete(&self, key: &K, expected: &[u8]) -> DbResult<()>

Compare-and-delete: if the current value == expected, delete the key. Returns Ok(()) on success, Err(CasMismatch) if current != expected, Err(KeyNotFound) if the key doesn’t exist.

Caveat: Holds the shard lock while reading the current value. On a block-cache miss this performs disk I/O under the lock, blocking all writes to the same shard. Pre-warm the cache to avoid latency spikes.

pub fn update( &self, key: &K, f: impl FnOnce(&[u8]) -> ByteView, ) -> DbResult<Option<ByteView>>

Atomically read-modify-write. Returns Some(new_value) if key existed, None otherwise. The closure receives the current value and returns a new ByteView. The closure must not be heavy (shard lock is held).

Caveat: Holds the shard lock while reading the current value. On a block-cache miss this performs disk I/O under the lock, blocking all writes to the same shard. Pre-warm the cache to avoid latency spikes.

pub fn fetch_update( &self, key: &K, f: impl FnOnce(&[u8]) -> ByteView, ) -> DbResult<Option<ByteView>>

Like update(), but returns Some(old_value) instead of the new one.

pub fn contains(&self, key: &K) -> bool

Check if a key exists.

pub fn entry_len(&self, key: &K) -> Option<u32>

Encoded value byte length for key, or None if absent. Reads only the in-memory index entry (DiskLoc::len); no disk I/O.

pub fn len(&self) -> usize

Total number of entries across all shards.

pub fn is_empty(&self) -> bool

pub fn sync_hints(&self) -> DbResult<()>

Write hint files for all active shard files. Call during graceful shutdown.

pub fn iter_view(&self) -> Option<VarMapIterView<'_, K, H>>

Ordered scan view. None unless the collection was opened with iterable(true). Direction follows Config::reversed (default DESC).

Occasional admin/maintenance API, not a hot path. Use it to browse a collection or bulk-clean stale rows (retain); for scan-heavy access reach for the *Tree sibling — a Map’s strength is O(1) point lookup, and the companion key index costs extra memory plus O(log N) per write. Unlike a Tree scan it is not a point-in-time snapshot: the k-way merge drops each shard lock between batches, so concurrent writes may be only partially reflected (run on a quiet DB if you need a coherent view).

pub fn scope<const N: usize>( &self, key: &K, ) -> Option<Scope<'_, MapScan<&Self>, N>>

Field-granular prefix / range / page builder. None unless opened iterable(true).

pub fn warmup(&self) -> DbResult<()>

Pre-populate the block cache with blocks containing live values.

Walks the index to collect unique block offsets, sorts them for sequential I/O, then reads each block into the cache. Only blocks with live data are loaded.

pub fn for_each(&self, f: impl FnMut(K, ByteView))

Read-only pass over all live entries. Order is unspecified; entries inserted/removed concurrently may be missed or visited once — intended for schema validation on a quiet database.

Lock order matches replay_init: snapshot keys under the per-shard index lock, release it, then read each value via self.try_get (which re-acquires the index lock only briefly). A key that is genuinely absent by read time was concurrently deleted — skipped silently; only a real read failure is logged with a warn.

pub fn migrate( &self, f: impl Fn(&K, &[u8]) -> MigrateAction<ByteView>, ) -> DbResult<usize>

Iterate all entries and optionally mutate them. Call once at startup.

• Keep — no change (fires on_init if H::NEEDS_INIT); not counted
• Update(new_value) — replace value (hook-free write, fires on_init)
• Delete — remove entry (hook-free tombstone, no hooks)

on_write is NEVER fired during migrate — see docs/hooks.md in the armdb crate.

A value read failure aborts the migration with Err (fail-loud): the step is not committed and will re-run on next open. Consequence: one unreadable entry blocks opening until the underlying I/O issue is fixed (or data is restored from a replica/backup) — consistency over availability by design.

Returns the number of mutated entries.

pub fn atomic<R>( &self, shard_key: &K, f: impl FnOnce(&mut VarMapShard<'_, K, H>) -> DbResult<R>, ) -> DbResult<R>

Atomically execute multiple operations on a single shard. All keys must route to the same shard as shard_key. The closure must be short — shard lock is held for its duration.

pub fn shard_for(&self, key: &K) -> usize

impl<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K>> VarMap<K, H>

pub fn start_replication_server( &self, bind_addr: SocketAddr, signal: ShutdownSignal, ) -> DbResult<ReplicationServer>

Install SPSC replication producers into every shard and start a ReplicationServer bound to bind_addr.

Single-call contract

Each call installs fresh SPSC producers, replacing any previously installed ones. Call this at most once per VarMap instance — a second call will orphan the in-flight producer of any active streaming connection on the first server, which will then observe an empty ring buffer and silently stop forwarding entries.

pub fn start_replication_client( &self, leader_addr: SocketAddr, registry: Arc<ReplicationRegistry>, signal: ShutdownSignal, ) -> DbResult<ReplicationClient>

Start a ReplicationClient that streams entries from leader_addr into registry. Symmetric to Self::start_replication_server.

key_len is derived from size_of::<K>().

impl<K, H> VarMap<K, H>

where K: Key + Send + Sync + Hash + Eq + 'static, H: WriteHook<K> + Send + Sync + 'static,

pub fn as_replication_target(self: &Arc<Self>) -> Box<dyn ReplicationTarget>

Wrap a shared handle to this map as a Box<dyn ReplicationTarget>.

The returned box holds an Arc clone — the caller retains full read access to the original map through the Arc while the registry owns the box. This is the intended pattern for follower-side wiring:

let follower = Arc::new(VarMap::<[u8; 8]>::open(path, cfg)?);
let registry = ReplicationRegistry::new(follower.as_replication_target());
// `follower` remains usable for .get() etc.

Trait Implementations

impl<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K>> CompactionIndex<K> for VarMap<K, H>

fn update_if_match(&self, key: &K, old_loc: DiskLoc, new_loc: DiskLoc) -> bool

If the current index points to old_loc, it is updated to new_loc and returns true.

fn invalidate_blocks(&self, shard_id: u8, file_id: u32, total_bytes: u64)

Invalidate cached blocks for a file after compaction replaces its contents.

fn contains_key(&self, key: &K) -> bool

Returns true if the key currently exists in the index (i.e. has a live Put).

fn is_live(&self, shard_id: u8, key: &K, loc: DiskLoc) -> bool

Returns true iff the index currently maps key to exactly loc — i.e. the entry at loc is the live (latest) version of key.

impl<K: Key + Send + Sync + Hash + Eq, H: WriteHook<K>> ReplicationTarget for VarMap<K, H>

Available on crate feature replication only.

fn apply_entry( &self, _shard_inner: &mut ShardInner, _shard_id: u8, file_id: u32, entry_offset: u64, header: &EntryHeader, key: &[u8], _value: &[u8], ) -> DbResult<ApplyOutcome>

Apply entry with known key/value split (streaming mode, O(1) routing). Returns the outcome so the registry can account dead bytes.

fn try_apply_entry( &self, shard_inner: &mut ShardInner, shard_id: u8, file_id: u32, entry_offset: u64, header: &EntryHeader, raw_after_header: &[u8], ) -> DbResult<ApplyOutcome>

Try to apply entry with CRC matching (catch-up mode). Returns ApplyOutcome::NotMatched if CRC fails (key belongs to a different target).

fn key_len(&self) -> usize

The key length (K) for this tree type.

fn mark_replication_follower(&self)

Disable TTL sweep and require rebuild before re-enabling after follower duty.