Struct IdIndex 

pub struct IdIndex { /* private fields */ }

Per-document ID index — atomic file-per-doc, sharded across 256 subdirs.

Write path: updates go to write_buf (DashMap, zero I/O, lock-free). Background ticker calls flush_write_buf() every 1s — Rayon-parallel disk writes. Read path: write_buf checked first (latest value), then disk. This eliminates per-PUT fs::rename from the hot path, fixing concurrent write contention.

Implementations

impl IdIndex

pub fn new(db_root: &Path) -> Result<IdIndex, Error>

pub fn in_memory() -> IdIndex

Create a pure in-memory id index — no disk I/O.

pub fn try_flush_write_buf(&self) -> Result<(), Error>

Flush the WAL write buffer to disk in parallel. Called by the background ticker. No-op for in-memory databases. Safe to call concurrently with writes. Flush the in-memory WAL to disk, reporting I/O failure to the caller. Every entry is attempted; the first error is returned after the pass.

DURABILITY INVARIANT: an entry is dropped from write_buf ONLY when its disk write actually succeeded. A failed write (ENOSPC, EIO, EROFS) leaves the entry buffered so the next tick retries it.

Before 2.8.6 this cleared the buffer unconditionally, which silently and permanently discarded acknowledged writes whenever a flush hit a full disk: put() had already returned Ok and the content-addressed object was durable (so verify() still counted it), but no id-index entry ever reached disk — so the row was simply absent on reopen, with no error anywhere. Reproduced on a full 22 MiB filesystem: 30 rows acknowledged, verify() reported 30 healthy objects, list() returned 0.

pub fn flush_write_buf(&self)

Back-compat wrapper around [try_flush_write_buf]: flushes and logs. Prefer the try_ form — a swallowed flush error is a lost write.

pub fn get(&self, coll: &str, id: &str) -> Option<String>

Get the current object hash for a document. Checks WAL write buffer first (most recent), then disk.

pub fn set(&self, coll: &str, id: &str, hash: &str) -> Result<(), Error>

Set the current object hash for a document. Disk mode: writes to WAL buffer only (zero I/O on hot path). Background ticker flushes WAL to disk every 1s via Rayon.

pub fn list_ids(&self, coll: &str) -> Vec<String>

List all doc IDs in a collection (memory map or disk + WAL merge).

pub fn remove(&self, coll: &str, id: &str) -> Result<(), Error>

Remove the id index entry for a document (tombstone / delete). Disk mode: writes a tombstone to the WAL buffer; flushed to disk on next ticker.

pub fn collections(&self) -> Vec<String>

List all known collections.

Overlays the WAL, exactly as ids() does. A collection whose first write is still sitting in write_buf has no directory on disk yet, so a read_dir-only implementation reports it as absent for up to a full flush tick.

That was a real bug, and a nasty one because it was invisible to a human at a terminal: type a PUT, type a query, and the 1s ticker has already fired in between. Only an automated caller — one that writes and reads in the same millisecond — ever sees the empty list. It surfaced through /cast, which checks the generated collection against this list and returned “collection does not exist” for a collection that had just been written successfully.

Tombstoned entries are excluded, but only when the collection has no surviving documents anywhere — a delete of one document must not hide the whole collection.