Struct IdIndex 

pub struct IdIndex { /* private fields */ }

Per-document ID index — atomic file-per-doc, sharded across 256 subdirs.

Write path: updates go to write_buf (DashMap, zero I/O, lock-free). Background ticker calls flush_write_buf() every 1s — Rayon-parallel disk writes. Read path: write_buf checked first (latest value), then disk. This eliminates per-PUT fs::rename from the hot path, fixing concurrent write contention.

Implementations

impl IdIndex

pub fn new(db_root: &Path) -> Result<IdIndex, Error>

pub fn in_memory() -> IdIndex

Create a pure in-memory id index — no disk I/O.

pub fn flush_write_buf(&self)

Flush the WAL write buffer to disk in parallel. Called by the background ticker. No-op for in-memory databases. Safe to call concurrently with writes.

pub fn get(&self, coll: &str, id: &str) -> Option<String>

Get the current object hash for a document. Checks WAL write buffer first (most recent), then disk.

pub fn set(&self, coll: &str, id: &str, hash: &str) -> Result<(), Error>

Set the current object hash for a document. Disk mode: writes to WAL buffer only (zero I/O on hot path). Background ticker flushes WAL to disk every 1s via Rayon.

pub fn list_ids(&self, coll: &str) -> Vec<String>

List all doc IDs in a collection (memory map or disk + WAL merge).

pub fn remove(&self, coll: &str, id: &str) -> Result<(), Error>

Remove the id index entry for a document (tombstone / delete). Disk mode: writes a tombstone to the WAL buffer; flushed to disk on next ticker.

pub fn collections(&self) -> Vec<String>

List all known collections.

Overlays the WAL, exactly as ids() does. A collection whose first write is still sitting in write_buf has no directory on disk yet, so a read_dir-only implementation reports it as absent for up to a full flush tick.

That was a real bug, and a nasty one because it was invisible to a human at a terminal: type a PUT, type a query, and the 1s ticker has already fired in between. Only an automated caller — one that writes and reads in the same millisecond — ever sees the empty list. It surfaced through /cast, which checks the generated collection against this list and returned “collection does not exist” for a collection that had just been written successfully.

Tombstoned entries are excluded, but only when the collection has no surviving documents anywhere — a delete of one document must not hide the whole collection.