# vsdb API Examples
This document provides examples for selected public APIs in the `vsdb` crate.
## Durability fences
`Mapx`, `MapxOrd`, and `MapxOrdRawKey` batches also expose `commit_sync()`.
This opt-in operation synchronizes the shard WAL before atomic publication;
ordinary `commit()` keeps its existing asynchronous default. An empty batch is
not a fence for earlier writes. No cross-collection transaction is implied.
An I/O failure may have left recoverable WAL data, so avoid automatic retries.
The same distinction applies to SlotDex and VecDex mutations. Preflight
rejection leaves that batch unapplied, and runtime caches update only after
successful commits. A persistence error can still leave a complete WAL batch
that recovery replays; an error does not establish that recovered disk state
is unchanged. Related rows recover together as one atomic batch.
`Mapx`, `MapxOrd`, `MapxOrdRawKey`, and `Orphan` expose `try_sync_wal()`:
```rust
use vsdb::MapxOrd;
# fn main() -> vsdb::Result<()> {
let mut state = MapxOrd::<u64, String>::new();
state.insert(&1, &"accepted".to_owned());
state.try_sync_wal()?;
# Ok(())
# }
```
The fence makes prior successful writes on the collection's shard durable
without forcing a memtable flush; it returns synchronization errors. Other
shards need their own fences. Use `collection.namespace().try_sync_wal()?` to
cover every shard of that namespace, including composite indexes such as
`SlotDex`. Shards synchronize sequentially, so an error can leave earlier
shards synchronized. These calls do not provide cross-collection transactions.
They are no-ops in read-only mode. The collection-level `sync_wal()` form
panics on errors; ordinary inserts and batch commits do not fsync individually.
## Basic collection read views
Immutable `iter` and `range` on `Mapx`, `MapxOrd` and `MapxOrdRawKey` retain a
committed view captured at iterator creation. A read-only use of a `shadow`
handle may therefore keep a range iterator while the sole writer updates the
map. Later writes, clears and flushes do not alter that iterator's results.
Separate point reads or iterators can observe different views; this does not
give a shared snapshot across collections. Mutable iteration still requires
coordination with every other writer to its keys. Composite indexes retain
their single-active-handle contract described below.
## Namespaces
Persistent collections support namespaces — independently-rooted engine instances
for physical placement and whole-directory removal without per-key traversal.
In-memory `MptCalc` / `SmtCalc` need no namespace; `VerMapWithProof` inherits
placement from its wrapped `VerMap`. Everyday constructors stay parameterless.
```rust,no_run
use vsdb::{Namespace, NamespaceOpts, basic::mapx::Mapx, basic::mapx_ord::MapxOrd};
// ---- Everyday tier: zero parameters, no names, no paths ----
let cold = Namespace::create().unwrap();
// Ambient placement: ordinary new() calls inside scope() use `cold`.
// Explicit new_in() and DagMap children use their explicit/parent namespace.
// Same namespace placement (not necessarily the same shard).
let index = Mapx::<u64, u64>::new_in(&archive.namespace());
assert_eq!(archive.namespace().id(), index.namespace().id());
// ---- Persist and recover via InstanceId ----
let id = archive.save_meta().unwrap();
// id.to_string() => "42@1" (map 42 in namespace 1)
let restored: Mapx<u64, String> = Mapx::from_meta(id).unwrap();
// Bare u64 from pre-namespace code still works for default-namespace maps.
let legacy: Mapx<u64, String> = Mapx::new();
let old_id = legacy.save_meta().unwrap().map_id;
let from_old: Mapx<u64, String> = Mapx::from_meta(old_id).unwrap();
// ---- Advanced tier (opt-in): explicit volume, shards, budget ----
let fast = Namespace::create_with(NamespaceOpts {
path: Some("/mnt/nvme/vsdb".into()),
shards: 8,
mem_budget_mb: Some(2048),
}).unwrap();
let hot: MapxOrd<i32, String> = MapxOrd::new_in(&fast);
// ---- Cross-namespace copy: `clone_in` (the `Clone` counterpart) ----
// Deep-copies into a brand-new instance placed in the target namespace
// (chunked; never whole-map in memory). Available on MapxRaw, Mapx,
// MapxOrd, MapxOrdRawKey, and Orphan.
let archive_copy = archive.clone_in(&Namespace::default_ns()).unwrap();
// Admin functions (from vsdb_core, re-exported by vsdb):
// Namespace::list() -> Result<Vec<NsInfo>>
// Namespace::close_by_id(id) -> Result<()>
// ns.close() -> consuming form; refusal returns the handle
// Namespace::destroy(id) -> Result<()>
// Namespace::relocate(id, new_path) -> Result<()>
```
Typed handles restored by serde or `from_meta` check a tag of the fully
qualified Rust type name, including generic parameters. Unrelated types with
the same short name are rejected. Moving or renaming a persisted type requires
an explicit migration; a type alias referring to the original type is fine.
This check is not a schema fingerprint: changing fields under the same type
name still requires application schema versioning and conversion. v16
`VSTYPE02` handles remain readable; new handles use `VSTYPE03`.
Key rules:
- `Mapx::new()` uses the current creation scope, or the default namespace outside one.
- Two `new_in(&ns)` calls can route to different shards; namespace placement alone does not provide a shared WAL.
- A composite structure (`VerMap`, `SlotDex`, …) always lives wholly inside one namespace.
- Cross-namespace atomic transactions do not exist (separate WALs).
- Reads, writes, and deserialization always route via the handle's own namespace —
ambient scope affects creation only.
- Memory sizing defaults to 2 GiB for the default namespace and 512 MiB for
each non-default namespace, without host-RAM/cgroup detection.
A nonzero `VsdbOptions::with_mem_budget_mb` (then `VSDB_MEM_BUDGET_MB`) configures the
default engine; `NamespaceOpts::mem_budget_mb` configures others. Units are
binary MiB. These inputs are not hard RSS limits: floors, caps, pinned
blocks, and runtime metadata affect actual memory use. See the
[memory design](../../docs/proposals/shared-mem-pool.md).
## Read-only mode
Configure read-only access once at process startup, then restore handles saved
by an earlier writer:
```rust,no_run
use vsdb::{
InstanceId, Mapx, OpenMode, VerMap, VsdbError, VsdbOptions,
vsdb_configure, vsdb_open_mode,
};
# fn main() -> vsdb::Result<()> {
vsdb_configure(VsdbOptions::read_only("/srv/my-app/vsdb"))?;
assert_eq!(vsdb_open_mode(), OpenMode::ReadOnly);
// Tokens saved by the writer (`save_meta`); illustrative values.
let map_id: InstanceId = "40960000".parse()?;
let map = Mapx::<String, String>::from_meta(map_id)?;
assert!(map.namespace().is_read_only());
let _ = map.get(&"key".to_owned());
let versioned_id: InstanceId = "40960001".parse()?;
let mut history = VerMap::<u64, String>::from_meta(versioned_id)?;
let main = history.branch_id("main").unwrap();
let _ = history.get(main, &7)?;
assert!(matches!(
history.insert(main, &8, &"blocked".to_owned()),
Err(VsdbError::ReadOnly { .. })
));
# Ok(())
# }
```
The setting covers every namespace opened by the process. Query APIs and
in-memory WAL recovery are supported without filesystem changes. Fallible
write APIs return `VsdbError::ReadOnly`, infallible collection mutations panic,
maintenance-only flush/GC writes are skipped, and explicit
trie-cache saves return the capability error. See the
[read-only mode guide](read-only.md) for the full operation matrix,
locking and snapshot rules, and recovery constraints.
## Mapx
`Mapx` is a hash map-like data structure that stores key-value pairs.
```rust
use vsdb::Mapx;
let mut map: Mapx<String, String> = Mapx::new();
// Insert some key-value pairs
map.insert(&"key1".to_string(), &"value1".to_string());
map.insert(&"key2".to_string(), &"value2".to_string());
// Get a value
assert_eq!(map.get("key1"), Some("value1".to_string()));
// Check if a key exists
assert!(map.contains_key("key2"));
// Iterate over the key-value pairs
for (key, value) in map.iter() {
println!("{}: {}", key, value);
}
// Remove a key-value pair
map.remove("key1");
```
Borrowed lookups use the stored key type's encoding. `String` accepts `str`;
byte keys (`Vec<u8>`, `Box<[u8]>`, `[u8; N]`) accept byte slices for `get`,
`get_mut`, `contains_key`, and `remove`. Array lookups validate the full slice,
including its length:
```rust
use vsdb::Mapx;
let mut map = Mapx::<[u8; 2], u64>::new();
map.insert(&[1, 0], &7);
assert_eq!(map.get(&[1, 0][..]), Some(7));
assert_eq!(map.get(&[0][..]), None);
```
Mapx normalizes byte-slice queries through the owning key codec because
postcard encodes arrays differently from variable-length byte sequences.
Other custom `KeyRef` forms must encode identically to their owning type.
Ordered maps instead use `OrderedKeyRef` / `KeyEnDeOrdered` encodings.
## MapxOrd
`MapxOrd` is a B-tree map-like data structure that stores key-value pairs in a sorted order.
```rust
use vsdb::MapxOrd;
let mut map: MapxOrd<i32, String> = MapxOrd::new();
// Insert some key-value pairs
map.insert(&3, &"three".to_string());
map.insert(&1, &"one".to_string());
map.insert(&2, &"two".to_string());
// Get a value
assert_eq!(map.get(&1), Some("one".to_string()));
// Iterate over the key-value pairs in sorted order
for (key, value) in map.iter() {
println!("{}: {}", key, value);
}
// Get the first and last key-value pairs
assert_eq!(map.first(), Some((1, "one".to_string())));
assert_eq!(map.last(), Some((3, "three".to_string())));
```
## VerMap
`VerMap` provides Git-style versioned storage with branching, commits, merge, and rollback (`rollback_to`).
For a detailed architecture guide with diagrams, see [Versioned Module — Architecture & Internals](versioned.md).
```rust
use vsdb::versioned::map::VerMap;
use vsdb::versioned::BranchId;
// 1. Create an empty versioned map (starts with a "main" branch).
let mut m: VerMap<u32, String> = VerMap::new();
let main = m.main_branch();
// 2. Write on the main branch and commit a snapshot.
m.insert(main, &1, &"hello".into()).unwrap();
m.commit(main).unwrap();
// 3. Fork a feature branch — cheap, no data copied.
let feat: BranchId = m.create_branch("feature", main).unwrap();
m.insert(feat, &1, &"updated".into()).unwrap();
m.commit(feat).unwrap();
// 4. Branches are isolated.
assert_eq!(m.get(main, &1).unwrap(), Some("hello".into()));
assert_eq!(m.get(feat, &1).unwrap(), Some("updated".into()));
// 5. Three-way merge: feature → main (source wins on conflict).
m.merge(feat, main).unwrap();
assert_eq!(m.get(main, &1).unwrap(), Some("updated".into()));
// 6. Clean up: deleting the branch reclaims unreachable commits and
// B+ tree nodes automatically — no manual gc() call required
// (gc() also supports an explicit full node sweep).
// Physical disk reclamation waits for compaction and is best-effort.
m.delete_branch(feat).unwrap();
```
## MptCalc / SmtCalc (Merkle Trie)
`MptCalc` and `SmtCalc` are stateless, in-memory Merkle trie implementations:
they hold an ephemeral trie for root computation while all persistence and
versioning is handled by an external store (e.g. `VerMap`).
```rust
use vsdb::trie::MptCalc;
// Build a trie
let mut mpt = MptCalc::new();
mpt.insert(b"key1", b"value1").unwrap();
mpt.insert(b"key2", b"value2").unwrap();
// MPT keys must be at most 1024 bytes (MAX_MPT_KEY_LEN); insert,
// batch_update, and from_entries return an error for longer keys.
// SMT keys are hashed to a fixed 256-bit path and have no length limit.
// Compute the 32-byte Merkle root hash
let root = mpt.root_hash().unwrap();
assert_eq!(root.len(), 32);
// Lookup
assert_eq!(mpt.get(b"key1").unwrap(), Some(b"value1".to_vec()));
// Remove
mpt.remove(b"key1").unwrap();
assert_eq!(mpt.get(b"key1").unwrap(), None);
// Batch update
mpt.batch_update(&[
(b"k1".as_ref(), Some(b"v1".as_ref())),
(b"k2".as_ref(), None), // remove
]).unwrap();
// Disposable cache (low-level API, used internally by VerMapWithProof):
let cache_root = std::env::temp_dir().join(format!("vsdb-trie-example-{}", std::process::id()));
let cache_dir = cache_root.as_path();
std::fs::create_dir_all(cache_dir).unwrap();
let cache_id = 1;
let sync_tag = 0;
mpt.save_cache(cache_dir, cache_id, sync_tag).unwrap();
let (mut loaded, tag, hash) = MptCalc::load_cache(cache_dir, cache_id).unwrap();
assert_eq!(tag, sync_tag);
assert_eq!(loaded.root_hash().unwrap(), hash);
// VerMapWithProof loads a cache automatically; saving requires save_cache(commit).
std::fs::remove_dir_all(cache_dir).unwrap();
```
### SmtCalc with Proofs
```rust
use vsdb::trie::SmtCalc;
let mut smt = SmtCalc::new();
smt.insert(b"alice", b"100").unwrap();
smt.insert(b"bob", b"200").unwrap();
let root = smt.root_hash().unwrap();
let root32: [u8; 32] = root.try_into().unwrap();
// Membership proof
let proof = smt.prove(b"alice").unwrap();
assert_eq!(proof.value(), Some(b"100".as_ref()));
assert!(SmtCalc::verify_proof(&root32, b"alice", &proof).unwrap());
// Non-membership proof
let proof = smt.prove(b"charlie").unwrap();
assert_eq!(proof.value(), None);
assert!(SmtCalc::verify_proof(&root32, b"charlie", &proof).unwrap());
```
### Proof transport
Both `MptProof` and `SmtProof` support serde and a versioned postcard transport:
```rust
use vsdb::{MptCalc, MptProof};
let mut trie = MptCalc::new();
trie.insert(b"account", b"10").unwrap();
let root: [u8; 32] = trie.root_hash().unwrap().try_into().unwrap();
let wire = trie.prove(b"account").unwrap().to_bytes().unwrap();
let received = MptProof::from_bytes(&wire).unwrap();
assert!(MptCalc::verify_proof(&root, b"account", &received).unwrap());
```
`to_bytes` is equivalent to `postcard::to_allocvec`; `from_bytes` also rejects
trailing bytes. Version-1 field order is `(format, key, value, nodes)` for MPT
and `(format, key_hash, leaf, siblings)` for SMT. The eight-byte format tags are
`VSMPTP01` and `VSSMTP01`; other kinds or versions are rejected. A successfully
decoded proof still needs verification against the expected root and key.
Trie cache files and application-defined older proof encodings are separate
formats and are not accepted by these decoders.
## VerMapWithProof
Integrates `VerMap` with MPT or SMT for versioned Merkle root computation.
Use `prove_at(branch, &key)` to synchronize and return a `ProofWithRoot` in one
call, including uncommitted changes. `prove_at_commit(commit, &key)` selects
committed history instead and returns an error for reclaimed commits. Both
backends use the same method names and ordered key codec.
```rust
use vsdb::{SmtCalc, VerMapWithProof};
let mut state = VerMapWithProof::<u64, String, SmtCalc>::new();
let main = state.map().main_branch();
state.map_mut().insert(main, &7, &"value".to_owned()).unwrap();
let result = state.prove_at(main, &7).unwrap();
assert!(VerMapWithProof::<u64, String, SmtCalc>::verify_key_proof(
&result.root_hash, &7, &result.proof,
).unwrap());
```
Earlier root queries and `save_cache` calls do not determine these methods'
proof context. The existing `prove`, `prove_key`, and `prove_mpt` methods still
use the last synchronized trie; they remain useful when proving several keys
against a root the caller has already selected. `ProofWithRoot` supports serde;
its embedded proof uses the versioned representation above.
```rust
use vsdb::trie::{MptCalc, VerMapWithProof};
let mut vmp: VerMapWithProof<Vec<u8>, Vec<u8>, MptCalc> = VerMapWithProof::new();
let main = vmp.map().main_branch();
// Write data and commit
vmp.map_mut().insert(main, &b"key1".to_vec(), &b"val1".to_vec()).unwrap();
let commit = vmp.map_mut().commit(main).unwrap();
// Compute the Merkle root (incrementally maintained)
let root = vmp.merkle_root(main).unwrap();
assert_eq!(root.len(), 32);
// Optional application checkpoint: serializes the entire trie, so do this
// periodically rather than on every block. Errors are returned to the caller.
vmp.save_cache(commit).unwrap();
// Construction automatically loads an available cache; missing/stale caches
// are rebuilt/caught up. Root calculation and Drop never save cache files.
```
`save_cache(commit)` synchronizes proofs to the selected committed state,
excluding dirty changes. Call `merkle_root(branch)` again before proving the
branch working state. Historical root queries return `CommitNotFound` for
reclaimed commits, even if a matching trie cache remains.
## Slotdex
`SlotDex` (in the `slotdex` module) is a skip-list-like index for efficient, timestamp-based paged queries.
```rust
use vsdb::{SlotDex64, slotdex::Order}; // SlotDex<u64, K> alias — slot type is u64
let mut db = SlotDex64::<String>::new(10u64, false).unwrap(); // tier_capacity must be >= 2
// Insert entries into slots (e.g., timestamps)
db.insert(100, "entry_a".to_string()).unwrap();
db.insert(100, "entry_b".to_string()).unwrap();
db.insert(200, "entry_c".to_string()).unwrap();
db.insert(300, "entry_d".to_string()).unwrap();
assert_eq!(db.total(), 4);
// Paged queries
let page = db.page(.., 2, 0, Order::Desc); // any slot range, page_size=2, page_index=0
assert_eq!(page, vec!["entry_d".to_string(), "entry_c".to_string()]);
// Slot-range queries
let entries = db.page(100..=200, 10, 0, Order::Asc);
assert_eq!(entries.len(), 3);
assert_eq!(db.count(100..300), 3);
// Remove
db.remove(100, &"entry_a".to_string()).unwrap();
assert_eq!(db.total(), 3);
// Bulk insertion (amortizes engine writes; ideal for imports/rebuilds)
db.insert_batch([(400u64, "entry_e".to_string()), (400, "entry_f".to_string())])
.unwrap();
assert_eq!(db.total(), 5);
```
`Order` controls slot order; keys within a slot retain ascending key order.
For a complete range scan or aggregation, use `iter(slots, order)`:
```rust
use vsdb::{SlotDex64, slotdex::Order};
let mut index = SlotDex64::<u64>::new(8, false).unwrap();
let first_ten: Vec<_> = index.iter(.., Order::Desc).take(10).collect();
assert_eq!(first_ten, (0..10).collect::<Vec<_>>());
```
The iterator decodes keys on demand and has no `u16` page-size ceiling. It keeps
ascending keys within each slot in both slot orders. Keep the index's single
active handle for the entire iteration; restored aliases must not mutate it.
Use `page` when seeking directly to an offset, and `count` for a row count that
does not need to decode the keys.
SlotDex stores counts and rows together in one atomic batch per mutation,
including the entire `insert_batch` call. Split large imports across calls
to bound temporary memory; earlier calls remain committed if a later call
fails. Restoring through serde or `from_meta` creates independent runtime
caches: retire the old active handle before mutation and
route reads/writes through one shared instance. Sequential writes through
separate restored handles can also invalidate cached state.
## VecDex
See the [VecDex guide](vecdex.md) for runnable examples, runtime metric
selection, filtered search, and recovery/ownership rules.