liboxen 0.59.0

Oxen is a fast data version control system, built with machine learning training data in mind. Designed to handle terabytes of data with ease, using a workflow similar to git. Version both structured and unstructured data of any modality: text, images, video, audio, CSV, Parquet, JSONL, model checkpoints, and more. liboxen is the embeddable core library behind the oxen CLI and server, which power fine tuning and inference pipelines for multimodal LLMs, image models, and video models on Oxen.ai.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

//! Reusable, low-level LMDB layer for oxen.
//!
//! Oxen subsystems using LMDB should not use this layer directly. Instead, they should provide a
//! thin wrapper around this layer that provides their own database names, key ordering, and value
//! framing.
//!
//! The pieces:
//!   * [`lmdb_error`] — [`LmdbLayerError`], the `OxenError`-independent leaf error type.
//!   * [`txn`] — the closure-scoped `with_read_txn`/`with_write_txn` brackets (auto
//!     commit-on-`Ok` / abort-on-`Err`), the txn-lifetime rule, and the `WithoutTls`/`MDB_NOTLS`
//!     choice (nestable reads + thread-pool-safe; `Send` read txns are a side effect).
//!   * [`lmdb_env`] — [`LmdbEnv`] (heed's `Env` pinned to `WithoutTls`) plus `open_lmdb_env`,
//!     which opens an env holding one database with a fixed per-store sparse map size (no runtime
//!     resize).
//!   * [`lmdb_db`] — [`LmdbDb`], a named sub-database with raw-bytes keys and opaque-bytes values
//!     copied out as `bytes::Bytes`, plus `open_db` to open one database in its own write txn.
//!   * [`env_registry`] — `LmdbEnvRegistry` (the ONE path-keyed, weak-retention env cache that
//!     enforces "at most one live env per canonical path") plus `open_shared_env`, the
//!     process-global entry point `LmdbStore` opens every env through.
//!   * [`store`] — the `LmdbStore` default-method lifecycle trait: an implementor supplies its
//!     env's location and map size, its database's name, and a handles slot, and inherits an env
//!     opened on first use plus `read`/`write`/`snapshot_to`. Its domain ops stay inherent.

pub mod env_registry;
pub mod lmdb_db;
pub mod lmdb_env;
pub mod lmdb_error;
pub mod store;
pub mod txn;

pub use env_registry::shared_env_is_live;
pub use lmdb_db::LmdbDb;
pub use lmdb_env::LmdbEnv;
pub use lmdb_error::LmdbLayerError;
pub use txn::{with_read_txn, with_write_txn};