Expand description
§PicoVolt (PVDB)
A compact embedded SQL database with page-backed storage and MVCC history.
PicoVolt’s Virtualization Layer Engine (VLE) shifts between a directory of
mutable append-only files (development mode) and an immutable,
memory-mappable single-file binary (.pvdb, production mode).
§Layout
| Layer | Modules | Responsibility |
|---|---|---|
| core | core::types, core::errors, core::value | byte layouts, errors, value model |
| storage | storage::page, storage::cas, storage::compress, storage::record, storage::vle | pages, dedup, compression, serialization, file router |
| engine | engine::mvcc, engine::wasm, engine::query, engine::compliance | snapshots, sandbox, SQL, optional host usage policies |
| surface | Database | dev/prod lifecycle, query, bake |
§Quick start
Low-level layouts round-trip through explicit little-endian encoders:
use picovolt::{RecordEnvelope, RowPageHeader, FileHeader, MAGIC_BYTES};
let env = RecordEnvelope::new(/* tx_inserted */ 1, /* prev_version */ 0);
assert!(env.is_active());
assert_eq!(RecordEnvelope::decode(&env.encode()).unwrap(), env);
let page = RowPageHeader::new(0);
assert_eq!(RowPageHeader::decode(&page.encode()).unwrap(), page);
let file = FileHeader::new(0x1000, 0x2000);
assert_eq!(&file.encode()[0..4], &MAGIC_BYTES);The high-level surface opens a workspace, runs SQL (including BEFORE tx
time-travel), and bakes a production monolith:
use picovolt::{Database, QueryResult};
let mut db = Database::open_dev("./demo.pv")?;
db.query("CREATE TABLE users (id, name)")?;
db.query("INSERT INTO users VALUES (1, 'alice')")?;
if let QueryResult::Rows { rows, .. } = db.query("SELECT * FROM users")? {
assert_eq!(rows.len(), 1);
}
db.bake("./app.pvdb")?; // compile to a single mmap'able file
let mut prod = Database::open_prod("./app.pvdb")?; // read-onlyRe-exports§
pub use journal::BlobChange;pub use journal::ChangeCommit;pub use journal::ChangeSink;pub use journal::CommitLogOptions;pub use journal::CommitLogStatus;pub use journal::PageChange;pub use journal::SnapshotCheckpoint;pub use journal::COMMIT_LOG_DIR;
Modules§
- core
- Core foundations: primitive types, fixed byte layouts, and the error taxonomy.
- encryption
- Native authenticated snapshots and encrypted single-writer vaults. Authenticated encrypted snapshots and native, single-writer encrypted vaults.
- engine
- Engine layer: transaction management and snapshot isolation, the WASM extension runtime, the SQL front-end, and an optional host usage-policy hook.
- journal
- Bounded, checksummed page undo journal and ordered physical change stream.
- migrate
- Best-effort import of a SQL dump (see
Database::import_sql). Best-effort import of a SQL dump (such as the output ofsqlite3 mydb .dump) into PicoVolt. - persistent
- Named persistent retrieval catalog and bounded binary codecs.
Named, optional retrieval indexes. Source rows, not these derived bytes, are
authoritative. The binary representation is specified in
docs/FORMAT.md. - retrieval
- Retrieval over one bounded SELECT snapshot. Mutations are rejected before execution.
- search
- Application-owned full-text indexes over a consistent PicoVolt query result.
- storage
- Storage layer: page structures, content-addressable storage, compression primitives, record serialization, and the Virtualization Layer Engine.
- upgrade
- History-preserving baked-image format migration.
Safe, history-preserving migration of baked
.pvdbimages. - vector
- Exact, bounded vector similarity indexes owned by the application. No embedding model, approximate graph, persistence or automatic SQL indexing.
Structs§
- Buffer
Pool Stats - Buffer-pool counters included in
DatabaseStats. - Cancellation
Token - A clonable, thread-safe cancellation signal.
- CasStats
- Content-addressed blob counters included in
DatabaseStats. - Columnar
Page Header - Header of a cold packed columnar page (
Page_Type = 0x02). - Compaction
Report - Result of one bounded cooperative cold-page compaction pass.
- Compliance
Monitor - The configurable thresholds and the audit entry point.
- Compression
Stats - Compression counters included in
TableStats. - Database
- A PicoVolt database handle.
- Database
Stats - A read-only physical and logical inventory returned by
Database::inspect_stats. - File
Header - The 24-byte header at the start of a baked monolithic
.pvdbfile. - Import
Report - The outcome of
Database::import_sql. - Index
Stats - Secondary-index counters included in
TableStats. - Interpreter
- The
pv-wasmengine. Compiles modules; holds no per-module state. - Migration
Options - Options for
migrate_file. - Migration
Plan - Validated, read-only description of a proposed migration.
- Migration
Report - Successful migration outcome.
- Prepared
Statement - A reusable, validated SQL template with positional
?parameters. - PvModule
- A decoded, ready-to-run WASM module.
- Query
Limits - Resource bounds for executing SQL supplied by an untrusted caller.
- Read
Transaction - An explicit, snapshot-stable read transaction.
- Record
Envelope - Transaction-management envelope wrapping every record or column-pointer entry.
- Request
Options - Per-operation resource and cancellation controls.
- RowPage
Header - Header of a hot slotted row-buffer page (
Page_Type = 0x01). - Runtime
Metrics - Usage metrics supplied by the host application at audit time.
- Shared
Database - A cloneable native handle for safely sharing one PicoVolt database across application threads or tasks.
- Shared
Database Options - Construction options for
SharedDatabase. - Snapshot
- A read view of the database “as of” a particular transaction id.
- Table
Stats - Per-table storage inventory included in
DatabaseStats. - TxManager
- Allocates transaction ids and hands out read snapshots.
- Wasm
Module - A compiled guest module, instantiable on demand.
- Wasm
Runtime - Owns a
wasmiEngine; compiles guest modules. - Write
Transaction - An explicit exclusive write transaction.
Enums§
- Compliance
Error - Errors raised by the optional, application-driven usage-policy hook
(
crate::engine::compliance). - Durability
- Durability policy for flushes.
- Page
Type - The Chameleon layout a page is currently using.
- PvError
- The unified error type for every fallible PicoVolt operation.
- Query
Result - The outcome of
Database::query. - Value
- A single cell value.
Constants§
- BLAK
E3_ HASH_ LEN - Length of a BLAKE3 digest, in bytes.
- CAS_
INLINE_ THRESHOLD - Payloads strictly larger than this many bytes are intercepted and redirected to Content-Addressable Storage, replaced inline by an 8-byte pointer. Payloads of this size or smaller are stored inline.
- CAS_
POINTER_ SIZE - Width of a CAS redirect pointer, in bytes.
- CHUNK_
CAP_ BYTES - Hard cap on a development-mode append-only chunk file: 64 MiB.
- COLD_
CONVERSION_ SECS - Recommended host maintenance interval for calling
Database::compact_step. PicoVolt 1.x remains single-owner and never starts an implicit background thread; applications may use a worker or timer they control. - DEFAULT_
SHARED_ QUEUE_ CAPACITY - Default number of accepted operations that may wait behind the worker.
- FILE_
HEADER_ SIZE - Size of the
.pvdbfile header:magic(4) + format_version(2) + flags(2) + manifest_offset(8) + cas_offset(8). - FORMAT_
VERSION - The newest on-disk
.pvdbformat version this build can read. A file whose version is greater is rejected rather than mis-parsed. - FORMAT_
VERSION_ BASE - The version stamped on a file that carries no binary index region, kept at 1 so such files stay readable by older builds.
- FORMAT_
VERSION_ COLUMNAR - The version required when any table page uses the integrated cold columnar layout.
- FORMAT_
VERSION_ COMMIT_ ANCHOR - Minimum format for a persisted monotonic commit-sequence anchor.
- FORMAT_
VERSION_ COMMIT_ LOG - Minimum workspace format for the production concurrency contract.
- FORMAT_
VERSION_ CONSTRAINTS - The version required by persisted uniqueness/nullability constraints.
- FORMAT_
VERSION_ INDEX - The version required by a binary secondary-index region without constraints.
- FORMAT_
VERSION_ RETRIEVAL - Minimum format for named persistent retrieval catalog objects.
- FORMAT_
VERSION_ SCHEMA - The version required by persisted literal defaults or
CHECKconstraints. - MAGIC_
BYTES - Magic bytes identifying a baked monolithic
.pvdbfile: ASCII"PVDB". - MANIFEST_
FILE - Manifest file name within a development workspace.
- NO_PAGE
- Sentinel page id meaning “no page” (end of an append-only page chain).
- PAGE_
CHECKSUM_ OFFSET - Byte offset of the per-page
u32integrity checksum within a page. The checksum covers the whole page except its own 4 bytes; a stored value of0means the page has not been stamped (a blank or directly-written page) and is accepted without verification. - PAGE_
HEADER_ SIZE - Fixed header size shared by both the row and columnar page layouts. The
trailing 4 bytes (at
PAGE_CHECKSUM_OFFSET) hold the per-page integrity checksum, written and verified by the storage layer rather than by the header codecs, so the page body begins after them. - PAGE_
SIZE - Size of a single physical page, in bytes. All storage I/O happens in units of this size.
- TRANSACTION_
BACKUP_ DIR - Private copy of the last committed workspace used for crash recovery.
- TRANSACTION_
LOCK_ FILE - Lock file used to distinguish a live filesystem transaction from a crash.
- TRANSACTION_
MARKER_ FILE - Recovery marker written before a filesystem transaction may mutate pages.
- TX_NULL
- Reserved transaction id meaning “no transaction”, used by
RecordEnvelope::tx_deletedto mark a record as still active.
Traits§
- Wasm
Exec - A uniform interface over WASM backends, the
wasmiengine and PicoVolt’s owncrate::engine::interpinterpreter, so call sites and differential tests can treat them interchangeably.
Functions§
- is_
record_ visible - Free-function form of
RecordEnvelope::is_visible, matching the spec’sis_record_visible(envelope, target_tx)signature. - migrate_
file - Migrate one baked image to the latest writer format. Publication is same-directory and no-clobber, so a failure leaves both source and any pre-existing destination untouched.
- pack_
addr - Pack a
(page_id, slot)pair into aRecordAddr. - plan_
file_ migration - Validate the full source image and return a dry-run plan without creating the destination or backup.
- pv_bake
- Compile a development workspace at
workspaceinto a monolith atout_path. - pv_
open_ dev - Open or create a development-mode database. See
Database::open_dev. - pv_
open_ prod - Open a production-mode (baked) database. See
Database::open_prod. - unpack_
addr - Unpack a
RecordAddrinto its(page_id, slot)parts.
Type Aliases§
- PageId
- Logical page identifier.
- Phys
Addr - Absolute physical address of a record (byte offset within the data block), used to chain MVCC versions together.
- Record
Addr - A stable physical record address: the page id packed with the slot index
(
page_id << 16 | slot). Records never move once written, so this is durable. - Result
- Convenience alias used throughout the crate.
- Row
- A logical row: an ordered list of cell values, one per column.
- TxId
- Transaction identifier. Monotonically increasing;
TX_NULLis reserved as the “unset” sentinel.