Skip to main content

Crate picovolt

Crate picovolt 

Source
Expand description

§PicoVolt (PVDB)

A compact embedded SQL database with page-backed storage and MVCC history. PicoVolt’s Virtualization Layer Engine (VLE) shifts between a directory of mutable append-only files (development mode) and an immutable, memory-mappable single-file binary (.pvdb, production mode).

§Layout

LayerModulesResponsibility
corecore::types, core::errors, core::valuebyte layouts, errors, value model
storagestorage::page, storage::cas, storage::compress, storage::record, storage::vlepages, dedup, compression, serialization, file router
engineengine::mvcc, engine::wasm, engine::query, engine::compliancesnapshots, sandbox, SQL, optional host usage policies
surfaceDatabasedev/prod lifecycle, query, bake

§Quick start

Low-level layouts round-trip through explicit little-endian encoders:

use picovolt::{RecordEnvelope, RowPageHeader, FileHeader, MAGIC_BYTES};

let env = RecordEnvelope::new(/* tx_inserted */ 1, /* prev_version */ 0);
assert!(env.is_active());
assert_eq!(RecordEnvelope::decode(&env.encode()).unwrap(), env);

let page = RowPageHeader::new(0);
assert_eq!(RowPageHeader::decode(&page.encode()).unwrap(), page);

let file = FileHeader::new(0x1000, 0x2000);
assert_eq!(&file.encode()[0..4], &MAGIC_BYTES);

The high-level surface opens a workspace, runs SQL (including BEFORE tx time-travel), and bakes a production monolith:

use picovolt::{Database, QueryResult};

let mut db = Database::open_dev("./demo.pv")?;
db.query("CREATE TABLE users (id, name)")?;
db.query("INSERT INTO users VALUES (1, 'alice')")?;

if let QueryResult::Rows { rows, .. } = db.query("SELECT * FROM users")? {
    assert_eq!(rows.len(), 1);
}

db.bake("./app.pvdb")?;                  // compile to a single mmap'able file
let mut prod = Database::open_prod("./app.pvdb")?; // read-only

Re-exports§

pub use journal::BlobChange;
pub use journal::ChangeCommit;
pub use journal::ChangeSink;
pub use journal::CommitLogOptions;
pub use journal::CommitLogStatus;
pub use journal::PageChange;
pub use journal::SnapshotCheckpoint;
pub use journal::COMMIT_LOG_DIR;

Modules§

core
Core foundations: primitive types, fixed byte layouts, and the error taxonomy.
encryption
Native authenticated snapshots and encrypted single-writer vaults. Authenticated encrypted snapshots and native, single-writer encrypted vaults.
engine
Engine layer: transaction management and snapshot isolation, the WASM extension runtime, the SQL front-end, and an optional host usage-policy hook.
journal
Bounded, checksummed page undo journal and ordered physical change stream.
migrate
Best-effort import of a SQL dump (see Database::import_sql). Best-effort import of a SQL dump (such as the output of sqlite3 mydb .dump) into PicoVolt.
persistent
Named persistent retrieval catalog and bounded binary codecs. Named, optional retrieval indexes. Source rows, not these derived bytes, are authoritative. The binary representation is specified in docs/FORMAT.md.
retrieval
Retrieval over one bounded SELECT snapshot. Mutations are rejected before execution.
search
Application-owned full-text indexes over a consistent PicoVolt query result.
storage
Storage layer: page structures, content-addressable storage, compression primitives, record serialization, and the Virtualization Layer Engine.
upgrade
History-preserving baked-image format migration. Safe, history-preserving migration of baked .pvdb images.
vector
Exact, bounded vector similarity indexes owned by the application. No embedding model, approximate graph, persistence or automatic SQL indexing.

Structs§

BufferPoolStats
Buffer-pool counters included in DatabaseStats.
CancellationToken
A clonable, thread-safe cancellation signal.
CasStats
Content-addressed blob counters included in DatabaseStats.
ColumnarPageHeader
Header of a cold packed columnar page (Page_Type = 0x02).
CompactionReport
Result of one bounded cooperative cold-page compaction pass.
ComplianceMonitor
The configurable thresholds and the audit entry point.
CompressionStats
Compression counters included in TableStats.
Database
A PicoVolt database handle.
DatabaseStats
A read-only physical and logical inventory returned by Database::inspect_stats.
FileHeader
The 24-byte header at the start of a baked monolithic .pvdb file.
ImportReport
The outcome of Database::import_sql.
IndexStats
Secondary-index counters included in TableStats.
Interpreter
The pv-wasm engine. Compiles modules; holds no per-module state.
MigrationOptions
Options for migrate_file.
MigrationPlan
Validated, read-only description of a proposed migration.
MigrationReport
Successful migration outcome.
PreparedStatement
A reusable, validated SQL template with positional ? parameters.
PvModule
A decoded, ready-to-run WASM module.
QueryLimits
Resource bounds for executing SQL supplied by an untrusted caller.
ReadTransaction
An explicit, snapshot-stable read transaction.
RecordEnvelope
Transaction-management envelope wrapping every record or column-pointer entry.
RequestOptions
Per-operation resource and cancellation controls.
RowPageHeader
Header of a hot slotted row-buffer page (Page_Type = 0x01).
RuntimeMetrics
Usage metrics supplied by the host application at audit time.
SharedDatabase
A cloneable native handle for safely sharing one PicoVolt database across application threads or tasks.
SharedDatabaseOptions
Construction options for SharedDatabase.
Snapshot
A read view of the database “as of” a particular transaction id.
TableStats
Per-table storage inventory included in DatabaseStats.
TxManager
Allocates transaction ids and hands out read snapshots.
WasmModule
A compiled guest module, instantiable on demand.
WasmRuntime
Owns a wasmi Engine; compiles guest modules.
WriteTransaction
An explicit exclusive write transaction.

Enums§

ComplianceError
Errors raised by the optional, application-driven usage-policy hook (crate::engine::compliance).
Durability
Durability policy for flushes.
PageType
The Chameleon layout a page is currently using.
PvError
The unified error type for every fallible PicoVolt operation.
QueryResult
The outcome of Database::query.
Value
A single cell value.

Constants§

BLAKE3_HASH_LEN
Length of a BLAKE3 digest, in bytes.
CAS_INLINE_THRESHOLD
Payloads strictly larger than this many bytes are intercepted and redirected to Content-Addressable Storage, replaced inline by an 8-byte pointer. Payloads of this size or smaller are stored inline.
CAS_POINTER_SIZE
Width of a CAS redirect pointer, in bytes.
CHUNK_CAP_BYTES
Hard cap on a development-mode append-only chunk file: 64 MiB.
COLD_CONVERSION_SECS
Recommended host maintenance interval for calling Database::compact_step. PicoVolt 1.x remains single-owner and never starts an implicit background thread; applications may use a worker or timer they control.
DEFAULT_SHARED_QUEUE_CAPACITY
Default number of accepted operations that may wait behind the worker.
FILE_HEADER_SIZE
Size of the .pvdb file header: magic(4) + format_version(2) + flags(2) + manifest_offset(8) + cas_offset(8).
FORMAT_VERSION
The newest on-disk .pvdb format version this build can read. A file whose version is greater is rejected rather than mis-parsed.
FORMAT_VERSION_BASE
The version stamped on a file that carries no binary index region, kept at 1 so such files stay readable by older builds.
FORMAT_VERSION_COLUMNAR
The version required when any table page uses the integrated cold columnar layout.
FORMAT_VERSION_COMMIT_ANCHOR
Minimum format for a persisted monotonic commit-sequence anchor.
FORMAT_VERSION_COMMIT_LOG
Minimum workspace format for the production concurrency contract.
FORMAT_VERSION_CONSTRAINTS
The version required by persisted uniqueness/nullability constraints.
FORMAT_VERSION_INDEX
The version required by a binary secondary-index region without constraints.
FORMAT_VERSION_RETRIEVAL
Minimum format for named persistent retrieval catalog objects.
FORMAT_VERSION_SCHEMA
The version required by persisted literal defaults or CHECK constraints.
MAGIC_BYTES
Magic bytes identifying a baked monolithic .pvdb file: ASCII "PVDB".
MANIFEST_FILE
Manifest file name within a development workspace.
NO_PAGE
Sentinel page id meaning “no page” (end of an append-only page chain).
PAGE_CHECKSUM_OFFSET
Byte offset of the per-page u32 integrity checksum within a page. The checksum covers the whole page except its own 4 bytes; a stored value of 0 means the page has not been stamped (a blank or directly-written page) and is accepted without verification.
PAGE_HEADER_SIZE
Fixed header size shared by both the row and columnar page layouts. The trailing 4 bytes (at PAGE_CHECKSUM_OFFSET) hold the per-page integrity checksum, written and verified by the storage layer rather than by the header codecs, so the page body begins after them.
PAGE_SIZE
Size of a single physical page, in bytes. All storage I/O happens in units of this size.
TRANSACTION_BACKUP_DIR
Private copy of the last committed workspace used for crash recovery.
TRANSACTION_LOCK_FILE
Lock file used to distinguish a live filesystem transaction from a crash.
TRANSACTION_MARKER_FILE
Recovery marker written before a filesystem transaction may mutate pages.
TX_NULL
Reserved transaction id meaning “no transaction”, used by RecordEnvelope::tx_deleted to mark a record as still active.

Traits§

WasmExec
A uniform interface over WASM backends, the wasmi engine and PicoVolt’s own crate::engine::interp interpreter, so call sites and differential tests can treat them interchangeably.

Functions§

is_record_visible
Free-function form of RecordEnvelope::is_visible, matching the spec’s is_record_visible(envelope, target_tx) signature.
migrate_file
Migrate one baked image to the latest writer format. Publication is same-directory and no-clobber, so a failure leaves both source and any pre-existing destination untouched.
pack_addr
Pack a (page_id, slot) pair into a RecordAddr.
plan_file_migration
Validate the full source image and return a dry-run plan without creating the destination or backup.
pv_bake
Compile a development workspace at workspace into a monolith at out_path.
pv_open_dev
Open or create a development-mode database. See Database::open_dev.
pv_open_prod
Open a production-mode (baked) database. See Database::open_prod.
unpack_addr
Unpack a RecordAddr into its (page_id, slot) parts.

Type Aliases§

PageId
Logical page identifier.
PhysAddr
Absolute physical address of a record (byte offset within the data block), used to chain MVCC versions together.
RecordAddr
A stable physical record address: the page id packed with the slot index (page_id << 16 | slot). Records never move once written, so this is durable.
Result
Convenience alias used throughout the crate.
Row
A logical row: an ordered list of cell values, one per column.
TxId
Transaction identifier. Monotonically increasing; TX_NULL is reserved as the “unset” sentinel.