This crate defines tools to implement datastructures that can live in main memory or on-disk, meaning that their natural habitat is memory-mapped files, but if that environment is threatened, they might seek refuge in lower-level environments.
One core building block of this library is the notion of virtual
memory pages, which are allocated and freed by an
externally-provided allocator (see how the
sanakirja crate does
this). The particular implementation used here is meant to allow a
transactional system with readers reading the structures
concurrently with one writer at a time.
At the moment, only B trees are implemented, as well as the following general traits:
LoadPageis a trait used to get a pointer to a page. In the most basic version, this may just return a pointer to the file, offset by the requested offset. In more sophisticated versions, this can be used to encrypt and compress pages.
AllocPageallocates and frees pages, because as datastructures need to be persisted on disk, we can’t rely on Rust’s memory management to do it for us. Users of this crate don’t have to worry about this though.
Moreover, two other traits can be used to store things on pages:
Storable is a simple trait that all
Sized + Ord types
without references can readily implement (the
macro does that). For types containing references to pages
allocated in the database, the comparison function can be
customised. Moreover, these types must supply an iterator over
these references, in order for reference-counting to work properly
when the datastructures referencing these types are forked.
Dynamically-sized types, or types that need to be represented in a
dynamically-sized way, can use the
An implementation of B trees. The core operations on B trees
(lookup, iterate, put and del) are generic in the actual
implementation of nodes, via the
BTreeMutPage. This allows for a simpler code for the
high-level functions, as well as specialised, high-performance
implementations for the nodes.
Representation of a mutable or shared page. This is an owned page
Vec in Rust’s std), but we do not know whether we can
mutate it or not.
An owned page on which we can write. This is just a wrapper around
CowPage to avoid checking the “dirty” bit at runtime.
Representation of a borrowed, or immutable page, like a slice in Rust.
There’s a hard-coded assumption that pages have 4K bytes. This is true for normal memory pages on almost all platforms.
Trait for allocating and freeing pages.
Trait for loading a page.
Types that can be stored on disk. This trait may be used in
Sized in order to determine the on-disk size,
UnsizedStorable when special arrangements are needed.
Types that can be stored on disk.