Struct Builder

pub struct Builder<M, S = Crc32>
where
    M: BaseTable,
{ /* private fields */ }

A write-ahead log builder.

Implementations

impl<M, S> Builder<M, S>

where M: BaseTable,

pub const fn with_lock_meta(self, lock_meta: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Set if lock the meta of the WAL in the memory to prevent OS from swapping out the header of WAL. When using memory map backed WAL, the meta of the WAL is in the header, meta is frequently accessed, lock (mlock on the header) the meta can reduce the page fault, but yes, this means that one WAL will have one page are locked in memory, and will not be swapped out. So, this is a trade-off between performance and memory usage.

Default is true.

This configuration has no effect on windows and vec backed WAL.

Example

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_lock_meta(false);

pub fn with_read(self, read: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for read access.

This option, when true, will indicate that the file should be read-able if opened.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_read(true);

pub fn with_write(self, write: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for write access.

This option, when true, will indicate that the file should be write-able if opened.

If the file already exists, any write calls on it will overwrite its contents, without truncating it.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_write(true);

pub fn with_append(self, append: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for the append mode.

This option, when true, means that writes will append to a file instead of overwriting previous contents. Note that setting .write(true).append(true) has the same effect as setting only .append(true).

For most filesystems, the operating system guarantees that all writes are atomic: no writes get mangled because another process writes at the same time.

One maybe obvious note when using append-mode: make sure that all data that belongs together is written to the file in one operation. This can be done by concatenating strings before passing them to write(), or using a buffered writer (with a buffer of adequate size), and calling flush() when the message is complete.

If a file is opened with both read and append access, beware that after opening, and after every write, the position for reading may be set at the end of the file. So, before writing, save the current position (using seek(SeekFrom::Current(opts))), and restore it before the next read.

Note

This function doesn’t create the file if it doesn’t exist. Use the Options::with_create method to do so.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_append(true);

pub fn with_truncate(self, truncate: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for truncating a previous file.

If a file is successfully opened with this option set it will truncate the file to opts length if it already exists.

The file must be opened with write access for truncate to work.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_write(true).with_truncate(true);

pub fn with_create(self, val: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option to create a new file, or open it if it already exists. If the file does not exist, it is created and set the lenght of the file to the given size.

In order for the file to be created, Options::with_write or Options::with_append access must be used.

See also std::fs::write() for a simple function to create a file with some given data.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_write(true).with_create(true);

pub fn with_create_new(self, val: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option to create a new file and set the file length to the given value, failing if it already exists.

No file is allowed to exist at the target location, also no (dangling) symlink. In this way, if the call succeeds, the file returned is guaranteed to be new.

This option is useful because it is atomic. Otherwise between checking whether a file exists and creating a new one, the file may have been created by another process (a TOCTOU race condition / attack).

If .with_create_new(true) is set, .with_create() and .with_truncate() are ignored.

The file must be opened with write or append access in order to create a new file.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new()
  .with_write(true)
  .with_create_new(true);

pub fn with_stack(self, stack: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Configures the anonymous memory map to be suitable for a process or thread stack.

This option corresponds to the MAP_STACK flag on Linux. It has no effect on Windows.

This option has no effect on file-backed memory maps and vec backed Wal.

Example

use orderwal::{Builder, multiple_version::LinkedTable};

let stack = Builder::<LinkedTable<[u8], [u8]>>::new().with_stack(true);

pub fn with_huge(self, page_bits: Option<u8>) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Configures the anonymous memory map to be allocated using huge pages.

This option corresponds to the MAP_HUGETLB flag on Linux. It has no effect on Windows.

The size of the requested page can be specified in page bits. If not provided, the system default is requested. The requested length should be a multiple of this, or the mapping will fail.

This option has no effect on file-backed memory maps and vec backed Wal.

Example

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_huge(Some(8));

pub fn with_populate(self, populate: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Populate (prefault) page tables for a mapping.

For a file mapping, this causes read-ahead on the file. This will help to reduce blocking on page faults later.

This option corresponds to the MAP_POPULATE flag on Linux. It has no effect on Windows.

This option has no effect on vec backed Wal.

Example

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_populate(true);

impl<M, S> Builder<M, S>

where M: BaseTable,

pub const fn lock_meta(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Get if lock the meta of the WAL in the memory to prevent OS from swapping out the header of WAL. When using memory map backed WAL, the meta of the WAL is in the header, meta is frequently accessed, lock (mlock on the header) the meta can reduce the page fault, but yes, this means that one WAL will have one page are locked in memory, and will not be swapped out. So, this is a trade-off between performance and memory usage.

Example

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_lock_meta(false);
assert_eq!(opts.lock_meta(), false);

pub const fn read(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with read access.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_read(true);
assert_eq!(opts.read(), true);

pub const fn write(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with write access.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_write(true);
assert_eq!(opts.write(), true);

pub const fn append(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with append access.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_append(true);
assert_eq!(opts.append(), true);

pub const fn truncate(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with truncate access.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_truncate(true);
assert_eq!(opts.truncate(), true);

pub const fn create(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be created if it does not exist.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_create(true);
assert_eq!(opts.create(), true);

pub const fn create_new(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be created if it does not exist and fail if it does.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_create_new(true);
assert_eq!(opts.create_new(), true);

pub const fn stack(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the memory map should be suitable for a process or thread stack.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_stack(true);
assert_eq!(opts.stack(), true);

pub const fn huge(&self) -> Option<u8>

Available on crate feature memmap and non-target_family="wasm" only.

Returns the page bits of the memory map.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_huge(Some(8));
assert_eq!(opts.huge(), Some(8));

pub const fn populate(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the memory map should populate (prefault) page tables for a mapping.

Examples

use orderwal::{Builder, multiple_version::LinkedTable};

let opts = Builder::<LinkedTable<[u8], [u8]>>::new().with_populate(true);
assert_eq!(opts.populate(), true);

impl<M, S> Builder<M, S>

where M: BaseTable,

pub fn map_anon<W>(self) -> Result<W, Error<W::Memtable>>

where W: Constructable<Memtable = M, Checksumer = S>,

Available on crate feature memmap and non-target_family="wasm" only.

Creates a new in-memory write-ahead log but backed by an anonymous mmap.

Example

use orderwal::{base::OrderWal, Builder};

let wal = Builder::new()
  .with_capacity(1024)
  .map_anon::<OrderWal<[u8], [u8]>>()
  .unwrap();

pub unsafe fn map<'a, W, P>(self, path: P) -> Result<W, Error<W::Memtable>>

where S: BuildChecksumer, P: AsRef<Path>, W: Constructable<Memtable = M, Checksumer = S> + Immutable, M::Key: Type + Ord + 'static, <M::Key as Type>::Ref<'a>: KeyRef<'a, M::Key>,

Available on crate feature memmap and non-target_family="wasm" only.

Opens a write-ahead log backed by a file backed memory map in read-only mode.

Safety

All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.

Example

use orderwal::{base::OrderWalReader, Builder};



let wal = unsafe {
  Builder::new()
    .map::<OrderWalReader<[u8], [u8]>, _>(&path)
    .unwrap()
};

pub unsafe fn map_with_path_builder<'a, W, PB, E>( self, path_builder: PB, ) -> Result<W, Either<E, Error<W::Memtable>>>

where PB: FnOnce() -> Result<PathBuf, E>, S: BuildChecksumer, W: Constructable<Memtable = M, Checksumer = S> + Immutable, M::Key: Type + Ord + 'static, <M::Key as Type>::Ref<'a>: KeyRef<'a, M::Key>,

Available on crate feature memmap and non-target_family="wasm" only.

Opens a write-ahead log backed by a file backed memory map in read-only mode.

Safety

All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.

Example

use orderwal::{base::OrderWalReader, Builder};



let wal = unsafe {
  Builder::new()
    .map_with_path_builder::<OrderWalReader<[u8], [u8]>, _, ()>(|| Ok(path))
    .unwrap()
};

pub unsafe fn map_mut<'a, W, P>(self, path: P) -> Result<W, Error<W::Memtable>>

where S: BuildChecksumer, P: AsRef<Path>, W: Constructable<Memtable = M, Checksumer = S>, M::Key: Type + Ord + 'static, <M::Key as Type>::Ref<'a>: KeyRef<'a, M::Key>,

Available on crate feature memmap and non-target_family="wasm" only.

Opens a write-ahead log backed by a file backed memory map.

Safety

All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.

Example

use orderwal::{base::OrderWal, Builder};

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("map_mut_with_path_builder_example.wal");

let wal = unsafe {
  Builder::new()
    .with_create_new(true)
    .with_read(true)
    .with_write(true)
    .with_capacity(1000)
    .map_mut::<OrderWal<[u8], [u8]>, _>(&path)
    .unwrap()
};

pub unsafe fn map_mut_with_path_builder<'a, W, PB, E>( self, path_builder: PB, ) -> Result<W, Either<E, Error<W::Memtable>>>

where PB: FnOnce() -> Result<PathBuf, E>, S: BuildChecksumer, W: Constructable<Memtable = M, Checksumer = S>, M::Key: Type + Ord + 'static, <M::Key as Type>::Ref<'a>: KeyRef<'a, M::Key>,

Available on crate feature memmap and non-target_family="wasm" only.

Opens a write-ahead log backed by a file backed memory map.

Safety

All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.

Example

use orderwal::{base::OrderWal, Builder};

let dir = tempfile::tempdir().unwrap();
  
let wal = unsafe {
  Builder::new()
    .with_create_new(true)
    .with_read(true)
    .with_write(true)
    .with_capacity(1000)
    .map_mut_with_path_builder::<OrderWal<[u8], [u8]>, _, ()>(
      || Ok(dir.path().join("map_mut_with_path_builder_example.wal")),
    )
    .unwrap()
};

impl<M> Builder<M>

where M: BaseTable, M::Options: Default,

pub fn new() -> Self

Returns a new write-ahead log builder with the given options.

impl<M, S> Builder<M, S>

where M: BaseTable,

pub fn with_checksumer<NS>(self, cks: NS) -> Builder<M, NS>

Returns a new write-ahead log builder with the new checksumer

Example

use orderwal::{Builder, Crc32, multiple_version::DefaultTable};

let opts = Builder::<DefaultTable<[u8], [u8]>>::new().with_checksumer(Crc32::new());

pub fn with_options(self, opts: Options) -> Self

Returns a new write-ahead log builder with the new options

Example

use orderwal::{Builder, Options, multiple_version::DefaultTable};

let opts = Builder::<DefaultTable<[u8], [u8]>>::new().with_options(Options::default());

pub fn with_memtable_options(self, opts: M::Options) -> Self

Returns a new write-ahead log builder with the new options

Example

use orderwal::{Builder, multiple_version::{ArenaTable, ArenaTableOptions}};

let opts = Builder::<ArenaTable<[u8], [u8]>>::new().with_memtable_options(ArenaTableOptions::default());

pub fn change_memtable<NM>(self) -> Builder<NM, S>

where NM: BaseTable, NM::Options: Default,

Returns a new write-ahead log builder with the new memtable.

Example

use orderwal::{Builder, multiple_version::{DefaultTable, ArenaTable}};

let opts = Builder::<ArenaTable<[u8], [u8]>>::new().change_memtable::<DefaultTable<[u8], [u8]>>();

pub fn change_memtable_with_options<NM>( self, opts: NM::Options, ) -> Builder<NM, S>

where NM: BaseTable,

Returns a new write-ahead log builder with the new memtable and its options

Example

use orderwal::{Builder, multiple_version::{DefaultTable, ArenaTable, ArenaTableOptions}};

let opts = Builder::<DefaultTable<[u8], [u8]>>::new().change_memtable_with_options::<ArenaTable<[u8], [u8]>>(ArenaTableOptions::default().with_capacity(1000));

pub const fn with_reserved(self, reserved: u32) -> Self

Set the reserved bytes of the WAL.

The reserved is used to configure the start position of the WAL. This is useful when you want to add some bytes as your own WAL’s header.

The default reserved is 0.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let opts = Builder::<DefaultTable<[u8], [u8]>>::new().with_reserved(8);

pub const fn reserved(&self) -> u32

Get the reserved of the WAL.

The reserved is used to configure the start position of the WAL. This is useful when you want to add some bytes as your own WAL’s header.

The default reserved is 0.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let opts = Builder::<DefaultTable<[u8], [u8]>>::new().with_reserved(8);

assert_eq!(opts.reserved(), 8);

pub const fn magic_version(&self) -> u16

Returns the magic version.

The default value is 0.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_magic_version(1);
assert_eq!(options.magic_version(), 1);

pub const fn capacity(&self) -> u32

Returns the capacity of the WAL.

The default value is 0.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_capacity(1000);
assert_eq!(options.capacity(), 1000);

pub const fn maximum_key_size(&self) -> KeySize

Returns the maximum key length.

The default value is u16::MAX.

Example

use orderwal::{Builder, KeySize, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_maximum_key_size(KeySize::with(1024));
assert_eq!(options.maximum_key_size(), KeySize::with(1024));

pub const fn maximum_value_size(&self) -> u32

Returns the maximum value length.

The default value is u32::MAX.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_maximum_value_size(1024);
assert_eq!(options.maximum_value_size(), 1024);

pub const fn sync(&self) -> bool

Returns true if the WAL syncs on write.

The default value is true.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new();
assert_eq!(options.sync(), true);

pub const fn with_capacity(self, cap: u32) -> Self

Sets the capacity of the WAL.

This configuration will be ignored when using file-backed memory maps.

The default value is 0.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_capacity(100);
assert_eq!(options.capacity(), 100);

pub const fn with_maximum_key_size(self, size: KeySize) -> Self

Sets the maximum key length.

Example

use orderwal::{Builder, KeySize, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_maximum_key_size(KeySize::with(1024));
assert_eq!(options.maximum_key_size(), KeySize::with(1024));

pub const fn with_maximum_value_size(self, size: u32) -> Self

Sets the maximum value length.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_maximum_value_size(1024);
assert_eq!(options.maximum_value_size(), 1024);

pub const fn with_sync(self, sync: bool) -> Self

Sets the WAL to sync on write.

The default value is true.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_sync(false);
assert_eq!(options.sync(), false);

pub const fn with_magic_version(self, version: u16) -> Self

Sets the magic version.

The default value is 0.

Example

use orderwal::{Builder, multiple_version::DefaultTable};

let options = Builder::<DefaultTable<[u8], [u8]>>::new().with_magic_version(1);
assert_eq!(options.magic_version(), 1);

impl<M, S> Builder<M, S>

where M: BaseTable,

pub fn alloc<W>(self) -> Result<W, Error<W::Memtable>>

where W: Constructable<Memtable = M, Checksumer = S>,

Creates a new in-memory write-ahead log backed by an aligned vec.

Example

use orderwal::{base::OrderWal, Builder};

let wal = Builder::new()
  .with_capacity(1024)
  .alloc::<OrderWal<[u8], [u8]>>()
  .unwrap();

Trait Implementations

impl<M> Default for Builder<M>

where M: BaseTable, M::Options: Default,

fn default() -> Self

Returns the “default value” for a type.