Struct Builder

Source
pub struct Builder<S = Crc32> { /* private fields */ }
Expand description

The builder to build a Log

Implementations§

Source§

impl<S> Builder<S>

Source

pub fn map_anon<C>(self, fid: C::Id) -> Result<C>
where C: Constructor<Checksumer = S> + Mutable,

Available on crate feature memmap and non-target_family="wasm" only.

Create a new in-memory value log which is backed by a anonymous memory map.

What the difference between this method and Builder::alloc?

  1. This method will use mmap anonymous to require memory from the OS directly. If you require very large contiguous memory regions, this method might be more suitable because it’s more direct in requesting large chunks of memory from the OS.

  2. Where as Builder::alloc will use an AlignedVec ensures we are working within Rust’s memory safety guarantees. Even if we are working with raw pointers with Box::into_raw, the backend Log will reclaim the ownership of this memory by converting it back to a Box when dropping the backend Log. Since AlignedVec uses heap memory, the data might be more cache-friendly, especially if you’re frequently accessing or modifying it.

§Example
use valog::{sync, unsync, Builder};

// Create a sync in-memory value log.
let map = Builder::new().with_capacity(1024).map_anon::<sync::ValueLog>(0u32).unwrap();

// Create a unsync in-memory value log.
let arena = Builder::new().with_capacity(1024).map_anon::<unsync::ValueLog>(0u32).unwrap();
Source

pub unsafe fn map<C, P: AsRef<Path>>(self, path: P, fid: C::Id) -> Result<C>
where C: Constructor<Checksumer = S> + Frozen,

Available on crate feature memmap and non-target_family="wasm" only.

Opens a read-only map which backed by file-backed memory map.

§Safety
  • All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.
§Example
use valog::{sync, Builder};

let dir = tempfile::tempdir().unwrap();
let map = unsafe {
  Builder::new()
    .with_capacity(1024)
    .with_create_new(true)
    .with_read(true)
    .with_write(true)
    .map_mut::<sync::ValueLog, _>(
      dir.path().join("map_example.vlog"),
      1u32
    )
    .unwrap()
};

drop(map);

let map = unsafe {
  Builder::new()
    .with_read(true)
    .map::<sync::ImmutableValueLog, _>(
      dir.path().join("map_example.vlog"),
      1u32,
    )
   .unwrap()
};
Source

pub unsafe fn map_with_path_builder<C, PB, E>( self, path_builder: PB, fid: C::Id, ) -> Result<C, Either<E, Error>>
where C: Constructor<Checksumer = S> + Frozen, PB: FnOnce() -> Result<PathBuf, E>,

Available on crate feature memmap and non-target_family="wasm" only.

Opens a read-only map which backed by file-backed memory map with a path builder.

§Safety
  • All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.
§Example
use valog::{sync, Builder};

let dir = tempfile::tempdir().unwrap();
let map = unsafe {
  Builder::new()
    .with_capacity(1024)
    .with_create_new(true)
    .with_read(true)
    .with_write(true)
    .map_mut::<sync::ValueLog, _>(
      dir.path().join("map_with_path_builder_example.vlog"),
      1u32
    )
    .unwrap()
};

drop(map);

let map = unsafe {
  Builder::new()
    .with_read(true)
    .map_with_path_builder::<sync::ImmutableValueLog, _, ()>(
      || Ok(dir.path().join("map_with_path_builder_example.vlog")),
      1u32,
    )
   .unwrap()
};
Source

pub unsafe fn map_mut<C, P: AsRef<Path>>(self, path: P, fid: C::Id) -> Result<C>
where C: Constructor<Checksumer = S> + Mutable,

Available on crate feature memmap and non-target_family="wasm" only.

Creates a new map or reopens a map which backed by a file backed memory map.

§Safety
  • All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.
§Example
use valog::{sync, Builder};

let dir = tempfile::tempdir().unwrap();
let map = unsafe {
  Builder::new()
    .with_capacity(1024)
    .with_create_new(true)
    .with_read(true)
    .with_write(true)
    .map_mut::<sync::ValueLog, _>(
      dir.path().join("map_mut_example.vlog"),
      1u32
    )
    .unwrap()
};
Source

pub unsafe fn map_mut_with_path_builder<C, PB, E>( self, path_builder: PB, fid: C::Id, ) -> Result<C, Either<E, Error>>
where C: Constructor<Checksumer = S> + Mutable, PB: FnOnce() -> Result<PathBuf, E>,

Available on crate feature memmap and non-target_family="wasm" only.

Creates a new map or reopens a map which backed by a file backed memory map with path builder.

§Safety
  • All file-backed memory map constructors are marked unsafe because of the potential for Undefined Behavior (UB) using the map if the underlying file is subsequently modified, in or out of process. Applications must consider the risk and take appropriate precautions when using file-backed maps. Solutions such as file permissions, locks or process-private (e.g. unlinked) files exist but are platform specific and limited.
§Example
use valog::{sync, Builder};

let dir = tempfile::tempdir().unwrap();
let map = unsafe {
  Builder::new()
    .with_capacity(1024)
    .with_create_new(true)
    .with_read(true)
    .with_write(true)
    .map_mut_with_path_builder::<sync::ValueLog, _, ()>(
      || Ok(dir.path().join("map_mut_with_path_builder_example.vlog")),
      1u32
    )
    .unwrap()
};
Source§

impl<C> Builder<C>

Source

pub fn with_read(self, read: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for read access.

This option, when true, will indicate that the file should be read-able if opened.

§Examples
use valog::Builder;

let opts = Builder::new().with_read(true);
Source

pub fn with_write(self, write: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for write access.

This option, when true, will indicate that the file should be write-able if opened.

If the file already exists, any write calls on it will overwrite its contents, without truncating it.

§Examples
use valog::Builder;

let opts = Builder::new().with_write(true);
Source

pub fn with_append(self, append: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for the append mode.

This option, when true, means that writes will append to a file instead of overwriting previous contents. Note that setting .write(true).append(true) has the same effect as setting only .append(true).

For most filesystems, the operating system guarantees that all writes are atomic: no writes get mangled because another process writes at the same time.

One maybe obvious note when using append-mode: make sure that all data that belongs together is written to the file in one operation. This can be done by concatenating strings before passing them to write(), or using a buffered writer (with a buffer of adequate size), and calling flush() when the message is complete.

If a file is opened with both read and append access, beware that after opening, and after every write, the position for reading may be set at the end of the file. So, before writing, save the current position (using seek(SeekFrom::Current(opts))), and restore it before the next read.

§Note

This function doesn’t create the file if it doesn’t exist. Use the Options::with_create method to do so.

§Examples
use valog::Builder;

let opts = Builder::new().with_append(true);
Source

pub fn with_truncate(self, truncate: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option for truncating a previous file.

If a file is successfully opened with this option set it will truncate the file to opts length if it already exists.

The file must be opened with write access for truncate to work.

§Examples
use valog::Builder;

let opts = Builder::new().with_write(true).with_truncate(true);
Source

pub fn with_create(self, val: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option to create a new file, or open it if it already exists. If the file does not exist, it is created and set the lenght of the file to the given size.

In order for the file to be created, Options::with_write or Options::with_append access must be used.

See also std::fs::write() for a simple function to create a file with some given data.

§Examples
use valog::Builder;

let opts = Builder::new().with_write(true).with_create(true);
Source

pub fn with_create_new(self, val: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Sets the option to create a new file and set the file length to the given value, failing if it already exists.

No file is allowed to exist at the target location, also no (dangling) symlink. In this way, if the call succeeds, the file returned is guaranteed to be new.

This option is useful because it is atomic. Otherwise between checking whether a file exists and creating a new one, the file may have been created by another process (a TOCTOU race condition / attack).

If .with_create_new(true) is set, .with_create() and .with_truncate() are ignored.

The file must be opened with write or append access in order to create a new file.

§Examples
use valog::Builder;

let file = Builder::new()
  .with_write(true)
  .with_create_new(true);
Source

pub fn with_offset(self, offset: u64) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Configures the memory map to start at byte offset from the beginning of the file.

This option has no effect on anonymous memory maps or vec backed Log.

By default, the offset is 0.

§Example
use valog::Builder;

let opts = Builder::new().with_offset(30);
Source

pub fn with_stack(self, stack: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Configures the anonymous memory map to be suitable for a process or thread stack.

This option corresponds to the MAP_STACK flag on Linux. It has no effect on Windows.

This option has no effect on file-backed memory maps and vec backed Log.

§Example
use valog::Builder;

let stack = Builder::new().with_stack(true);
Source

pub fn with_huge(self, page_bits: Option<u8>) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Configures the anonymous memory map to be allocated using huge pages.

This option corresponds to the MAP_HUGETLB flag on Linux. It has no effect on Windows.

The size of the requested page can be specified in page bits. If not provided, the system default is requested. The requested length should be a multiple of this, or the mapping will fail.

This option has no effect on file-backed memory maps and vec backed Log.

§Example
use valog::Builder;

let stack = Builder::new().with_huge(Some(8));
Source

pub fn with_populate(self, populate: bool) -> Self

Available on crate feature memmap and non-target_family="wasm" only.

Populate (prefault) page tables for a mapping.

For a file mapping, this causes read-ahead on the file. This will help to reduce blocking on page faults later.

This option corresponds to the MAP_POPULATE flag on Linux. It has no effect on Windows.

This option has no effect on vec backed Log.

§Example
use valog::Builder;

let opts = Builder::new().with_populate(true);
Source§

impl<C> Builder<C>

Source

pub const fn read(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with read access.

§Examples
use valog::Builder;

let opts = Builder::new().with_read(true);
assert_eq!(opts.read(), true);
Source

pub const fn write(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with write access.

§Examples
use valog::Builder;

let opts = Builder::new().with_write(true);
assert_eq!(opts.write(), true);
Source

pub const fn append(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with append access.

§Examples
use valog::Builder;

let opts = Builder::new().with_append(true);
assert_eq!(opts.append(), true);
Source

pub const fn truncate(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be opened with truncate access.

§Examples
use valog::Builder;

let opts = Builder::new().with_truncate(true);
assert_eq!(opts.truncate(), true);
Source

pub const fn create(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be created if it does not exist.

§Examples
use valog::Builder;

let opts = Builder::new().with_create(true);
assert_eq!(opts.create(), true);
Source

pub const fn create_new(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the file should be created if it does not exist and fail if it does.

§Examples
use valog::Builder;

let opts = Builder::new().with_create_new(true);
assert_eq!(opts.create_new(), true);
Source

pub const fn offset(&self) -> u64

Available on crate feature memmap and non-target_family="wasm" only.

Returns the offset of the memory map.

§Examples
use valog::Builder;

let opts = Builder::new().with_offset(30);
assert_eq!(opts.offset(), 30);
Source

pub const fn stack(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the memory map should be suitable for a process or thread stack.

§Examples
use valog::Builder;

let opts = Builder::new().with_stack(true);
assert_eq!(opts.stack(), true);
Source

pub const fn huge(&self) -> Option<u8>

Available on crate feature memmap and non-target_family="wasm" only.

Returns the page bits of the memory map.

§Examples
use valog::Builder;

let opts = Builder::new().with_huge(Some(8));
assert_eq!(opts.huge(), Some(8));
Source

pub const fn populate(&self) -> bool

Available on crate feature memmap and non-target_family="wasm" only.

Returns true if the memory map should populate (prefault) page tables for a mapping.

§Examples
use valog::Builder;

let opts = Builder::new().with_populate(true);
assert_eq!(opts.populate(), true);
Source§

impl Builder

Source

pub fn new() -> Self

Create a new Builder with default values.

Source§

impl<S> Builder<S>

Source

pub fn with_checksumer<NS>(self, cks: NS) -> Builder<NS>

Returns a new map builder with the new BuildChecksumer.

§Example
use valog::{Builder, checksum::Crc32};

let builder = Builder::new().with_checksumer(Crc32::new());
Source

pub const fn with_options(self, opts: Options) -> Self

Returns a new map builder with the new Options.

§Example
use valog::{Builder, options::Options};

let builder = Builder::new().with_options(Options::new().with_capacity(1024));
Source

pub const fn with_reserved(self, reserved: u32) -> Self

Set the reserved bytes of the Log.

The reserved is used to configure the start position of the Log. This is useful when you want to add some bytes before the Log, e.g. when using the memory map file backed Log, you can set the reserved to the size to 8 to store a 8 bytes checksum.

The default reserved is 0.

§Example
use valog::Builder;

let builder = Builder::new().with_reserved(8);
Source

pub const fn with_sync(self, sync: bool) -> Self

Set if flush the data to the disk when new value is inserted.

Default is true.

§Example
use valog::Builder;

let opts = Builder::new().with_sync(false);
Source

pub const fn with_lock_meta(self, lock_meta: bool) -> Self

Set if lock the meta of the Log in the memory to prevent OS from swapping out the first page of Log. When using memory map backed Log, the meta of the Log is in the first page, meta is frequently accessed, lock (mlock on the first page) the meta can reduce the page fault, but yes, this means that one SkipMap will have one page are locked in memory, and will not be swapped out. So, this is a trade-off between performance and memory usage.

Default is true.

This configuration has no effect on windows and vec backed Log.

§Example
use valog::Builder;

let opts = Builder::new().with_lock_meta(false);
Source

pub const fn with_magic_version(self, magic_version: u16) -> Self

Set the magic version of the value log.

This is used by the application using value log to ensure that it doesn’t open the value log with incompatible data format.

The default value is 0.

§Example
use valog::Builder;

let builder = Builder::new().with_magic_version(1);
Source

pub const fn with_freelist(self, freelist: Freelist) -> Self

Set the Freelist kind of the value log.

The default value is Freelist::Optimistic.

§Example
use valog::{Builder, options::Freelist};

let builder = Builder::new().with_freelist(Freelist::Optimistic);
Source

pub const fn with_unify(self, unify: bool) -> Self

Set if use the unify memory layout of the value log.

File backed value log has different memory layout with other kind backed value log, set this value to true will unify the memory layout of the value log, which means all kinds of backed value log will have the same memory layout.

This value will be ignored if the value log is backed by a file backed memory map.

The default value is false.

§Example
use valog::Builder;

let builder = Builder::new().with_unify(true);
Source

pub const fn with_maximum_value_size(self, size: u32) -> Self

Sets the maximum size of the value.

Default is u32::MAX.

§Example
use valog::Builder;

let builder = Builder::new().with_maximum_value_size(1024);
Source

pub const fn with_capacity(self, capacity: u32) -> Self

Sets the capacity of the underlying Log.

Default is 1024. This configuration will be ignored if the map is memory-mapped.

§Example
use valog::Builder;

let builder = Builder::new().with_capacity(1024);
Source

pub const fn reserved(&self) -> u32

Get the reserved of the Log.

The reserved is used to configure the start position of the Log. This is useful when you want to add some bytes before the Log, e.g. when using the memory map file backed Log, you can set the reserved to the size to 8 to store a 8 bytes checksum.

The default reserved is 0.

§Example
use valog::Builder;

let builder = Builder::new().with_reserved(8);

assert_eq!(builder.reserved(), 8);
Source

pub const fn sync(&self) -> bool

Get if flush the data to the disk when new value is inserted.

Default is true.

§Example
use valog::Builder;

let builder = Builder::new().with_sync(false);

assert_eq!(builder.sync(), false);
Source

pub const fn lock_meta(&self) -> bool

Get if lock the meta of the Log in the memory to prevent OS from swapping out the first page of Log. When using memory map backed Log, the meta of the Log is in the first page, meta is frequently accessed, lock (mlock on the first page) the meta can reduce the page fault, but yes, this means that one SkipMap will have one page are locked in memory, and will not be swapped out. So, this is a trade-off between performance and memory usage.

§Example
use valog::Builder;

let opts = Builder::new().with_lock_meta(false);

assert_eq!(opts.lock_meta(), false);
Source

pub const fn maximum_value_size(&self) -> u32

Returns the maximum size of the value.

Default is u32::MAX. The maximum size of the value is u32::MAX - header.

§Example
use valog::Builder;

let builder = Builder::default().with_maximum_value_size(1024);
assert_eq!(builder.maximum_value_size(), 1024);
Source

pub const fn capacity(&self) -> u32

Returns the configuration of underlying Log size.

Default is 1024. This configuration will be ignored if the map is memory-mapped.

§Example
use valog::Builder;

let builder = Builder::new().with_capacity(1024);
assert_eq!(builder.capacity(), 1024);
Source

pub const fn unify(&self) -> bool

Get if use the unify memory layout of the value log.

File backed value log has different memory layout with other kind backed value log, set this value to true will unify the memory layout of the value log, which means all kinds of backed value log will have the same memory layout.

This value will be ignored if the value log is backed by a file backed memory map.

The default value is false.

§Example
use valog::Builder;

let builder = Builder::new().with_unify(true);

assert_eq!(builder.unify(), true);
Source

pub const fn magic_version(&self) -> u16

Get the magic version of the value log.

This is used by the application using value log to ensure that it doesn’t open the value log with incompatible data format.

The default value is 0.

§Example
use valog::Builder;

let builder = Builder::new().with_magic_version(1);

assert_eq!(builder.magic_version(), 1);
Source

pub const fn freelist(&self) -> Freelist

Get the Freelist kind of the value log.

The default value is Freelist::Optimistic.

§Example
use valog::{Builder, options::Freelist};

let builder = Builder::new().with_freelist(Freelist::Optimistic);

assert_eq!(builder.freelist(), Freelist::Optimistic);
Source§

impl<S: BuildChecksumer> Builder<S>

Source

pub fn alloc<C>(self, fid: C::Id) -> Result<C, Error>
where C: Constructor<Checksumer = S> + Mutable,

Create a new in-memory value log which is backed by a AlignedVec.

What the difference between this method and Builder::map_anon?

  1. This method will use an AlignedVec ensures we are working within Rust’s memory safety guarantees. Even if we are working with raw pointers with Box::into_raw, the backend Log will reclaim the ownership of this memory by converting it back to a Box when dropping the backend Log. Since AlignedVec uses heap memory, the data might be more cache-friendly, especially if you’re frequently accessing or modifying it.

  2. Where as Builder::map_anon will use mmap anonymous to require memory from the OS. If you require very large contiguous memory regions, mmap might be more suitable because it’s more direct in requesting large chunks of memory from the OS.

§Example
use valog::{Builder, sync, unsync};

// Create a sync in-memory value log.
let map = Builder::new().with_capacity(1024).alloc::<sync::ValueLog>(1u32).unwrap();

// Create a unsync in-memory value log.
let arena = Builder::new().with_capacity(1024).alloc::<unsync::ValueLog>(1u32).unwrap();

Trait Implementations§

Source§

impl Default for Builder

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

§

impl<S> Freeze for Builder<S>
where S: Freeze,

§

impl<S> RefUnwindSafe for Builder<S>
where S: RefUnwindSafe,

§

impl<S> Send for Builder<S>
where S: Send,

§

impl<S> Sync for Builder<S>
where S: Sync,

§

impl<S> Unpin for Builder<S>
where S: Unpin,

§

impl<S> UnwindSafe for Builder<S>
where S: UnwindSafe,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more