Struct AppendOnlyFile

pub struct AppendOnlyFile<T> { /* private fields */ }

Wrapper for a file that can be read at any position (random read) but for which writes are append only. Reads are backed by a memory map (mmap(2)), relying on the operating system for fast access and caching. The memory map is reallocated to expand it when new writes are flushed.

Despite being append-only, the file can still be pruned and truncated. The former simply happens by rewriting it, ignoring some of the data. The latter by truncating the underlying file and re-creating the mmap.

Implementations

impl<T> AppendOnlyFile<T>

where T: Debug + Readable + Writeable,

pub fn open<P>( path: P, size_info: SizeInfo, version: ProtocolVersion, ) -> Result<AppendOnlyFile<T>>

where P: AsRef<Path> + Debug,

Open a file (existing or not) as append-only, backed by a mmap.

pub fn init(&mut self) -> Result<()>

(Re)init an underlying file and its associated memmap. Taking care to initialize the mmap_offset_cache for each element.

pub fn append(&mut self, bytes: &mut [u8]) -> Result<()>

Append data to the file. Until the append-only file is synced, data is only written to memory.

pub fn rewind(&mut self, pos: u64)

Rewinds the data file back to a previous position. We simply “rewind” the buffer_start_pos to the specified position. Note: We do not currently support rewinding within the buffer itself.

pub fn flush(&mut self) -> Result<()>

Syncs all writes (fsync), reallocating the memory map to make the newly written data accessible.

pub fn discard(&mut self)

Discard the current non-flushed data.

pub fn read(&self, pos: u64) -> Result<&[u8]>

Read the bytes representing the element at the given position (0-indexed). Uses the offset cache to determine the offset to read from and the size in bytes to actually read. Leverages the memory map.

pub fn as_temp_file(&self) -> Result<File>

Create a new tempfile containing the contents of this append only file. This allows callers to see a consistent view of the data without locking the append only file.

pub fn write_tmp_pruned(&self, prune_pos: &[u64]) -> Result<()>

Saves a copy of the current file content, skipping data at the provided prune positions. prune_pos must be ordered.

pub fn replace_with_tmp(&mut self) -> Result<()>

Replace the underlying file with the file at tmp path. Rebuild and initialize from the new file.

pub fn release(&mut self)

Release underlying file handles.

pub fn size(&self) -> Result<u64>

Current size of the file in bytes.

pub fn path(&self) -> &Path

Path of the underlying file