Struct queue_file::QueueFile

pub struct QueueFile { /* private fields */ }

QueueFile is a lightning-fast, transactional, file-based FIFO.

Addition and removal from an instance is an O(1) operation and is atomic. Writes are synchronous by default; data will be written to disk before an operation returns.

The underlying file. Uses a ring buffer to store entries. Designed so that a modification isn’t committed or visible until we write the header. The header is much smaller than a segment. So long as the underlying file system supports atomic segment writes, changes to the queue are atomic. Storing the file length ensures we can recover from a failed expansion (i.e. if setting the file length succeeds but the process dies before the data can be copied).

Example

use queue_file::QueueFile;

let mut qf = QueueFile::open("example.qf")
    .expect("cannot open queue file");
let data = "Welcome to QueueFile!".as_bytes();

qf.add(&data).expect("add failed");

if let Ok(Some(bytes)) = qf.peek() {
    assert_eq!(data, bytes.as_ref());
}

qf.remove().expect("remove failed");

File format

  16-32 bytes      Header
  ...              Data

This implementation supports two versions of the header format.

Versioned Header (32 bytes):
  1 bit            Versioned indicator [0 = legacy, 1 = versioned]
  31 bits          Version, always 1
  8 bytes          File length
  4 bytes          Element count
  8 bytes          Head element position
  8 bytes          Tail element position

Legacy Header (16 bytes):
  1 bit            Legacy indicator, always 0
  31 bits          File length
  4 bytes          Element count
  4 bytes          Head element position
  4 bytes          Tail element position

Each element stored is represented by:

Element:
  4 bytes          Data length
  ...              Data

Implementations

impl QueueFile

pub fn with_capacity<P: AsRef<Path>>(
    path: P,
    capacity: u64
) -> Result<QueueFile, Error>

pub fn open<P: AsRef<Path>>(path: P) -> Result<QueueFile, Error>

pub fn open_legacy<P: AsRef<Path>>(path: P) -> Result<QueueFile, Error>

pub fn get_overwrite_on_remove(&self) -> bool

Returns true if removing an element will also overwrite data with zero bytes.

pub fn set_overwrite_on_remove(&mut self, value: bool)

If set to true removing an element will also overwrite data with zero bytes.

pub fn get_sync_writes(&self) -> bool

Returns true if every write to file will be followed by sync_data() call.

pub fn set_sync_writes(&mut self, value: bool)

If set to true every write to file will be followed by sync_data() call.

pub fn get_skip_write_header_on_add(&self) -> bool

Returns true if skips header update upon adding enabled.

pub fn set_skip_write_header_on_add(&mut self, value: bool)

If set to true skips header update upon adding.

pub fn is_empty(&self) -> bool

Returns true if this queue contains no entries.

pub fn size(&self) -> usize

Returns the number of elements in this queue.

pub fn sync_all(&mut self) -> Result<(), Error>

Synchronizes the underlying file, look at File::sync_all doc for more info.

pub fn add_n(
    &mut self,
    elems: impl IntoIterator<Item = impl AsRef<[u8]>> + Clone
) -> Result<(), Error>

pub fn add(&mut self, buf: &[u8]) -> Result<(), Error>

Adds an element to the end of the queue.

pub fn peek(&mut self) -> Result<Option<Box<[u8]>>, Error>

Reads the eldest element. Returns OK(None) if the queue is empty.

pub fn remove(&mut self) -> Result<(), Error>

Removes the eldest element.

pub fn remove_n(&mut self, n: usize) -> Result<(), Error>

Removes the eldest n elements.

pub fn clear(&mut self) -> Result<(), Error>

Clears this queue. Truncates the file to the initial size.

pub fn iter(&mut self) -> Iter<'_>

Returns an iterator over elements in this queue.

pub fn file_len(&self) -> u64

Returns the amount of bytes used by the backed file. Always >= Self::used_bytes.

pub fn used_bytes(&self) -> u64

Returns the amount of bytes used by the queue.

Trait Implementations

impl Debug for QueueFile

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for QueueFile

fn drop(&mut self)

Executes the destructor for this type.