Struct queue_file::QueueFile
source · pub struct QueueFile { /* private fields */ }
Expand description
QueueFile
is a lightning-fast, transactional, file-based FIFO.
Addition and removal from an instance is an O(1) operation and is atomic. Writes are synchronous by default; data will be written to disk before an operation returns.
The underlying file. Uses a ring buffer to store entries. Designed so that a modification isn’t committed or visible until we write the header. The header is much smaller than a segment. So long as the underlying file system supports atomic segment writes, changes to the queue are atomic. Storing the file length ensures we can recover from a failed expansion (i.e. if setting the file length succeeds but the process dies before the data can be copied).
Example
use queue_file::QueueFile;
let mut qf = QueueFile::open(path)
.expect("cannot open queue file");
let data = "Welcome to QueueFile!".as_bytes();
qf.add(&data).expect("add failed");
if let Ok(Some(bytes)) = qf.peek() {
assert_eq!(data, bytes.as_ref());
}
qf.remove().expect("remove failed");
File format
16-32 bytes Header
... Data
This implementation supports two versions of the header format.
Versioned Header (32 bytes):
1 bit Versioned indicator [0 = legacy, 1 = versioned]
31 bits Version, always 1
8 bytes File length
4 bytes Element count
8 bytes Head element position
8 bytes Tail element position
Legacy Header (16 bytes):
1 bit Legacy indicator, always 0
31 bits File length
4 bytes Element count
4 bytes Head element position
4 bytes Tail element position
Each element stored is represented by:
Element:
4 bytes Data length
... Data
Implementations§
source§impl QueueFile
impl QueueFile
sourcepub const fn overwrite_on_remove(&self) -> bool
pub const fn overwrite_on_remove(&self) -> bool
Returns true if removing an element will also overwrite data with zero bytes.
pub const fn get_overwrite_on_remove(&self) -> bool
sourcepub fn set_overwrite_on_remove(&mut self, value: bool)
pub fn set_overwrite_on_remove(&mut self, value: bool)
If set to true removing an element will also overwrite data with zero bytes.
sourcepub const fn sync_writes(&self) -> bool
pub const fn sync_writes(&self) -> bool
Returns true if every write to file will be followed by sync_data()
call.
pub const fn get_sync_writes(&self) -> bool
sourcepub fn set_sync_writes(&mut self, value: bool)
pub fn set_sync_writes(&mut self, value: bool)
If set to true every write to file will be followed by sync_data()
call.
sourcepub const fn skip_write_header_on_add(&self) -> bool
pub const fn skip_write_header_on_add(&self) -> bool
Returns true if skips header update upon adding enabled.
pub const fn get_skip_write_header_on_add(&self) -> bool
sourcepub fn set_skip_write_header_on_add(&mut self, value: bool)
pub fn set_skip_write_header_on_add(&mut self, value: bool)
If set to true skips header update upon adding.
sourcepub fn set_read_buffer_size(&mut self, size: usize)
pub fn set_read_buffer_size(&mut self, size: usize)
Changes buffer size used for data reading.
pub const fn cache_offset_policy(&self) -> Option<OffsetCacheKind>
pub const fn get_cache_offset_policy(&self) -> Option<OffsetCacheKind>
pub fn set_cache_offset_policy( &mut self, kind: impl Into<Option<OffsetCacheKind>> )
sourcepub fn sync_all(&mut self) -> Result<(), Error>
pub fn sync_all(&mut self) -> Result<(), Error>
Synchronizes the underlying file, look at File::sync_all
doc for more info.
pub fn add_n( &mut self, elems: impl IntoIterator<Item = impl AsRef<[u8]>> + Clone ) -> Result<(), Error>
sourcepub fn add(&mut self, buf: &[u8]) -> Result<(), Error>
pub fn add(&mut self, buf: &[u8]) -> Result<(), Error>
Adds an element to the end of the queue.
sourcepub fn peek(&mut self) -> Result<Option<Box<[u8]>>, Error>
pub fn peek(&mut self) -> Result<Option<Box<[u8]>>, Error>
Reads the eldest element. Returns OK(None)
if the queue is empty.
sourcepub fn clear(&mut self) -> Result<(), Error>
pub fn clear(&mut self) -> Result<(), Error>
Clears this queue. Truncates the file to the initial size.
sourcepub fn iter(&mut self) -> Iter<'_> ⓘ
pub fn iter(&mut self) -> Iter<'_> ⓘ
Returns an iterator over elements in this queue.
Example
let mut qf = QueueFile::open(path).expect("failed to open queue");
let items = vec![vec![1, 2], vec![], vec![3]];
qf.add_n(&items).expect("failed to add elements to queue");
let stored = qf.iter().map(Vec::from).collect::<Vec<_>>();
assert_eq!(items, stored);
sourcepub const fn file_len(&self) -> u64
pub const fn file_len(&self) -> u64
Returns the amount of bytes used by the backed file.
Always >= Self::used_bytes
.
sourcepub const fn used_bytes(&self) -> u64
pub const fn used_bytes(&self) -> u64
Returns the amount of bytes used by the queue.
sourcepub fn into_inner_file(self) -> File
pub fn into_inner_file(self) -> File
Returns underlying file of the queue.