[−][src]Struct esl01_indexedlog::log::Log
An append-only storage with indexes and integrity checks.
The Log
is backed by a directory in the filesystem. The
directory includes:
- An append-only "log" file. It can be seen as a serialization result of an append-only list of byte slices. Each byte slice has a checksum.
- Multiple user-defined indexes. Each index has an append-only on-disk radix-tree representation and a small, separate, non-append-only checksum file.
- A small "metadata" file which records the logic lengths (in bytes) for the log and index files.
Reading is lock-free because the log and indexes are append-only. Writes are buffered in memory. Flushing in-memory parts to disk requires taking a flock on the directory.
Fields
dir: GenericPath
Methods
impl Log
[src]
pub fn open<P: AsRef<Path>>(dir: P, index_defs: Vec<IndexDef>) -> Result<Self>
[src]
Construct Log
at given directory. Incrementally build up specified
indexes.
Create the Log
if it does not exist.
See OpenOptions::open
for details.
pub fn append<T: AsRef<[u8]>>(&mut self, data: T) -> Result<()>
[src]
Append an entry in-memory. Update related indexes in-memory.
The memory part is not shared. Therefore other Log
instances won't see
the change immediately.
To write in-memory entries and indexes to disk, call Log::sync
.
pub fn clear_dirty(&mut self) -> Result<()>
[src]
Remove dirty (in-memory) state. Restore the Log
to the state as
if it's just loaded from disk without modifications.
pub fn try_clone(&self) -> Result<Self>
[src]
Return a cloned Log
with pending in-memory changes.
pub fn try_clone_without_dirty(&self) -> Result<Self>
[src]
Return a cloned Log
without pending in-memory changes.
This is logically equivalent to calling clear_dirty
immediately
on the result after try_clone
, but potentially cheaper.
pub fn sync(&mut self) -> Result<u64>
[src]
Load the latest data from disk. Write in-memory entries to disk.
After writing, update on-disk indexes. These happen in a same critical section, protected by a lock on the directory.
Even if Log::append
is never called, this function has a side effect
updating the Log
to contain latest entries on disk.
Other Log
instances living in a same process or other processes won't
be notified about the change and they can only access the data
"snapshotted" at open time.
Return the size of the updated primary log file in bytes.
For in-memory-only Logs, this function does nothing, and returns 0.
pub fn is_changed(&self) -> bool
[src]
Check if the log is changed on disk.
pub fn flush(&mut self) -> Result<u64>
[src]
Renamed. Use Log::sync
instead.
pub fn slice_to_bytes(&self, slice: &[u8]) -> Bytes
[src]
Convert a slice to Bytes
.
Do not copy the slice if it's from the main on-disk buffer.
pub fn index_slice_to_bytes(&self, index_id: usize, slice: &[u8]) -> Bytes
[src]
Convert a slice to Bytes
.
Do not copy the slice if it's from the specified index buffer.
pub fn rebuild_indexes(self, force: bool) -> Result<String>
[src]
Rebuild indexes.
If force
is false
, then indexes that pass the checksum check
will not be rebuilt. Otherwise, they will be rebuilt regardless.
Setting force
to true
might reduce the size used by the index
files. But that is more expensive.
The function consumes the Log
object, since it is hard to recover
from an error case.
Return message useful for human consumption.
pub fn lookup<K: AsRef<[u8]>>(
&self,
index_id: usize,
key: K
) -> Result<LogLookupIter>
[src]
&self,
index_id: usize,
key: K
) -> Result<LogLookupIter>
Look up an entry using the given index. The index_id
is the index of
index_defs
passed to Log::open
.
Return an iterator of Result<&[u8]>
, in reverse insertion order.
pub fn lookup_prefix<K: AsRef<[u8]>>(
&self,
index_id: usize,
prefix: K
) -> Result<LogRangeIter>
[src]
&self,
index_id: usize,
prefix: K
) -> Result<LogRangeIter>
Look up keys and entries using the given prefix.
The index_id
is the index of index_defs
passed to Log::open
.
Return an iterator that yields (key, iter)
, where key
is the full
key, iter
is LogLookupIter
that allows iteration through matched
entries.
pub fn lookup_range<'a>(
&self,
index_id: usize,
range: impl RangeBounds<&'a [u8]>
) -> Result<LogRangeIter>
[src]
&self,
index_id: usize,
range: impl RangeBounds<&'a [u8]>
) -> Result<LogRangeIter>
Look up keys and entries by querying a specified index about a specified range.
The index_id
is the index of index_defs
defined by OpenOptions
.
Return an iterator that yields (key, iter)
, where key
is the full
key, iter
is LogLookupIter
that allows iteration through entries
matching that key.
pub fn lookup_prefix_hex<K: AsRef<[u8]>>(
&self,
index_id: usize,
hex_prefix: K
) -> Result<LogRangeIter>
[src]
&self,
index_id: usize,
hex_prefix: K
) -> Result<LogRangeIter>
Look up keys and entries using the given hex prefix. The length of the hex string can be odd.
Return an iterator that yields (key, iter)
, where key
is the full
key, iter
is LogLookupIter
that allows iteration through matched
entries.
pub fn iter(&self) -> LogIter
[src]
Return an iterator for all entries.
pub fn iter_dirty(&self) -> LogIter
[src]
Return an iterator for in-memory entries that haven't been flushed to disk.
For in-memory Logs, this is the same as Log::iter
.
pub fn index_func<'a>(
&self,
index_id: usize,
entry: &'a [u8]
) -> Result<Vec<Cow<'a, [u8]>>>
[src]
&self,
index_id: usize,
entry: &'a [u8]
) -> Result<Vec<Cow<'a, [u8]>>>
Applies the given index function to the entry data and returns the index keys.
pub fn path(&self) -> &GenericPath
[src]
Return the reference to the GenericPath
used to crate the Log
.
Trait Implementations
Auto Trait Implementations
impl !RefUnwindSafe for Log
impl Send for Log
impl Sync for Log
impl Unpin for Log
impl !UnwindSafe for Log
Blanket Implementations
impl<T> Any for T where
T: 'static + ?Sized,
[src]
T: 'static + ?Sized,
impl<T> Borrow<T> for T where
T: ?Sized,
[src]
T: ?Sized,
impl<T> BorrowMut<T> for T where
T: ?Sized,
[src]
T: ?Sized,
fn borrow_mut(&mut self) -> &mut T
[src]
impl<T> From<T> for T
[src]
impl<T, U> Into<U> for T where
U: From<T>,
[src]
U: From<T>,
impl<T, U> TryFrom<U> for T where
U: Into<T>,
[src]
U: Into<T>,
type Error = Infallible
The type returned in the event of a conversion error.
fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>
[src]
impl<T, U> TryInto<U> for T where
U: TryFrom<T>,
[src]
U: TryFrom<T>,
type Error = <U as TryFrom<T>>::Error
The type returned in the event of a conversion error.
fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>
[src]
impl<V, T> VZip<V> for T where
V: MultiLane<T>,
V: MultiLane<T>,