pub struct Storage<K>where
for<'a> K: Key<'a>,{ /* private fields */ }
Expand description
A main storage struct.
Storage has a generic parameter K representing the type of Key
.
To perform read/write operations K must implement Key
trait.
Examples
use pearl::{Storage, Builder, Key, ArrayKey};
#[tokio::main]
async fn main() {
let mut storage: Storage<ArrayKey<8>> = Builder::new()
.work_dir("/tmp/pearl/")
.max_blob_size(1_000_000)
.max_data_in_blob(1_000_000_000)
.blob_file_name_prefix("pearl-test")
.build()
.unwrap();
storage.init().await.unwrap();
}
Implementations§
source§impl<K> Storage<K>where
for<'a> K: Key<'a> + 'static,
impl<K> Storage<K>where for<'a> K: Key<'a> + 'static,
sourcepub async fn init_lazy(&mut self) -> Result<()>
pub async fn init_lazy(&mut self) -> Result<()>
init_lazy()
used to prepare all environment to further work, but unlike init
doesn’t set active blob, which means that first write may take time..
Storage works in directory provided to builder. If directory don’t exist, storage creates it, otherwise tries to init existing storage.
Errors
Returns error in case of failures with IO operations or if some of the required params are missed.
sourcepub async fn has_active_blob(&self) -> bool
pub async fn has_active_blob(&self) -> bool
FIXME: maybe it would be better to add check of is_pending
state of observer for all
sync operations and return result in more appropriate way for that case (change Result
sourcepub async fn try_create_active_blob(&self) -> Result<()>
pub async fn try_create_active_blob(&self) -> Result<()>
Creates active blob
NOTICE! This function works in current thread, so it may take time. To perform this
asyncronously, use [create_active_blob_in_background()
]
Returns true if new blob was created else false
Errors
Fails if it’s not possible to create new blob
[create_active_blob_in_background()
]: struct.Storage.html#method.create_active_blob_async
sourcepub async fn create_active_blob_in_background(&self)
pub async fn create_active_blob_in_background(&self)
Creates active blob
NOTICE! This function returns immediately, so you can’t check result of operation. If you
want be sure about operation’s result, use [try_create_active_blob()
]
[try_create_active_blob()
]: struct.Storage.html#method.try_create_active_blob
sourcepub async fn try_close_active_blob(&self) -> Result<()>
pub async fn try_close_active_blob(&self) -> Result<()>
Dumps active blob
NOTICE! This function works in current thread, so it may take time. To perform this
asyncronously, use [close_active_blob_in_background()
]
Returns true if blob was really dumped else false
Errors
Fails if there are some errors during dump
[close_active_blob_in_background()
]: struct.Storage.html#method.create_active_blob_async
sourcepub async fn close_active_blob_in_background(&self)
pub async fn close_active_blob_in_background(&self)
Dumps active blob
NOTICE! This function returns immediately, so you can’t check result of operation. If you
want be sure about operation’s result, use [try_close_active_blob()
]
sourcepub async fn try_restore_active_blob(&self) -> Result<()>
pub async fn try_restore_active_blob(&self) -> Result<()>
Sets last blob from closed blobs as active if there is no active blobs
NOTICE! This function works in current thread, so it may take time. To perform this
asyncronously, use [restore_active_blob_in_background()
]
Returns true if last blob was set as active as false
Errors
Fails if active blob is set or there is no closed blobs
[restore_active_blob_in_background()
]: struct.Storage.html#method.restore_active_blob_async
sourcepub async fn restore_active_blob_in_background(&self)
pub async fn restore_active_blob_in_background(&self)
Sets last blob from closed blobs as active if there is no active blobs
NOTICE! This function returns immediately, so you can’t check result of operation. If you
want be sure about operation’s result, use [try_restore_active_blob()
]
[try_restore_active_blob()
]: struct.Storage.html#method.try_restore_active_blob
sourcepub async fn write(&self, key: impl AsRef<K>, value: Bytes) -> Result<()>
pub async fn write(&self, key: impl AsRef<K>, value: Bytes) -> Result<()>
Writes data
to active blob asyncronously. If active blob reaches it limit, creates new
and closes old.
NOTICE! First write into storage without active blob may take more time due to active blob
creation
Examples
use pearl::{Builder, Storage, ArrayKey};
async fn write_data(storage: Storage<ArrayKey<8>>) {
let key = ArrayKey::<8>::default();
let data = b"async written to blob".to_vec().into();
storage.write(key, data).await;
}
Errors
Fails with the same errors as write_with
sourcepub async fn write_with(
&self,
key: impl AsRef<K>,
value: Bytes,
meta: Meta
) -> Result<()>
pub async fn write_with( &self, key: impl AsRef<K>, value: Bytes, meta: Meta ) -> Result<()>
Similar to write
but with metadata
Examples
use pearl::{Builder, Meta, Storage, ArrayKey};
async fn write_data(storage: Storage<ArrayKey<8>>) {
let key = ArrayKey::<8>::default();
let data = b"async written to blob".to_vec().into();
let mut meta = Meta::new();
meta.insert("version".to_string(), b"1.0".to_vec());
storage.write_with(&key, data, meta).await;
}
Errors
Fails if duplicates are not allowed and record already exists.
sourcepub async fn free_excess_resources(&self) -> usize
pub async fn free_excess_resources(&self) -> usize
Free all resources that may be freed without work interruption NOTICE! This function frees part of the resources in separate thread, so actual resources may be freed later
sourcepub async fn inactive_index_memory(&self) -> usize
pub async fn inactive_index_memory(&self) -> usize
Get size in bytes of inactive indexes
sourcepub async fn index_memory(&self) -> usize
pub async fn index_memory(&self) -> usize
Get size in bytes of all freeable resources
sourcepub async fn read_with(
&self,
key: impl AsRef<K>,
meta: &Meta
) -> Result<ReadResult<Bytes>>
pub async fn read_with( &self, key: impl AsRef<K>, meta: &Meta ) -> Result<ReadResult<Bytes>>
Reads data matching given key and metadata
Examples
use pearl::{Builder, Meta, Storage, ArrayKey};
async fn read_data(storage: Storage<ArrayKey<8>>) {
let key = ArrayKey::<8>::default();
let mut meta = Meta::new();
meta.insert("version".to_string(), b"1.0".to_vec());
let data = storage.read_with(&key, &meta).await;
}
Errors
Return error if record is not found.
sourcepub async fn read_all_with_deletion_marker(
&self,
key: impl AsRef<K>
) -> Result<Vec<Entry>>
pub async fn read_all_with_deletion_marker( &self, key: impl AsRef<K> ) -> Result<Vec<Entry>>
Returns entries with matching key and deletion marker, if any
Errors
Fails after any disk IO errors.
sourcepub async fn blobs_count(&self) -> usize
pub async fn blobs_count(&self) -> usize
blob_count
returns exact number of closed blobs plus one active, if there is some.
It locks on inner structure, so it much slower than next_blob_id
.
Examples
use pearl::{Builder, Storage, ArrayKey};
async fn check_blobs_count(storage: Storage<ArrayKey<8>>) {
assert_eq!(storage.blobs_count().await, 1);
}
sourcepub fn corrupted_blobs_count(&self) -> usize
pub fn corrupted_blobs_count(&self) -> usize
blob_count
returns exact number of corrupted blobs.
sourcepub async fn active_index_memory(&self) -> usize
pub async fn active_index_memory(&self) -> usize
active_index_memory
returns the amount of memory used by blob to store active indices
sourcepub async fn disk_used(&self) -> u64
pub async fn disk_used(&self) -> u64
disk_used
returns amount of disk space occupied by storage related files
sourcepub fn next_blob_id(&self) -> usize
pub fn next_blob_id(&self) -> usize
Returns next blob ID. If pearl dir structure wasn’t changed from the outside,
returned number is equal to blobs_count
. But this method doesn’t require
lock. So it is much faster than blobs_count
.
sourcepub async fn contains(
&self,
key: impl AsRef<K>
) -> Result<ReadResult<BlobRecordTimestamp>>
pub async fn contains( &self, key: impl AsRef<K> ) -> Result<ReadResult<BlobRecordTimestamp>>
contains
is used to check whether a key is in storage.
Slower than check_bloom
, because doesn’t prevent disk IO operations.
contains
returns either “definitely in storage” or “definitely not”.
Also returns creation timestamp
Errors
Fails because of any IO errors
sourcepub async fn check_filters(&self, key: impl AsRef<K>) -> Option<bool>
pub async fn check_filters(&self, key: impl AsRef<K>) -> Option<bool>
check_filters
is used to check whether a key is in storage.
Range (min-max test) and bloom filters are used.
If bloom filter opt out and range filter passes, returns None
.
False positive results are possible, but false negatives are not.
In other words, check_filters
returns either “possibly in storage” or “definitely not”.
sourcepub async fn records_count(&self) -> usize
pub async fn records_count(&self) -> usize
Total records count in storage.
sourcepub async fn records_count_detailed(&self) -> Vec<(usize, usize)>
pub async fn records_count_detailed(&self) -> Vec<(usize, usize)>
Records count per blob. Format: (blob_id
, count). Last value is from active blob.
sourcepub async fn records_count_in_active_blob(&self) -> Option<usize>
pub async fn records_count_in_active_blob(&self) -> Option<usize>
Records count in active blob. Returns None if active blob not set or any IO error occured.
sourcepub async fn fsyncdata(&self) -> IOResult<()>
pub async fn fsyncdata(&self) -> IOResult<()>
Syncronizes data and metadata of the active blob with the filesystem.
Like tokio::std::fs::File::sync_data
, this function will attempt to ensure that in-core data reaches the filesystem before returning.
May not syncronize file metadata to the file system.
Errors
Fails because of any IO errors.
sourcepub async fn force_update_active_blob(
&self,
predicate: fn(_: Option<ActiveBlobStat>) -> bool
)
pub async fn force_update_active_blob( &self, predicate: fn(_: Option<ActiveBlobStat>) -> bool )
Force updates active blob on new one to dump index of old one on disk and free RAM.
This function was used previously instead of [close_active_blob_in_background()
]
Creates new active blob.
Errors
Fails because of any IO errors.
Or if there are some problems with syncronization.
[close_active_blob_in_background()
]: struct.Storage.html#method.close_active_blob_async