Struct pearl::Storage

pub struct Storage<K: Key> { /* fields omitted */ }

A main storage struct.

This type is clonable, cloning it will only create a new reference, not a new storage. Storage has a type parameter K. To perform read/write operations K must implement Key trait.

Examples

use pearl::{Storage, Builder, Key};

#[tokio::main]
async fn main() {
    let mut storage: Storage<String> = Builder::new()
        .work_dir("/tmp/pearl/")
        .max_blob_size(1_000_000)
        .max_data_in_blob(1_000_000_000)
        .blob_file_name_prefix("pearl-test")
        .build()
        .unwrap();
    storage.init().await.unwrap();
}

Implementations

impl<K: Key> Storage<K>

pub async fn init(&mut self) -> Result<()>

init() used to prepare all environment to further work.

Storage works in directory provided to builder. If directory don’t exist, storage creates it, otherwise tries to init existing storage.

Errors

Returns error in case of failures with IO operations or if some of the required params are missed.

pub async fn write(&self, key: impl AsRef<K>, value: Vec<u8>) -> Result<()>

Writes data to active blob asyncronously. If active blob reaches it limit, creates new and closes old.

Examples

async fn write_data() {
    let key = 42u64.to_be_bytes().to_vec();
    let data = b"async written to blob".to_vec();
    storage.write(key, data).await
}

Errors

Fails with the same errors as write_with

pub async fn write_with(
    &self,
    key: impl AsRef<K>,
    value: Vec<u8>,
    meta: Meta
) -> Result<()>

Similar to write but with metadata

Examples

async fn write_data() {
    let key = 42u64.to_be_bytes().to_vec();
    let data = b"async written to blob".to_vec();
    let meta = Meta::new();
    meta.insert("version".to_string(), b"1.0".to_vec());
    storage.write_with(&key, data, meta).await
}

Errors

Fails if duplicates are not allowed and record already exists.

pub async fn read(&self, key: impl AsRef<K>) -> Result<Vec<u8>>

Reads the first found data matching given key.

Examples

async fn read_data() {
    let key = 42u64.to_be_bytes().to_vec();
    let data = storage.read(key).await;
}

Errors

Same as read_with

pub async fn read_with(
    &self,
    key: impl AsRef<K>,
    meta: &Meta
) -> Result<Vec<u8>>

Reads data matching given key and metadata

Examples

async fn read_data() {
    let key = 42u64.to_be_bytes().to_vec();
    let meta = Meta::new();
    meta.insert("version".to_string(), b"1.0".to_vec());
    let data = storage.read(&key, &meta).await;
}

Errors

Return error if record is not found.

pub async fn read_all(&self, key: impl AsRef<K>) -> Result<Vec<Entry>>

Returns entries with matching key

Errors

Fails after any disk IO errors.

pub async fn close(self) -> Result<()>

Stop blob updater and release lock file

Errors

Fails because of any IO errors

pub async fn blobs_count(&self) -> usize

blob_count returns exact number of closed blobs plus one active, if there is some. It locks on inner structure, so it much slower than next_blob_id.

Examples

use pearl::Builder;

let mut storage = Builder::new().work_dir("/tmp/pearl/").build::<f64>();
storage.init().await;
assert_eq!(storage.blobs_count(), 1);

pub async fn index_memory(&self) -> usize

index_memory returns the amount of memory used by blob to store indices

pub fn next_blob_id(&self) -> usize

Returns next blob ID. If pearl dir structure wasn’t changed from the outside, returned number is equal to blobs_count. But this method doesn’t require lock. So it is much faster than blobs_count.

pub async fn contains(&self, key: K) -> Result<bool>

contains is used to check whether a key is in storage. Slower than check_bloom, because doesn’t prevent disk IO operations. contains returns either “definitely in storage” or “definitely not”.

Errors

Fails because of any IO errors

pub async fn check_bloom(&self, key: impl AsRef<K>) -> Option<bool>

check_bloom is used to check whether a key is in storage. If bloom filter opt out, returns None. Uses bloom filter under the hood, so false positive results are possible, but false negatives are not. In other words, check_bloom returns either “possibly in storage” or “definitely not”.

pub async fn records_count(&self) -> usize

Total records count in storage.

pub async fn records_count_detailed(&self) -> Vec<(usize, usize)>

Records count per blob. Format: (blob_id, count). Last value is from active blob.

pub async fn records_count_in_active_blob(&self) -> Option<usize>

Records count in active blob. Returns None if active blob not set or any IO error occured.

pub async fn fsyncdata(&self) -> IOResult<()>

Syncronizes data and metadata of the active blob with the filesystem. Like tokio::std::fs::File::sync_data, this function will attempt to ensure that in-core data reaches the filesystem before returning. May not syncronize file metadata to the file system.

Errors

Fails because of any IO errors.

pub async fn close_active_blob(&self)

Force closes active blob to dump index on disk and free RAM. Creates new active blob.

Errors

Fails because of any IO errors. Or if there are any problems with syncronization.

Trait Implementations

impl<K: Key> Clone for Storage<K>

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<K: Debug + Key> Debug for Storage<K>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.