Trait fjall::GarbageCollection

source · 
pub trait GarbageCollection {
    // Required methods
    fn gc_scan(&self) -> Result<GcReport>;
    fn gc_with_space_amp_target(&self, factor: f32) -> Result<u64>;
    fn gc_with_staleness_threshold(&self, threshold: f32) -> Result<u64>;
    fn gc_drop_stale_segments(&self) -> Result<u64>;
}
Expand description

Functions for garbage collection strategies

These functions are to be used with a key-value separated partition.

Required Methods§

source

fn gc_scan(&self) -> Result<GcReport>

Collects statistics about blob fragmentation inside the partition.

§Errors

Will return Err if an IO error occurs.

§Panics

Panics if the partition is not KV-separated.

source

fn gc_with_space_amp_target(&self, factor: f32) -> Result<u64>

Rewrites blobs in order to achieve the given space amplification factor.

§Examples
let opts = PartitionCreateOptions::default().with_kv_separation(Default::default());
let blobs = keyspace.open_partition("my_blobs", opts)?;

blobs.insert("a", "hello".repeat(1_000))?;
blobs.insert("b", "hello".repeat(1_000))?;
blobs.insert("c", "hello".repeat(1_000))?;
blobs.insert("d", "hello".repeat(1_000))?;
blobs.insert("e", "hello".repeat(1_000))?;
blobs.remove("a")?;
blobs.remove("b")?;
blobs.remove("c")?;
blobs.remove("d")?;

let report = blobs.gc_scan()?;
assert_eq!(0.8, report.stale_ratio());
assert_eq!(5.0, report.space_amp());
assert_eq!(5, report.total_blobs);
assert_eq!(4, report.stale_blobs);
assert_eq!(0, report.stale_segment_count);
assert_eq!(1, report.segment_count);

let bytes_freed = blobs.gc_with_space_amp_target(1.5)?;
assert!(bytes_freed > 0);

let report = blobs.gc_scan()?;
assert_eq!(0.0, report.stale_ratio());
assert_eq!(1.0, report.space_amp());
assert_eq!(1, report.total_blobs);
assert_eq!(0, report.stale_blobs);
assert_eq!(0, report.stale_segment_count);
assert_eq!(1, report.segment_count);
§Errors

Will return Err if an IO error occurs.

§Panics

Panics if the partition is not KV-separated.

source

fn gc_with_staleness_threshold(&self, threshold: f32) -> Result<u64>

Rewrites blobs that have reached a given staleness threshold.

§Examples
let opts = PartitionCreateOptions::default().with_kv_separation(Default::default());
let blobs = keyspace.open_partition("my_blobs", opts)?;

blobs.insert("a", "hello".repeat(1_000))?;
blobs.insert("b", "hello".repeat(1_000))?;
blobs.insert("c", "hello".repeat(1_000))?;
blobs.insert("d", "hello".repeat(1_000))?;
blobs.insert("e", "hello".repeat(1_000))?;
blobs.remove("a")?;
blobs.remove("b")?;
blobs.remove("c")?;
blobs.remove("d")?;

let report = blobs.gc_scan()?;
assert_eq!(0.8, report.stale_ratio());
assert_eq!(5.0, report.space_amp());
assert_eq!(5, report.total_blobs);
assert_eq!(4, report.stale_blobs);
assert_eq!(0, report.stale_segment_count);
assert_eq!(1, report.segment_count);

let bytes_freed = blobs.gc_with_staleness_threshold(0.5)?;
assert!(bytes_freed > 0);

let report = blobs.gc_scan()?;
assert_eq!(0.0, report.stale_ratio());
assert_eq!(1.0, report.space_amp());
assert_eq!(1, report.total_blobs);
assert_eq!(0, report.stale_blobs);
assert_eq!(0, report.stale_segment_count);
assert_eq!(1, report.segment_count);
§Errors

Will return Err if an IO error occurs.

§Panics

Panics if the partition is not KV-separated.

Panics if the threshold is negative.

Values above 1.0 will be treated as 1.0. If you want to drop only fully stale segments, use [GarbageCollector::drop_stale_segments] instead.

source

fn gc_drop_stale_segments(&self) -> Result<u64>

Drops fully stale segments.

This is called implicitly by other garbage collection strategies.

§Examples
let opts = PartitionCreateOptions::default().with_kv_separation(Default::default());
let blobs = keyspace.open_partition("my_blobs", opts)?;

blobs.insert("a", "hello".repeat(1_000))?;
assert!(blobs.contains_key("a")?);

blobs.remove("a")?;
assert!(!blobs.contains_key("a")?);

let report = blobs.gc_scan()?;
assert_eq!(1.0, report.stale_ratio());
assert_eq!(1, report.stale_blobs);
assert_eq!(1, report.stale_segment_count);
assert_eq!(1, report.segment_count);

let bytes_freed = blobs.gc_drop_stale_segments()?;
assert!(bytes_freed > 0);

let report = blobs.gc_scan()?;
assert_eq!(0.0, report.stale_ratio());
assert_eq!(0, report.stale_blobs);
assert_eq!(0, report.stale_segment_count);
assert_eq!(0, report.segment_count);
§Errors

Will return Err if an IO error occurs.

§Panics

Panics if the partition is not KV-separated.

Implementors§