Skip to main content

Compactor

Struct Compactor 

Source
pub struct Compactor {
    pub base_path: PathBuf,
    pub write_budget_bytes: u64,
    pub cohort_window_secs: u64,
}
Expand description

Drives deathtime-cohort compaction for an EdgeStore database.

The compactor groups segments by their cohort bucket, waits until all records in a cohort are past their death time, then rewrites only the live records into new segments (removing dead ones). No in-place writes are performed; all output is append-oriented.

Fields§

§base_path: PathBuf

Base directory of the EdgeStore database being compacted.

§write_budget_bytes: u64

Maximum bytes the compactor may write per compact_cycle call.

§cohort_window_secs: u64

Cohort window width in seconds (matches EdgestoreConfig::cohort_window_secs).

Implementations§

Source§

impl Compactor

Source

pub fn new( base_path: PathBuf, write_budget_bytes: u64, cohort_window_secs: u64, ) -> Self

Create a new Compactor.

base_path — database directory. write_budget_bytes— write-amplification cap per compaction pass. cohort_window_secs— must match the value used when segments were written.

Source

pub fn identify_cohorts( segments: &[SegmentMeta], now_nanos: i64, ) -> Vec<CohortInfo>

Group segments into cohorts and sort them for compaction priority.

Fully-expired cohorts (now_nanos > max_death_time_nanos) come first, ordered by lowest max_death_time_nanos. Partially-expired cohorts follow.

Source

pub fn collect_expired_cohort( &self, manifest: &mut Manifest, cohort: &CohortInfo, stats: &mut CompactionStats, ) -> Result<(), EdgestoreError>

Remove all segment files and manifest entries for a fully-expired cohort.

Zero live records are relocated (all were expired). Missing files are logged and skipped — not treated as errors. Caller is responsible for ensuring no pinned segments are in the cohort. Delete all files belonging to an expired cohort.

Source

pub fn compact_partial_cohort( &self, manifest: &mut Manifest, cohort: &CohortInfo, now_nanos: i64, next_segment_id: SegmentId, stats: &mut CompactionStats, ) -> Result<(), EdgestoreError>

Compact a partially-expired cohort: read all entries, filter dead ones, write survivors to a new segment, remove old segments.

If no entries survive, the cohort is treated as fully expired (files deleted, no output segment written). Rewrite a partially-expired cohort, keeping only live entries.

Source

pub fn compact_cycle( &self, manifest: &mut Manifest, now_nanos: i64, pinned_segment_ids: &HashSet<SegmentId>, ) -> Result<CompactionStats, EdgestoreError>

Run one full compaction cycle against the manifest.

  • Fully-expired cohorts are collected first (zero live relocation).
  • Partially-expired cohorts are compacted by removing dead records.
  • Stops when bytes_written >= write_budget_bytes.
  • Segments in pinned_segment_ids are never removed or rewritten.

Run a full compaction cycle over all cohorts.

Trait Implementations§

Source§

impl Debug for Compactor

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V