Skip to main content

CheckpointManager

diskann_disk::build::chunking::checkpoint

Trait CheckpointManager 

Source 
pub trait CheckpointManager:
    Send
    + Sync
    + CheckpointManagerClone {
    // Required methods
    fn get_resumption_point(&self, stage: WorkStage) -> ANNResult<Option<usize>>;
    fn update(
        &mut self,
        progress: Progress,
        next_stage: WorkStage,
    ) -> ANNResult<()>;
    fn mark_as_invalid(&mut self) -> ANNResult<()>;
}
Expand description

This trait provides functionalities to get and set checkpoint records ..for tracking the progress and state in a chunkable index build process. it needs to be marked as send and sync because it will be used in a multi-threaded environment. However, during the index build process DiskANN will not have parallel requests trying to access the checkpoint record.

Required Methods§

Source

fn get_resumption_point(&self, stage: WorkStage) -> ANNResult<Option<usize>>

Gets the resumption point for a given work stage.

Returns the offset where processing should resume for the specified stage. Returns None if:

  • No checkpoint exists for the stage
  • The stage has already completed
Source

fn update(&mut self, progress: Progress, next_stage: WorkStage) -> ANNResult<()>

Updates the checkpoint record with progress information

§Arguments
  • progress - The current progress (Completed or Processed amount)
  • next_stage - If provided and progress is Completed, advances to this stage.
Source

fn mark_as_invalid(&mut self) -> ANNResult<()>

Marks the checkpoint as invalid for current stage.

When a checkpoint is marked as invalid:

  • Future calls to get_resumption_point(curent_stage) will return offset 0
  • This forces processing to restart from the beginning of the stage
  • Protects against partial/incomplete work if a crash occurs

Implementors§