Struct CheckpointManager 

pub struct CheckpointManager {
    pub interval_secs: f64,
    /* private fields */
}

Handles saving and loading crawler checkpoints to disk.

You do not create this directly; the CrawlerEngine manages it internally. It writes to <crawldir>/checkpoint.json and supports configurable save intervals so you can trade off between checkpoint frequency and disk I/O overhead.

Fields

interval_secs: f64

The minimum interval in seconds between automatic checkpoint saves.

Implementations

impl CheckpointManager

pub fn new(crawldir: impl Into<PathBuf>, interval_secs: f64) -> Result<Self>

Creates a new checkpoint manager writing to the given directory.

interval_secs controls how often the engine auto-saves during the crawl loop. A value of 0.0 disables periodic saves (checkpoints are still taken on pause). Returns an error if interval_secs is negative.

pub fn has_checkpoint(&self) -> bool

Returns true if a checkpoint file exists on disk. The engine checks this at startup to decide whether to resume from a previous run or start fresh.

pub fn save(&self, data: &CheckpointData) -> Result<()>

Atomically writes checkpoint data to disk using a temporary file and rename. The two-step write ensures that a crash during serialization cannot corrupt an existing checkpoint. The crawl directory is created automatically if it does not exist.

pub fn load(&self) -> Result<Option<CheckpointData>>

Loads checkpoint data from disk, returning None if no checkpoint exists. If the file exists but cannot be deserialized (e.g., it was corrupted), a warning is logged and None is returned rather than propagating an error, so the crawl can start fresh.

pub fn cleanup(&self) -> Result<()>

Deletes the checkpoint file from disk if it exists. The engine calls this after a crawl completes normally (not paused) to avoid accidentally resuming a finished crawl on the next run.