Skip to main content

Crate shardline_gc

Crate shardline_gc 

Source
Expand description

Garbage collection for Shardline chunk storage.

This crate provides local filesystem garbage collection including orphan chunk discovery, quarantine lifecycle management, and retention window enforcement.

§Quick start

GC is driven by a LocalGcOptions mode selection and reports back through LocalGcReport. The options and report types are pure and safe to build directly:

use shardline_gc::{
    LocalGcDiagnostics, LocalGcOptions, LocalGcReport, MINIMUM_GC_RETENTION_SECONDS,
};

// Default options are a no-op dry run: nothing is marked or deleted.
let options = LocalGcOptions::default();
assert_eq!(options.mode_name(), "dry-run");
assert!(!options.mark && !options.sweep);

// A real cleanup run marks orphans and sweeps expired quarantine entries,
// with a retention window that protects in-flight uploads.
let options = LocalGcOptions::mark_and_sweep(MINIMUM_GC_RETENTION_SECONDS);
assert_eq!(options.mode_name(), "mark-and-sweep");

// Reports and diagnostics are plain data you can inspect after a run.
let report = LocalGcReport::default();
assert_eq!(report.deleted_chunks, 0);
let diagnostics = LocalGcDiagnostics {
    report,
    retention_report: Vec::new(),
    orphan_inventory: Vec::new(),
};
assert!(diagnostics.orphan_inventory.is_empty());

To run GC against real storage, drive run_gc_with_stores with explicit record, index, and object-store adapters (the shardline-server crate wires this up from its own configuration).

Structs§

GcOrphanInventoryEntry
One currently orphaned object after a GC run.
GcRetentionReportEntry
One active retention-window entry after a GC run.
LocalGcDiagnostics
Detailed GC diagnostics intended for operators and automation.
LocalGcOptions
Local filesystem garbage-collection execution options.
LocalGcReport
Local filesystem garbage-collection report.
LocalRecordStore
Re-exported index types for convenience. Local SQLite implementation of the record-store contract.
ServerFrontendParseError
Invalid server frontend token.

Enums§

GcError
Garbage collection runtime failure.
GcOrphanQuarantineState
Quarantine state for one orphaned object.
ServerFrontend
Runtime protocol frontend selection.
ServerFrontendKind
Runtime protocol frontend selection.

Constants§

DEFAULT_LOCAL_GC_RETENTION_SECONDS
Default retention window for new local quarantine candidates. Default retention window for new local quarantine candidates.
MINIMUM_GC_RETENTION_SECONDS
Minimum retention window in seconds for GC quarantine entries.

Functions§

quarantine_record_path
Returns the quarantine record path for the given hash.
quarantine_root
Returns the quarantine root directory for the given storage root.
run_gc_with_oci_tombstones
Runs GC with OCI tombstone reclamation enabled.
run_gc_with_stores
Runs garbage collection against provided record, index, and object stores.
run_local_gc
Runs local filesystem garbage collection.
run_local_gc_diagnostics
Runs local filesystem garbage collection and returns operator diagnostics.
run_postgres_gc_with_remote_delete_protection
Runs Postgres GC with durable admission around S3 object deletes.