Struct FileUploadSession 

pub struct FileUploadSession { /* private fields */ }

Manages the translation of files between the MerkleDB / pointer file format and the materialized version.

This class handles the clean operations. It’s meant to be a single atomic session that succeeds or fails as a unit; i.e. all files get uploaded on finalization, and all shards and xorbs needed to reconstruct those files are properly uploaded and registered.

Implementations

impl FileUploadSession

pub async fn new( config: Arc<TranslatorConfig>, ) -> Result<Arc<FileUploadSession>>

pub async fn dry_run( config: Arc<TranslatorConfig>, ) -> Result<Arc<FileUploadSession>>

pub async fn upload_files( self: &Arc<Self>, files_and_sha256: impl IntoIterator<Item = (impl AsRef<Path>, Sha256Policy)> + Send, ) -> Result<Vec<XetFileInfo>>

pub fn start_clean( self: &Arc<Self>, tracking_name: Option<Arc<str>>, size: Option<u64>, sha256: Sha256Policy, ) -> Result<(UniqueID, SingleFileCleaner)>

Start to clean one file. When cleaning multiple files, each file should be associated with one Cleaner. This allows to launch multiple clean task simultaneously.

The caller is responsible for memory usage management, the parameter “buffer_size” indicates the maximum number of Vec in the internal buffer.

If a sha256 is provided via Sha256Policy::Provided, the value will be directly used in shard upload to avoid redundant computation. Sha256Policy::Skip skips SHA-256 computation entirely and no metadata_ext is included in the shard.

pub async fn spawn_upload_from_path( self: &Arc<Self>, file_path: PathBuf, sha256: Sha256Policy, ) -> Result<(UniqueID, JoinHandle<Result<(XetFileInfo, DeduplicationMetrics)>>)>

Spawns a task that reads file_path and uploads it.

Returns the tracking ID and a join handle for the spawned task.

pub async fn spawn_upload_bytes( self: &Arc<Self>, bytes: Vec<u8>, sha256: Sha256Policy, tracking_name: Option<Arc<str>>, ) -> Result<(UniqueID, JoinHandle<Result<(XetFileInfo, DeduplicationMetrics)>>)>

Spawns a task that uploads bytes as a single file.

Returns the tracking ID and a join handle for the spawned task.

pub async fn checkpoint(self: &Arc<Self>) -> Result<()>

pub fn progress(&self) -> &Arc<GroupProgress>

pub fn report(&self) -> GroupProgressReport

pub fn item_report(&self, id: UniqueID) -> Option<ItemProgressReport>

pub fn item_reports(&self) -> HashMap<UniqueID, ItemProgressReport>

pub async fn finalize(self: Arc<Self>) -> Result<DeduplicationMetrics>

pub async fn finalize_with_report( self: Arc<Self>, ) -> Result<(DeduplicationMetrics, GroupProgressReport)>

pub async fn finalize_with_file_info( self: Arc<Self>, ) -> Result<(DeduplicationMetrics, Vec<MDBFileInfo>)>