Skip to main content

upload_ranges

xet_data::processing::range_upload

Function upload_ranges 

Source 
pub async fn upload_ranges(
    config: Arc<TranslatorConfig>,
    cas_client: Arc<dyn Client>,
    original_hash: MerkleHash,
    original_size: u64,
    dirty_inputs: Vec<DirtyInput>,
) -> Result<XetFileInfo>
Expand description

Upload an edited version of an existing file, reusing the unchanged regions from the original file’s CAS segments and only re-uploading the parts the caller actually rewrites.

dirty_inputs is a list of edits expressed in the original file’s coordinates. Each edit replaces original_range with new_length bytes from its reader; resize edits (including pure inserts and pure deletes) are supported. The output file size is derived from the inputs.

§When to use

  • In-place edit: original_range.len() == new_length, file size unchanged.
  • Resize edit: any new_length. Replaces original_range.len() original bytes with new_length new bytes.
  • Pure insert: original_range.start == original_range.end, new_length > 0.
  • Pure delete: original_range.start < original_range.end, new_length == 0.
  • Append: original_range == original_size..original_size, new_length > 0.
  • Truncate to N: original_range == N..original_size, new_length == 0.
  • No change: empty dirty_inputs. Returns the original hash without any CAS call.

§Arguments

  • config - Translator configuration for creating upload sessions.
  • cas_client - CAS client for fetching original file metadata and downloading boundary bytes.
  • original_hash - Merkle hash of the original file in CAS.
  • original_size - Size of the original file in bytes.
  • dirty_inputs - Edits to apply. Must be sorted by original_range.start and non-overlapping (prev.original_range.end <= next.original_range.start). Each reader must yield exactly new_length bytes. Each reader is consumed exactly once.

§Limitations

The composed file has no SHA-256 metadata (metadata_ext = None), since recomputing it would require reading the full file. This means upload_ranges is only suitable for contexts that don’t require SHA-256 verification (e.g. HF buckets, xet-native repos), not for Git LFS-backed repos that verify SHA-256 on download.