Skip to main content

split_async

composefs_oci::tar

Function split_async 

Source 
pub async fn split_async<ObjectID: FsVerityHashValue>(
    tar_stream: impl AsyncRead + Unpin,
    repo: Arc<Repository<ObjectID>>,
    content_type: u64,
) -> Result<(ObjectID, ImportStats)>
Expand description

Asynchronously splits a tar archive into a composefs split stream.

Processes the tar stream asynchronously with parallel object storage. Large files are streamed to O_TMPFILE via a channel, and their fs-verity digests are computed in background blocking tasks. This avoids blocking the async runtime while allowing multiple files to be processed concurrently.

Concurrency is limited to available_parallelism() to avoid overwhelming the system with too many concurrent I/O operations.

Files larger than INLINE_CONTENT_MAX_V0 are stored externally in the object store, while smaller files and metadata are stored inline in the split stream.

ยงArguments

  • tar_stream - The async buffered tar stream to read from
  • repo - The repository for creating tmpfiles and storing objects
  • content_type - The content type identifier for the splitstream

Returns the fs-verity object ID of the stored splitstream and import statistics.