Trait StorageHandler 

pub trait StorageHandler: AsAny {
    // Required methods
    fn list_from(
        &self,
        path: &Url,
    ) -> DeltaResult<Box<dyn Iterator<Item = DeltaResult<FileMeta>>>>;
    fn read_files(
        &self,
        files: Vec<FileSlice>,
    ) -> DeltaResult<Box<dyn Iterator<Item = DeltaResult<Bytes>>>>;
    fn copy_atomic(&self, src: &Url, dest: &Url) -> DeltaResult<()>;
    fn put(&self, path: &Url, data: Bytes, overwrite: bool) -> DeltaResult<()>;
    fn head(&self, path: &Url) -> DeltaResult<FileMeta>;
}

Provides file system related functionalities to Delta Kernel.

Delta Kernel uses this handler whenever it needs to access the underlying file system where the Delta table is present. Connector implementation of this trait can hide filesystem specific details from Delta Kernel.

Required Methods

fn list_from( &self, path: &Url, ) -> DeltaResult<Box<dyn Iterator<Item = DeltaResult<FileMeta>>>>

Recursively list files whose full path is lexicographically greater than (UTF-8 sorting) the given path, restricted to descendants of path’s parent directory. The result must be sorted by the full path (UTF-8 byte order).

The listing is recursive: files in nested subdirectories are included, not just files directly under the parent. For example, listing from dir/0001.json may return dir/0002.json, dir/sub/0003.json, and dir/sub/nested/0004.json, all interleaved in lexicographic order.

The parent directory is derived from path:

• If path is directory-like (ends with /), the parent is path itself and the result contains all files at or below that directory.
• Otherwise, the parent is the directory containing path, and only files (at any depth under that parent) whose full path sorts strictly greater than path are returned.

fn read_files( &self, files: Vec<FileSlice>, ) -> DeltaResult<Box<dyn Iterator<Item = DeltaResult<Bytes>>>>

Read data specified by the start and end offset from the file.

fn copy_atomic(&self, src: &Url, dest: &Url) -> DeltaResult<()>

Copy a file atomically from source to destination. If the destination file already exists, it must return Err(Error::FileAlreadyExists).

fn put(&self, path: &Url, data: Bytes, overwrite: bool) -> DeltaResult<()>

Write data to the specified path.

If overwrite is false and the file already exists, this must return Err(Error::FileAlreadyExists).

fn head(&self, path: &Url) -> DeltaResult<FileMeta>

Perform a HEAD request for the given file at a Url, returning the file metadata.

If the file does not exist, this must return an Err with Error::FileNotFound.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl StorageHandler for MeteredStorageHandler

impl StorageHandler for PlanBasedStorageHandler

Available on crate feature declarative-plans and crate feature default-engine-base and (crate features arrow-conversion or declarative-plans or default-engine-base) only.