pub trait ObjectStore:
Display
+ Send
+ Sync
+ Debug
+ 'static {
// Required methods
fn put_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
payload: PutPayload,
opts: PutOptions,
) -> Pin<Box<dyn Future<Output = Result<PutResult, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait;
fn put_multipart_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
opts: PutMultipartOptions,
) -> Pin<Box<dyn Future<Output = Result<Box<dyn MultipartUpload>, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait;
fn get_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: GetOptions,
) -> Pin<Box<dyn Future<Output = Result<GetResult, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait;
fn delete_stream(
&self,
locations: Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>,
) -> Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>;
fn list(
&self,
prefix: Option<&Path>,
) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>;
fn list_with_delimiter<'life0, 'life1, 'async_trait>(
&'life0 self,
prefix: Option<&'life1 Path>,
) -> Pin<Box<dyn Future<Output = Result<ListResult, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait;
fn copy_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: CopyOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait;
// Provided methods
fn get_ranges<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
location: &'life1 Path,
ranges: &'life2 [Range<u64>],
) -> Pin<Box<dyn Future<Output = Result<Vec<Bytes>, Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait { ... }
fn list_with_offset(
&self,
prefix: Option<&Path>,
offset: &Path,
) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>> { ... }
fn rename_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: RenameOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>
where 'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait { ... }
}Expand description
The deltalake crate is currently just a meta-package shim for deltalake-core Universal API for object store services.
See the module-level documentation for a high level overview and
examples. See ObjectStoreExt for additional convenience methods.
§Contract
This trait is a contract between object store implementations
(e.g. providers, wrappers) and the object_store crate itself. It is
intended to be the minimum API required for an object store.
The ObjectStoreExt acts as an API/contract between object_store and
the store users and provides additional methods that may be simpler to use
but overlap in functionality with ObjectStore.
§Clone
If a store implements Clone, that will only clone the handle to the underlying data. It will NOT clone/fork the
actual key-value data. Hence, the cloned instance and the original instance share the same state.
§Minimal Default Implementations
There are only a few default implementations for methods in this trait by
design. This was different from versions prior to 0.13.0, which had many
more default implementations. Default implementations are convenient for
users, but error-prone for implementors as they require keeping the
convenience APIs correctly in sync.
As of version 0.13.0, most methods on ObjectStore must be implemented, and
the convenience methods have been moved to the ObjectStoreExt trait as
described above. See #385 for more details.
§Wrappers
If you wrap an ObjectStore – e.g. to add observability – you SHOULD
implement all trait methods. This ensures that defaults implementations
that are overwritten by the wrapped store are also used by the wrapper.
For example:
struct MyStore {
...
}
#[async_trait]
impl ObjectStore for MyStore {
// implement custom ranges handling
async fn get_ranges(
&self,
location: &Path,
ranges: &[Range<u64>],
) -> Result<Vec<Bytes>> {
...
}
...
}
struct Wrapper {
inner: Arc<dyn ObjectStore>,
}
#[async_trait]
#[deny(clippy::missing_trait_methods)]
impl ObjectStore for Wrapper {
// If we would not implement this method,
// we would get the trait default and not
// use the actual implementation of `inner`.
async fn get_ranges(
&self,
location: &Path,
ranges: &[Range<u64>],
) -> Result<Vec<Bytes>> {
...
}
...
}To automatically detect this issue, use
#[deny(clippy::missing_trait_methods)].
§Upgrade Guide for 0.13.0
Upgrading to object_store 0.13.0 from an earlier version typically involves:
- Add a
useforObjectStoreExtto solve the error
error[E0599]: no method named `put` found for reference `&dyn object_store::ObjectStore` in the current scope
--> datafusion/datasource/src/url.rs:993:14- Remove any (now) redundant implementations (such as
ObjectStore::put) from anyObjectStoreimplementations to resolve the error
error[E0407]: method `put` is not a member of trait `ObjectStore`
--> datafusion/datasource/src/url.rs:1103:9
|-
Convert
ObjectStore::deletetoObjectStore::delete_stream(see documentation on that method for details and examples) -
Combine
ObjectStore::copyandObjectStore::copy_if_not_existsimplementations intoObjectStore::copy_opts(see documentation on that method for details and examples) -
Update
object_store::Error::NotImplementedto include the name of the missing method
For example, change instances of
object_store::Error::NotImplementedto
object_store::Error::NotImplemented {
operation: "put".to_string(),
implementer: "RequestCountingObjectStore".to_string(),
};Required Methods§
Sourcefn put_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
payload: PutPayload,
opts: PutOptions,
) -> Pin<Box<dyn Future<Output = Result<PutResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
fn put_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
payload: PutPayload,
opts: PutOptions,
) -> Pin<Box<dyn Future<Output = Result<PutResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
Save the provided payload to location with the given options
The operation is guaranteed to be atomic, it will either successfully
write the entirety of payload to location, or fail. No clients
should be able to observe a partially written object
Sourcefn put_multipart_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
opts: PutMultipartOptions,
) -> Pin<Box<dyn Future<Output = Result<Box<dyn MultipartUpload>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
fn put_multipart_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
opts: PutMultipartOptions,
) -> Pin<Box<dyn Future<Output = Result<Box<dyn MultipartUpload>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
Perform a multipart upload with options
Client should prefer ObjectStore::put_opts for small payloads, as streaming uploads
typically require multiple separate requests. See MultipartUpload for more information
For more advanced multipart uploads see MultipartStore
Sourcefn get_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: GetOptions,
) -> Pin<Box<dyn Future<Output = Result<GetResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
fn get_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: GetOptions,
) -> Pin<Box<dyn Future<Output = Result<GetResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
Perform a get request with options
§Example
This example uses a basic local filesystem object store to get an object with a specific etag. On the local filesystem, supplying an invalid etag will error. Versioned object stores will return the specified object version, if it exists.
async fn get_opts_example() {
let tmp = tempdir().unwrap();
let store = LocalFileSystem::new_with_prefix(tmp.path()).unwrap();
let location = Path::from("example.txt");
let content = b"Hello, Object Store!";
// Put the object into the store
store
.put(&location, content.as_ref().into())
.await
.expect("Failed to put object");
// Get the object from the store to figure out the right etag
let result: object_store::GetResult = store.get(&location).await.expect("Failed to get object");
let etag = result.meta.e_tag.expect("ETag should be present");
// Get the object from the store with range and etag
let bytes = store
.get_opts(
&location,
GetOptions::new()
.with_if_match(Some(etag.clone())),
)
.await
.expect("Failed to get object with range and etag")
.bytes()
.await
.expect("Failed to read bytes");
println!(
"Retrieved with ETag {}: {}",
etag,
String::from_utf8_lossy(&bytes)
);
// Show that if the etag does not match, we get an error
let wrong_etag = "wrong-etag".to_string();
match store
.get_opts(
&location,
GetOptions::new().with_if_match(Some(wrong_etag))
)
.await
{
Ok(_) => println!("Unexpectedly succeeded with wrong ETag"),
Err(e) => println!("On a non-versioned object store, getting an invalid ETag ('wrong-etag') results in an error as expected: {}", e),
}
}To retrieve a range of bytes from a versioned object, specify the range in the GetOptions supplied to this method.
async fn get_opts_range_example() {
let tmp = tempdir().unwrap();
let store = LocalFileSystem::new_with_prefix(tmp.path()).unwrap();
let location = Path::from("example.txt");
let content = b"Hello, Object Store!";
// Put the object into the store
store
.put(&location, content.as_ref().into())
.await
.expect("Failed to put object");
// Get the object from the store to figure out the right etag
let result: object_store::GetResult = store.get(&location).await.expect("Failed to get object");
let etag = result.meta.e_tag.expect("ETag should be present");
// Get the object from the store with range and etag
let bytes = store
.get_opts(
&location,
GetOptions::new()
.with_range(Some(0..5))
.with_if_match(Some(etag.clone())),
)
.await
.expect("Failed to get object with range and etag")
.bytes()
.await
.expect("Failed to read bytes");
println!(
"Retrieved range [0-5] with ETag {}: {}",
etag,
String::from_utf8_lossy(&bytes)
);
// Show that if the etag does not match, we get an error
let wrong_etag = "wrong-etag".to_string();
match store
.get_opts(
&location,
GetOptions::new().with_range(Some(0..5)).with_if_match(Some(wrong_etag))
)
.await
{
Ok(_) => println!("Unexpectedly succeeded with wrong ETag"),
Err(e) => println!("On a non-versioned object store, getting an invalid ETag ('wrong-etag') results in an error as expected: {}", e),
}
}Sourcefn delete_stream(
&self,
locations: Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>,
) -> Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>
fn delete_stream( &self, locations: Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>, ) -> Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>
Delete all the objects at the specified locations
When supported, this method will use bulk operations that delete more than one object per a request. Otherwise, the implementation may call the single object delete method for each location.
§Bulk Delete Support
The following backends support native bulk delete operations:
- AWS (S3): Uses the native DeleteObjects API with batches of up to 1000 objects
- Azure: Uses the native Blob Batch API with batches of up to 256 objects
The following backends use concurrent individual delete operations:
- GCP: Performs individual delete requests with up to 10 concurrent operations
- HTTP: Performs individual delete requests with up to 10 concurrent operations
- Local: Performs individual file deletions with up to 10 concurrent operations
- Memory: Performs individual in-memory deletions sequentially
The returned stream yields the results of the delete operations in the same order as the input locations. However, some errors will be from an overall call to a bulk delete operation, and not from a specific location.
If the object did not exist, the result may be an error or a success,
depending on the behavior of the underlying store. For example, local
filesystems, GCP, and Azure return an error, while S3 and in-memory will
return Ok. If it is an error, it will be Error::NotFound.
// Create two objects
store.put(&Path::from("foo"), "foo".into()).await?;
store.put(&Path::from("bar"), "bar".into()).await?;
// List object
let locations = store.list(None).map_ok(|m| m.location).boxed();
// Delete them
store.delete_stream(locations).try_collect::<Vec<Path>>().await?;Note: Before version 0.13, delete_stream has a default implementation
that deletes each object with up to 10 concurrent requests. This default
behavior has been removed, and each implementation must now provide its
own delete_stream implementation explicitly. The following example
shows how to implement delete_stream to get the previous default
behavior.
fn delete_stream(
&self,
locations: BoxStream<'static, Result<Path>>,
) -> BoxStream<'static, Result<Path>> {
let client = Arc::clone(&self.client);
locations
.map(move |location| {
let client = Arc::clone(&client);
async move {
let location = location?;
client.delete(&location).await?;
Ok(location)
}
})
.buffered(10)
.boxed()
}Sourcefn list(
&self,
prefix: Option<&Path>,
) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
fn list( &self, prefix: Option<&Path>, ) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
List all the objects with the given prefix.
Prefixes are evaluated on a path segment basis, i.e. foo/bar is a prefix of foo/bar/x but not of
foo/bar_baz/x. List is recursive, i.e. foo/bar/more/x will be included.
Note: the order of returned ObjectMeta is not guaranteed
For more advanced listing see PaginatedListStore
Sourcefn list_with_delimiter<'life0, 'life1, 'async_trait>(
&'life0 self,
prefix: Option<&'life1 Path>,
) -> Pin<Box<dyn Future<Output = Result<ListResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
fn list_with_delimiter<'life0, 'life1, 'async_trait>(
&'life0 self,
prefix: Option<&'life1 Path>,
) -> Pin<Box<dyn Future<Output = Result<ListResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
Self: 'async_trait,
List objects with the given prefix and an implementation specific delimiter. Returns common prefixes (directories) in addition to object metadata.
Prefixes are evaluated on a path segment basis, i.e. foo/bar is a prefix of foo/bar/x but not of
foo/bar_baz/x. List is not recursive, i.e. foo/bar/more/x will not be included.
Sourcefn copy_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: CopyOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait,
fn copy_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: CopyOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait,
Copy an object from one path to another in the same object store.
Provided Methods§
Sourcefn get_ranges<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
location: &'life1 Path,
ranges: &'life2 [Range<u64>],
) -> Pin<Box<dyn Future<Output = Result<Vec<Bytes>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait,
fn get_ranges<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
location: &'life1 Path,
ranges: &'life2 [Range<u64>],
) -> Pin<Box<dyn Future<Output = Result<Vec<Bytes>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait,
Return the bytes that are stored at the specified location in the given byte ranges
Sourcefn list_with_offset(
&self,
prefix: Option<&Path>,
offset: &Path,
) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
fn list_with_offset( &self, prefix: Option<&Path>, offset: &Path, ) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
List all the objects with the given prefix and a location greater than offset
Some stores, such as S3 and GCS, may be able to push offset down to reduce
the number of network requests required.
This returns an exclusive offset, i.e. objects at exactly offset will not be included.
Note: the order of returned ObjectMeta is not guaranteed
For more advanced listing see PaginatedListStore
Sourcefn rename_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: RenameOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait,
fn rename_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: RenameOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Self: 'async_trait,
Move an object from one path to another in the same object store.
By default, this is implemented as a copy and then delete source. It may not check when deleting source that it was the same object that was originally copied.
Implementations on Foreign Types§
Source§impl ObjectStore for S3StorageBackend
impl ObjectStore for S3StorageBackend
fn put_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
bytes: PutPayload,
options: PutOptions,
) -> Pin<Box<dyn Future<Output = Result<PutResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
S3StorageBackend: 'async_trait,
fn put_multipart_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: PutMultipartOptions,
) -> Pin<Box<dyn Future<Output = Result<Box<dyn MultipartUpload>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
S3StorageBackend: 'async_trait,
fn get_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: GetOptions,
) -> Pin<Box<dyn Future<Output = Result<GetResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
S3StorageBackend: 'async_trait,
fn get_ranges<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
location: &'life1 Path,
ranges: &'life2 [Range<u64>],
) -> Pin<Box<dyn Future<Output = Result<Vec<Bytes>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
S3StorageBackend: 'async_trait,
fn delete_stream( &self, locations: Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>, ) -> Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>
fn list( &self, prefix: Option<&Path>, ) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
fn list_with_offset( &self, prefix: Option<&Path>, offset: &Path, ) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
fn list_with_delimiter<'life0, 'life1, 'async_trait>(
&'life0 self,
prefix: Option<&'life1 Path>,
) -> Pin<Box<dyn Future<Output = Result<ListResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
S3StorageBackend: 'async_trait,
fn copy_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: CopyOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
S3StorageBackend: 'async_trait,
fn rename_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: RenameOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
S3StorageBackend: 'async_trait,
Source§impl ObjectStore for HdfsObjectStore
impl ObjectStore for HdfsObjectStore
Source§fn put_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
payload: PutPayload,
opts: PutOptions,
) -> Pin<Box<dyn Future<Output = Result<PutResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
fn put_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
payload: PutPayload,
opts: PutOptions,
) -> Pin<Box<dyn Future<Output = Result<PutResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
Save the provided bytes to the specified location
To make the operation atomic, we write to a temporary file .{filename}.tmp.{i} and rename
on a successful write, where i is an integer that is incremented until a non-existent file
is found.
Source§fn put_multipart_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
_opts: PutMultipartOptions,
) -> Pin<Box<dyn Future<Output = Result<Box<dyn MultipartUpload>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
fn put_multipart_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
_opts: PutMultipartOptions,
) -> Pin<Box<dyn Future<Output = Result<Box<dyn MultipartUpload>, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
Create a multipart writer that writes to a temporary file in a background task, and renames to the final destination on complete.
Source§fn get_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: GetOptions,
) -> Pin<Box<dyn Future<Output = Result<GetResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
fn get_opts<'life0, 'life1, 'async_trait>(
&'life0 self,
location: &'life1 Path,
options: GetOptions,
) -> Pin<Box<dyn Future<Output = Result<GetResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
Reads data for the specified location.
Source§fn delete_stream(
&self,
locations: Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>,
) -> Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>
fn delete_stream( &self, locations: Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>, ) -> Pin<Box<dyn Stream<Item = Result<Path, Error>> + Send>>
Delete a stream of objects.
Source§fn list(
&self,
prefix: Option<&Path>,
) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
fn list( &self, prefix: Option<&Path>, ) -> Pin<Box<dyn Stream<Item = Result<ObjectMeta, Error>> + Send>>
List all the objects with the given prefix.
Prefixes are evaluated on a path segment basis, i.e. foo/bar/ is a prefix of foo/bar/x but not of
foo/bar_baz/x.
Note: the order of returned ObjectMeta is not guaranteed
Source§fn list_with_delimiter<'life0, 'life1, 'async_trait>(
&'life0 self,
prefix: Option<&'life1 Path>,
) -> Pin<Box<dyn Future<Output = Result<ListResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
fn list_with_delimiter<'life0, 'life1, 'async_trait>(
&'life0 self,
prefix: Option<&'life1 Path>,
) -> Pin<Box<dyn Future<Output = Result<ListResult, Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
HdfsObjectStore: 'async_trait,
List objects with the given prefix and an implementation specific delimiter. Returns common prefixes (directories) in addition to object metadata.
Prefixes are evaluated on a path segment basis, i.e. foo/bar/ is a prefix of foo/bar/x but not of
foo/bar_baz/x.
Source§fn copy_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: CopyOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
HdfsObjectStore: 'async_trait,
fn copy_opts<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
from: &'life1 Path,
to: &'life2 Path,
options: CopyOptions,
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
HdfsObjectStore: 'async_trait,
Copy an object from one path to another with options.