Struct opendal::Object

source · 
pub struct Object { /* private fields */ }
Expand description

Object is the handler for all object related operations.

Notes

Object will cache part of object metadata that pre-fetch by list or stat operations. It’s better to reuse the same object whenever possible.

Implementations§

source§

impl Object

source

pub fn new(op: Operator, path: &str) -> Self

Creates a new Object with normalized path.

  • All path will be converted into relative path (without any leading /)
  • Path endswith / means it’s a dir path.
  • Otherwise, it’s a file path.
source

pub fn operator(&self) -> Operator

Fetch the operator that used by this object.

source

pub fn id(&self) -> String

ID of object.

ID is the unique id of object in the underlying backend. In different backend, the id could have different meaning.

For example:

  • In fs: id is the absolute path of file, like /path/to/dir/test_object.
  • In s3: id is the full object key, like path/to/dir/test_object
Example
use anyhow::Result;
use futures::io;
use opendal::Operator;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let id = op.object("test").id();

    Ok(())
}
source

pub fn path(&self) -> &str

Path of object. Path is relative to operator’s root. Only valid in current operator.

The value is the same with Metadata::path().

Example
use anyhow::Result;
use futures::io;
use opendal::Operator;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let path = op.object("test").path();

    Ok(())
}
source

pub fn name(&self) -> &str

Name of object. Name is the last segment of path.

If this object is a dir, Name MUST endswith / Otherwise, Name MUST NOT endswith /.

The value is the same with Metadata::name().

Example
use anyhow::Result;
use futures::io;
use opendal::Operator;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let name = op.object("test").name();

    Ok(())
}
source

pub async fn create(&self) -> Result<()>

Create an empty object, like using the following linux commands:

  • touch path/to/file
  • mkdir path/to/dir/
Behavior
  • Create on existing dir will succeed.
  • Create on existing file will overwrite and truncate it.
Examples
Create an empty file
let mut o = op.object("path/to/file");
let _ = o.create().await?;
Create a dir
let mut o = op.object("path/to/dir/");
let _ = o.create().await?;
source

pub fn blocking_create(&self) -> Result<()>

Create an empty object, like using the following linux commands:

  • touch path/to/file
  • mkdir path/to/dir/
Behavior
  • Create on existing dir will succeed.
  • Create on existing file will overwrite and truncate it.
Examples
Create an empty file
let mut o = op.object("path/to/file");
let _ = o.blocking_create()?;
Create a dir
let mut o = op.object("path/to/dir/");
let _ = o.blocking_create()?;
source

pub async fn read(&self) -> Result<Vec<u8>>

Read the whole object into a bytes.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Object::reader

Examples
let mut o = op.object("path/to/file");
let bs = o.read().await?;
source

pub fn blocking_read(&self) -> Result<Vec<u8>>

Read the whole object into a bytes.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Object::blocking_reader

Examples
let bs = o.blocking_read()?;
source

pub async fn range_read(&self, range: impl RangeBounds<u64>) -> Result<Vec<u8>>

Read the specified range of object into a bytes.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Object::range_reader

Notes
  • The returning contnet’s length may be smaller than the range specified.
Examples
let mut o = op.object("path/to/file");
let bs = o.range_read(1024..2048).await?;
source

pub fn blocking_range_read( &self, range: impl RangeBounds<u64> ) -> Result<Vec<u8>>

Read the specified range of object into a bytes.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Object::blocking_range_reader

Examples
let bs = o.blocking_range_read(1024..2048)?;
source

pub async fn reader(&self) -> Result<ObjectReader>

Create a new reader which can read the whole object.

Examples
let o = op.object("path/to/file");
let r = o.reader().await?;
source

pub fn blocking_reader(&self) -> Result<BlockingObjectReader>

Create a new reader which can read the whole object.

Examples
let o = op.object("path/to/file");
let r = o.blocking_reader()?;
source

pub async fn range_reader( &self, range: impl RangeBounds<u64> ) -> Result<ObjectReader>

Create a new reader which can read the specified range.

Notes
  • The returning contnet’s length may be smaller than the range specified.
Examples
let o = op.object("path/to/file");
let r = o.range_reader(1024..2048).await?;
source

pub fn blocking_range_reader( &self, range: impl RangeBounds<u64> ) -> Result<BlockingObjectReader>

Create a new reader which can read the specified range.

Examples
let o = op.object("path/to/file");
let r = o.blocking_range_reader(1024..2048)?;
source

pub async fn decompress_read(&self) -> Result<Option<Vec<u8>>>

Read the whole object into a bytes with auto detected compress algorithm.

If we can’t find the correct algorithm, we return Ok(None) instead.

Feature

This function needs to enable feature compress.

Examples
let o = op.object("path/to/file.gz");
let bs = o.decompress_read().await?.expect("must read succeed");
source

pub async fn decompress_reader(&self) -> Result<Option<impl Read>>

Create a reader with auto-detected compress algorithm.

If we can’t find the correct algorithm, we will return Ok(None).

Feature

This function needs to enable feature compress.

Examples
let o = op.object("path/to/file.gz");
let r = o.decompress_reader().await?;
source

pub async fn decompress_read_with( &self, algo: CompressAlgorithm ) -> Result<Vec<u8>>

Read the whole object into a bytes with specific compress algorithm.

Feature

This function needs to enable feature compress.

Examples
let o = op.object("path/to/file.gz");
let bs = o.decompress_read_with(CompressAlgorithm::Gzip).await?;
source

pub async fn decompress_reader_with( &self, algo: CompressAlgorithm ) -> Result<impl Read>

Create a reader with specific compress algorithm.

Feature

This function needs to enable feature compress.

Examples
let o = op.object("path/to/file.gz");
let r = o.decompress_reader_with(CompressAlgorithm::Gzip).await?;
source

pub async fn write(&self, bs: impl Into<Vec<u8>>) -> Result<()>

Write bytes into object.

Notes
  • Write will make sure all bytes has been written, or an error will be returned.
Examples
use bytes::Bytes;

let mut o = op.object("path/to/file");
let _ = o.write(vec![0; 4096]).await?;
source

pub async fn write_with( &self, args: OpWrite, bs: impl Into<Vec<u8>> ) -> Result<()>

Write data with option described in OpenDAL rfc-0661

Notes
  • Write will make sure all bytes has been written, or an error will be returned.
Examples
use bytes::Bytes;
use opendal::ops::OpWrite;

let mut o = op.object("path/to/file");
let bs = b"hello, world!".to_vec();
let args = OpWrite::new(bs.len() as u64).with_content_type("text/plain");
let _ = o.write_with(args, bs).await?;
source

pub fn blocking_write(&self, bs: impl Into<Vec<u8>>) -> Result<()>

Write bytes into object.

Notes
  • Write will make sure all bytes has been written, or an error will be returned.
Examples
use bytes::Bytes;

let mut o = op.object("path/to/file");
let _ = o.blocking_write(vec![0; 4096])?;
source

pub fn blocking_write_with( &self, args: OpWrite, bs: impl Into<Vec<u8>> ) -> Result<()>

Write data with option described in OpenDAL rfc-0661

Notes
  • Write will make sure all bytes has been written, or an error will be returned.
Examples
use bytes::Bytes;
use opendal::ops::OpWrite;

let mut o = op.object("hello.txt");
let bs = b"hello, world!".to_vec();
let ow = OpWrite::new(bs.len() as u64).with_content_type("text/plain");
let _ = o.blocking_write_with(ow, bs)?;
source

pub async fn write_from(&self, size: u64, br: impl Read + 'static) -> Result<()>

Write data into object from a input::Read.

Notes
  • Write will make sure all bytes has been written, or an error will be returned.
Examples
use bytes::Bytes;
use futures::io::Cursor;

let mut o = op.object("path/to/file");
let r = Cursor::new(vec![0; 4096]);
let _ = o.write_from(4096, r).await?;
source

pub fn blocking_write_from( &self, size: u64, br: impl BlockingRead + 'static ) -> Result<()>

Write data into object from a input::BlockingRead.

Notes
  • Write will make sure all bytes has been written, or an error will be returned.
Examples
use std::io::Cursor;

use bytes::Bytes;

let mut o = op.object("path/to/file");
let r = Cursor::new(vec![0; 4096]);
let _ = o.blocking_write_from(4096, r)?;
source

pub async fn delete(&self) -> Result<()>

Delete object.

Notes
  • Delete not existing error won’t return errors.
Examples
op.object("test").delete().await?;
source

pub fn blocking_delete(&self) -> Result<()>

Delete object.

Notes
  • Delete not existing error won’t return errors.
Examples
op.object("test").blocking_delete()?;
source

pub async fn list(&self) -> Result<ObjectLister>

List current dir object.

This function will create a new handle to list objects.

An error will be returned if object path doesn’t end with /.

Examples
let o = op.object("path/to/dir/");
let mut ds = o.list().await?;
while let Some(mut de) = ds.try_next().await? {
    let meta = de
        .metadata({
            use opendal::ObjectMetakey::*;
            Mode
        })
        .await?;
    match meta.mode() {
        ObjectMode::FILE => {
            println!("Handling file")
        }
        ObjectMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        ObjectMode::Unknown => continue,
    }
}
source

pub fn blocking_list(&self) -> Result<BlockingObjectLister>

List current dir object.

This function will create a new handle to list objects.

An error will be returned if object path doesn’t end with /.

Examples
let o = op.object("path/to/dir/");
let mut ds = o.blocking_list()?;
while let Some(mut de) = ds.next() {
    let meta = de?.blocking_metadata({
        use opendal::ObjectMetakey::*;
        Mode
    })?;
    match meta.mode() {
        ObjectMode::FILE => {
            println!("Handling file")
        }
        ObjectMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        ObjectMode::Unknown => continue,
    }
}
source

pub async fn scan(&self) -> Result<ObjectLister>

List dir in flat way.

This function will create a new handle to list objects.

An error will be returned if object path doesn’t end with /.

Examples
let o = op.object("path/to/dir/");
let mut ds = o.scan().await?;
while let Some(mut de) = ds.try_next().await? {
    let meta = de
        .metadata({
            use opendal::ObjectMetakey::*;
            Mode
        })
        .await?;
    match meta.mode() {
        ObjectMode::FILE => {
            println!("Handling file")
        }
        ObjectMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        ObjectMode::Unknown => continue,
    }
}
source

pub fn blocking_scan(&self) -> Result<BlockingObjectLister>

List dir in flat way.

This function will create a new handle to list objects.

An error will be returned if object path doesn’t end with /.

Examples
let o = op.object("path/to/dir/");
let mut ds = o.blocking_list()?;
while let Some(mut de) = ds.next() {
    let meta = de?.blocking_metadata({
        use opendal::ObjectMetakey::*;
        Mode
    })?;
    match meta.mode() {
        ObjectMode::FILE => {
            println!("Handling file")
        }
        ObjectMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        ObjectMode::Unknown => continue,
    }
}
source

pub async fn stat(&self) -> Result<ObjectMetadata>

Get current object’s metadata without cache directly.

Notes

Use stat if you:

  • Want detect the outside changes of object.
  • Don’t want to read from cached object metadata.

You may want to use metadata if you are working with objects returned by ObjectLister. It’s highly possible that metadata you want has already been cached.

Examples
use opendal::ErrorKind;
if let Err(e) = op.object("test").stat().await {
    if e.kind() == ErrorKind::ObjectNotFound {
        println!("object not exist")
    }
}
source

pub fn blocking_stat(&self) -> Result<ObjectMetadata>

Get current object’s metadata without cache directly.

Notes

Use stat if you:

  • Want detect the outside changes of object.
  • Don’t want to read from cached object metadata.

You may want to use metadata if you are working with objects returned by ObjectLister. It’s highly possible that metadata you want has already been cached.

Examples
use opendal::ErrorKind;
if let Err(e) = op.object("test").blocking_stat() {
    if e.kind() == ErrorKind::ObjectNotFound {
        println!("object not exist")
    }
}
source

pub async fn metadata( &self, flags: impl Into<FlagSet<ObjectMetakey>> ) -> Result<Arc<ObjectMetadata>>

Get current object’s metadata with cache.

metadata will check the given query with already cached metadata first. And query from storage if not found.

Notes

Use metadata if you are working with objects returned by ObjectLister. It’s highly possible that metadata you want has already been cached.

You may want to use stat, if you:

  • Want detect the outside changes of object.
  • Don’t want to read from cached object metadata.
Behavior

Visiting not fetched metadata will lead to panic in debug build. It must be a bug, please fix it instead.

Examples
Query already cached metadata

By query metadata with None, we can only query in-memory metadata cache. In this way, we can make sure that no API call will send.

let meta = op.object("test").metadata(None).await?;
// content length COULD be correct.
let _ = meta.content_length();
// etag COULD be correct.
let _ = meta.etag();
Query content length and content type
use opendal::ObjectMetakey;

let meta = op
    .object("test")
    .metadata({
        use ObjectMetakey::*;
        ContentLength | ContentType
    })
    .await?;
// content length MUST be correct.
let _ = meta.content_length();
// etag COULD be correct.
let _ = meta.etag();
Query all metadata

By query metadata with Complete, we can make sure that we have fetched all metadata of this object.

use opendal::ObjectMetakey;

let meta = op
    .object("test")
    .metadata({ ObjectMetakey::Complete })
    .await?;
// content length MUST be correct.
let _ = meta.content_length();
// etag MUST be correct.
let _ = meta.etag();
source

pub fn blocking_metadata( &self, flags: impl Into<FlagSet<ObjectMetakey>> ) -> Result<Arc<ObjectMetadata>>

Get current object’s metadata with cache in blocking way.

metadata will check the given query with already cached metadata first. And query from storage if not found.

Notes

Use metadata if you are working with objects returned by ObjectLister. It’s highly possible that metadata you want has already been cached.

You may want to use stat, if you:

  • Want detect the outside changes of object.
  • Don’t want to read from cached object metadata.
Behavior

Visiting not fetched metadata will lead to panic in debug build. It must be a bug, please fix it instead.

Examples
Query already cached metadata

By query metadata with None, we can only query in-memory metadata cache. In this way, we can make sure that no API call will send.

let meta = op.object("test").blocking_metadata(None)?;
// content length COULD be correct.
let _ = meta.content_length();
// etag COULD be correct.
let _ = meta.etag();
Query content length and content type
use opendal::ObjectMetakey;

let meta = op.object("test").blocking_metadata({
    use ObjectMetakey::*;
    ContentLength | ContentType
})?;
// content length MUST be correct.
let _ = meta.content_length();
// etag COULD be correct.
let _ = meta.etag();
Query all metadata

By query metadata with Complete, we can make sure that we have fetched all metadata of this object.

use opendal::ObjectMetakey;

let meta = op
    .object("test")
    .blocking_metadata({ ObjectMetakey::Complete })?;
// content length MUST be correct.
let _ = meta.content_length();
// etag MUST be correct.
let _ = meta.etag();
source

pub async fn is_exist(&self) -> Result<bool>

Check if this object exists or not.

Example
use anyhow::Result;
use futures::io;
use opendal::Operator;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let _ = op.object("test").is_exist().await?;

    Ok(())
}
source

pub fn blocking_is_exist(&self) -> Result<bool>

Check if this object exists or not.

Example
use anyhow::Result;
use opendal::Operator;
fn test(op: Operator) -> Result<()> {
    let _ = op.object("test").blocking_is_exist()?;

    Ok(())
}
source

pub fn presign_stat(&self, expire: Duration) -> Result<PresignedRequest>

Presign an operation for stat(head).

Example
use anyhow::Result;
use futures::io;
use opendal::Operator;
use time::Duration;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let signed_req = op.object("test").presign_stat(Duration::hours(1))?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;
source

pub fn presign_read(&self, expire: Duration) -> Result<PresignedRequest>

Presign an operation for read.

Example
use anyhow::Result;
use futures::io;
use opendal::Operator;
use time::Duration;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let signed_req = op.object("test.txt").presign_read(Duration::hours(1))?;
  • signed_req.method(): GET
  • signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
  • signed_req.headers(): { "host": "s3.amazonaws.com" }

We can download this object via curl or other tools without credentials:

curl "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -O /tmp/test.txt
source

pub fn presign_write(&self, expire: Duration) -> Result<PresignedRequest>

Presign an operation for write.

Example
use anyhow::Result;
use futures::io;
use opendal::Operator;
use time::Duration;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let signed_req = op.object("test.txt").presign_write(Duration::hours(1))?;
  • signed_req.method(): PUT
  • signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
  • signed_req.headers(): { "host": "s3.amazonaws.com" }

We can upload file as this object via curl or other tools without credential:

curl -X PUT "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -d "Hello, World!"
source

pub fn presign_write_with( &self, op: OpWrite, expire: Duration ) -> Result<PresignedRequest>

Presign an operation for write with option described in OpenDAL rfc-0661

You can pass OpWrite to this method to specify the content length and content type.

Example
use anyhow::Result;
use futures::io;
use opendal::ops::OpWrite;
use opendal::Operator;
use time::Duration;

#[tokio::main]
async fn test(op: Operator) -> Result<()> {
    let args = OpWrite::new(0).with_content_type("text/csv");
    let signed_req = op.object("test").presign_write_with(args, Duration::hours(1))?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;
source

pub fn to_multipart(&self, upload_id: &str) -> ObjectMultipart

Construct a multipart with existing upload id.

source

pub async fn create_multipart(&self) -> Result<ObjectMultipart>

Create a new multipart for current path.

Trait Implementations§

source§

impl Clone for Object

source§

fn clone(&self) -> Object

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl Debug for Object

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

§

impl !RefUnwindSafe for Object

§

impl Send for Object

§

impl Sync for Object

§

impl Unpin for Object

§

impl !UnwindSafe for Object

Blanket Implementations§

source§

impl<T> Any for Twhere T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for Twhere T: ?Sized,

const: unstable · source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for Twhere T: ?Sized,

const: unstable · source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> CompatExt for T

source§

fn compat(self) -> Compat<T>

Applies the Compat adapter by value. Read more
source§

fn compat_ref(&self) -> Compat<&T>

Applies the Compat adapter by shared reference. Read more
source§

fn compat_mut(&mut self) -> Compat<&mut T>

Applies the Compat adapter by mutable reference. Read more
source§

impl<T> From<T> for T

const: unstable · source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for Twhere U: From<T>,

const: unstable · source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

§

impl<T> Pointable for T

§

const ALIGN: usize = mem::align_of::<T>()

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<T> Same<T> for T

§

type Output = T

Should always be Self
source§

impl<T> ToOwned for Twhere T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for Twhere U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
const: unstable · source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for Twhere U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
const: unstable · source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for Twhere V: MultiLane<T>,

§

fn vzip(self) -> V

source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more