Struct opendal::ObjectReader

pub struct ObjectReader { /* private fields */ }

ObjectReader is the public API for users.

Usage

ObjectReader implements the following APIs:

• AsyncRead
• AsyncSeek
• Stream<Item = <io::Result<Bytes>>>

For reading data, we can use AsyncRead and Stream. The mainly different is where the copy happend.

AsyncRead requires user to prepare a buffer for ObjectReader to fill. And Stream will stream out a Bytes for user to decide when to copy it’s content.

For example, users may have their only CPU/IO bound workers and don’t want to do copy inside IO workers. They can use Stream to get a Bytes and consume it in side CPU workers inside.

Besides, Stream COULD reduce an extra copy if underlying reader is stream based (like services s3, azure which based on HTTP).

Notes

All implementions of ObjectReader should be zero cost. In our cases, which means others must pay the same cost for the same feature provide by us.

For examples, call read without seek should always act the same as calling read on plain reader.

Read is Seekable

We use internal AccessorHint::ReadIsSeekable to decide the most suitable implementations.

If there is a hint that ReadIsSeekable, we will open it with given args directy. Otherwise, we will pick a seekable reader implementation based on input range for it.

• Some(offset), Some(size) => RangeReader
• Some(offset), None and None, None => OffsetReader
• None, Some(size) => get the total size first to convert as RangeReader

No matter which reader we use, we will make sure the read operation is zero cost.

Read is Streamable

We use internal AccessorHint::ReadIsStreamable to decide the most suitable implementations.

If there is a hint that ReadIsStreamable, we will use existing reader directly. Otherwise, we will use transform this reader as a stream.

Consume instead of Drop

Normally, if reader is seekable, we need to drop current reader and start a new read call.

We can consume the data if the seek position is close enough. For example, users try to seek to Current(1), we can just read the data can consume it.

In this way, we can reduce the extra cost of dropping reader.

Trait Implementations

impl AsyncRead for ObjectReader

fn poll_read(
    self: Pin<&mut Self>,
    cx: &mut Context<'_>,
    buf: &mut [u8]
) -> Poll<Result<usize>>

Attempt to read from the AsyncRead into buf.

fn poll_read_vectored(
    self: Pin<&mut Self>,
    cx: &mut Context<'_>,
    bufs: &mut [IoSliceMut<'_>]
) -> Poll<Result<usize, Error>>

Attempt to read from the AsyncRead into bufs using vectored IO operations.

impl AsyncSeek for ObjectReader

fn poll_seek(
    self: Pin<&mut Self>,
    cx: &mut Context<'_>,
    pos: SeekFrom
) -> Poll<Result<u64>>

Attempt to seek to an offset, in bytes, in a stream.

impl Read for ObjectReader

fn poll_read(
    &mut self,
    cx: &mut Context<'_>,
    buf: &mut [u8]
) -> Poll<Result<usize>>

Read bytes asynchronously.

fn poll_seek(&mut self, cx: &mut Context<'_>, pos: SeekFrom) -> Poll<Result<u64>>

Seek asynchronously.

fn poll_next(&mut self, cx: &mut Context<'_>) -> Poll<Option<Result<Bytes>>>

Stream Bytes from underlying reader.

fn inner(&mut self) -> Option<&mut Reader>

Return the inner output bytes reader if there is one.

impl Stream for ObjectReader

type Item = Result<Bytes, Error>

Values yielded by the stream.

fn poll_next(
    self: Pin<&mut Self>,
    cx: &mut Context<'_>
) -> Poll<Option<Self::Item>>

Attempt to pull out the next value of this stream, registering the current task for wakeup if the value is not yet available, and returning None if the stream is exhausted.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the stream.