Struct BufReader 

pub struct BufReader { /* private fields */ }

Available on crate feature arrow-59 and crate feature tokio and (crate features arrow-58 or arrow-59) only.

An async-buffered reader compatible with the tokio IO traits

Internally this maintains a buffer of the requested size, and uses ObjectStoreExt::get_range to populate its internal buffer once depleted. This buffer is cleared on seek.

Whilst simple, this interface will typically be outperformed by the native ObjectStore methods that better map to the network APIs. This is because most object stores have very high first-byte latencies, on the order of 100-200ms, and so avoiding unnecessary round-trips is critical to throughput.

Systems looking to sequentially scan a file should instead consider using ObjectStoreExt::get, or ObjectStore::get_opts, or ObjectStoreExt::get_range to read a particular range.

Systems looking to read multiple ranges of a file should instead consider using ObjectStore::get_ranges, which will optimise the vectored IO.

Implementations

impl BufReader

pub fn new(store: Arc<dyn ObjectStore>, meta: &ObjectMeta) -> BufReader

Create a new BufReader from the provided ObjectMeta and ObjectStore

pub fn with_capacity( store: Arc<dyn ObjectStore>, meta: &ObjectMeta, capacity: usize, ) -> BufReader

Create a new BufReader from the provided ObjectMeta, ObjectStore, and capacity

Trait Implementations

impl AsyncBufRead for BufReader

fn poll_fill_buf( self: Pin<&mut BufReader>, cx: &mut Context<'_>, ) -> Poll<Result<&[u8], Error>>

Attempts to return the contents of the internal buffer, filling it with more data from the inner reader if it is empty.

fn consume(self: Pin<&mut BufReader>, amt: usize)

Tells this buffer that amt bytes have been consumed from the buffer, so they should no longer be returned in calls to poll_read.

impl AsyncRead for BufReader

fn poll_read( self: Pin<&mut BufReader>, cx: &mut Context<'_>, out: &mut ReadBuf<'_>, ) -> Poll<Result<(), Error>>

Attempts to read from the AsyncRead into buf.

impl AsyncSeek for BufReader

fn start_seek( self: Pin<&mut BufReader>, position: SeekFrom, ) -> Result<(), Error>

Attempts to seek to an offset, in bytes, in a stream.

fn poll_complete( self: Pin<&mut BufReader>, _cx: &mut Context<'_>, ) -> Poll<Result<u64, Error>>

Waits for a seek operation to complete.

impl Debug for BufReader

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.