Struct ParallelBgzfReader 

pub struct ParallelBgzfReader<R: Read> { /* private fields */ }

A BGZF reader that decompresses blocks across cores.

Blocks are independent, so inflating them is embarrassingly parallel: one thread reads compressed blocks — which is cheap, mostly memcpy — and a batch is then inflated on the rayon pool and served in order.

This trades seeking for throughput. Use it for a full pass over a compressed file; use BgzfReader when you need Seek, which by its nature has to decompress one block at a time.

use fastx::bgzf::ParallelBgzfReader;
use std::io::Read;

let file = std::fs::File::open("reads.fq.gz")?;
let mut reader = ParallelBgzfReader::new(file);
let mut all = Vec::new();
reader.read_to_end(&mut all)?;

Implementations

impl<R: Read> ParallelBgzfReader<R>

pub fn new(inner: R) -> ParallelBgzfReader<R>

Wrap a reader, inflating one batch of blocks per core at a time.

pub fn blocks_per_batch(self, blocks: usize) -> Self

How many blocks to inflate at a time. Larger batches balance load better and cost blocks × 64 KiB of memory.

pub fn into_inner(self) -> R

Unwrap the underlying reader.

Trait Implementations

impl<R: Read> Read for ParallelBgzfReader<R>

Available on crate feature parallel only.

fn read(&mut self, buf: &mut [u8]) -> Result<usize>

Pull some bytes from this source into the specified buffer, returning how many bytes were read.

fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize, Error>

Like read, except that it reads into a slice of buffers.

fn is_read_vectored(&self) -> bool

🔬This is a nightly-only experimental API. (can_vector)

Determines if this Reader has an efficient read_vectored implementation.

fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize, Error>

Reads all bytes until EOF in this source, placing them into buf.

fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>

Reads all bytes until EOF in this source, appending them to buf.

fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Error>

Reads the exact number of bytes required to fill buf.

fn read_buf(&mut self, buf: BorrowedCursor<'_, u8>) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Pull some bytes from this source into the specified buffer.

fn read_buf_exact( &mut self, cursor: BorrowedCursor<'_, u8>, ) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Reads the exact number of bytes required to fill cursor.

fn by_ref(&mut self) -> &mut Self

where Self: Sized,

Creates a “by reference” adapter for this instance of Read.

fn bytes(self) -> Bytes<Self>

where Self: Sized,

Transforms this Read instance to an Iterator over its bytes.

fn chain<R>(self, next: R) -> Chain<Self, R>

where R: Read, Self: Sized,

Creates an adapter which will chain this stream with another.

fn take(self, limit: u64) -> Take<Self>

where Self: Sized,

Creates an adapter which will read at most limit bytes from it.

fn read_array<const N: usize>(&mut self) -> Result<[u8; N], Error>

where Self: Sized,

🔬This is a nightly-only experimental API. (read_array)

Read and return a fixed array of bytes from this source.

fn read_le<T>(&mut self) -> Result<T, Error>

where T: FromEndianBytes, Self: Sized,

🔬This is a nightly-only experimental API. (read_le)

Read and return a type (e.g. an integer) in little-endian order.

fn read_be<T>(&mut self) -> Result<T, Error>

where T: FromEndianBytes, Self: Sized,

🔬This is a nightly-only experimental API. (read_le)

Read and return a type (e.g. an integer) in big-endian order.