bgzf 0.4.0

Utility library for working with explicitly BGZF compressed data
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

# bgzf

<p align="center">
  <a href="https://github.com/fulcrumgenomics/bgzf/actions?query=workflow%3ACheck"><img src="https://github.com/fulcrumgenomics/bgzf/actions/workflows/build_and_test.yml/badge.svg" alt="Build Status"></a>
  <img src="https://img.shields.io/crates/l/bgzf.svg" alt="license">
  <a href="https://crates.io/crates/bgzf"><img src="https://img.shields.io/crates/v/bgzf.svg?colorB=319e8c" alt="Version info"></a><br>
</p>

This library provides both high level readers and writers for the BGZF format as well as lower level compressor and decompressor functions.

Bgzf is a multi-gzip format that adds an extra field to the header indicating how large the complete block (with header and footer) is.


<p>
<a href="https://fulcrumgenomics.com">
<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/fulcrumgenomics/bgzf/main/.github/logos/fulcrumgenomics-dark.svg">
  <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/fulcrumgenomics/bgzf/main/.github/logos/fulcrumgenomics-light.svg">
  <img alt="Fulcrum Genomics" src="https://raw.githubusercontent.com/fulcrumgenomics/bgzf/main/.github/logos/fulcrumgenomics-light.svg" height="100">
</picture>
</a>
</p>

[Visit us at Fulcrum Genomics](https://www.fulcrumgenomics.com) to learn more about how we can power your Bioinformatics with bgzf and beyond.

<a href="mailto:contact@fulcrumgenomics.com?subject=[GitHub inquiry]"><img src="https://img.shields.io/badge/Email_us-%2338b44a.svg?&style=for-the-badge&logo=gmail&logoColor=white"/></a>
<a href="https://www.fulcrumgenomics.com"><img src="https://img.shields.io/badge/Visit_Us-%2326a8e0.svg?&style=for-the-badge&logo=wordpress&logoColor=white"/></a>

## Documentation and Examples

Please see the generated [Rust Docs](https://docs.rs/bgzf).

## Multithreading

The default `Reader`/`Writer` are single-threaded. Enable the optional
`multithreading-simple` feature to also get `MultithreadedReader` and
`MultithreadedWriter`, which (de)compress blocks in parallel on a dedicated pool of
worker threads while preserving block order:

```toml
[dependencies]
bgzf = { version = "0.4", features = ["multithreading-simple"] }
```

```rust
use std::fs::File;
use std::num::NonZero;
use bgzf::MultithreadedReader;

// The inner reader is any `Read + Send + 'static` source — here, a BGZF file on disk.
let input = File::open("example.bgzf")?;
let reader = MultithreadedReader::with_worker_count(NonZero::new(4).unwrap(), input);
```

Each reader/writer owns its own worker threads (hence *simple*), as distinguished from a
future `multithreading-pooled` feature that would share one thread pool across many
instances. Read-ahead/write-ahead depth is decoupled from the worker count, so even a
single-worker instance pipelines rather than stalling. Decompressed output is byte-identical
to `Reader`, and compressed output is byte-identical to `Writer`, at every worker count.

## Benchmarks

Run the compression benchmarks with:

```bash
cargo bench
```

This runs [Criterion](https://github.com/bheisler/criterion.rs) benchmarks measuring:
- Single block compression at various levels
- Writer throughput
- Reader throughput and read/write round-trips (including store-only)

The multithreaded reader/writer benchmarks (single-threaded vs. 1/2/4 workers) require the
feature:

```bash
cargo bench --features multithreading-simple --bench multithreaded
```