crabz 0.6.1

Parallel Compression
crabz-0.6.1 is not a library.

🦀 crabz

A cross platform, fast, compression and decompression tool.

Synopsis

This is currently a proof of concept CLI tool using the gzp crate.

Supported formats:

  • Gzip
  • Zlib
  • Mgzip
  • BGZF
  • Raw Deflate
  • Snap

Install

  • Homebrew / Linuxbrew
brew tap sstadick/crabz
brew install crabz
  • Debian (Ubuntu)
curl -LO https://github.com/sstadick/crabz/releases/download/<latest>/crabz-linux-amd64.deb
sudo dpkg -i crabz-linux-amd64.deb
  • Cargo
cargo install crabz
  • Conda
conda install -c bioconda crabz

Usage

❯ crabz -h              
Seth Stadick
Compress and decompress files

USAGE:
    crabz [FLAGS] [OPTIONS] [FILE]

FLAGS:
    -d, --decompress    Flag to switch to decompressing inputs. Note: this flag may change in future releases
    -h, --help          Prints help information
    -V, --version       Prints version information

OPTIONS:
    -l, --compression-level <compression-level>        Compression level [default: 6]
    -p, --compression-threads <compression-threads>
            Number of compression threads to use, or if decompressing a format that allow for multi-threaded
            decompression, the number to use. Note that > 4 threads for decompression doesn't seem to help [default:
            32]
    -f, --format <format>
            The format to use [default: gzip]  [possible values: gzip, bgzf, mgzip,
            zlib, deflate, snap]
    -o, --output <output>                              Output path to write to, empty or "-" to write to stdout

ARGS:
    <FILE>    Input file to read from, empty or "-" to read from stdin

Benchmarks

These benchmarks use the data in bench-data catted together 100 times. Run with bash ./benchmark.sh data.txt.

Benchmark system specs: Ubuntu 20 AMD Ryzen 9 3950X 16-Core Processor w/ 64 GB DDR4 memory and 1TB NVMe Drive

pigz v2.4 installed via apt on Ubuntu

Takeaways:

  • crabz with zlib backend is pretty much identical to pigz
  • crabz with zlib-ng backend is roughly 30-50% faster than pigz
  • crabz with rust backend is roughly 5-10% faster than pigz

It is already known that zlib-ng is faster than zlib, so none of this is groundbreaking. However, I think crabz gets an an edge due to the following:

  • crabz with deflate_rust backend is using all Rust only code, which is in theory more secure / safe.
  • crabz with zlib-ng is easier to install than pigz with a zlib-ng backend
  • crabz supports more formats than pigz
  • crabz is cross platform and can run on windows

With regards to block formats like Mgzip and BGZF, crabz is using libdeflater by default which excels at compressing and decompression known-sized blocks. This makes block compression formats very fast at a small loss to the compression ratio.

Comparing crabz against tools like bgzip, which also defaults to libdeflater as a backend shows them within a few percent of eachother.

As crabz is just a wrapper for the gzp library, the most exciting thing about these benchmarks is that gzp is on par with best in class CLI tools for multi-threaded compression and decompression as a library.

Flate2 zlib-ng backend

Compression

Command Mean [s] Min [s] Max [s] Relative
crabz -p 1 -c 3 < ./data.txt 6.450 ± 0.069 6.328 6.540 16.86 ± 0.24
pigz -p 1 -3 < ./data.txt 11.404 ± 0.152 11.186 11.717 29.81 ± 0.49
crabz -p 2 -c 3 < ./data.txt 3.437 ± 0.017 3.418 3.461 8.98 ± 0.10
pigz -p 2 -3 < ./data.txt 5.868 ± 0.031 5.826 5.927 15.34 ± 0.17
crabz -p 4 -c 3 < ./data.txt 1.741 ± 0.008 1.729 1.752 4.55 ± 0.05
pigz -p 4 -3 < ./data.txt 2.952 ± 0.008 2.939 2.960 7.72 ± 0.08
crabz -p 8 -c 3 < ./data.txt 0.889 ± 0.004 0.882 0.895 2.32 ± 0.02
pigz -p 8 -3 < ./data.txt 1.505 ± 0.008 1.493 1.520 3.93 ± 0.04
crabz -p 16 -c 3 < ./data.txt 0.485 ± 0.014 0.457 0.502 1.27 ± 0.04
pigz -p 16 -3 < ./data.txt 0.775 ± 0.011 0.764 0.797 2.02 ± 0.04
crabz -p 32 -c 3 < ./data.txt 0.383 ± 0.004 0.375 0.388 1.00
pigz -p 32 -3 < ./data.txt 0.699 ± 0.029 0.668 0.770 1.83 ± 0.08
crabz -p 1 -c 6 < ./data.txt 10.367 ± 0.211 10.106 10.642 27.10 ± 0.61
pigz -p 1 -6 < ./data.txt 26.734 ± 0.345 26.234 27.135 69.89 ± 1.12
crabz -p 2 -c 6 < ./data.txt 5.366 ± 0.036 5.299 5.429 14.03 ± 0.16
pigz -p 2 -6 < ./data.txt 13.589 ± 0.083 13.428 13.679 35.52 ± 0.40
crabz -p 4 -c 6 < ./data.txt 2.719 ± 0.021 2.694 2.757 7.11 ± 0.09
pigz -p 4 -6 < ./data.txt 6.887 ± 0.013 6.871 6.916 18.00 ± 0.17
crabz -p 8 -c 6 < ./data.txt 1.381 ± 0.007 1.372 1.397 3.61 ± 0.04
pigz -p 8 -6 < ./data.txt 3.479 ± 0.008 3.463 3.488 9.09 ± 0.09
crabz -p 16 -c 6 < ./data.txt 0.745 ± 0.022 0.727 0.804 1.95 ± 0.06
pigz -p 16 -6 < ./data.txt 1.818 ± 0.036 1.765 1.874 4.75 ± 0.10
crabz -p 32 -c 6 < ./data.txt 0.549 ± 0.006 0.538 0.557 1.44 ± 0.02
pigz -p 32 -6 < ./data.txt 1.187 ± 0.011 1.172 1.210 3.10 ± 0.04
crabz -p 1 -c 9 < ./data.txt 30.114 ± 0.196 29.842 30.420 78.72 ± 0.90
pigz -p 1 -9 < ./data.txt 51.369 ± 0.164 51.246 51.698 134.29 ± 1.33
crabz -p 2 -c 9 < ./data.txt 15.371 ± 0.070 15.202 15.443 40.18 ± 0.42
pigz -p 2 -9 < ./data.txt 26.452 ± 0.085 26.253 26.576 69.15 ± 0.69
crabz -p 4 -c 9 < ./data.txt 7.729 ± 0.022 7.699 7.768 20.20 ± 0.20
pigz -p 4 -9 < ./data.txt 13.365 ± 0.047 13.271 13.449 34.94 ± 0.35
crabz -p 8 -c 9 < ./data.txt 3.901 ± 0.006 3.889 3.910 10.20 ± 0.10
pigz -p 8 -9 < ./data.txt 6.749 ± 0.014 6.737 6.781 17.64 ± 0.17
crabz -p 16 -c 9 < ./data.txt 2.039 ± 0.024 1.997 2.071 5.33 ± 0.08
pigz -p 16 -9 < ./data.txt 3.486 ± 0.054 3.426 3.574 9.11 ± 0.17
crabz -p 32 -c 9 < ./data.txt 1.337 ± 0.072 1.220 1.411 3.49 ± 0.19
pigz -p 32 -9 < ./data.txt 2.203 ± 0.114 2.082 2.378 5.76 ± 0.30

Decompression

Command Mean [s] Min [s] Max [s] Relative
crabz -d < ./data.3.txt.gz 1.422 ± 0.010 1.411 1.437 1.03 ± 0.02
pigz -d < ./data.3.txt.gz 1.674 ± 0.031 1.621 1.705 1.21 ± 0.03
crabz -d < ./data.6.txt.gz 1.403 ± 0.016 1.389 1.427 1.01 ± 0.02
pigz -d < ./data.6.txt.gz 1.724 ± 0.026 1.697 1.766 1.24 ± 0.02
crabz -d < ./data.9.txt.gz 1.385 ± 0.018 1.359 1.416 1.00
pigz -d < ./data.9.txt.gz 1.745 ± 0.044 1.684 1.797 1.26 ± 0.04

Flate2 zlib backend

Compression

Command Mean [s] Min [s] Max [s] Relative
crabz -p 1 -c 3 < ./data.txt 11.248 ± 0.247 11.085 11.532 20.23 ± 0.45
pigz -p 1 -3 < ./data.txt 11.296 ± 0.171 11.104 11.434 20.32 ± 0.31
crabz -p 2 -c 3 < ./data.txt 5.681 ± 0.040 5.645 5.725 10.22 ± 0.08
pigz -p 2 -3 < ./data.txt 5.926 ± 0.015 5.916 5.944 10.66 ± 0.04
crabz -p 4 -c 3 < ./data.txt 2.891 ± 0.007 2.883 2.895 5.20 ± 0.02
pigz -p 4 -3 < ./data.txt 2.966 ± 0.013 2.955 2.980 5.34 ± 0.03
crabz -p 8 -c 3 < ./data.txt 1.461 ± 0.003 1.459 1.465 2.63 ± 0.01
pigz -p 8 -3 < ./data.txt 1.509 ± 0.004 1.505 1.512 2.71 ± 0.01
crabz -p 16 -c 3 < ./data.txt 0.784 ± 0.010 0.775 0.795 1.41 ± 0.02
pigz -p 16 -3 < ./data.txt 0.772 ± 0.010 0.765 0.784 1.39 ± 0.02
crabz -p 32 -c 3 < ./data.txt 0.556 ± 0.002 0.554 0.557 1.00
pigz -p 32 -3 < ./data.txt 0.743 ± 0.047 0.694 0.786 1.34 ± 0.08
crabz -p 1 -c 6 < ./data.txt 26.366 ± 0.154 26.189 26.469 47.42 ± 0.31
pigz -p 1 -6 < ./data.txt 26.688 ± 0.103 26.579 26.783 48.00 ± 0.23
crabz -p 2 -c 6 < ./data.txt 13.443 ± 0.069 13.400 13.523 24.18 ± 0.14
pigz -p 2 -6 < ./data.txt 13.605 ± 0.059 13.567 13.673 24.47 ± 0.13
crabz -p 4 -c 6 < ./data.txt 6.833 ± 0.005 6.828 6.837 12.29 ± 0.03
pigz -p 4 -6 < ./data.txt 6.866 ± 0.028 6.834 6.884 12.35 ± 0.06
crabz -p 8 -c 6 < ./data.txt 3.446 ± 0.000 3.445 3.446 6.20 ± 0.02
pigz -p 8 -6 < ./data.txt 3.482 ± 0.002 3.480 3.483 6.26 ± 0.02
crabz -p 16 -c 6 < ./data.txt 1.822 ± 0.012 1.813 1.835 3.28 ± 0.02
pigz -p 16 -6 < ./data.txt 1.771 ± 0.004 1.767 1.776 3.19 ± 0.01
crabz -p 32 -c 6 < ./data.txt 1.178 ± 0.008 1.171 1.187 2.12 ± 0.02
pigz -p 32 -6 < ./data.txt 1.184 ± 0.001 1.184 1.185 2.13 ± 0.01
crabz -p 1 -c 9 < ./data.txt 52.122 ± 0.288 51.790 52.293 93.75 ± 0.58
pigz -p 1 -9 < ./data.txt 53.031 ± 0.071 52.951 53.085 95.39 ± 0.29
crabz -p 2 -c 9 < ./data.txt 26.287 ± 0.047 26.249 26.339 47.28 ± 0.15
pigz -p 2 -9 < ./data.txt 26.409 ± 0.238 26.190 26.662 47.50 ± 0.45
crabz -p 4 -c 9 < ./data.txt 13.373 ± 0.051 13.317 13.419 24.05 ± 0.11
pigz -p 4 -9 < ./data.txt 13.414 ± 0.035 13.383 13.451 24.13 ± 0.09
crabz -p 8 -c 9 < ./data.txt 6.733 ± 0.003 6.731 6.736 12.11 ± 0.03
pigz -p 8 -9 < ./data.txt 6.763 ± 0.004 6.761 6.767 12.16 ± 0.03
crabz -p 16 -c 9 < ./data.txt 3.487 ± 0.034 3.450 3.517 6.27 ± 0.06
pigz -p 16 -9 < ./data.txt 3.459 ± 0.021 3.434 3.473 6.22 ± 0.04
crabz -p 32 -c 9 < ./data.txt 2.088 ± 0.008 2.081 2.097 3.76 ± 0.02
pigz -p 32 -9 < ./data.txt 2.107 ± 0.023 2.090 2.133 3.79 ± 0.04

Decompression

Flate2 rust backend

Compression

Command Mean [s] Min [s] Max [s] Relative
crabz -p 1 -c 3 < ./data.txt 10.167 ± 0.164 10.050 10.355 18.57 ± 0.33
pigz -p 1 -3 < ./data.txt 11.338 ± 0.071 11.292 11.420 20.71 ± 0.21
crabz -p 2 -c 3 < ./data.txt 4.912 ± 0.013 4.898 4.920 8.97 ± 0.08
pigz -p 2 -3 < ./data.txt 5.876 ± 0.047 5.826 5.919 10.73 ± 0.12
crabz -p 4 -c 3 < ./data.txt 2.463 ± 0.018 2.447 2.482 4.50 ± 0.05
pigz -p 4 -3 < ./data.txt 2.967 ± 0.008 2.958 2.972 5.42 ± 0.05
crabz -p 8 -c 3 < ./data.txt 1.255 ± 0.005 1.250 1.261 2.29 ± 0.02
pigz -p 8 -3 < ./data.txt 1.509 ± 0.002 1.507 1.511 2.76 ± 0.02
crabz -p 16 -c 3 < ./data.txt 0.705 ± 0.030 0.673 0.731 1.29 ± 0.05
pigz -p 16 -3 < ./data.txt 0.780 ± 0.015 0.768 0.797 1.42 ± 0.03
crabz -p 32 -c 3 < ./data.txt 0.547 ± 0.004 0.544 0.552 1.00
pigz -p 32 -3 < ./data.txt 0.755 ± 0.025 0.726 0.771 1.38 ± 0.05
crabz -p 1 -c 6 < ./data.txt 27.064 ± 0.288 26.863 27.394 49.44 ± 0.66
pigz -p 1 -6 < ./data.txt 27.034 ± 0.090 26.938 27.117 49.38 ± 0.43
crabz -p 2 -c 6 < ./data.txt 12.400 ± 0.083 12.321 12.487 22.65 ± 0.24
pigz -p 2 -6 < ./data.txt 13.619 ± 0.074 13.558 13.702 24.88 ± 0.24
crabz -p 4 -c 6 < ./data.txt 6.279 ± 0.023 6.263 6.305 11.47 ± 0.10
pigz -p 4 -6 < ./data.txt 6.879 ± 0.020 6.867 6.901 12.57 ± 0.11
crabz -p 8 -c 6 < ./data.txt 3.189 ± 0.010 3.178 3.198 5.83 ± 0.05
pigz -p 8 -6 < ./data.txt 3.477 ± 0.007 3.470 3.483 6.35 ± 0.05
crabz -p 16 -c 6 < ./data.txt 1.756 ± 0.015 1.740 1.771 3.21 ± 0.04
pigz -p 16 -6 < ./data.txt 1.799 ± 0.024 1.779 1.827 3.29 ± 0.05
crabz -p 32 -c 6 < ./data.txt 1.192 ± 0.011 1.183 1.205 2.18 ± 0.03
pigz -p 32 -6 < ./data.txt 1.196 ± 0.016 1.183 1.214 2.19 ± 0.03
crabz -p 1 -c 9 < ./data.txt 44.907 ± 0.283 44.585 45.116 82.03 ± 0.84
pigz -p 1 -9 < ./data.txt 53.109 ± 1.049 52.373 54.311 97.02 ± 2.07
crabz -p 2 -c 9 < ./data.txt 19.977 ± 0.159 19.819 20.136 36.49 ± 0.41
pigz -p 2 -9 < ./data.txt 26.562 ± 0.134 26.407 26.643 48.52 ± 0.46
crabz -p 4 -c 9 < ./data.txt 10.397 ± 0.484 10.070 10.953 18.99 ± 0.90
pigz -p 4 -9 < ./data.txt 13.346 ± 0.040 13.300 13.372 24.38 ± 0.21
crabz -p 8 -c 9 < ./data.txt 5.100 ± 0.021 5.076 5.114 9.32 ± 0.08
pigz -p 8 -9 < ./data.txt 6.754 ± 0.016 6.736 6.767 12.34 ± 0.10
crabz -p 16 -c 9 < ./data.txt 2.716 ± 0.014 2.708 2.732 4.96 ± 0.05
pigz -p 16 -9 < ./data.txt 3.444 ± 0.038 3.420 3.487 6.29 ± 0.09
crabz -p 32 -c 9 < ./data.txt 1.747 ± 0.009 1.740 1.758 3.19 ± 0.03
pigz -p 32 -9 < ./data.txt 2.086 ± 0.008 2.077 2.093 3.81 ± 0.03

Decompression

Command Mean [s] Min [s] Max [s] Relative
crabz -d < ./data.3.txt.gz 1.599 ± 0.014 1.573 1.615 1.00
pigz -d < ./data.3.txt.gz 1.696 ± 0.020 1.654 1.725 1.06 ± 0.02
crabz -d < ./data.6.txt.gz 1.615 ± 0.012 1.586 1.626 1.01 ± 0.01
pigz -d < ./data.6.txt.gz 1.760 ± 0.030 1.687 1.797 1.10 ± 0.02
crabz -d < ./data.9.txt.gz 1.613 ± 0.014 1.596 1.641 1.01 ± 0.01
pigz -d < ./data.9.txt.gz 1.767 ± 0.012 1.748 1.787 1.11 ± 0.01

Block Formats with libdeflater

Decompression

Command Mean [s] Min [s] Max [s] Relative
crabz -p 1 -d -f mgzip ./bdata.3.txt.gz > data.txt 1.221 ± 0.164 1.073 1.397 2.32 ± 0.31
pigz -d -c ./bdata.3.txt.gz > data.txt 2.415 ± 0.063 2.347 2.472 4.58 ± 0.14
crabz -p 1 -d -f mgzip ./bdata.6.txt.gz > data.txt 1.256 ± 0.063 1.200 1.325 2.38 ± 0.13
pigz -d -c ./bdata.6.txt.gz > data.txt 2.513 ± 0.052 2.467 2.569 4.77 ± 0.13
crabz -p 1 -d -f mgzip ./bdata.9.txt.gz > data.txt 1.147 ± 0.065 1.094 1.219 2.18 ± 0.13
pigz -d -c ./bdata.9.txt.gz > data.txt 2.394 ± 0.118 2.262 2.488 4.54 ± 0.24
crabz -p 1 -d -f mgzip ./bdata.12.txt.gz > data.txt 1.165 ± 0.074 1.106 1.248 2.21 ± 0.15
pigz -d -c ./bdata.12.txt.gz > data.txt 2.457 ± 0.067 2.408 2.534 4.66 ± 0.15
crabz -p 2 -d -f mgzip ./bdata.3.txt.gz > data.txt 0.634 ± 0.008 0.628 0.642 1.20 ± 0.03
pigz -d -c ./bdata.3.txt.gz > data.txt 2.379 ± 0.012 2.368 2.391 4.51 ± 0.08
crabz -p 2 -d -f mgzip ./bdata.6.txt.gz > data.txt 0.645 ± 0.015 0.629 0.658 1.22 ± 0.03
pigz -d -c ./bdata.6.txt.gz > data.txt 2.438 ± 0.073 2.356 2.497 4.62 ± 0.16
crabz -p 2 -d -f mgzip ./bdata.9.txt.gz > data.txt 0.659 ± 0.015 0.644 0.674 1.25 ± 0.04
pigz -d -c ./bdata.9.txt.gz > data.txt 2.451 ± 0.075 2.400 2.538 4.65 ± 0.16
crabz -p 2 -d -f mgzip ./bdata.12.txt.gz > data.txt 0.656 ± 0.015 0.647 0.673 1.24 ± 0.04
pigz -d -c ./bdata.12.txt.gz > data.txt 2.450 ± 0.045 2.412 2.500 4.65 ± 0.12
crabz -p 4 -d -f mgzip ./bdata.3.txt.gz > data.txt 0.577 ± 0.024 0.554 0.603 1.10 ± 0.05
pigz -d -c ./bdata.3.txt.gz > data.txt 2.459 ± 0.052 2.420 2.518 4.66 ± 0.13
crabz -p 4 -d -f mgzip ./bdata.6.txt.gz > data.txt 0.559 ± 0.024 0.531 0.576 1.06 ± 0.05
pigz -d -c ./bdata.6.txt.gz > data.txt 2.538 ± 0.044 2.502 2.587 4.81 ± 0.12
crabz -p 4 -d -f mgzip ./bdata.9.txt.gz > data.txt 0.552 ± 0.011 0.539 0.560 1.05 ± 0.03
pigz -d -c ./bdata.9.txt.gz > data.txt 2.402 ± 0.018 2.385 2.420 4.56 ± 0.08
crabz -p 4 -d -f mgzip ./bdata.12.txt.gz > data.txt 0.592 ± 0.040 0.546 0.616 1.12 ± 0.08
pigz -d -c ./bdata.12.txt.gz > data.txt 2.525 ± 0.038 2.484 2.558 4.79 ± 0.11
crabz -p 8 -d -f mgzip ./bdata.3.txt.gz > data.txt 0.563 ± 0.013 0.548 0.571 1.07 ± 0.03
pigz -d -c ./bdata.3.txt.gz > data.txt 2.490 ± 0.126 2.369 2.621 4.72 ± 0.25
crabz -p 8 -d -f mgzip ./bdata.6.txt.gz > data.txt 0.552 ± 0.018 0.533 0.569 1.05 ± 0.04
pigz -d -c ./bdata.6.txt.gz > data.txt 2.531 ± 0.115 2.417 2.647 4.80 ± 0.23
crabz -p 8 -d -f mgzip ./bdata.9.txt.gz > data.txt 0.603 ± 0.029 0.583 0.636 1.14 ± 0.06
pigz -d -c ./bdata.9.txt.gz > data.txt 2.483 ± 0.042 2.435 2.515 4.71 ± 0.11
crabz -p 8 -d -f mgzip ./bdata.12.txt.gz > data.txt 0.527 ± 0.009 0.519 0.537 1.00
pigz -d -c ./bdata.12.txt.gz > data.txt 2.524 ± 0.093 2.417 2.583 4.79 ± 0.19
crabz -p 16 -d -f mgzip ./bdata.3.txt.gz > data.txt 0.603 ± 0.058 0.551 0.665 1.14 ± 0.11
pigz -d -c ./bdata.3.txt.gz > data.txt 2.392 ± 0.007 2.384 2.397 4.54 ± 0.08
crabz -p 16 -d -f mgzip ./bdata.6.txt.gz > data.txt 0.611 ± 0.065 0.565 0.686 1.16 ± 0.13
pigz -d -c ./bdata.6.txt.gz > data.txt 2.593 ± 0.148 2.427 2.712 4.92 ± 0.29
crabz -p 16 -d -f mgzip ./bdata.9.txt.gz > data.txt 0.564 ± 0.027 0.541 0.594 1.07 ± 0.05
pigz -d -c ./bdata.9.txt.gz > data.txt 2.426 ± 0.023 2.404 2.450 4.60 ± 0.09
crabz -p 16 -d -f mgzip ./bdata.12.txt.gz > data.txt 0.601 ± 0.020 0.582 0.623 1.14 ± 0.04
pigz -d -c ./bdata.12.txt.gz > data.txt 2.528 ± 0.022 2.507 2.550 4.80 ± 0.09
crabz -p 32 -d -f mgzip ./bdata.3.txt.gz > data.txt 0.595 ± 0.019 0.577 0.614 1.13 ± 0.04
pigz -d -c ./bdata.3.txt.gz > data.txt 2.544 ± 0.107 2.422 2.621 4.83 ± 0.22
crabz -p 32 -d -f mgzip ./bdata.6.txt.gz > data.txt 0.601 ± 0.021 0.586 0.626 1.14 ± 0.05
pigz -d -c ./bdata.6.txt.gz > data.txt 2.519 ± 0.114 2.435 2.649 4.78 ± 0.23
crabz -p 32 -d -f mgzip ./bdata.9.txt.gz > data.txt 0.565 ± 0.023 0.539 0.579 1.07 ± 0.05
pigz -d -c ./bdata.9.txt.gz > data.txt 2.487 ± 0.064 2.415 2.540 4.72 ± 0.15
crabz -p 32 -d -f mgzip ./bdata.12.txt.gz > data.txt 0.557 ± 0.013 0.548 0.571 1.06 ± 0.03
pigz -d -c ./bdata.12.txt.gz > data.txt 2.505 ± 0.105 2.442 2.626 4.75 ± 0.22

TODOs

  • Test with jemalloc
  • Add some form of auto format detection, even just by file extension