rbase64

A fast multi-threaded base64 encoding / decoding library and CLI tool, made in Rust.

Have you ever wanted to base64 encode tens of gigabytes of data, but felt your encoder was not fast enough? Me neither. Either way, rbase64 is up to the task.

The goal of rbase64 is to provide a fast implementation of base64 encoding. The library is thoroughly tested and optimized for high throughput. As of v2.0.2, rbase64 is able to achieve up to 11.019 GiB/s encoding and 11.979 GiB/s decoding rates (measured in MacBook Air M1 2020 laptop, 10~20 MiB batches).

The project also comes with a command line interface powered by the library encoder. While limited by I/O, it still achieves high performance compared to other alternatives (for example, GNU base64 by a factor of 11x).

For details, see Performance.

Usage

Library

Add rbase64 to your Cargo.toml dependencies:

[dependencies]
rbase64 = "2.0.2"

Sample usage:

use rbase64;

fn main() {
    let a = b"Hello world";
    let b = "SGVsbG8gd29ybGQ=";

    assert_eq!(rbase64::encode(a), b);
    assert_eq!(&rbase64::decode(b).unwrap(), a);
}

CLI

Installation:

$ cargo install rbase64 --features cli

Usage: rbase64 [OPTIONS]

Rbase64 CLI is designed to have the same API as classic GNU base64, however, it does not append newline characters to file or stdout output.

Basic usage:

$ echo -n "Hello world" | rbase64
SGVsbG8gd29ybGQ=
$ echo -n "SGVsbG8gd29ybGQ=" | rbase64 --decode
Hello world

Performance

Library

Performance is measured using benchmarks with Criterion. Each bench measures encoding/decoding speed and throughput at varying input sizes in bytes. Measurements below were taken by running benchmarks on a MacBook Air M1 2020 laptop. Results are compared against three of the top most widely used Rust base64 encoding libraries in cargo.io.

Encoding

Decoding

CLI

Manual benchmarks using randomized binary files. Compared against classic GNU base64, and race64 (C).

• GNU base64

$ time (cat random-1gb.bin | base64 | pv -a > /dev/null)
[ 164MiB/s]
( cat random-1gb.bin | base64 | pv -a > /dev/null; )  7.89s user 0.96s system 106% cpu 8.317 total

$ time (cat random-1gb.b64 | base64 --decode | pv -a > /dev/null)
[ 105MiB/s]
( cat random-1gb.b64 | base64 --decode | pv -a > /dev/null; )  9.16s user 1.01s system 104% cpu 9.699 total

• race64 (C)

$ time (cat random-1gb.bin | ./race64 | pv -a > /dev/null)
[ 898MiB/s]
( cat random-1gb.bin | ./race64 | pv -a > /dev/null; )  0.88s user 1.14s system 128% cpu 1.566 total

$ time (cat random-1gb.b64 | ./race64 -d | pv -a > /dev/null)
[ 723MiB/s]
( cat random-1gb.b64 | ./race64 -d | pv -a > /dev/null; )  0.87s user 0.95s system 127% cpu 1.426 total

• rbase64

$ time (cat random-1gb.bin | rbase64 | pv -a > /dev/null)
[1.71GiB/s]
( cat random-1gb.bin | rbase64 | pv -a > /dev/null; )  0.56s user 0.40s system 121% cpu 0.788 total

$ time (cat random-1gb.b64 | rbase64 --decode | pv -a > /dev/null)
[1.16GiB/s]
( cat random-1gb.b64 | rbase64 --decode | pv -a > /dev/null; )  0.59s user 0.46s system 119% cpu 0.871 total

• GNU base64

$ time (cat random-10gb.bin | base64 | pv -a > /dev/null)
[ 154MiB/s]
( cat random-10gb.bin | base64 | pv -a > /dev/null; )  78.74s user 10.96s system 101% cpu 1:28.34 total

$ time (cat random-10gb.b64 | base64 --decode | pv -a > /dev/null)
[ 107MiB/s]
( cat random-10gb.b64 | base64 --decode | pv -a > /dev/null; )  91.02s user 10.00s system 105% cpu 1:35.39 total

• race64 (C)

$ time (cat random-10gb.bin | ./race64 | pv -a > /dev/null)
[ 818MiB/s]
( cat random-10gb.bin | ./race64 | pv -a > /dev/null; )  8.56s user 13.17s system 128% cpu 16.917 total

$ time (cat random-10gb.b64 | ./race64 -d | pv -a > /dev/null)
[ 724MiB/s]
( cat random-10gb.b64 | ./race64 -d | pv -a > /dev/null; )  8.42s user 9.05s system 123% cpu 14.152 total

• rbase64

$ time (cat random-10gb.bin | rbase64 | pv -a > /dev/null)
[1.64GiB/s]
( cat random-10gb.bin | rbase64 | pv -a > /dev/null; )  5.30s user 4.22s system 117% cpu 8.125 total

$ time (cat random-10gb.b64 | rbase64 --decode | pv -a > /dev/null)
[1.18GiB/s]
( cat random-10gb.b64 | rbase64 --decode | pv -a > /dev/null; )  5.58s user 4.40s system 117% cpu 8.491 total

For reference, here's the throughput of piping a file via cat with no processing:

$ time (cat random-10gb.b64 | pv -a > /dev/null)
[2.31GiB/s]
( cat random-10gb.b64 | pv -a > /dev/null; )  0.88s user 4.72s system 95% cpu 5.858 total

License

This project is dual-licensed under MIT and Apache 2.0.