Crate crc32_v2

Expand description

§⚡ CRC32-V2

The fastest pure-software, no_std-compatible CRC-32 implementation in Rust, slicing-by-16 tables, zero heap allocation in the hot path, and native Python / Node.js bindings, all written in 100% safe Rust 🗿.

§🚀 Installation

Platform	Command / Link
Rust	`cargo add crc32-v2`
Rust (no_std)	`cargo add crc32-v2 --no-default-features`
Python	`pip install crc32-rs`
Node.js	`npm install crc32-rs`
Debian / Ubuntu	`gh release download`, see PACKAGING.md
RHEL / Fedora	`gh release download`, see PACKAGING.md

§🔭 Features

Feature	Description
`no_std` + `alloc`	Embeddable in bare-metal, WASM, and embedded targets, no standard library required
100% safe Rust	`#![forbid(unsafe_code)]` enforced crate-wide; zero `unsafe` blocks
`crc32`	Byte-at-a-time baseline (~343 MiB/s); compatible with zlib, PKZIP, Ethernet, and FDDI
`crc32_little`	Slicing-by-4 (~833 MiB/s); 2.4× faster than byte-at-a-time
`crc32_little_8`	Slicing-by-8 (~1,004 MiB/s); 2.9× faster than byte-at-a-time
`crc32_little_16`	Slicing-by-16 (~1,282 MiB/s); 3.7× fastest pure-software path
`crc32_big`	Big-endian (unreflected) variant for hardware CRC controllers
`crc32_combine`	Merge two independently computed CRCs in O(log n) via GF(2) matrix squaring
Streaming `Digest`	Incremental checksum with zero-copy buffering
Build-time tables	`crc32-codegen` subcrate generates all 16 lookup tables at compile time
Python bindings	Via PyO3 / maturin (`pip install crc32-rs`)
Node.js bindings	Via napi-rs (`npm install crc32-rs`)

§🦀 Rust

§One-shot checksum

use crc32_v2::crc32;

let data = b"Hello, world!";
println!("CRC-32: {:#010X}", crc32(0, data));  // CRC-32: 0xEBE6C6E6
assert_eq!(crc32(0, b"123456789"), 0xCBF43926);

§Slicing-by-16 (fastest pure-software path, ~1,282 MiB/s)

use crc32_v2::byfour::crc32_little_16;

let data: Vec<u8> = (0..1_048_576).map(|i| i as u8).collect();
let crc = crc32_little_16(0, &data);
println!("CRC-32: {crc:#010X}");
assert_eq!(crc32_little_16(0, b"123456789"), 0xCBF43926);

§Slicing-by-8 (~1,004 MiB/s)

use crc32_v2::byfour::crc32_little_8;

let crc = crc32_little_8(0, b"Hello, world!");
assert_eq!(crc, 0xEBE6_C6E6);

§Slicing-by-4 (~833 MiB/s)

use crc32_v2::byfour::crc32_little;

let crc = crc32_little(0, b"Hello, world!");
assert_eq!(crc, 0xEBE6_C6E6);

§Combining two checksums without the original data

use crc32_v2::{crc32, crc32_combine};

let crc1 = crc32(0, b"Hello, ");
let crc2 = crc32(0, b"world!");
let combined = crc32_combine(crc1, crc2, b"world!".len() as u64);
assert_eq!(combined, crc32(0, b"Hello, world!"));

§Streaming `Digest`

use crc32_v2::Digest;

let mut digest = Digest::new();
digest.update(b"Hello, ");
digest.update(b"world!");
println!("CRC-32: {:#010X}", digest.finalize()); // 0xEBE6C6E6
let bytes: [u8; 4] = digest.digest();            // [0xEB, 0xE6, 0xC6, 0xE6]

§`no_std` usage

[dependencies]
crc32-v2 = { version = "0.2.0", default-features = false }

#![no_std]

use crc32_v2::{crc32, byfour::crc32_little_16, Digest};

let crc = crc32_little_16(0, b"embedded payload");
assert!(crc != 0);

§🐍 Python

See PYTHON.md for the full API reference.

pip install crc32-rs

from crc32_rs import crc32, crc32_little_16, crc32_bytes, crc32_hex, Digest

print(hex(crc32(b"Hello, world!")))             # 0xebe6c6e6
print(hex(crc32_little_16(b"Hello, world!")))   # 0xebe6c6e6
print(crc32_bytes(b"Hello, world!").hex())      # ebe6c6e6
print(crc32_hex(b"Hello, world!"))              # ebe6c6e6

d = Digest()
d.update(b"Hello, ")
d.update(b"world!")
print(hex(d.finalize()))   # 0xebe6c6e6
print(d.digest().hex())    # ebe6c6e6
print(repr(d))             # Digest(crc=0xEBE6C6E6)

§🟩 Node.js

See NODE.md for the full API reference.

npm install crc32-rs

const { crc32, crc32Little16, Digest } = require("crc32-rs");

console.log(crc32(Buffer.from("Hello, world!")).toString(16)); // ebe6c6e6
console.log(crc32Little16(Buffer.from("Hello, world!")).toString(16)); // ebe6c6e6

§📊 Benchmarks

Measured with cargo bench (lto="fat", opt-level=3, codegen-units=1, overflow-checks=false) on Linux x86-64 (Rust 1.94.1). Full methodology in BENCHMARKS.md.

§Throughput across payload sizes

Method	1 B	64 B	1 KiB	64 KiB	1 MiB	Throughput
`crc32_v2::crc32`	2 ns	167 ns	2,856 ns	188,134 ns	2,916,562 ns	~343 MiB/s
`crc32_v2::crc32_little`	3 ns	73 ns	1,086 ns	82,129 ns	1,199,953 ns	~833 MiB/s
`crc32_v2::crc32_little_8`	3 ns	61 ns	922 ns	56,486 ns	1,004,746 ns	~1,004 MiB/s
`crc32_v2::crc32_little_16`	3 ns	39 ns	753 ns	48,101 ns	781,535 ns	~1,282 MiB/s
`crc32fast::hash` (SIMD)	11 ns	20 ns	102 ns	5,802 ns	89,204 ns	~11,300 MiB/s

§Cross-library comparison: 1 MiB payload

Library	Method	Time	Throughput	Implementation
`crc32fast`	`hash`	89,204 ns	~11,300 MiB/s	SIMD (`pclmulqdq`) / Rust
`zlib-rs`	`crc32`	96,968 ns	~10,390 MiB/s	SIMD / Rust
Python (`zlib`)	`zlib.crc32`	355,270 ns	~2,815 MiB/s	SIMD / C extension
`crc32-v2`	`crc32_little_16`	781,535 ns	~1,282 MiB/s	Pure safe Rust
Python (`crcmod`)	`crcmod`	2,865,836 ns	~349 MiB/s	Non-SIMD / C extension

§Python overhead: small inputs

Payload	Python `zlib.crc32`	Rust `crc32-v2`	Speedup
1 B	~297 ns	~2 ns	~148×
64 B	~301 ns	~39 ns	~7.7×

Key takeaways

crc32_little_16 achieves ~1,282 MiB/s, the fastest pure-software, safe-Rust CRC-32 path, 3.7× faster than byte-at-a-time.

For inputs ≤ 1 byte, crc32 (2 ns) wins because slicing variants pay an alignment-prologue cost.

crc32fast achieves ~11 GiB/s via runtime hardware SIMD. For no_std, embedded, or WASM targets, use crc32-v2.

Python interpreter overhead dominates for small payloads: Rust is 148× faster for 1-byte inputs.

§🗂 Module Structure

Module	Contents
`tables`	`crc32`: byte-at-a-time CRC-32 function
`byfour`	`crc32_little` / `crc32_little_8` / `crc32_little_16` / `crc32_big`
`combine`	`crc32_combine`: O(log n) GF(2) matrix merging
`digest`	`Digest`: streaming incremental interface
`crc32tables`	16 LE slicing tables + BE table (generated by `crc32-codegen` at build time)

§🔒 Safety

#![forbid(unsafe_code)] is enforced at the crate root. Every byte of the implementation, 16-table generation, slicing loops, GF(2) matrix arithmetic, and the streaming Digest, is written in safe Rust. The no_std build path carries the same safety guarantees.

§📚 Further Reading

BENCHMARKS.md: Full benchmark methodology and measured results
RUST.md: Complete Rust API reference with complexity table
PYTHON.md: Python bindings guide
NODE.md: Node.js bindings guide
PACKAGING.md: Debian and RPM packaging guide
A Painless Guide to CRC Error Detection Algorithms
zlib: crc32.c: the C source this crate is ported from
crc32fast: SIMD-accelerated CRC-32 for Rust
crc: generic CRC for many widths and polynomials

[!NOTE] This project is a successor of the awesome crc32 crate created by the kawaii engineers at Microsoft 👉👈.

§📄 License

Licensed under the MIT License.

§crc32-v2 Rust Documentation 🦀

The crc32-v2 Rust crate provides a pure-software, 100%-safe-Rust CRC-32 implementation with slicing-by-4/8/16 high-throughput variants, a streaming Digest type, O(log n) CRC combining, and native Python / Node.js bindings.

§📦 Installation

[dependencies]
crc32-v2 = "0.2.0"

§🗂 Module Structure

Module	Contents
`tables`	`crc32`: byte-at-a-time CRC-32 (IEEE 802.3, ~343 MiB/s)
`byfour`	`crc32_little` / `crc32_little_8` / `crc32_little_16` / `crc32_big`
`combine`	`crc32_combine`: O(log n) GF(2) matrix merging
`digest`	`Digest`: streaming incremental checksum
`crc32tables`	16 LE slicing tables + BE table (build-time, `crc32-codegen`)

§🛠 Core Functions

§`crc32`: One-shot checksum

use crc32_v2::crc32;

assert_eq!(crc32(0, b"123456789"), 0xCBF43926);

let crc = crc32(crc32(0, b"Hello, "), b"world!");
assert_eq!(crc, crc32(0, b"Hello, world!"));

§`crc32_little_16`: Fastest pure-software path (~1.3 GiB/s)

use crc32_v2::byfour::crc32_little_16;

assert_eq!(crc32_little_16(0, b"123456789"), 0xCBF43926);

let data: Vec<u8> = (0u8..=255).cycle().take(1_048_576).collect();
let crc = crc32_little_16(0, &data);
println!("CRC-32: {crc:#010X}");

§`crc32_little_8`: Slicing-by-8 (~1 GiB/s)

use crc32_v2::byfour::crc32_little_8;

assert_eq!(crc32_little_8(0, b"123456789"), 0xCBF43926);

§`crc32_little`: Slicing-by-4 (~833 MiB/s)

use crc32_v2::byfour::crc32_little;

assert_eq!(crc32_little(0, b"123456789"), 0xCBF43926);

§`crc32_big`: Big-endian (unreflected)

use crc32_v2::byfour::crc32_big;

let empty: &[u8] = &[];
assert_eq!(crc32_big(0, empty), 0);

let full = crc32_big(0, b"Hello, world!");
let chained = crc32_big(crc32_big(0, b"Hello, "), b"world!");
assert_eq!(full, chained);

§`crc32_combine`: Merge two CRCs in O(log n)

use crc32_v2::{crc32, crc32_combine};

let crc1 = crc32(0, b"Hello, ");
let crc2 = crc32(0, b"world!");
let combined = crc32_combine(crc1, crc2, b"world!".len() as u64);
assert_eq!(combined, crc32(0, b"Hello, world!"));

§`Digest`: Streaming incremental checksum

use crc32_v2::Digest;

let mut digest = Digest::new();
digest.update(b"Hello, ");
digest.update(b"world!");
assert_eq!(digest.finalize(), 0xEBE6C6E6);

let bytes: [u8; 4] = digest.digest();
assert_eq!(u32::from_be_bytes(bytes), 0xEBE6C6E6);

let existing = crc32_v2::crc32(0, b"prefix:");
let mut d = Digest::with_initial(existing);
d.update(b" suffix");
assert_eq!(d.finalize(), crc32_v2::crc32(existing, b" suffix"));

§📖 API Complexity Table

Method	Time	Notes
`crc32(start, buf)`	O(n)	byte-at-a-time, ~343 MiB/s
`crc32_little(start, buf)`	O(n)	slicing-by-4, ~833 MiB/s
`crc32_little_8(start, buf)`	O(n)	slicing-by-8, ~1 GiB/s
`crc32_little_16(start, buf)`	O(n)	slicing-by-16, ~1.3 GiB/s
`crc32_big(start, buf)`	O(n)	big-endian (unreflected)
`crc32_combine(c1, c2, len2)`	O(log n)	GF(2) repeated squaring
`Digest::new()`	O(1)
`Digest::update(data)`	O(n)	n = data.len()
`Digest::finalize()`	O(1)	does not reset
`Digest::digest()`	O(1)	returns `[u8; 4]` (big-endian)
`Digest::reset()`	O(1)

§🏗 Build-time Code Generation: `crc32-codegen`

The 16 little-endian slicing tables and the big-endian table are generated at compile-time by the crc32-codegen subcrate and written to $OUT_DIR/crc32tables.rs. The main crate includes the generated file via include!().

This pattern (subcrate + build.rs) means that the tables are always consistent with the source, and no generated files are committed to the repository.

§📊 Running Benchmarks

# All benchmark groups
cargo bench --bench benchmark
cargo bench --bench nano_benchmark
cargo bench --bench compare_benchmark

# Quick run
cargo bench -- --quick

Results are written to target/criterion/ as interactive HTML reports. See BENCHMARKS.md for measured throughput numbers.

§🔒 Safety

#![forbid(unsafe_code)] is enforced crate-wide. All CRC-32 algorithms (table lookups, slicing loops, GF(2) matrix arithmetic) are pure safe Rust.

§🔗 See Also

docs.rs/crc32-v2
BENCHMARKS.md: measured throughput vs zlib-rs, crc32fast, Python
PACKAGING.md: Debian and RPM packaging guide
PYTHON.md: Python bindings guide
NODE.md: Node.js bindings guide
A Painless Guide to CRC Error Detection Algorithms
zlib - crc32.c

Re-exports§

pub use combine::crc32_combine;
pub use digest::Digest;
pub use tables::crc32;

Modules§

byfour: Multi-Width CRC-32 Implementations
combine: GF(2) Matrix Arithmetic and CRC Combination
crc32tables: Generated CRC-32 lookup tables, produced by crc32-codegen at build time.
digest: Streaming CRC-32 Digest
nodenode and std: Node.js Bindings
tables: CRC-32 Byte-at-a-Time Table Lookup

Crate crc32_v2

Crate crc32_v2 Copy item path

§⚡ CRC32-V2

§🚀 Installation

§🔭 Features

§🦀 Rust

§One-shot checksum

§Slicing-by-16 (fastest pure-software path, ~1,282 MiB/s)

§Slicing-by-8 (~1,004 MiB/s)

§Slicing-by-4 (~833 MiB/s)

§Combining two checksums without the original data

§Streaming Digest

§no_std usage

§🐍 Python

§🟩 Node.js

§📊 Benchmarks

§Throughput across payload sizes

§Cross-library comparison: 1 MiB payload

§Python overhead: small inputs

§🗂 Module Structure

§🔒 Safety

§📚 Further Reading

§📄 License

§crc32-v2 Rust Documentation 🦀

§📦 Installation

§🗂 Module Structure

§🛠 Core Functions

§crc32: One-shot checksum

§crc32_little_16: Fastest pure-software path (~1.3 GiB/s)

§crc32_little_8: Slicing-by-8 (~1 GiB/s)

§crc32_little: Slicing-by-4 (~833 MiB/s)

§crc32_big: Big-endian (unreflected)

§crc32_combine: Merge two CRCs in O(log n)

§Digest: Streaming incremental checksum

§📖 API Complexity Table

§🏗 Build-time Code Generation: crc32-codegen

§📊 Running Benchmarks

§🔒 Safety

§🔗 See Also

Re-exports§

Modules§

Crate crc32_v2

§Streaming `Digest`

§`no_std` usage

§`crc32`: One-shot checksum

§`crc32_little_16`: Fastest pure-software path (~1.3 GiB/s)

§`crc32_little_8`: Slicing-by-8 (~1 GiB/s)

§`crc32_little`: Slicing-by-4 (~833 MiB/s)

§`crc32_big`: Big-endian (unreflected)

§`crc32_combine`: Merge two CRCs in O(log n)

§`Digest`: Streaming incremental checksum

§🏗 Build-time Code Generation: `crc32-codegen`