Skip to main content

Module crc32

frozen_core

Module crc32 

Source
Expand description

Implementation of CRC32C (Castagnoli polynomial) to compute a 32-bit cyclic redundancy check (CRC) using Castagnoli polynomial, intended for data integrity verification for torn writes and curruption detection

[!WARNING] We assume little-endian target architecture, while big-endian architectures are not supported

[!IMPORTANT] The generated 32-bit CRC is not cryptographically secure, it’s intended use only is for data integrity in IO ops

§What is CRC?

Cyclic Redundancy Check (CRC) is an err detecting code, e.g. 32 bits, used for data integrity in networking protocols and storage systems, to check whether data has been altered, partially written/send or corrupted

For a given buffer of data, e.g. [u8; 0x100], we first compute a 32-bit crc, send/store it w/ the data, and when received/read, again compute crc on the buffer and match it w/ the original, to check for integrity

§CRC32C

CRC32C is a specific version of crc algo, which generates 32-bit crc using the Castagnoli polynomial

It uses polynomial arithmetic in GF(2), the generator polynomial for CRC32C is 0x1EDC6F41, and we use the reflected form 0x82F63B78, which is required by little-endian streaming algorithms

§Benchmarks

Look at following benches for the throughout and latency measurements,

| Mode | Size    | Time (ns / µs)        | Throughput (GiB/s) |
|:----:|:-------:|:---------------------:|:------------------:|
| 1x   | 4 KiB   | 318 ns                | 11.97              |
| 2x   | 4 KiB   | 340 ns                | 11.24              |
| 4x   | 4 KiB   | 451 ns                | 8.46               |
| 1x   | 64 KiB  | 5.44 µs               | 11.23              |
| 2x   | 64 KiB  | 5.26 µs               | 11.60              |
| 4x   | 64 KiB  | 7.50 µs               | 8.14               |
| 1x   | 1 MiB   | 89.80 µs              | 10.88              |
| 2x   | 1 MiB   | 90.56 µs              | 10.78              |
| 4x   | 1 MiB   | 120.04 µs             | 8.13               |

Environment used for benching,

  • OS: NixOS (WSL2)
  • Architecture: x86_64
  • Memory: 8 GiB RAM (DDR4)
  • Backend: Hardware (SSE4.2)
  • Rust: rustc 1.86.0 w/ cargo 1.86.0
  • Kernel: Linux 6.6.87.2-microsoft-standard-WSL2
  • CPU: Intel® Core™ i5-10300H @ 2.50GHz (4C / 8T)

§Example

let crc = frozen_core::crc32::Crc32C::default();

let b0: [u8; 8] = *b"12345678";
let b1: [u8; 8] = *b"ABCDEFGH";
let b2: [u8; 8] = *b"abcdefgh";
let b3: [u8; 8] = *b"zyxwvuts";

assert_eq!(crc.crc(&b0), crc.crc(&b0));
assert_eq!(crc.crc_2x([&b0, &b1]),crc.crc_2x([&b0, &b1]));
assert_eq!(crc.crc_4x([&b0, &b1, &b2, &b3]),crc.crc_4x([&b0, &b1, &b2, &b3]));

Structs§

Crc32C
Implementation of CRC32C (Castagnoli polynomial) to compute a 32-bit cyclic redundancy check (CRC) using Castagnoli polynomial