hashcrew

Overview

Hashcrew is a zero-dependency Rust library for fast, deterministic hashing in non-cryptographic applications. It provides allocation-free one-shot APIs, incremental state where the algorithm supports it, stable cross-platform digests for identical raw byte streams, and hardware-accelerated XXH3 kernels.

Every implementation supports no_std. XXH3 inputs longer than 240 bytes use a dedicated kernel layer with scalar, little-endian AArch64 NEON, x86-64 SSE2, and x86-64 AVX2 backends; the other algorithms use compact portable Rust cores.

[!WARNING]

These algorithms are not cryptographically secure. Deterministic hashers are also unsuitable for hash tables exposed to attacker-controlled keys because they do not protect against deliberate hash flooding.

Getting started

Hash families are opt-in Cargo features. For example, enable xxHash with:

cargo add hashcrew --features xxhash

No features are enabled by default, and every family works in no_std builds. Enable std explicitly when you need standard I/O adapters or XXH3 runtime CPU detection:

[dependencies]
hashcrew = { version = "0.2", features = ["std", "xxhash"] }

Import the algorithm family when the complete input is already in memory:

use hashcrew::xxhash::xxh3_64;

assert_ne!(xxh3_64(b"hashcrew"), 0);

Use a state type when data arrives incrementally:

use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64;

let mut hash = Xxh3_64::new();
hash.update(b"hash");
hash.update(b"crew");

assert_eq!(hash.digest(), xxh3_64(b"hashcrew"));

Custom XXH3 secrets can be borrowed or moved into the streaming state. Owning the storage is useful when a factory or component needs to return a self-contained hasher:

use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64_with_secret;

let secret = [0xa5; 192];
let expected = xxh3_64_with_secret(b"hashcrew", &secret).unwrap();
let mut hash = Xxh3_64::with_secret(secret).unwrap();
hash.update(b"hashcrew");

assert_eq!(hash.digest(), expected);

All public APIs are grouped under the cityhash, xxhash, murmur, fnv, and md5 modules. Each module keeps its one-shot functions, streaming states, builders, and configuration together.

Feature flags

No features are enabled by default. Each family feature exposes its same-named module. Enable multiple families together, such as features = ["xxhash", "md5"], and add std when its adapters or runtime CPU detection are needed.

The std feature does not enable any hash family. All families work without it; feature selection does not change digest values, and every configuration remains dependency-free and allocation-free.

API model

Hashcrew exposes the same algorithm at different integration boundaries. Pick the narrowest interface that matches where the bytes come from:

Hasher only supports a u64 result, so 128-bit states deliberately preserve their complete output: MD5 returns [u8; 16] in standard digest byte order, while the other 128-bit algorithms return u128. CityHash has neither a state nor standard adapters because it cannot hash incrementally with bounded memory.

Algorithm and capability map

The table names the canonical module-level function for complete input. A trailing * means the family also provides explicitly named seeded, custom-secret, or custom-offset-basis forms.

Hashcrew implements all three variants from the original MurmurHash3 family under their reference-qualified x86_32, x86_128, and x64_128 names. These architecture labels distinguish algorithms and do not restrict which target can run them. cityhash128_to_64 reduces an existing 128-bit CityHash value; it does not hash a new byte slice.

Choosing an algorithm

Use XXH3 for a new general-purpose checksum, cache key, or trusted-input hash table unless interoperability requires another family. Choose a 128-bit result when the application hashes enough distinct values for 64-bit collision probability to matter. XXH32, XXH64, CityHash, MurmurHash3, and FNV-1a are primarily useful for matching an existing format, protocol, or data set; their different outputs are not interchangeable.

MD5 is provided for compatibility with existing formats and protocols that require its standard digest. Its cryptographic security is broken. Call hashcrew::md5::md5(input) for complete input, or use hashcrew::md5::Md5 for streaming; both return the same 16 digest bytes without a RustCrypto dependency.

Streaming input

Call update when the application already has byte slices, as in the getting-started example above. With the std feature, every streaming state can also be used as the destination of std::io::copy or another producer that accepts std::io::Write.

The adapter treats every written byte as hash input; it accepts the complete buffer and has nothing to flush. It does not write the digest anywhere. Finish the producer first, then call digest on the state:

use std::io::{self, Cursor};
use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64;

let mut source = Cursor::new(b"hashcrew");
let mut hash = Xxh3_64::new();
io::copy(&mut source, &mut hash).unwrap();

assert_eq!(hash.digest(), xxh3_64(b"hashcrew"));

This adapter is only needed for std interoperability. The direct update API is available in both std and no_std builds.

Hash tables

The 32-bit and 64-bit streaming states implement core::hash::Hasher, with matching BuildHasher types for trusted-input hash tables:

use std::collections::HashMap;
use hashcrew::xxhash::Xxh3_64Builder;

let mut counts = HashMap::with_hasher(Xxh3_64Builder::with_seed(7));
counts.insert("hashcrew", 1);

assert_eq!(counts["hashcrew"], 1);

CityHash is intentionally one-shot. Its digest depends on the complete input length and tail, so a streaming facade would have to retain the entire message and would not provide bounded-memory incremental hashing.

XXH3 accepts custom secrets of at least 136 bytes and returns an error for shorter inputs. Its seed-and-secret APIs follow the reference contract: inputs up to 240 bytes use the seed, while longer inputs use the custom secret. Custom secrets and non-standard FNV offset bases alter deterministic output; neither makes these algorithms cryptographically secure.

Portability

Raw and streaming digests are stable across platforms for identical byte streams. Rust's Hash and BuildHasher adapters use typed encodings that can vary across platforms and compiler versions. They can also add framing bytes to strings and slices, so builder.hash_one(value) need not match hashing value.as_bytes() or the slice directly. Use one-shot functions or update with a defined byte encoding for persistent checksums and cross-language protocols.

Integer digests still need an explicit output byte order: xxHash's canonical format uses to_be_bytes(), while FNV's RFC format uses to_le_bytes(). MurmurHash3's to_le_bytes() reproduces the reference output on little-endian systems; for CityHash, follow the consuming format's word and byte order. MD5 already returns its standard digest bytes; its module documentation shows how to format them as hexadecimal with leading zeroes.

Target-guaranteed CPU features are selected at compile time. Other std builds cache runtime feature detection; no_std builds use compile-time features only and otherwise fall back to the scalar kernel. hashcrew::xxhash::kernel::selected_backend() reports the selected XXH3 backend.

Examples and benchmarks

Runnable examples live in the examples workspace crate. The benchmarks crate contains one-shot and streaming comparisons with independent implementations; see its benchmark guide for filters, input sizes, and the complete case matrix.

Repository workflows use the active Rust toolchain. cargo x lint selects nightly for Clippy, rustfmt, and rustdoc; its documentation check uses all features and the same docsrs configuration as docs.rs. cargo x miri also selects nightly. Use cargo x --help to list the workflows, or run common workflows with:

cargo x check
cargo x test
cargo x bench
cargo +nightly x miri

cargo x check validates empty, individual, and combined family configurations with and without std. Cross-target checks use --target <triple> and compile the library; add --no-std for targets without the standard library, and --rustflags "-C target-feature=..." to validate a specific hardware backend.

See the release guide for checks on stable and the MSRV.

Correctness

Integration tests compare CityHash, xxHash, MurmurHash3, and MD5 with independent implementations. FNV-1a and MD5 also have RFC vectors in the library tests; FNV-1a 64 has an independent implementation comparison. The suite covers boundary lengths, multiple seeds, custom secrets, custom FNV offset bases, randomized inputs, streaming partitions, available hardware backends, and both std and no_std builds.

Minimum Rust version policy

This crate's minimum supported rustc version is 1.85.0.

The current policy is that the minimum Rust version required to use this crate can be increased in minor version updates. For example, if crate 1.0 requires Rust 1.85.0, then crate 1.0.z for all values of z will also require Rust 1.85.0 or newer. However, crate 1.y for y > 0 may require a newer minimum version of Rust.

License and acknowledgements

This project is licensed under Apache License, Version 2.0. The license file also records incorporated third-party code and its copyright notices and terms.