hashcrew

Overview

Hashcrew is a zero-dependency Rust library for fast, deterministic, non-cryptographic hashing. It provides allocation-free one-shot APIs, incremental state where the algorithm supports it, stable cross-platform digests for identical raw byte streams, and hardware-accelerated XXH3 kernels.

Every implementation supports no_std. XXH3 inputs longer than 240 bytes use a dedicated kernel layer with scalar, little-endian AArch64 NEON, x86-64 SSE2, and x86-64 AVX2 backends; the other algorithms use compact portable Rust cores.

[!WARNING]

These algorithms are not cryptographically secure. Deterministic hashers are also unsuitable for hash tables exposed to attacker-controlled keys because they do not protect against deliberate hash flooding.

Getting started

cargo add hashcrew

Disable the default std feature for bare-metal and other no_std targets:

[dependencies]
hashcrew = { version = "0.1", default-features = false }

Import the algorithm family when the complete input is already in memory:

use hashcrew::{cityhash, fnv, murmur, xxhash};

let data = b"hashcrew";
let city = cityhash::cityhash64(data);
let xxh3 = xxhash::xxh3_64(data);
let murmur = murmur::murmur3_x64_128(data, 42);
let fnv = fnv::fnv1a_64(data);

assert_ne!(city, 0);
assert_ne!(xxh3, 0);
assert_ne!(murmur, 0);
assert_ne!(fnv, 0);

Use a state type when data arrives incrementally:

use hashcrew::murmur::Murmur3X64_128;
use hashcrew::murmur::murmur3_x64_128;

let mut hash = Murmur3X64_128::with_seed(42);
hash.update(b"hash");
hash.update(b"crew");

assert_eq!(hash.digest(), murmur3_x64_128(b"hashcrew", 42));

Custom XXH3 secrets can be borrowed or moved into the streaming state. Owning the storage is useful when a factory or component needs to return a self-contained hasher:

use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64_with_secret;

let secret = [0xa5; 192];
let expected = xxh3_64_with_secret(b"hashcrew", &secret).unwrap();
let mut hash = Xxh3_64::with_secret(secret).unwrap();
hash.update(b"hashcrew");

assert_eq!(hash.digest(), expected);

All public APIs are grouped under the cityhash, xxhash, murmur, and fnv modules. Each module keeps its one-shot functions, streaming states, builders, and configuration together.

API model

Hashcrew exposes the same algorithm at different integration boundaries. Pick the narrowest interface that matches where the bytes come from:

Hasher only supports a u64 result, so 128-bit states deliberately expose digest() -> u128 instead of truncating their output. CityHash has neither a state nor standard adapters because it cannot hash incrementally with bounded memory.

Algorithm and capability map

The table names the canonical module-level function for complete input. A trailing * means the family also provides explicitly named seeded, custom-secret, or custom-offset-basis forms.

Hashcrew implements all three variants from the original MurmurHash3 family under their reference-qualified x86_32, x86_128, and x64_128 names. These architecture labels distinguish algorithms and do not restrict which target can run them. cityhash128_to_64 reduces an existing 128-bit CityHash value; it does not hash a new byte slice.

Choosing an algorithm

Use XXH3 for a new general-purpose checksum, cache key, or trusted-input hash table unless interoperability requires another family. Choose a 128-bit result when the application hashes enough distinct values for 64-bit collision probability to matter. XXH32, XXH64, CityHash, MurmurHash3, and FNV-1a are primarily useful for matching an existing format, protocol, or data set; their different outputs are not interchangeable.

Streaming input

Call update when the application already has byte slices, as in the getting-started example above. With the default std feature, every streaming state can also be used as the destination of std::io::copy or another producer that accepts std::io::Write.

The adapter treats every written byte as hash input; it accepts the complete buffer and has nothing to flush. It does not write the digest anywhere. Finish the producer first, then call digest on the state:

use std::io::{self, Cursor};
use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64;

let mut source = Cursor::new(b"hashcrew");
let mut hash = Xxh3_64::new();
io::copy(&mut source, &mut hash).unwrap();

assert_eq!(hash.digest(), xxh3_64(b"hashcrew"));

This adapter is only needed for std interoperability. The direct update API is available in both std and no_std builds.

Hash tables

The 32-bit and 64-bit streaming states implement core::hash::Hasher, with matching BuildHasher types for trusted-input hash tables:

use std::collections::HashMap;
use hashcrew::xxhash::Xxh3_64Builder;

let mut counts = HashMap::with_hasher(Xxh3_64Builder::with_seed(7));
counts.insert("hashcrew", 1);

assert_eq!(counts["hashcrew"], 1);

CityHash is intentionally one-shot. Its digest depends on the complete input length and tail, so a streaming facade would have to retain the entire message and would not provide bounded-memory incremental hashing.

XXH3 accepts custom secrets of at least 136 bytes and returns an error for shorter inputs. Its seed-and-secret APIs follow the reference contract: inputs up to 240 bytes use the seed, while longer inputs use the custom secret. Custom secrets and non-standard FNV offset bases alter deterministic output; neither makes these algorithms cryptographically secure.

Portability

Raw and streaming digests are stable across platforms for identical byte streams. Rust's Hash and BuildHasher adapters use native typed encodings, including platform endianness and usize width, and are not a portable serialization format.

Target-guaranteed CPU features are selected at compile time. Other std builds cache runtime feature detection; no_std builds use compile-time features only and otherwise fall back to the scalar kernel. hashcrew::xxhash::kernel::selected_backend() reports the selected XXH3 backend.

Examples and benchmarks

Runnable examples live in the examples workspace crate. The benchmarks crate contains one-shot and streaming comparisons with independent implementations; see its benchmark guide for filters, input sizes, and the complete case matrix.

Use the repository workflow commands to run them:

cargo x test
cargo x bench

Correctness

Integration tests compare CityHash, xxHash, and MurmurHash3 with independent implementations, and verify FNV-1a against RFC vectors plus an independent 64-bit implementation. The suite covers boundary lengths, multiple seeds, custom secrets, custom FNV offset bases, randomized inputs, streaming partitions, available hardware backends, and both std and no_std builds.

Minimum Supported Rust Version (MSRV)

Hashcrew's minimum supported rustc version is 1.85.0. The MSRV may be increased in a minor release.

License and acknowledgements

This project is licensed under Apache License, Version 2.0. See THIRD_PARTY_NOTICES.md for the specifications, implementations, and development-only comparison dependencies that informed Hashcrew.