# hashcrew
[![Crates.io][crates-badge]][crates-url]
[![Documentation][docs-badge]][docs-url]
[![MSRV 1.85][msrv-badge]](https://www.whatrustisit.com)
[![Apache 2.0 licensed][license-badge]][license-url]
[![Build Status][actions-badge]][actions-url]
[crates-badge]: https://img.shields.io/crates/v/hashcrew.svg
[crates-url]: https://crates.io/crates/hashcrew
[docs-badge]: https://docs.rs/hashcrew/badge.svg
[docs-url]: https://docs.rs/hashcrew
[msrv-badge]: https://img.shields.io/badge/MSRV-1.85-green?logo=rust
[license-badge]: https://img.shields.io/crates/l/hashcrew
[license-url]: https://www.apache.org/licenses/LICENSE-2.0
[actions-badge]: https://github.com/fast/hashcrew/actions/workflows/ci.yml/badge.svg
[actions-url]: https://github.com/fast/hashcrew/actions/workflows/ci.yml
## Overview
Hashcrew is a zero-dependency Rust library for fast, deterministic hashing in non-cryptographic applications. It provides allocation-free one-shot APIs, incremental state where the algorithm supports it, stable cross-platform digests for identical raw byte streams, and hardware-accelerated XXH3 kernels.
Every implementation supports `no_std`. XXH3 inputs longer than 240 bytes use a dedicated kernel layer with scalar, little-endian AArch64 NEON, x86-64 SSE2, and x86-64 AVX2 backends; the other algorithms use compact portable Rust cores.
> [!WARNING]
>
> These algorithms are not cryptographically secure. Deterministic hashers are also unsuitable for hash tables exposed to attacker-controlled keys because they do not protect against deliberate hash flooding.
## Getting started
Hash families are opt-in Cargo features. For example, enable xxHash with:
```shell
cargo add hashcrew --features xxhash
```
No features are enabled by default, and every family works in `no_std` builds. Enable `std` explicitly when you need standard I/O adapters or XXH3 runtime CPU detection:
```toml
[dependencies]
hashcrew = { version = "0.2", features = ["std", "xxhash"] }
```
Import the algorithm family when the complete input is already in memory:
```rust
use hashcrew::xxhash::xxh3_64;
assert_ne!(xxh3_64(b"hashcrew"), 0);
```
Use a state type when data arrives incrementally:
```rust
use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64;
let mut hash = Xxh3_64::new();
hash.update(b"hash");
hash.update(b"crew");
assert_eq!(hash.digest(), xxh3_64(b"hashcrew"));
```
Custom XXH3 secrets can be borrowed or moved into the streaming state. Owning the storage is useful when a factory or component needs to return a self-contained hasher:
```rust
use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64_with_secret;
let secret = [0xa5; 192];
let expected = xxh3_64_with_secret(b"hashcrew", &secret).unwrap();
let mut hash = Xxh3_64::with_secret(secret).unwrap();
hash.update(b"hashcrew");
assert_eq!(hash.digest(), expected);
```
All public APIs are grouped under the [`cityhash`](https://docs.rs/hashcrew/*/hashcrew/cityhash/), [`xxhash`](https://docs.rs/hashcrew/*/hashcrew/xxhash/), [`murmur`](https://docs.rs/hashcrew/*/hashcrew/murmur/), [`fnv`](https://docs.rs/hashcrew/*/hashcrew/fnv/), and [`md5`](https://docs.rs/hashcrew/*/hashcrew/md5/) modules. Each module keeps its one-shot functions, streaming states, builders, and configuration together.
## Feature flags
No features are enabled by default. Each family feature exposes its same-named module. Enable multiple families together, such as `features = ["xxhash", "md5"]`, and add `std` when its adapters or runtime CPU detection are needed.
| `cityhash` | CityHash32, CityHash64, and CityHash128 |
| `fnv` | FNV-1a 32 and 64 |
| `md5` | MD5 |
| `murmur` | MurmurHash3 x86_32, x86_128, and x64_128 |
| `xxhash` | XXH32, XXH64, XXH3-64, and XXH3-128 |
| `std` | `std::io::Write` adapters and XXH3 runtime CPU detection |
The `std` feature does not enable any hash family. All families work without it; feature selection does not change digest values, and every configuration remains dependency-free and allocation-free.
## API model
Hashcrew exposes the same algorithm at different integration boundaries. Pick the narrowest interface that matches where the bytes come from:
| One complete byte slice | A module-level function such as `xxh3_64(input)` | Computes and returns the digest immediately without constructing a state. |
| Byte slices arriving incrementally | A state such as `Xxh3_64`: construct, call `update`, then call `digest` | Retains bounded working state; `digest` reads the current result and allows further updates. |
| A file, socket, decoder, or another `std::io` source | The same state through `std::io::Write` with the `std` feature | Treats every written byte as input; finish the producer, then call `digest` separately. |
| A Rust hash collection or generic `Hash` caller | A state through `Hasher`, usually constructed by its matching builder | Accepts Rust's typed `Hash` encoding and returns a `u64` from `Hasher::finish`. |
`Hasher` only supports a `u64` result, so 128-bit states deliberately preserve their complete output: MD5 returns `[u8; 16]` in standard digest byte order, while the other 128-bit algorithms return `u128`. CityHash has neither a state nor standard adapters because it cannot hash incrementally with bounded memory.
## Algorithm and capability map
The table names the canonical module-level function for complete input. A trailing `*` means the family also provides explicitly named seeded, custom-secret, or custom-offset-basis forms.
| CityHash32 | `cityhash32` | — | `u32` | — |
| CityHash64 | `cityhash64*` | — | `u64` | — |
| CityHash128 | `cityhash128*` | — | `u128` | — |
| XXH32 | `xxh32` | `Xxh32` | `u32` | `Xxh32` / `Xxh32Builder` |
| XXH64 | `xxh64` | `Xxh64` | `u64` | `Xxh64` / `Xxh64Builder` |
| XXH3-64 | `xxh3_64*` | `Xxh3_64` | `u64` | `Xxh3_64` / `Xxh3_64Builder` or secret builder |
| XXH3-128 | `xxh3_128*` | `Xxh3_128` | `u128` | — |
| MurmurHash3 x86_32 | `murmur3_x86_32` | `Murmur3X86_32` | `u32` | `Murmur3X86_32` / `Murmur3X86_32Builder` |
| MurmurHash3 x86_128 | `murmur3_x86_128` | `Murmur3X86_128` | `u128` | — |
| MurmurHash3 x64_128 | `murmur3_x64_128` | `Murmur3X64_128` | `u128` | — |
| FNV-1a 32 | `fnv1a_32*` | `Fnv1a32` | `u32` | `Fnv1a32` / `Fnv1a32Builder` |
| FNV-1a 64 | `fnv1a_64*` | `Fnv1a64` | `u64` | `Fnv1a64` / `Fnv1a64Builder` |
| MD5 | `md5` | `Md5` | `[u8; 16]` | — |
Hashcrew implements all three variants from the original MurmurHash3 family under their reference-qualified `x86_32`, `x86_128`, and `x64_128` names. These architecture labels distinguish algorithms and do not restrict which target can run them. `cityhash128_to_64` reduces an existing 128-bit CityHash value; it does not hash a new byte slice.
## Choosing an algorithm
Use XXH3 for a new general-purpose checksum, cache key, or trusted-input hash table unless interoperability requires another family. Choose a 128-bit result when the application hashes enough distinct values for 64-bit collision probability to matter. XXH32, XXH64, CityHash, MurmurHash3, and FNV-1a are primarily useful for matching an existing format, protocol, or data set; their different outputs are not interchangeable.
MD5 is provided for compatibility with existing formats and protocols that require its standard digest. Its cryptographic security is broken. Call `hashcrew::md5::md5(input)` for complete input, or use `hashcrew::md5::Md5` for streaming; both return the same 16 digest bytes without a RustCrypto dependency.
## Streaming input
Call `update` when the application already has byte slices, as in the getting-started example above. With the `std` feature, every streaming state can also be used as the destination of `std::io::copy` or another producer that accepts `std::io::Write`.
The adapter treats every written byte as hash input; it accepts the complete buffer and has nothing to flush. It does not write the digest anywhere. Finish the producer first, then call `digest` on the state:
```rust
use std::io::{self, Cursor};
use hashcrew::xxhash::Xxh3_64;
use hashcrew::xxhash::xxh3_64;
let mut source = Cursor::new(b"hashcrew");
let mut hash = Xxh3_64::new();
io::copy(&mut source, &mut hash).unwrap();
assert_eq!(hash.digest(), xxh3_64(b"hashcrew"));
```
This adapter is only needed for `std` interoperability. The direct `update` API is available in both `std` and `no_std` builds.
## Hash tables
The 32-bit and 64-bit streaming states implement `core::hash::Hasher`, with matching `BuildHasher` types for trusted-input hash tables:
```rust
use std::collections::HashMap;
use hashcrew::xxhash::Xxh3_64Builder;
let mut counts = HashMap::with_hasher(Xxh3_64Builder::with_seed(7));
counts.insert("hashcrew", 1);
assert_eq!(counts["hashcrew"], 1);
```
CityHash is intentionally one-shot. Its digest depends on the complete input length and tail, so a streaming facade would have to retain the entire message and would not provide bounded-memory incremental hashing.
XXH3 accepts custom secrets of at least 136 bytes and returns an error for shorter inputs. Its seed-and-secret APIs follow the reference contract: inputs up to 240 bytes use the seed, while longer inputs use the custom secret. Custom secrets and non-standard FNV offset bases alter deterministic output; neither makes these algorithms cryptographically secure.
## Portability
Raw and streaming digests are stable across platforms for identical byte streams. Rust's `Hash` and `BuildHasher` adapters use typed encodings that can vary across platforms and compiler versions. They can also add framing bytes to strings and slices, so `builder.hash_one(value)` need not match hashing `value.as_bytes()` or the slice directly. Use one-shot functions or `update` with a defined byte encoding for persistent checksums and cross-language protocols.
Integer digests still need an explicit output byte order: xxHash's canonical format uses `to_be_bytes()`, while FNV's RFC format uses `to_le_bytes()`. MurmurHash3's `to_le_bytes()` reproduces the reference output on little-endian systems; for CityHash, follow the consuming format's word and byte order. MD5 already returns its standard digest bytes; its [module documentation](https://docs.rs/hashcrew/*/hashcrew/md5/#hexadecimal-output) shows how to format them as hexadecimal with leading zeroes.
Target-guaranteed CPU features are selected at compile time. Other `std` builds cache runtime feature detection; `no_std` builds use compile-time features only and otherwise fall back to the scalar kernel. [`hashcrew::xxhash::kernel::selected_backend()`](https://docs.rs/hashcrew/*/hashcrew/xxhash/kernel/fn.selected_backend.html) reports the selected XXH3 backend.
## Examples and benchmarks
Runnable examples live in the [`examples`](examples) workspace crate. The [`benchmarks`](benchmarks) crate contains one-shot and streaming comparisons with independent implementations; see its [benchmark guide](benchmarks/README.md) for filters, input sizes, and the complete case matrix.
Repository workflows use the active Rust toolchain. `cargo x lint` selects nightly for Clippy, rustfmt, and rustdoc; its documentation check uses all features and the same `docsrs` configuration as docs.rs. `cargo x miri` also selects nightly. Use `cargo x --help` to list the workflows, or run common workflows with:
```shell
cargo x check
cargo x test
cargo x bench
cargo +nightly x miri
```
`cargo x check` validates empty, individual, and combined family configurations with and without `std`. Cross-target checks use `--target <triple>` and compile the library; add `--no-std` for targets without the standard library, and `--rustflags "-C target-feature=..."` to validate a specific hardware backend.
See the [release guide](RELEASE.md) for checks on stable and the MSRV.
## Correctness
Integration tests compare CityHash, xxHash, MurmurHash3, and MD5 with independent implementations. FNV-1a and MD5 also have RFC vectors in the library tests; FNV-1a 64 has an independent implementation comparison. The suite covers boundary lengths, multiple seeds, custom secrets, custom FNV offset bases, randomized inputs, streaming partitions, available hardware backends, and both `std` and `no_std` builds.
## Minimum Rust version policy
This crate's minimum supported `rustc` version is `1.85.0`.
The current policy is that the minimum Rust version required to use this crate can be increased in minor version updates. For example, if `crate 1.0` requires Rust 1.85.0, then `crate 1.0.z` for all values of `z` will also require Rust 1.85.0 or newer. However, `crate 1.y` for `y > 0` may require a newer minimum version of Rust.
## License and acknowledgements
This project is licensed under [Apache License, Version 2.0](LICENSE). The license file also records incorporated third-party code and its copyright notices and terms.