Batch-hash multiple files with BLAKE2b using the best strategy for the workload.
Samples a few files to estimate total data size. For small workloads, uses
single-core SIMD batch hashing (blake2b_hash_files_many) to avoid stat and
thread spawn overhead. For larger workloads, uses multi-core work-stealing
parallelism where each worker calls blake2b_hash_file (with I/O pipelining
for large files on Linux).
Returns results in input order.
Hash data and write hex result directly into an output buffer.
Returns the number of hex bytes written. Avoids String allocation
on the critical single-file fast path.
out must be at least 128 bytes for BLAKE2b (64 * 2), 64 for SHA256, 32 for MD5.
Hash a file by path. Uses I/O pipelining for large files on Linux,
mmap with HUGEPAGE hints as fallback, single-read for small files,
and streaming read for non-regular files.
Hash a file without fstat — just open, read until EOF, hash.
For many-file workloads (100+ tiny files), skipping fstat saves ~5µs/file.
Uses a two-tier buffer strategy: small stack buffer (4KB) for the initial read,
then falls back to a larger stack buffer (64KB) or streaming hash for bigger files.
For benchmark’s 55-byte files: one read() fills the 4KB buffer, hash immediately.
Hash a single file using raw Linux syscalls for minimum overhead.
Bypasses Rust’s File abstraction entirely: raw open/fstat/read/close.
For the single-file fast path, this eliminates OpenOptions builder,
CString heap allocation, File wrapper overhead, and Read trait dispatch.
Hash a single file using raw syscalls and write hex directly to output buffer.
Returns number of hex bytes written.
This is the absolute minimum-overhead path for single-file hashing:
raw open + fstat + read + hash + hex encode, with zero String allocation.
Batch-hash multiple files: pre-read all files into memory in parallel,
then hash all data in parallel. Optimal for many small files where per-file
overhead (open/read/close syscalls) dominates over hash computation.
Batch-hash multiple files with SHA-256/MD5 using work-stealing parallelism.
Files are sorted by size (largest first) so the biggest files start processing
immediately. Each worker thread grabs the next unprocessed file via atomic index,
eliminating tail latency from uneven file sizes.
Returns results in input order.
Fast parallel hash for multi-file workloads. Skips the stat-all-and-sort phase
of hash_files_parallel() and uses hash_file_nostat() per worker to minimize
per-file syscall overhead. For 100 tiny files, this eliminates ~200 stat() calls
(100 from the sort phase + 100 from open_and_stat inside each worker).
Returns results in input order.
Parse a BSD-style tag line: “ALGO (filename) = hash”
Returns (expected_hash, filename, optional_bits).
bits is the hash length parsed from the algo name (e.g., BLAKE2b-256 -> Some(256)).
Issue readahead hints for a list of file paths to warm the page cache.
Uses POSIX_FADV_WILLNEED which is non-blocking and batches efficiently.
Only issues hints for files >= 1MB; small files are read fast enough
that the fadvise syscall overhead isn’t worth it.
Check if parallel hashing is worthwhile for the given file paths.
Always parallelize with 2+ files — rayon’s thread pool is lazily initialized
once and reused, so per-file work-stealing overhead is negligible (~1µs).
Removing the stat()-based size check eliminates N extra syscalls for N files.
Build and write the standard GNU hash output line in a single write() call.
Format: “hash filename\n” or “hash *filename\n” (binary mode).
For escaped filenames: “\hash escaped_filename\n”.