Crate tape_sha256 

SHA-256 in bulk: many independent messages at once, or iterated chains

One hash cannot be vectorised, since each of the 64 rounds depends on the last. Independent messages can, though: run them in lockstep, one per SIMD lane, and every round becomes an elementwise vector operation.

Worth reaching for when you have many messages and need none of them early. The canonical case is a Merkle tree, where the whole bottom level can be hashed in one pass.

use tape_sha256::hash_many;

let msgs: Vec<&[u8]> = vec![b"one", b"two", b"three"];
let mut out = vec![[0u8; 32]; msgs.len()];
hash_many(&msgs, &mut out);

When every leaf shares a domain-separation prefix, hash_many_prefixed avoids materialising prefix || body per message:

use tape_sha256::hash_many_prefixed;

const LEAF: &[u8] = b"\x00SOLANA_MERKLE_SHREDS_LEAF";
let leaves: Vec<&[u8]> = vec![&[1u8; 64], &[2u8; 64]];
let mut out = vec![[0u8; 32]; leaves.len()];
hash_many_prefixed(LEAF, &leaves, &mut out);

Correctness

Output is bit-identical to any conforming SHA-256. Every backend is gated against the independent sha2 crate over lengths covering all block and padding edge cases, and each SIMD backend is cross-checked against the portable one.

Backend selection

By default the best kernel for the running CPU is picked and falls back gracefully. The scalar, avx2, avx512, and neon features pin one kernel at build time instead, skipping detection; a pinned kernel the CPU lacks will fault, so pin only what the whole fleet supports.

Using both threads of a core

These entry points are stateless, so two threads may call them concurrently on disjoint halves of a batch. Doing that on the two SMT siblings of one physical core measured 1.5x the single-thread rate on Zen 5 (9.83 vs 14.84 us for 64 Merkle leaves), because a single thread at ~2 instructions per cycle is limited by its own dependency chains rather than by issue width, and the sibling fills the gaps.

This crate never spawns or pins a thread. Thread topology is the caller’s policy, not a hashing library’s. Split the batch, hand each half to a thread you have pinned, and join:

use tape_sha256::hash_many;

let (m0, m1) = msgs.split_at(msgs.len() / 2);
let (o0, o1) = out.split_at_mut(msgs.len() / 2);
std::thread::scope(|s| {
    s.spawn(|| hash_many(m0, o0));
    s.spawn(|| hash_many(m1, o1));
});

How much this is worth depends heavily on the microarchitecture.

Three further caveats decide whether this is worth anything:

The two threads must be siblings of the same physical core. Pinned to different cores this is ordinary multicore parallelism, which spends a core to get it; the 1.5x claim is specifically about using a sibling that would otherwise idle. Sibling pairs are listed in /sys/devices/system/cpu/cpu<N>/topology/thread_siblings_list. Getting the pairing wrong degrades silently, so it is worth asserting.

The sibling has to actually be idle. Under a fully loaded process there is no spare thread to claim and the gain goes away. Use an equal split of the same kernel: an uneven or mixed-kernel split leaves a straggler and measured worse than a single thread.

Hash chains

Everything above is one pass over messages that already exist. A chain instead feeds each digest back in as the next message, which is what Solana’s proof of history is.

hash_chain is the exception to the whole premise of this crate: one chain has no independent work to lane up, so it ignores the multi-buffer kernels and drives the CPU’s SHA-256 unit at its latency floor.

hash_chains puts the premise back. Independent chains, one per verified entry, say run in lockstep exactly like independent messages do, which trades that latency floor for a throughput one. See both.

Structs

Message

One message to hash: an optional shared prefix plus a body

Functions

backend

Name of the active backend, for logging and benchmarks

chain_backend

Name of the kernel hash_chain uses

hash_chain

Hashes seed, then that digest, and so on n times

hash_chains

Hashes many independent chains at once, one per lane

hash_many

Hashes every message in msgs, writing one digest per message to out

hash_many_prefixed

Hashes prefix || body for each body in bodies

hash_messages

Hashes pre-built messages, for callers that want per-message prefixes

hash_pairs

Hashes prefix || left || right for each pair, without joining them first

lane_width

Number of messages the active backend hashes per pass