sha3_kernel_hasher/lib.rs
1//! # SHA3-Kernel-Hasher
2//!
3//! **Production-grade SHA3-512 (FIPS 202) in pure Rust with dual-mode
4//! compilation for userspace and Windows kernel drivers.**
5//!
6//! ## Overview
7//!
8//! This crate provides a self-contained implementation of the SHA3-512
9//! cryptographic hash function as standardised in FIPS PUB 202 (National
10//! Institute of Standards and Technology, 2015). It includes the complete
11//! Keccak-f\[1600\] permutation, correct SHA3 domain separation and
12//! `pad10*1` padding, an incremental (streaming) API, and helper modules
13//! for hex encoding, constant-time comparison, and I/O-based hashing.
14//!
15//! ## Algorithm
16//!
17//! SHA3-512 is built on the *sponge construction* (Bertoni et al., 2011)
18//! operating on a 1600-bit state partitioned into a 576-bit rate (*r*)
19//! and a 1024-bit capacity (*c*). The construction proceeds in two
20//! phases:
21//!
22//! 1. **Absorb** — The message is padded with domain byte `0x06` and the
23//! `pad10*1` rule (FIPS 202, §6.1), split into 72-byte blocks, and
24//! each block is XORed into the state followed by Keccak-f\[1600\].
25//! 2. **Squeeze** — The first 512 bits (64 bytes) of the post-absorb
26//! state are extracted as the digest.
27//!
28//! The Keccak-f\[1600\] permutation applies 24 rounds, each consisting
29//! of five step mappings — θ (theta), ρ (rho), π (pi), χ (chi), and
30//! ι (iota) — to a 5×5 matrix of 64-bit lanes.
31//!
32//! ## Security Properties
33//!
34//! | Property | Security (bits) |
35//! |----------|----------------|
36//! | Collision resistance | 256 |
37//! | Preimage resistance | 512 |
38//! | Second preimage resistance | 512 |
39//!
40//! SHA3 is structurally distinct from the Merkle-Damgård family and is
41//! immune to length-extension attacks (Kelsey and Schneier, 2005).
42//!
43//! ## Modules
44//!
45//! | Module | Description |
46//! |--------|-------------|
47//! | [`hex`] | `Sha3Digest` wrapper with `Display`, `LowerHex`, `UpperHex`, hex encode/decode |
48//! | [`ct`] | Constant-time byte comparison (Bernstein, Lange and Schwabe, 2012) |
49//! | `io` | Streaming `std::io::Read` adapter and `hash_file` convenience (std only) |
50//! | [`kernel_safe`] | Processor state save/restore for kernel SIMD usage |
51//!
52//! ## Compilation Modes
53//!
54//! - **`std`** (default) — Full standard library, runtime SIMD detection,
55//! `io` module enabled.
56//! - **`kernel`** — `no_std` + `alloc` for Windows kernel-mode drivers.
57//! The `io` module is disabled; SIMD defaults to scalar.
58//!
59//! ## SIMD Status
60//!
61//! Runtime detection probes for AVX-512F/BW/DQ, AVX2+BMI2, and SSE4.2 via
62//! `is_x86_feature_detected!`, and is exposed through [`SimdFeatures`] and
63//! [`Sha3_512Kernel::simd_features`] so a caller can query what the current
64//! CPU supports.
65//!
66//! **AVX-512 single-message permutation (`std` + `x86_64` only):** real,
67//! not a stub. Setting [`PerformanceConfig::use_avx512`] to `true`
68//! dispatches every Keccak-f\[1600\] call in `hash()`/`update()`/
69//! `finalize()` to a vectorized implementation (`src/simd_avx512.rs`)
70//! when the CPU supports AVX-512F, cross-validated bit-for-bit against
71//! the scalar implementation (zero state, 2000 random states, 50 chained
72//! permutations, and the same XKCP known-answer values the scalar tests
73//! check). **It defaults to `false` because it is not a measured
74//! throughput win**: `cargo bench --bench sha3_bench` on this container's
75//! Xeon shows the AVX-512 path within noise of scalar at 64 KiB payloads,
76//! ~3-4% *slower* at 1 MiB, and ~10% slower at 4 KiB. A single
77//! Keccak-f\[1600\] instance has no independent lanes for one 512-bit
78//! register to exploit — π's cross-row mixing needs real shuffle/blend
79//! instructions (originally a memory-gather, which measured ~25-30%
80//! slower still; see `simd_avx512`'s module docs for that history) that
81//! scalar code simply doesn't pay for. This matches the wider Keccak-SIMD
82//! literature: single-instance vectorization is rarely profitable —
83//! real throughput gains come from hashing multiple independent messages
84//! in parallel, which is exactly what `Sha3_512Kernel::hash_many` does.
85//!
86//! **Multi-message batching (`hash_many`, `std` + `x86_64` only): a real
87//! win.** `Sha3_512Kernel::hash_many` hashes a batch of independent
88//! messages using the SIMD-parallel permutations in `src/simd_parallel.rs`
89//! — independent messages give the vector width actual parallel work,
90//! unlike single-message hashing above. Measured end to end
91//! (`cargo bench --bench sha3_bench sha3_512_hash_many`, 64 independent
92//! 4 KiB messages, this container's Xeon):
93//!
94//! | Path | Throughput | vs. scalar loop |
95//! |------|-----------|------------------|
96//! | scalar (one `hash()` per message) | ~182 MiB/s | 1.0x |
97//! | AVX2×4 batching | ~220 MiB/s | ~1.2x |
98//! | AVX-512×8 batching | ~1004 MiB/s | **~5.5x** |
99//!
100//! AVX-512×8 is a genuine, substantial win — 8 independent messages
101//! give an 8-lane register real parallel work with no gather/shuffle
102//! needed (ρ+π reduce to array indexing plus a uniform per-register
103//! rotate — see `simd_parallel`'s module docs for why this layout is so
104//! much more favorable than single-message vectorization). AVX2×4 is
105//! correct and a modest win, but far short of AVX-512: AVX2 has no
106//! 64-bit rotate instruction, so ρ is emulated as shift-left +
107//! shift-right + or, which eats most of the 4-way parallelism's benefit.
108//! Both are controlled by the same
109//! [`PerformanceConfig::use_avx512`]/[`PerformanceConfig::use_avx2`]
110//! flags as `hash()`; when neither applies, `hash_many` falls back to
111//! hashing each message individually via the scalar path.
112//!
113//! `prefetch_data`, `parallel_chunks`, and `chunk_size` remain no-ops.
114//!
115//! ## Correctness Validation
116//!
117//! - NIST CAVP SHA3-512 test vectors (empty, `0xCC`, `0x41FB`, `0xA3`×200)
118//! - XKCP Keccak-f\[1600\] known-answer tests (lanes 0–4)
119//! - Cross-validated against RustCrypto `sha3` v0.10 for 20 input lengths
120//! - Incremental vs. one-shot, determinism, avalanche, boundary conditions
121//!
122//! ## Quick Start
123//!
124//! ```rust
125//! use sha3_kernel_hasher::{Sha3_512Kernel, Sha3Digest};
126//!
127//! let mut hasher = Sha3_512Kernel::new();
128//!
129//! // One-shot with typed digest
130//! let digest: Sha3Digest = hasher.hash_digest(b"hello");
131//! println!("{}", digest);
132//!
133//! // Incremental hashing
134//! hasher.reset();
135//! hasher.update(b"hello");
136//! hasher.update(b" world");
137//! let digest = hasher.finalize_digest();
138//! println!("{:X}", digest);
139//! ```
140//!
141//! ## References
142//!
143//! - Bertoni, G. et al. (2011) *The Keccak Reference*, v3.0.
144//! <https://keccak.team/files/Keccak-reference-3.0.pdf>
145//! - NIST (2015) *SHA-3 Standard*. FIPS PUB 202.
146//! doi:10.6028/NIST.FIPS.202
147//! - Bernstein, D.J. (2005) *Cache-timing attacks on AES*.
148//! <https://cr.yp.to/antiforgery/cachetiming-20050414.pdf>
149//! - Bernstein, D.J., Lange, T. and Schwabe, P. (2012)
150//! 'The Security Impact of a New Cryptographic Library',
151//! *LATINCRYPT 2012*, LNCS 7533, pp. 159–176.
152//! - Kelsey, J. and Schneier, B. (2005) 'Second Preimages on n-Bit
153//! Hash Functions for Much Less than 2^n Work', *EUROCRYPT 2005*,
154//! LNCS 3494, pp. 474–490.
155//! - Kocher, P.C. (1996) 'Timing Attacks on Implementations of
156//! Diffie-Hellman, RSA, DSS, and Other Systems', *CRYPTO '96*,
157//! LNCS 1109, pp. 104–113.
158
159#![cfg_attr(feature = "kernel", no_std)]
160#![warn(missing_docs)]
161
162#[cfg(feature = "kernel")]
163extern crate alloc;
164
165#[cfg(feature = "kernel")]
166use alloc::vec::Vec;
167#[cfg(not(feature = "kernel"))]
168use std::vec::Vec;
169
170#[cfg(feature = "serde")]
171use serde::{Deserialize, Serialize};
172
173pub mod ct;
174pub mod hex;
175mod keccak;
176
177#[cfg(not(feature = "kernel"))]
178pub mod io;
179
180// AVX-512 requires `is_x86_feature_detected!`, which is a `std` macro (no
181// `core` equivalent) — so this is gated out of the `kernel` (`no_std`)
182// build, same as `io` above, and out of non-x86_64 targets entirely.
183#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
184mod simd_avx512;
185
186// Multi-message parallel permutation (AVX2x4 / AVX-512x8) backing
187// `hash_many`. Same cfg gate as `simd_avx512` — see its comment above.
188#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
189mod simd_parallel;
190
191pub use hex::Sha3Digest;
192// Only the non-AVX-512 `permute()` fallback below calls this directly; the
193// x86_64+std build's `permute()` reaches the scalar path indirectly via
194// `simd_avx512::keccak_f1600_dispatch` instead.
195#[cfg(any(not(target_arch = "x86_64"), feature = "kernel"))]
196use keccak::keccak_f1600;
197
198/// Test-only call counters proving which permutation path actually ran.
199///
200/// Digest *output* is identical whether the scalar or AVX-512 path runs
201/// (that's the whole correctness contract in `simd_avx512.rs`), so an
202/// output-only test can't tell "the config flag is wired up" apart from
203/// "the config flag is silently ignored and everything happens to still
204/// be correct scalar code." These counters close that gap.
205#[cfg(test)]
206static AVX512_PATH_CALLS: core::sync::atomic::AtomicUsize = core::sync::atomic::AtomicUsize::new(0);
207#[cfg(test)]
208static SCALAR_PATH_CALLS: core::sync::atomic::AtomicUsize = core::sync::atomic::AtomicUsize::new(0);
209
210/// Single call site for the Keccak-f\[1600\] permutation used by the
211/// sponge (`Sha3_512State::absorb`/`finalize`). Chooses AVX-512 when
212/// `use_avx512` is `true` *and* the running CPU actually supports it
213/// (checked again, cheaply, inside [`simd_avx512::keccak_f1600_dispatch`]
214/// via `is_x86_feature_detected!`); falls back to the scalar
215/// implementation otherwise. On non-x86_64 targets, or the `kernel`
216/// (`no_std`) build where the AVX-512 module is compiled out entirely
217/// (see its `mod` declaration above), `use_avx512` is accepted but has
218/// no effect — there is no vector path to dispatch to.
219#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
220#[inline]
221fn permute(state: &mut [u64; 25], use_avx512: bool) {
222 #[cfg(test)]
223 {
224 use core::sync::atomic::Ordering;
225 if use_avx512 && is_x86_feature_detected!("avx512f") {
226 AVX512_PATH_CALLS.fetch_add(1, Ordering::Relaxed);
227 } else {
228 SCALAR_PATH_CALLS.fetch_add(1, Ordering::Relaxed);
229 }
230 }
231 simd_avx512::keccak_f1600_dispatch(state, use_avx512);
232}
233
234#[cfg(any(not(target_arch = "x86_64"), feature = "kernel"))]
235#[inline]
236fn permute(state: &mut [u64; 25], _use_avx512: bool) {
237 #[cfg(test)]
238 {
239 use core::sync::atomic::Ordering;
240 SCALAR_PATH_CALLS.fetch_add(1, Ordering::Relaxed);
241 }
242 keccak_f1600(state);
243}
244
245/// SHA3-512 rate in bytes: r = (1600 − 2×512) / 8 = 72.
246const SHA3_512_RATE: usize = 72;
247
248/// SHA3 domain separation byte (FIPS 202 §6.1).
249const SHA3_DOMAIN_BYTE: u8 = 0x06;
250
251/// SHA3-512 kernel hasher with SIMD optimisation
252pub struct Sha3_512Kernel {
253 state: Sha3_512State,
254 /// SIMD feature detection
255 features: SimdFeatures,
256 /// Performance configuration
257 config: PerformanceConfig,
258}
259
260/// SIMD feature detection results, as probed once via
261/// `is_x86_feature_detected!` and returned by
262/// [`Sha3_512Kernel::simd_features`].
263#[derive(Debug, Clone, Copy)]
264pub struct SimdFeatures {
265 /// Whether the running CPU supports AVX-512F/BW/DQ. Gates both the
266 /// single-message AVX-512 permutation and the AVX-512×8 batch path.
267 pub avx512_available: bool,
268 /// Whether the running CPU supports AVX2 + BMI2. Gates the AVX2×4
269 /// batch path used by `Sha3_512Kernel::hash_many` (`std` +
270 /// `x86_64` builds only — not an intra-doc link here since the
271 /// method doesn't exist under `kernel`/non-x86_64 builds).
272 pub avx2_available: bool,
273 /// Whether the running CPU supports SSE4.2. Currently detected but
274 /// not dispatched to by any permutation path.
275 pub sse42_available: bool,
276 /// Whether the running CPU supports BMI2 (`RORX`/`SHRX` etc.),
277 /// required alongside AVX2 for the AVX2×4 batch path.
278 pub bmi2_available: bool,
279}
280
281/// Performance configuration for hasher.
282///
283/// **`use_avx512`/`use_avx2` are real and wired up** on `std` + `x86_64`
284/// builds, but they mean different things depending on which method you
285/// call:
286///
287/// - **`Sha3_512Kernel::hash`/`update`/`finalize`** (single message):
288/// dispatch every Keccak-f\[1600\] permutation to
289/// [`simd_avx512`](crate) when `use_avx512` is `true` and the CPU
290/// supports AVX-512F (falling back to scalar otherwise; `use_avx2` has
291/// no effect here — see below). This is **not a clear win, which is
292/// why `use_avx512` defaults to `false`**: measured on this crate's own
293/// `cargo bench --bench sha3_bench` (this container's Xeon), it's
294/// within noise of scalar at 64 KiB payloads, ~3-4% *slower* at 1 MiB,
295/// and ~10% slower at 4 KiB. A single Keccak-f\[1600\] instance has no
296/// independent lanes for a 512-bit-wide register to parallelize
297/// *within* — π's cross-row mixing costs real shuffle/blend
298/// instructions that scalar code doesn't pay. Set it `true` here only
299/// if you've benchmarked your own workload and it actually helps.
300/// - **`Sha3_512Kernel::hash_many`** (a batch of independent messages):
301/// dispatches to AVX-512×8 or AVX2×4 batched permutations in
302/// `src/simd_parallel.rs`. Here the flags select a **real, measured
303/// win** — independent messages give the vector width genuine parallel
304/// work. See the crate-level "SIMD Status" section for full numbers;
305/// in short, AVX-512×8 measured ~5.5x over a scalar loop, AVX2×4 only
306/// ~1.2x (AVX2 has no 64-bit rotate instruction, so ρ is emulated via
307/// shift+shift+or, eating most of its 4-way parallelism).
308///
309/// `prefetch_data`, `parallel_chunks`, and `chunk_size` remain no-ops —
310/// nothing reads them yet.
311#[derive(Debug, Clone, Copy)]
312pub struct PerformanceConfig {
313 /// Real and wired up (`std` + `x86_64` only) for both `hash()` and
314 /// `hash_many()` — see the struct-level docs above; the two methods
315 /// get very different returns on setting this `true`.
316 pub use_avx512: bool,
317 /// No-op for `hash()` (AVX2 cannot accelerate a single-message
318 /// permutation — see struct docs); real, and a modest win, for
319 /// `hash_many()`.
320 pub use_avx2: bool,
321 /// No-op today; reserved for a future prefetch optimization.
322 pub prefetch_data: bool,
323 /// No-op today; reserved for a future parallel-chunk optimization.
324 pub parallel_chunks: bool,
325 /// No-op today.
326 pub chunk_size: usize,
327}
328
329impl Default for PerformanceConfig {
330 fn default() -> Self {
331 Self {
332 // Not a measured win on this crate's own benchmarks (see the
333 // struct docs) — opt-in, not default-on.
334 use_avx512: false,
335 use_avx2: true,
336 prefetch_data: true,
337 parallel_chunks: true,
338 chunk_size: 1024,
339 }
340 }
341}
342
343/// SHA3-512 sponge state (64-byte aligned for SIMD compatibility)
344#[repr(C, align(64))]
345#[derive(Debug, Clone)]
346pub struct Sha3_512State {
347 /// Keccak state: 5×5 matrix of 64-bit lanes (1600 bits)
348 pub state: [u64; 25],
349 /// Current position in the absorb buffer (0..rate)
350 pub pos: usize,
351 /// Rate in bytes (72 for SHA3-512)
352 pub rate: usize,
353 /// Absorb buffer — accumulates input until a full rate-block is ready
354 pub buffer: [u8; 200],
355}
356
357#[cfg(feature = "serde")]
358mod state_serde {
359 use super::Sha3_512State;
360 use core::fmt;
361 use serde::de::{self, MapAccess, Visitor};
362 use serde::ser::SerializeStruct;
363 use serde::{Deserialize, Deserializer, Serialize, Serializer};
364
365 #[cfg(feature = "kernel")]
366 use alloc::vec::Vec;
367 #[cfg(not(feature = "kernel"))]
368 use std::vec::Vec;
369
370 impl Serialize for Sha3_512State {
371 fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
372 let mut s = serializer.serialize_struct("Sha3_512State", 4)?;
373 s.serialize_field("state", &self.state.as_slice())?;
374 s.serialize_field("pos", &self.pos)?;
375 s.serialize_field("rate", &self.rate)?;
376 s.serialize_field("buffer", &self.buffer.as_slice())?;
377 s.end()
378 }
379 }
380
381 impl<'de> Deserialize<'de> for Sha3_512State {
382 fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
383 #[derive(serde::Deserialize)]
384 #[serde(field_identifier, rename_all = "lowercase")]
385 enum Field {
386 State,
387 Pos,
388 Rate,
389 Buffer,
390 }
391
392 struct StateVisitor;
393
394 impl<'de> Visitor<'de> for StateVisitor {
395 type Value = Sha3_512State;
396
397 fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
398 f.write_str("struct Sha3_512State")
399 }
400
401 fn visit_map<M: MapAccess<'de>>(
402 self,
403 mut map: M,
404 ) -> Result<Sha3_512State, M::Error> {
405 let mut state_lanes: Option<Vec<u64>> = None;
406 let mut pos: Option<usize> = None;
407 let mut rate: Option<usize> = None;
408 let mut buffer_vec: Option<Vec<u8>> = None;
409
410 while let Some(key) = map.next_key::<Field>()? {
411 match key {
412 Field::State => {
413 state_lanes = Some(map.next_value::<Vec<u64>>()?);
414 }
415 Field::Pos => {
416 pos = Some(map.next_value::<usize>()?);
417 }
418 Field::Rate => {
419 rate = Some(map.next_value::<usize>()?);
420 }
421 Field::Buffer => {
422 buffer_vec = Some(map.next_value::<Vec<u8>>()?);
423 }
424 }
425 }
426
427 let sv = state_lanes.ok_or_else(|| de::Error::missing_field("state"))?;
428 let pv = pos.ok_or_else(|| de::Error::missing_field("pos"))?;
429 let rv = rate.ok_or_else(|| de::Error::missing_field("rate"))?;
430 let bv = buffer_vec.ok_or_else(|| de::Error::missing_field("buffer"))?;
431
432 let mut state = [0u64; 25];
433 if sv.len() != 25 {
434 return Err(de::Error::invalid_length(sv.len(), &"25"));
435 }
436 state.copy_from_slice(&sv);
437
438 let mut buffer = [0u8; 200];
439 if bv.len() != 200 {
440 return Err(de::Error::invalid_length(bv.len(), &"200"));
441 }
442 buffer.copy_from_slice(&bv);
443
444 Ok(Sha3_512State {
445 state,
446 pos: pv,
447 rate: rv,
448 buffer,
449 })
450 }
451 }
452
453 deserializer.deserialize_struct(
454 "Sha3_512State",
455 &["state", "pos", "rate", "buffer"],
456 StateVisitor,
457 )
458 }
459 }
460}
461
462/// Hash result type — 64 bytes (512 bits)
463pub type Sha3_512Hash = [u8; 64];
464
465/// Memory region descriptor for kernel hashing.
466///
467/// Passed to [`Sha3_512Kernel::hash_memory_regions`], which hashes each
468/// region for which `is_readable` is `true` and `size` is a nonzero value
469/// not exceeding [`MAX_HASH_SIZE`], skipping the rest. The caller is
470/// responsible for the region actually being valid, mapped memory at
471/// `base_address` for `size` bytes — see [`kernel_safe`] and
472/// [`Sha3_512Kernel::hash_memory_region`] for the `unsafe` contract this
473/// descriptor feeds into.
474#[derive(Debug, Clone)]
475#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
476pub struct MemoryRegion {
477 /// Starting virtual address of the region.
478 pub base_address: u64,
479 /// Length of the region in bytes.
480 pub size: usize,
481 /// Whether the region is safe to read. Regions with `is_readable ==
482 /// false` are skipped by [`Sha3_512Kernel::hash_memory_regions`].
483 pub is_readable: bool,
484 /// Whether the region is writable. Informational only — not
485 /// consulted by the hashing path, which only reads.
486 pub is_writable: bool,
487 /// Whether the region is executable. Informational only — not
488 /// consulted by the hashing path, which only reads.
489 pub is_executable: bool,
490}
491
492/// Maximum safe hash size for kernel operations (256 MiB)
493pub const MAX_HASH_SIZE: usize = 256 * 1024 * 1024;
494
495/// Default chunk size for parallel processing
496pub const DEFAULT_CHUNK_SIZE: usize = 1024;
497
498impl Sha3_512Kernel {
499 /// Create a new SHA3-512 kernel hasher with default configuration.
500 pub fn new() -> Self {
501 Self::with_config(PerformanceConfig::default())
502 }
503
504 /// Create a hasher with a custom performance configuration.
505 ///
506 /// # Examples
507 ///
508 /// ```
509 /// use sha3_kernel_hasher::{PerformanceConfig, Sha3_512Kernel};
510 ///
511 /// let mut hasher = Sha3_512Kernel::with_config(PerformanceConfig {
512 /// use_avx2: true,
513 /// ..Default::default()
514 /// });
515 /// let digest = hasher.hash(b"hello");
516 /// assert_eq!(digest.len(), 64);
517 /// ```
518 pub fn with_config(config: PerformanceConfig) -> Self {
519 let features = Self::detect_simd_features();
520 Self {
521 state: Sha3_512State::new(),
522 features,
523 config,
524 }
525 }
526
527 /// Hash `data` in a single call, returning the 64-byte SHA3-512 digest.
528 ///
529 /// Runs the AVX-512-vectorized Keccak-f\[1600\] permutation when
530 /// `self.config.use_avx512` is `true` *and* the CPU actually supports
531 /// AVX-512F (checked at each permutation call); otherwise runs the
532 /// scalar path. `self.config.use_avx2`/`prefetch_data`/
533 /// `parallel_chunks`/`chunk_size` remain no-ops — see the crate-level
534 /// "SIMD Status" section for why AVX2 alone can't accelerate a
535 /// single-message permutation.
536 pub fn hash(&mut self, data: &[u8]) -> Sha3_512Hash {
537 self.state.reset();
538 self.state.absorb(data, self.config.use_avx512);
539 self.state.finalize(self.config.use_avx512)
540 }
541
542 /// Hash a raw memory region (kernel-compatible).
543 ///
544 /// # Safety
545 ///
546 /// The caller must guarantee that `[base_address, base_address + size)`
547 /// is a valid, readable memory region for the lifetime of this call.
548 /// Returns `[0u8; 64]` if `size` is zero or exceeds `MAX_HASH_SIZE`.
549 ///
550 /// # Examples
551 ///
552 /// ```
553 /// use sha3_kernel_hasher::Sha3_512Kernel;
554 ///
555 /// let buffer = vec![0xABu8; 4096];
556 /// let mut hasher = Sha3_512Kernel::new();
557 /// let digest = hasher.hash_memory_region(buffer.as_ptr() as u64, buffer.len());
558 /// assert_eq!(digest, hasher.hash(&buffer));
559 /// ```
560 pub fn hash_memory_region(&mut self, base_address: u64, size: usize) -> Sha3_512Hash {
561 if size == 0 || size > MAX_HASH_SIZE {
562 return [0u8; 64];
563 }
564 let memory_slice = unsafe { core::slice::from_raw_parts(base_address as *const u8, size) };
565 self.hash(memory_slice)
566 }
567
568 /// Hash multiple memory regions, returning one digest per readable region.
569 pub fn hash_memory_regions(&mut self, regions: &[MemoryRegion]) -> Vec<Sha3_512Hash> {
570 regions
571 .iter()
572 .filter(|r| r.is_readable && r.size > 0 && r.size <= MAX_HASH_SIZE)
573 .map(|region| self.hash_memory_region(region.base_address, region.size))
574 .collect()
575 }
576
577 /// Hash multiple independent messages, using SIMD-parallel batches
578 /// when `self.config` allows and the CPU supports it.
579 ///
580 /// Unlike `hash()`'s [`PerformanceConfig::use_avx512`] (opt-in, not a
581 /// measured win — see the crate-level "SIMD Status" section),
582 /// **AVX-512×8 batching here is a real, measured win**: independent
583 /// messages give the vector width actual parallel work, unlike a
584 /// single permutation. Measured on this container's Xeon (raw
585 /// permutation throughput, `simd_parallel::timing_smoke`): AVX-512×8
586 /// is ~5.3x faster than 8 sequential scalar permutations. AVX2×4 is
587 /// implemented and correct but measured only ~1.0x (no real gain) —
588 /// AVX2 has no 64-bit rotate instruction, so ρ is emulated as
589 /// shift-left + shift-right + or, and that overhead roughly cancels
590 /// out the 4-way parallelism. Both are controlled by the same
591 /// [`PerformanceConfig::use_avx512`]/[`PerformanceConfig::use_avx2`]
592 /// flags as `hash()`; when neither applies (disabled in config, not
593 /// supported by the CPU, `kernel` build, or non-x86_64 target),
594 /// messages are hashed individually via the scalar path.
595 ///
596 /// Messages are batched in groups of the selected width (8 or 4);
597 /// within a group, variable-length messages are supported directly
598 /// — each message's own FIPS 202 `pad10*1` padding determines its own
599 /// block count, and its digest is captured the instant its own last
600 /// block is permuted, even while other lanes in the same group still
601 /// have blocks left. See `hash_batch` below for the algorithm.
602 ///
603 /// Returns one digest per input message, in the same order.
604 ///
605 /// # Examples
606 ///
607 /// ```
608 /// # #[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
609 /// # {
610 /// use sha3_kernel_hasher::Sha3_512Kernel;
611 ///
612 /// let hasher = Sha3_512Kernel::new();
613 /// let messages: [&[u8]; 3] = [b"first", b"second", b"third"];
614 /// let digests = hasher.hash_many(&messages);
615 /// assert_eq!(digests.len(), 3);
616 /// # }
617 /// ```
618 #[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
619 pub fn hash_many(&self, messages: &[&[u8]]) -> Vec<Sha3_512Hash> {
620 let width = if self.config.use_avx512 && is_x86_feature_detected!("avx512f") {
621 8
622 } else if self.config.use_avx2 && is_x86_feature_detected!("avx2") {
623 4
624 } else {
625 1
626 };
627
628 let mut results = Vec::with_capacity(messages.len());
629 for chunk in messages.chunks(width.max(1)) {
630 match width {
631 8 => results.extend(hash_batch_x8(chunk)),
632 4 => results.extend(hash_batch_x4(chunk)),
633 _ => {
634 for m in chunk {
635 results.push(Sha3_512Kernel::new().hash(m));
636 }
637 }
638 }
639 }
640 results
641 }
642
643 /// Detect available SIMD features at runtime (userspace only).
644 #[cfg(all(target_arch = "x86_64", not(feature = "kernel")))]
645 fn detect_simd_features() -> SimdFeatures {
646 SimdFeatures {
647 avx512_available: is_x86_feature_detected!("avx512f")
648 && is_x86_feature_detected!("avx512bw")
649 && is_x86_feature_detected!("avx512dq"),
650 avx2_available: is_x86_feature_detected!("avx2") && is_x86_feature_detected!("bmi2"),
651 sse42_available: is_x86_feature_detected!("sse4.2"),
652 bmi2_available: is_x86_feature_detected!("bmi2"),
653 }
654 }
655
656 /// In kernel mode, default to scalar (SIMD requires KeSaveFloatingPointState).
657 /// SIMD can be enabled via PerformanceConfig after verifying processor state.
658 #[cfg(any(not(target_arch = "x86_64"), feature = "kernel"))]
659 fn detect_simd_features() -> SimdFeatures {
660 SimdFeatures {
661 avx512_available: false,
662 avx2_available: false,
663 sse42_available: false,
664 bmi2_available: false,
665 }
666 }
667
668 /// Get the SIMD features detected on the current CPU at construction
669 /// time (see [`SimdFeatures`] for what each flag gates).
670 ///
671 /// # Examples
672 ///
673 /// ```
674 /// use sha3_kernel_hasher::Sha3_512Kernel;
675 ///
676 /// let hasher = Sha3_512Kernel::new();
677 /// let features = hasher.simd_features();
678 /// println!("AVX-512 available: {}", features.avx512_available);
679 /// ```
680 pub fn simd_features(&self) -> SimdFeatures {
681 self.features
682 }
683
684 /// Get the hasher's current [`PerformanceConfig`].
685 pub fn config(&self) -> PerformanceConfig {
686 self.config
687 }
688
689 /// Replace the hasher's [`PerformanceConfig`].
690 ///
691 /// # Examples
692 ///
693 /// ```
694 /// use sha3_kernel_hasher::{PerformanceConfig, Sha3_512Kernel};
695 ///
696 /// let mut hasher = Sha3_512Kernel::new();
697 /// hasher.update_config(PerformanceConfig {
698 /// use_avx2: true,
699 /// ..hasher.config()
700 /// });
701 /// ```
702 pub fn update_config(&mut self, config: PerformanceConfig) {
703 self.config = config;
704 }
705
706 /// Hash `data` and return a [`Sha3Digest`] wrapper with formatting traits.
707 ///
708 /// This is a convenience method equivalent to
709 /// `Sha3Digest::new(self.hash(data))`.
710 pub fn hash_digest(&mut self, data: &[u8]) -> Sha3Digest {
711 Sha3Digest::new(self.hash(data))
712 }
713
714 /// Finalize current hash (for incremental hashing).
715 ///
716 /// Applies SHA3 padding and extracts the digest. After calling this
717 /// method, the hasher must be `reset()` before reuse.
718 pub fn finalize(&mut self) -> Sha3_512Hash {
719 self.state.finalize(self.config.use_avx512)
720 }
721
722 /// Finalize and return a [`Sha3Digest`] wrapper with formatting traits.
723 pub fn finalize_digest(&mut self) -> Sha3Digest {
724 Sha3Digest::new(self.finalize())
725 }
726
727 /// Reset hasher state for a new message.
728 pub fn reset(&mut self) {
729 self.state.reset();
730 }
731
732 /// Update hasher with additional data (incremental / streaming).
733 ///
734 /// Can be called multiple times; the final digest is obtained
735 /// by calling `finalize()`.
736 pub fn update(&mut self, data: &[u8]) {
737 self.state.absorb(data, self.config.use_avx512);
738 }
739}
740
741impl Default for Sha3_512Kernel {
742 fn default() -> Self {
743 Self::new()
744 }
745}
746
747// ---------------------------------------------------------------------------
748// hash_many — SIMD-parallel batch hashing of independent messages.
749//
750// This operates on raw `[u64; 25]` states directly rather than going
751// through `Sha3_512State`, because a batch of W independent sponges needs
752// W state arrays live at once for the vector primitives in
753// `simd_parallel` to operate on — `Sha3_512State`'s single-sponge,
754// streaming (`pos`/`buffer`) design isn't the right shape for that, and
755// retrofitting it would complicate the streaming API this function
756// doesn't need. `pad_message`/`xor_block_into`/`squeeze` below
757// deliberately mirror `Sha3_512State::absorb`/`finalize`'s byte-level
758// logic exactly (same rate, same pad10*1 rule, same little-endian word
759// order) — `tests::hash_many_matches_hash_for_every_length_0_to_300`
760// cross-checks the two against each other so they can't silently drift.
761// ---------------------------------------------------------------------------
762
763#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
764fn pad_message(msg: &[u8]) -> Vec<u8> {
765 let rate = SHA3_512_RATE;
766 let full_blocks = msg.len() / rate;
767 let total_len = (full_blocks + 1) * rate;
768 let mut buf = vec![0u8; total_len];
769 buf[..msg.len()].copy_from_slice(msg);
770 buf[msg.len()] = SHA3_DOMAIN_BYTE;
771 let last = total_len - 1;
772 buf[last] |= 0x80;
773 buf
774}
775
776#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
777fn xor_block_into(state: &mut [u64; 25], block: &[u8]) {
778 let words = SHA3_512_RATE / 8;
779 for (i, lane) in state.iter_mut().enumerate().take(words) {
780 let off = i * 8;
781 let word = u64::from_le_bytes([
782 block[off],
783 block[off + 1],
784 block[off + 2],
785 block[off + 3],
786 block[off + 4],
787 block[off + 5],
788 block[off + 6],
789 block[off + 7],
790 ]);
791 *lane ^= word;
792 }
793}
794
795#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
796fn squeeze(state: &[u64; 25]) -> Sha3_512Hash {
797 let mut hash = [0u8; 64];
798 for i in 0..8 {
799 hash[i * 8..(i + 1) * 8].copy_from_slice(&state[i].to_le_bytes());
800 }
801 hash
802}
803
804/// Batch up to 8 independent messages through `simd_parallel`'s
805/// AVX-512×8 permutation. `chunk.len()` must be `<= 8` — lanes beyond
806/// `chunk.len()` stay all-zero and are never read.
807///
808/// # Algorithm
809///
810/// Each message is padded independently ([`pad_message`]), so messages
811/// of different lengths need different numbers of blocks. Blocks are
812/// processed in lockstep across all lanes — every lane that still has a
813/// block at `block_idx` gets it XORed in before the shared vector
814/// permutation call; a lane with fewer blocks simply contributes no XOR
815/// once its own blocks are exhausted (it still gets permuted along with
816/// the others — harmless, since nothing reads its state again after its
817/// digest is captured). The instant a lane reaches its own last block
818/// (`block_idx == num_blocks[i] - 1`), its digest is squeezed out
819/// immediately, before any further block-XOR/permute rounds run for
820/// lanes that aren't done yet — that ordering is what makes variable-
821/// length batching correct.
822#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
823fn hash_batch_x8(chunk: &[&[u8]]) -> Vec<Sha3_512Hash> {
824 debug_assert!(chunk.len() <= 8);
825 let padded: Vec<Vec<u8>> = chunk.iter().map(|m| pad_message(m)).collect();
826 let num_blocks: Vec<usize> = padded.iter().map(|p| p.len() / SHA3_512_RATE).collect();
827 let max_blocks = num_blocks.iter().copied().max().unwrap_or(0);
828
829 let mut states = [[0u64; 25]; 8];
830 let mut results = vec![[0u8; 64]; chunk.len()];
831
832 for block_idx in 0..max_blocks {
833 for i in 0..chunk.len() {
834 if block_idx < num_blocks[i] {
835 let start = block_idx * SHA3_512_RATE;
836 xor_block_into(&mut states[i], &padded[i][start..start + SHA3_512_RATE]);
837 }
838 }
839 // SAFETY: this function is only reached from `hash_many`, which
840 // checked `is_x86_feature_detected!("avx512f")` before selecting
841 // width 8.
842 unsafe { simd_parallel::keccak_f1600_x8_avx512(&mut states) };
843 for i in 0..chunk.len() {
844 if block_idx == num_blocks[i] - 1 {
845 results[i] = squeeze(&states[i]);
846 }
847 }
848 }
849 results
850}
851
852/// Same as [`hash_batch_x8`], batching up to 4 messages through
853/// `simd_parallel`'s AVX2×4 permutation.
854#[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
855fn hash_batch_x4(chunk: &[&[u8]]) -> Vec<Sha3_512Hash> {
856 debug_assert!(chunk.len() <= 4);
857 let padded: Vec<Vec<u8>> = chunk.iter().map(|m| pad_message(m)).collect();
858 let num_blocks: Vec<usize> = padded.iter().map(|p| p.len() / SHA3_512_RATE).collect();
859 let max_blocks = num_blocks.iter().copied().max().unwrap_or(0);
860
861 let mut states = [[0u64; 25]; 4];
862 let mut results = vec![[0u8; 64]; chunk.len()];
863
864 for block_idx in 0..max_blocks {
865 for i in 0..chunk.len() {
866 if block_idx < num_blocks[i] {
867 let start = block_idx * SHA3_512_RATE;
868 xor_block_into(&mut states[i], &padded[i][start..start + SHA3_512_RATE]);
869 }
870 }
871 // SAFETY: this function is only reached from `hash_many`, which
872 // checked `is_x86_feature_detected!("avx2")` before selecting
873 // width 4.
874 unsafe { simd_parallel::keccak_f1600_x4_avx2(&mut states) };
875 for i in 0..chunk.len() {
876 if block_idx == num_blocks[i] - 1 {
877 results[i] = squeeze(&states[i]);
878 }
879 }
880 }
881 results
882}
883
884// ---------------------------------------------------------------------------
885// Sha3_512State — core sponge implementation
886// ---------------------------------------------------------------------------
887
888impl Default for Sha3_512State {
889 fn default() -> Self {
890 Self {
891 state: [0u64; 25],
892 pos: 0,
893 rate: SHA3_512_RATE,
894 buffer: [0u8; 200],
895 }
896 }
897}
898
899impl Sha3_512State {
900 /// Create a new SHA3-512 sponge state (all-zero per FIPS 202 §4).
901 pub fn new() -> Self {
902 Self::default()
903 }
904
905 /// Reset the sponge to the initial all-zero state.
906 pub fn reset(&mut self) {
907 self.state = [0u64; 25];
908 self.pos = 0;
909 self.buffer = [0u8; 200];
910 }
911
912 /// Absorb arbitrary-length input data into the sponge.
913 ///
914 /// Input is buffered until a full rate-block (72 bytes) is accumulated,
915 /// then XORed into the state and followed by a Keccak-f\[1600\]
916 /// permutation.
917 fn absorb(&mut self, data: &[u8], use_avx512: bool) {
918 let mut offset = 0;
919 let len = data.len();
920
921 // If we have buffered data, fill the buffer first
922 if self.pos > 0 {
923 let remaining = self.rate - self.pos;
924 let to_copy = core::cmp::min(remaining, len);
925 self.buffer[self.pos..self.pos + to_copy].copy_from_slice(&data[..to_copy]);
926 self.pos += to_copy;
927 offset += to_copy;
928
929 if self.pos == self.rate {
930 self.xor_block_into_state();
931 permute(&mut self.state, use_avx512);
932 self.pos = 0;
933 }
934 }
935
936 // Process full rate-blocks directly from input (zero-copy)
937 while offset + self.rate <= len {
938 // XOR rate bytes from data directly into state lanes
939 let block = &data[offset..offset + self.rate];
940 let words = self.rate / 8; // 9 words for SHA3-512
941 for i in 0..words {
942 let off = i * 8;
943 let word = u64::from_le_bytes([
944 block[off],
945 block[off + 1],
946 block[off + 2],
947 block[off + 3],
948 block[off + 4],
949 block[off + 5],
950 block[off + 6],
951 block[off + 7],
952 ]);
953 self.state[i] ^= word;
954 }
955 permute(&mut self.state, use_avx512);
956 offset += self.rate;
957 }
958
959 // Buffer any remaining bytes
960 let tail = len - offset;
961 if tail > 0 {
962 self.buffer[..tail].copy_from_slice(&data[offset..]);
963 self.pos = tail;
964 }
965 }
966
967 /// Apply SHA3 padding, absorb the final block, and squeeze the digest.
968 ///
969 /// Implements the SHA3 padding rule from FIPS 202 §6.1:
970 /// 1. Append domain separation byte `0x06`
971 /// 2. Pad with zeros
972 /// 3. Set the high bit of the last byte in the rate block (`0x80`)
973 /// 4. XOR the padded block into state
974 /// 5. Apply Keccak-f\[1600\]
975 /// 6. Extract 64 bytes in little-endian lane order
976 fn finalize(&mut self, use_avx512: bool) -> Sha3_512Hash {
977 // --- SHA3 padding (FIPS 202 §6.1) ---
978 // Clear buffer beyond current position
979 for i in self.pos..self.rate {
980 self.buffer[i] = 0;
981 }
982 // Domain separation byte
983 self.buffer[self.pos] = SHA3_DOMAIN_BYTE;
984 // pad10*1: set high bit of the last byte in the rate block
985 self.buffer[self.rate - 1] |= 0x80;
986
987 // Absorb the padded final block
988 self.xor_block_into_state();
989 permute(&mut self.state, use_avx512);
990
991 // --- Squeeze phase ---
992 // Extract 64 bytes (8 lanes × 8 bytes) in little-endian order
993 let mut hash = [0u8; 64];
994 for i in 0..8 {
995 hash[i * 8..(i + 1) * 8].copy_from_slice(&self.state[i].to_le_bytes());
996 }
997 hash
998 }
999
1000 /// XOR the current rate-block from the buffer into the sponge state.
1001 ///
1002 /// Reads `rate / 8` little-endian 64-bit words from `self.buffer`
1003 /// and XORs them into the first `rate / 8` lanes of `self.state`.
1004 #[inline]
1005 fn xor_block_into_state(&mut self) {
1006 let words = self.rate / 8; // 9 for SHA3-512
1007 for i in 0..words {
1008 let off = i * 8;
1009 let word = u64::from_le_bytes([
1010 self.buffer[off],
1011 self.buffer[off + 1],
1012 self.buffer[off + 2],
1013 self.buffer[off + 3],
1014 self.buffer[off + 4],
1015 self.buffer[off + 5],
1016 self.buffer[off + 6],
1017 self.buffer[off + 7],
1018 ]);
1019 self.state[i] ^= word;
1020 }
1021 }
1022}
1023
1024/// Kernel-safe memory operations
1025pub mod kernel_safe {
1026
1027 /// Save processor state for SIMD operations
1028 ///
1029 /// # Safety
1030 ///
1031 /// This function is unsafe because it directly manipulates processor registers.
1032 /// The caller must ensure:
1033 /// - The processor supports the required SIMD instructions (AVX2/AVX-512)
1034 /// - No other code is currently using SIMD registers
1035 /// - The saved state is properly restored before returning to normal execution
1036 /// - The function is called only from kernel mode or with appropriate privileges
1037 #[cfg(target_arch = "x86_64")]
1038 pub unsafe fn save_processor_state() -> ProcessorState {
1039 ProcessorState::save()
1040 }
1041
1042 /// Restore processor state after SIMD operations
1043 ///
1044 /// # Safety
1045 ///
1046 /// This function is unsafe because it directly manipulates processor registers.
1047 /// The caller must ensure:
1048 /// - The state was previously saved by `save_processor_state()`
1049 /// - No SIMD registers have been modified since the save
1050 /// - The processor is in the same state as when the save occurred
1051 /// - The function is called only from kernel mode or with appropriate privileges
1052 #[cfg(target_arch = "x86_64")]
1053 pub unsafe fn restore_processor_state(state: ProcessorState) {
1054 state.restore();
1055 }
1056
1057 /// Processor state structure
1058 #[cfg(target_arch = "x86_64")]
1059 #[repr(C)]
1060 pub struct ProcessorState {
1061 // XMM/YMM/ZMM registers
1062 simd_registers: [u8; 512],
1063 // Control registers
1064 control_registers: [u64; 8],
1065 }
1066
1067 #[cfg(target_arch = "x86_64")]
1068 impl ProcessorState {
1069 /// Save current processor state to memory
1070 ///
1071 /// # Safety
1072 ///
1073 /// This function is unsafe because it directly accesses processor registers.
1074 /// The caller must ensure:
1075 /// - The processor supports XSAVE/XSAVEOPT instructions
1076 /// - No other code is currently executing SIMD operations
1077 /// - Sufficient memory is allocated for the state structure
1078 /// - The function is called only from kernel mode or with appropriate privileges
1079 pub unsafe fn save() -> Self {
1080 // In real implementation, use XSAVE/XSAVEOPT
1081 Self {
1082 simd_registers: [0u8; 512],
1083 control_registers: [0u64; 8],
1084 }
1085 }
1086
1087 /// Restore processor state from saved memory
1088 ///
1089 /// # Safety
1090 ///
1091 /// This function is unsafe because it directly modifies processor registers.
1092 /// The caller must ensure:
1093 /// - The state was previously saved by the `save()` method
1094 /// - No processor registers have been modified since the save
1095 /// - The processor is in the same state as when the save occurred
1096 /// - The function is called only from kernel mode or with appropriate privileges
1097 pub unsafe fn restore(self) {
1098 // In real implementation, use XRSTOR
1099 }
1100 }
1101
1102 #[cfg(not(target_arch = "x86_64"))]
1103 pub struct ProcessorState;
1104
1105 #[cfg(not(target_arch = "x86_64"))]
1106 pub unsafe fn save_processor_state() -> ProcessorState {
1107 ProcessorState
1108 }
1109
1110 #[cfg(not(target_arch = "x86_64"))]
1111 pub unsafe fn restore_processor_state(_state: ProcessorState) {
1112 // No-op for non-x86_64
1113 }
1114}
1115
1116/// Performance utilities (available in std mode only)
1117#[cfg(not(feature = "kernel"))]
1118pub mod performance {
1119 use super::*;
1120
1121 /// Benchmark hasher performance
1122 pub fn benchmark_hasher<F>(hasher_fn: F, data: &[u8]) -> BenchmarkResult
1123 where
1124 F: Fn(&[u8]) -> Sha3_512Hash,
1125 {
1126 let start = std::time::Instant::now();
1127 let hash = hasher_fn(data);
1128 let duration = start.elapsed();
1129
1130 BenchmarkResult {
1131 hash,
1132 duration,
1133 throughput: (data.len() as f64) / duration.as_secs_f64(),
1134 }
1135 }
1136
1137 /// Benchmark result
1138 #[derive(Debug)]
1139 pub struct BenchmarkResult {
1140 /// Computed hash
1141 pub hash: Sha3_512Hash,
1142 /// Time taken
1143 pub duration: std::time::Duration,
1144 /// Throughput in bytes/second
1145 pub throughput: f64,
1146 }
1147}
1148
1149#[cfg(test)]
1150mod tests {
1151 use super::*;
1152
1153 /// Helper: encode byte array to lowercase hex string.
1154 fn bytes_to_hex(bytes: &[u8]) -> String {
1155 bytes.iter().map(|b| format!("{:02x}", b)).collect()
1156 }
1157
1158 // ===================================================================
1159 // NIST CAVP SHA3-512 Test Vectors (FIPS 202)
1160 // Source: https://csrc.nist.gov/projects/cryptographic-algorithm-
1161 // validation-program/secure-hashing#sha3vsha3vss
1162 // ===================================================================
1163
1164 /// NIST ShortMsg vector: empty message (Len = 0)
1165 #[test]
1166 fn test_nist_sha3_512_empty() {
1167 let expected = "a69f73cca23a9ac5c8b567dc185a756e\
1168 97c982164fe25859e0d1dcc1475c80a6\
1169 15b2123af1f5f94c11e3e9402c3ac558\
1170 f500199d95b6d3e301758586281dcd26";
1171 let mut hasher = Sha3_512Kernel::new();
1172 let hash = hasher.hash(b"");
1173 assert_eq!(
1174 bytes_to_hex(&hash),
1175 expected,
1176 "SHA3-512('') must match NIST CAVP vector"
1177 );
1178 }
1179
1180 /// NIST ShortMsg vector: single byte 0xCC (Len = 8)
1181 #[test]
1182 fn test_nist_sha3_512_one_byte_cc() {
1183 let expected = "3939fcc8b57b63612542da31a834e5dc\
1184 c36e2ee0f652ac72e02624fa2e5adeec\
1185 c7dd6bb3580224b4d6138706fc6e8059\
1186 7b528051230b00621cc2b22999eaa205";
1187 let mut hasher = Sha3_512Kernel::new();
1188 let hash = hasher.hash(&[0xCC]);
1189 assert_eq!(
1190 bytes_to_hex(&hash),
1191 expected,
1192 "SHA3-512(0xCC) must match NIST CAVP vector"
1193 );
1194 }
1195
1196 /// NIST ShortMsg vector: two bytes 0x41, 0xFB (Len = 16)
1197 #[test]
1198 fn test_nist_sha3_512_two_bytes() {
1199 let expected = "aa092865a40694d91754dbc767b5202c\
1200 546e226877147a95cb8b4c8f8709fe8c\
1201 d6905256b089da37896ea5ca19d2cd9a\
1202 b94c7192fc39f7cd4d598975a3013c69";
1203 let mut hasher = Sha3_512Kernel::new();
1204 let hash = hasher.hash(&[0x41, 0xFB]);
1205 assert_eq!(
1206 bytes_to_hex(&hash),
1207 expected,
1208 "SHA3-512(0x41FB) must match NIST CAVP vector"
1209 );
1210 }
1211
1212 /// NIST LongMsg vector: 200 bytes of 0xA3 repeated
1213 /// This exercises multi-block absorb (200 > rate of 72).
1214 #[test]
1215 fn test_nist_sha3_512_200_bytes_a3() {
1216 let expected = "e76dfad22084a8b1467fcf2ffa58361b\
1217 ec7628edf5f3fdc0e4805dc48caeeca8\
1218 1b7c13c30adf52a3659584739a2df46b\
1219 e589c51ca1a4a8416df6545a1ce8ba00";
1220 let input = vec![0xA3u8; 200];
1221 let mut hasher = Sha3_512Kernel::new();
1222 let hash = hasher.hash(&input);
1223 assert_eq!(
1224 bytes_to_hex(&hash),
1225 expected,
1226 "SHA3-512(0xA3 × 200) must match NIST CAVP vector"
1227 );
1228 }
1229
1230 // ===================================================================
1231 // Incremental (streaming) API tests
1232 // ===================================================================
1233
1234 /// Verify that incremental hashing produces the same result
1235 /// as single-call hashing.
1236 #[test]
1237 fn test_incremental_matches_oneshot() {
1238 let data = b"The quick brown fox jumps over the lazy dog";
1239 let mut oneshot = Sha3_512Kernel::new();
1240 let expected = oneshot.hash(data);
1241
1242 let mut incremental = Sha3_512Kernel::new();
1243 incremental.reset();
1244 incremental.update(&data[..10]);
1245 incremental.update(&data[10..30]);
1246 incremental.update(&data[30..]);
1247 let result = incremental.finalize();
1248
1249 assert_eq!(result, expected, "Incremental must match oneshot");
1250 }
1251
1252 /// Verify byte-at-a-time feeding produces correct digest.
1253 #[test]
1254 fn test_incremental_byte_at_a_time() {
1255 let data = b"abcdefghijklmnopqrstuvwxyz";
1256 let mut oneshot = Sha3_512Kernel::new();
1257 let expected = oneshot.hash(data);
1258
1259 let mut inc = Sha3_512Kernel::new();
1260 inc.reset();
1261 for &b in data.iter() {
1262 inc.update(&[b]);
1263 }
1264 let result = inc.finalize();
1265 assert_eq!(result, expected, "Byte-at-a-time must match oneshot");
1266 }
1267
1268 // ===================================================================
1269 // Determinism and edge cases
1270 // ===================================================================
1271
1272 /// Same input must always produce the same output.
1273 #[test]
1274 fn test_deterministic() {
1275 let mut hasher = Sha3_512Kernel::new();
1276 let data = b"Bunker SHA3-512 determinism test";
1277 let h1 = hasher.hash(data);
1278 let h2 = hasher.hash(data);
1279 assert_eq!(h1, h2, "Hash must be deterministic");
1280 }
1281
1282 /// Different inputs must produce different outputs.
1283 #[test]
1284 fn test_avalanche() {
1285 let mut hasher = Sha3_512Kernel::new();
1286 let h1 = hasher.hash(b"input_a");
1287 let h2 = hasher.hash(b"input_b");
1288 assert_ne!(h1, h2, "Different inputs must avalanche");
1289 }
1290
1291 /// Hash of the empty message must NOT be all zeros.
1292 #[test]
1293 fn test_empty_not_zero() {
1294 let mut hasher = Sha3_512Kernel::new();
1295 let hash = hasher.hash(b"");
1296 assert_ne!(hash, [0u8; 64], "SHA3-512('') must not be all-zero");
1297 }
1298
1299 /// Boundary: exactly one rate-block (72 bytes).
1300 #[test]
1301 fn test_exact_rate_block() {
1302 let data = vec![0x42u8; 72];
1303 let mut hasher = Sha3_512Kernel::new();
1304 let h1 = hasher.hash(&data);
1305 let h2 = hasher.hash(&data);
1306 assert_eq!(h1, h2);
1307 assert_ne!(h1, [0u8; 64]);
1308 }
1309
1310 /// Boundary: rate-block + 1 byte (73 bytes → 2 blocks).
1311 #[test]
1312 fn test_rate_plus_one() {
1313 let data_72 = vec![0x42u8; 72];
1314 let data_73 = vec![0x42u8; 73];
1315 let mut hasher = Sha3_512Kernel::new();
1316 let h72 = hasher.hash(&data_72);
1317 let h73 = hasher.hash(&data_73);
1318 assert_ne!(h72, h73, "72 vs 73 bytes must differ");
1319 }
1320
1321 /// Large payload: 1 MiB of zeros.
1322 #[test]
1323 fn test_large_payload() {
1324 let data = vec![0u8; 1024 * 1024];
1325 let mut hasher = Sha3_512Kernel::new();
1326 let h1 = hasher.hash(&data);
1327 let h2 = hasher.hash(&data);
1328 assert_eq!(h1, h2);
1329 assert_ne!(h1, [0u8; 64]);
1330 }
1331
1332 // ===================================================================
1333 // SIMD feature detection
1334 // ===================================================================
1335
1336 #[test]
1337 fn test_simd_detection() {
1338 let hasher = Sha3_512Kernel::new();
1339 let features = hasher.simd_features();
1340 #[cfg(target_arch = "x86_64")]
1341 {
1342 // On x86_64, at minimum SSE4.2 should be available on any
1343 // CPU manufactured after 2008.
1344 assert!(
1345 features.sse42_available || features.avx2_available || features.avx512_available,
1346 "At least one SIMD tier must be detected on x86_64"
1347 );
1348 }
1349 }
1350
1351 // ===================================================================
1352 // PerformanceConfig::use_avx512 must actually change the permutation
1353 // path `hash()` takes — not just be stored and ignored. Output alone
1354 // can't distinguish "wired up" from "silently ignored, still
1355 // correct" since both paths are correct by construction; these
1356 // tests assert on `AVX512_PATH_CALLS`/`SCALAR_PATH_CALLS`, which
1357 // `permute()` increments on every real permutation call.
1358 // ===================================================================
1359
1360 #[test]
1361 #[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
1362 fn hash_dispatches_to_avx512_path_when_configured_and_available() {
1363 if !is_x86_feature_detected!("avx512f") {
1364 eprintln!("skipping: no avx512f on this host");
1365 return;
1366 }
1367 AVX512_PATH_CALLS.store(0, core::sync::atomic::Ordering::Relaxed);
1368
1369 let mut hasher = Sha3_512Kernel::with_config(PerformanceConfig {
1370 use_avx512: true,
1371 ..PerformanceConfig::default()
1372 });
1373 let vector_digest = hasher.hash(b"the quick brown fox jumps over the lazy dog");
1374
1375 assert!(
1376 AVX512_PATH_CALLS.load(core::sync::atomic::Ordering::Relaxed) > 0,
1377 "hash() with use_avx512: true must actually invoke the AVX-512 \
1378 permutation path on hardware that supports it, not just store \
1379 the flag"
1380 );
1381
1382 // Correctness: must still match the scalar-forced hasher exactly.
1383 let mut scalar_hasher = Sha3_512Kernel::with_config(PerformanceConfig {
1384 use_avx512: false,
1385 ..PerformanceConfig::default()
1386 });
1387 let scalar_digest = scalar_hasher.hash(b"the quick brown fox jumps over the lazy dog");
1388 assert_eq!(
1389 vector_digest, scalar_digest,
1390 "AVX-512-dispatched hash() must match scalar-dispatched hash() bit for bit"
1391 );
1392 }
1393
1394 #[test]
1395 #[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
1396 fn hash_dispatches_to_scalar_path_when_use_avx512_is_false() {
1397 SCALAR_PATH_CALLS.store(0, core::sync::atomic::Ordering::Relaxed);
1398
1399 let mut hasher = Sha3_512Kernel::with_config(PerformanceConfig {
1400 use_avx512: false,
1401 ..PerformanceConfig::default()
1402 });
1403 hasher.hash(b"some data");
1404
1405 assert!(
1406 SCALAR_PATH_CALLS.load(core::sync::atomic::Ordering::Relaxed) > 0,
1407 "hash() with use_avx512: false must take the scalar path"
1408 );
1409 }
1410
1411 #[test]
1412 #[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
1413 fn incremental_update_also_uses_configured_dispatch() {
1414 // absorb() is called on every full rate-block during multi-block
1415 // incremental hashing, not just once in finalize() — verify the
1416 // dispatch flag reaches that call site too, using a payload that
1417 // spans multiple 72-byte rate blocks.
1418 if !is_x86_feature_detected!("avx512f") {
1419 eprintln!("skipping: no avx512f on this host");
1420 return;
1421 }
1422 AVX512_PATH_CALLS.store(0, core::sync::atomic::Ordering::Relaxed);
1423
1424 let mut hasher = Sha3_512Kernel::with_config(PerformanceConfig {
1425 use_avx512: true,
1426 ..PerformanceConfig::default()
1427 });
1428 let data = vec![0xABu8; 72 * 5 + 13]; // 5 full blocks + a partial tail
1429 hasher.update(&data[..72 * 3]);
1430 hasher.update(&data[72 * 3..]);
1431 let vector_digest = hasher.finalize_digest();
1432
1433 let mut scalar_hasher = Sha3_512Kernel::with_config(PerformanceConfig {
1434 use_avx512: false,
1435 ..PerformanceConfig::default()
1436 });
1437 let scalar_digest = scalar_hasher.hash_digest(&data);
1438
1439 assert!(
1440 AVX512_PATH_CALLS.load(core::sync::atomic::Ordering::Relaxed) >= 5,
1441 "expected at least 5 AVX-512 permutation calls (one per full \
1442 rate-block plus finalize), got {}",
1443 AVX512_PATH_CALLS.load(core::sync::atomic::Ordering::Relaxed)
1444 );
1445 assert_eq!(vector_digest, scalar_digest);
1446 }
1447
1448 // ===================================================================
1449 // Memory region hashing
1450 // ===================================================================
1451
1452 #[test]
1453 fn test_memory_hashing() {
1454 let mut hasher = Sha3_512Kernel::new();
1455 let data = vec![0u8; 1024];
1456 let h1 = hasher.hash(&data);
1457 let h2 = hasher.hash_memory_region(data.as_ptr() as u64, data.len());
1458 assert_eq!(h1, h2, "Memory region hash must match slice hash");
1459 }
1460
1461 #[test]
1462 fn test_memory_hashing_zero_size() {
1463 let mut hasher = Sha3_512Kernel::new();
1464 let hash = hasher.hash_memory_region(0x1000, 0);
1465 assert_eq!(hash, [0u8; 64], "Zero-size region must return zero hash");
1466 }
1467
1468 #[test]
1469 fn test_memory_hashing_oversized() {
1470 let mut hasher = Sha3_512Kernel::new();
1471 let hash = hasher.hash_memory_region(0x1000, MAX_HASH_SIZE + 1);
1472 assert_eq!(hash, [0u8; 64], "Oversized region must return zero hash");
1473 }
1474
1475 // ===================================================================
1476 // Cross-validation against the `sha3` reference crate
1477 // ===================================================================
1478
1479 /// Verify our implementation matches the RustCrypto `sha3` crate
1480 /// for a variety of input lengths, covering all absorb edge cases.
1481 #[test]
1482 fn test_cross_validate_vs_sha3_crate() {
1483 use sha3::{Digest, Sha3_512 as RefSha3_512};
1484 let test_lengths: &[usize] = &[
1485 0, 1, 2, 31, 32, 63, 64, 71, 72, 73, 100, 143, 144, 145, 200, 255, 256, 512, 1000, 4096,
1486 ];
1487 let mut hasher = Sha3_512Kernel::new();
1488
1489 for &len in test_lengths {
1490 let data: Vec<u8> = (0..len).map(|i| (i & 0xFF) as u8).collect();
1491 let our_hash = hasher.hash(&data);
1492
1493 let mut ref_hasher = RefSha3_512::new();
1494 ref_hasher.update(&data);
1495 let ref_hash = ref_hasher.finalize();
1496
1497 assert_eq!(
1498 &our_hash[..],
1499 &ref_hash[..],
1500 "Mismatch vs sha3 crate at len={}",
1501 len
1502 );
1503 }
1504 }
1505
1506 // ===================================================================
1507 // hash_many — SIMD-parallel batch hashing
1508 // ===================================================================
1509
1510 #[cfg(all(not(feature = "kernel"), target_arch = "x86_64"))]
1511 mod hash_many_tests {
1512 use super::*;
1513
1514 fn force(use_avx512: bool, use_avx2: bool) -> Sha3_512Kernel {
1515 Sha3_512Kernel::with_config(PerformanceConfig {
1516 use_avx512,
1517 use_avx2,
1518 ..PerformanceConfig::default()
1519 })
1520 }
1521
1522 /// Every one of `hash_many`'s three dispatch paths (AVX-512x8,
1523 /// AVX2x4, scalar fallback) must match `hash()` called
1524 /// individually, across every rate-block boundary from 0 to
1525 /// just past 4 full blocks (0..=300 bytes covers the empty
1526 /// message, the single-byte, every tail length 1..71, exact
1527 /// multiples of the 72-byte rate, and multi-block messages).
1528 #[test]
1529 fn hash_many_matches_hash_for_every_length_0_to_300() {
1530 let lengths: Vec<usize> = (0..=300).collect();
1531 let messages: Vec<Vec<u8>> = lengths
1532 .iter()
1533 .map(|&len| (0..len).map(|i| (i & 0xFF) as u8).collect())
1534 .collect();
1535 let refs: Vec<&[u8]> = messages.iter().map(|m| m.as_slice()).collect();
1536
1537 let mut scalar_hasher = Sha3_512Kernel::new();
1538 let expected: Vec<Sha3_512Hash> =
1539 messages.iter().map(|m| scalar_hasher.hash(m)).collect();
1540
1541 for (label, hasher) in [
1542 ("avx512", force(true, false)),
1543 ("avx2", force(false, true)),
1544 ("scalar", force(false, false)),
1545 ] {
1546 if label == "avx512" && !is_x86_feature_detected!("avx512f") {
1547 eprintln!("skipping hash_many({label}): no avx512f on this host");
1548 continue;
1549 }
1550 if label == "avx2" && !is_x86_feature_detected!("avx2") {
1551 eprintln!("skipping hash_many({label}): no avx2 on this host");
1552 continue;
1553 }
1554 let results = hasher.hash_many(&refs);
1555 assert_eq!(results.len(), messages.len());
1556 for (i, &len) in lengths.iter().enumerate() {
1557 assert_eq!(
1558 results[i], expected[i],
1559 "hash_many({label}) mismatch at message length {len}"
1560 );
1561 }
1562 }
1563 }
1564
1565 /// Deliberately mixes wildly different lengths in one batch (one
1566 /// message finishes in a single block, another needs several) to
1567 /// exercise the "extract on completion, keep permuting other
1568 /// lanes" scheduling in `hash_batch_x8`/`hash_batch_x4` — this is
1569 /// the part that's wrong if a short message's digest gets
1570 /// overwritten by a later round meant for a longer sibling lane.
1571 #[test]
1572 fn hash_many_handles_mixed_lengths_in_one_batch() {
1573 let batch: Vec<Vec<u8>> = vec![
1574 vec![], // 1 block
1575 vec![0xAB; 5], // 1 block
1576 vec![0xCD; 72], // exact 1 rate-block of data -> 2 blocks total
1577 vec![0xEF; 72 * 3 + 17], // multi-block, uneven tail
1578 b"the quick brown fox".to_vec(), // 1 block
1579 ];
1580 let refs: Vec<&[u8]> = batch.iter().map(|m| m.as_slice()).collect();
1581
1582 let mut scalar_hasher = Sha3_512Kernel::new();
1583 let expected: Vec<Sha3_512Hash> = batch.iter().map(|m| scalar_hasher.hash(m)).collect();
1584
1585 if is_x86_feature_detected!("avx512f") {
1586 let results = force(true, false).hash_many(&refs);
1587 assert_eq!(results, expected, "AVX-512x8 mixed-length batch mismatch");
1588 }
1589 if is_x86_feature_detected!("avx2") {
1590 let results = force(false, true).hash_many(&refs);
1591 assert_eq!(results, expected, "AVX2x4 mixed-length batch mismatch");
1592 }
1593 }
1594
1595 /// A batch larger than the SIMD width (8 or 4) must be chunked
1596 /// correctly, not just handle a single group.
1597 #[test]
1598 fn hash_many_handles_batches_larger_than_simd_width() {
1599 let batch: Vec<Vec<u8>> = (0..37)
1600 .map(|i| (0..(i * 3 + 1)).map(|b| (b & 0xFF) as u8).collect())
1601 .collect();
1602 let refs: Vec<&[u8]> = batch.iter().map(|m| m.as_slice()).collect();
1603
1604 let mut scalar_hasher = Sha3_512Kernel::new();
1605 let expected: Vec<Sha3_512Hash> = batch.iter().map(|m| scalar_hasher.hash(m)).collect();
1606
1607 let results = Sha3_512Kernel::new().hash_many(&refs);
1608 assert_eq!(results, expected);
1609 }
1610
1611 #[test]
1612 fn hash_many_empty_input_returns_empty() {
1613 let results = Sha3_512Kernel::new().hash_many(&[]);
1614 assert!(results.is_empty());
1615 }
1616 }
1617}
1618
1619/// Criterion benchmark harness for `cargo bench --features benchmarks`.
1620///
1621/// Not part of the crate's stable public API — it exists so
1622/// `cargo bench` can find a `main` entry point, and is only compiled
1623/// when the `benchmarks` feature is enabled. See the "Performance"
1624/// section of the crate-level docs for the throughput numbers this
1625/// harness produces.
1626///
1627/// `main` below is generated by the `criterion_main!` macro and can't
1628/// carry its own doc comment, hence the module-level `allow`.
1629#[cfg(feature = "benchmarks")]
1630#[allow(missing_docs)]
1631pub mod benchmarks {
1632 use super::*;
1633 #[cfg(feature = "kernel")]
1634 use alloc::vec;
1635 use criterion::{black_box, criterion_group, criterion_main, Criterion};
1636 #[cfg(not(feature = "kernel"))]
1637 use std::vec;
1638
1639 fn bench_sha3_512(c: &mut Criterion) {
1640 let mut hasher = Sha3_512Kernel::new();
1641 let data = vec![0u8; 1024 * 1024]; // 1MB
1642
1643 c.bench_function("sha3_512_hash", |b| {
1644 b.iter(|| black_box(hasher.hash(&data)))
1645 });
1646 }
1647
1648 criterion_group!(benches, bench_sha3_512);
1649 criterion_main!(benches);
1650}