1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
// # What is this "aligned buffer" business?
//
// We need a byte array with an alignemnt of OUTPUT_RATE_BYTES that the Aegis
// trait uses. OUTPUT_RATE_BYTES depends on AEGIS algorithm variant. The byte
// width of the SIMD vector we use with the variant is _always_ less than or
// equal to OUTPUT_RATE_BYTES, so reading from a byte array aligned to
// OUTPUT_RATE_BYTES ensures we make _aligned_ (instead of _unaligned_)
// reads[^1] from the byte array when copying into that SIMD vector.
//
// So we pass OUTPUT_RATE_BYTES as the BYTES parameter for AlignedBuf.
//
// To get a properly aligned byte array, we have to put it in a struct and then
// use `#[align(N)]` to align it to N bytes... but we don't want to end up with
// any padding bytes in our struct, so `#[align(N)]` must match the width of the
// internal byte array.
//
// Rust does not allow using a const generic parameter for that N in
// `#[align(N)]`. Thus we use a macro to generate AlignedBuffer[16|32|64|128]
// structs, all of which implement the AlignedBuf trait.
//
// The Aegis trait uses the AlignedBuf trait as an associated type. Our
// Aegis128L and Aegis256 structs then use AlignedBufHolder and AlignedBufRouter
// to choose the correct AlignedBufferXX struct to "fill in" the AlignedBuf
// associated type.
//
// Confused? Perfectly understandable; we're hitting some rough edge cases of
// the Rust type system that require significant boilerplate to work around.
//
// # Preserving Alignment
//
// _References_ to the array field inside AlignedBufferXX (or the struct itself)
// are guaranteed to be aligned correctly.
//
// ...BUT ONLY REFERENCES! If you pass the array _by value_ to some function,
// you lose the alignment guarantee. Passing the AlignedBufferXX struct by-value
// is fine and the alignment is preserved.
//
// TL;DR: Use the `as_ref()`/`as_mut()` methods on AlignedBufferXX (or through
// the AlignedBuf trait) which return a (mutable) reference to the internal
// array. Do as much work as possible _through the reference_ to get any perf
// benefits from aligned reads.
//
// NOTE: It is NOT possible to somehow transmute an unaligned array reference
// like `&[u8; 16]` to an `&AlignedBuffer16`. Data at an unaligned address
// _must be copied_ to an aligned address to get an aligned reference to it.
// Thus you'll need to call `AlignedBuffer16::from_array([u8; 16])`.
//
// [^1]: On modern CPUs the perf cost of reading unaligned data is effectively
// zero. Here's why:
//
// - Unaligned reads that are within a cache line (64 B) are free.
// - Unaligned reads which cross a cache line have a tiny cost, BUT:
// - Big out-of-order execution windows on modern CPUs hide the latency.
// - Crossing an OS page boundary (4 KiB) has a small cost, but hitting that is
// rare enough to not matter.
//
// So yes, there are many systems in place that will hide this cost, but it's
// still better to just make aligned reads in the first place. And on older or
// embedded CPUs unaligned reads are still an issue.
use crateconst_assert;
use size_of;
/// A trait representing an array of `BYTES` bytes that are also aligned
/// exactly to the provided `BYTES`.
/// Part of the necessary machinery that maps a number of bytes to the correct
/// `AlignedBufferXX` type.
///
/// Here's how to use the machinery:
///
/// ```ignore
/// <AlignedBufHolder as AlignedBufRouter<BYTES>>::AlignedBuf
/// ```
///
/// That expression should resolve to the correct `AlignedBufferXX` type.
///
/// - `<AlignedBufHolder as AlignedBufRouter<32>>::AlignedBuf`
/// resolves to `AlignedBuffer32`
/// - `<AlignedBufHolder as AlignedBufRouter<64>>::AlignedBuf`
/// resolves to `AlignedBuffer64`
///
/// etc.
/// See the comment on [`AlignedBufRouter`].
;
// Takes a number of bytes and produces an AlignedBufferXX type where XX is the
// number of bytes provided.
//
// Call example:
//
// gen_aligned_buffer!(16);
// We need an AlignedBufferXX for each OUTPUT_RATE_BYTES that we end up using.
gen_aligned_buffer!;
gen_aligned_buffer!;
gen_aligned_buffer!;
gen_aligned_buffer!;