1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
//! The NIST prime curves, P-256 and P-384.
//!
//! These are the FIPS-approved curves, and the ones most of the deployed world
//! uses: TLS, X.509, JWT/JOSE, COSE, WebAuthn, and Secure Boot all default to
//! one of them.
//!
//! # One implementation, two curves
//!
//! Every NIST prime curve has `a = -3` and a prime of the form that admits the
//! same Montgomery arithmetic, so the field, the group law, ECDSA, and ECDH are
//! written once here and instantiated per curve:
//!
//! | module | contents |
//! |---|---|
//! | [`arith`] | Montgomery arithmetic, generic over limb count |
//! | [`point`] | the group law, in Jacobian coordinates |
//! | [`ecdsa`] | signing and verification, with RFC 6979 nonces |
//! | [`ecdh`] | key agreement |
//!
//! Sharing matters more here than it looks: a bug fixed in a duplicated group
//! law gets fixed once and forgotten once. The per-curve modules supply only
//! constants and widths.
//!
//! # Why the group law is shaped the way it is
//!
//! These curves have no Montgomery ladder to fall back on, so scalar
//! multiplication is a fixed-window loop -- five doublings and an addition per
//! digit, the same for every scalar. [`point::Point::add`] is total: it
//! computes both the addition and the doubling and selects between them
//! without branching. That costs roughly a third more field multiplications
//! than a formula with exceptional cases, so the loops use it only where an
//! exceptional case can arise; elsewhere they use the incomplete formula,
//! with the identity handled by selection, and say in a comment why equal
//! inputs cannot reach it (`Point::mul_scalar`, `gentable::mul`).
//!
//! # P-256 on x86-64
//!
//! [`p256_field`] specialises P-256's reduction to its prime; on x86-64 that is
//! assembly (`p256_x86`), and with BMI2 and ADX the point formulas are each one
//! block of it (`p256_x86_point`), selected through hooks on [`point::Curve`]
//! that other curves leave at their default. `safegcd` inverts P-256's two
//! rings and P-384's field on 64-bit targets.
//!
//! # P-384 on x86-64
//!
//! With BMI2 and ADX, P-384's field operations are specialised to its prime
//! too: Montgomery reduction takes two `mulx` per word where a generic
//! modulus takes six, squaring computes each cross product once, and the
//! doubling folds its small multiples into three fused operations. They and
//! the point formulas are generated into `p384_x86_point` and dispatched from
//! `p384_x86`; the scalar ring's multiplication stays generic in the modulus
//! (`mont_x86`).
//!
//! # Without AVX-512 IFMA
//!
//! Where the CPU has BMI2 and ADX but not IFMA, `p256_x86::mul_scalar` and
//! `p384_x86::mul_scalar` run [`point::Point::mul_scalar`]'s loop on one
//! buffer, the accumulator's slots shared by the in-place doubling and the
//! addition, so a digit's five doublings run back to back with nothing
//! copied; P-384's window is affine, its additions mixed. P-256's generator
//! table runs the same way (`p256_x86::table_mul`).
// With the modules below, the only places in this crate allowed `unsafe`:
// inline assembly for P-256 and P-384 on x86-64. See the modules, and
// `lib.rs`, for why.
pub
// The same, for whole point formulas; generated.
pub
// P-384's multiplications in the same assembly; generated, and its dispatch.
pub
pub
pub
// P-256's and P-384's point formulas on AVX-512 IFMA, generated, and the
// scalar multiplications built on them, selected at run time.
pub
pub
// Inputs on both sides of the fused operations' fold, for the tests of the
// x86-64 assembly above.
// Verification's `u1*G + u2*Q`: variable time, for public inputs only.
pub
pub
// Its chain on the P-256 and P-384 formulas' assembly; also variable time.
pub
// Its divsteps in assembly, each decision a conditional move.
pub