1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
//! The bit counts a machine has one instruction for.
//!
//! Design: `spec/optimizer/20-idioms-and-libcalls.md` section 20.6 and tamnd/rucc#310.
//!
//! Two places ask the same question of a target. The code generator decides which counts it
//! leaves for a rule and which it writes out as shifts, masks and a multiply, and the loop deletion
//! pass decides whether a loop that counts bits is worth turning into one count in front of it. If
//! each read its own table the two could disagree, and the pass would put a count where the code
//! generator then writes out a dozen instructions in place of a loop that went round twice. So the
//! table is a fact about the target, it is here for the reason [`crate::BitInsts`] is, and both
//! read it.
//!
//! A count can be in the base architecture rather than in an extension, which is how AArch64 has
//! both of its: `clz` for the leading zeros, and `rbit` then `clz` for the trailing ones, which is two
//! instructions and still a long way short of the arithmetic. Those have no feature to name and are
//! on everywhere.
//!
//! A count can also be guarded, which is how x86-64 has its two zero counts on a processor without
//! `lzcnt` or BMI. `bsr` and `bsf` find the bit, but what they leave for a zero is not the same on
//! every processor, so a rule takes one only where the count is written as a choice between the
//! width, for a zero, and the count, for anything else. A conditional move after the search is that
//! choice, so a guarded count is three or four instructions: much better than the arithmetic, and
//! not one instruction, which is why the loop deletion pass does not count it as one.
//!
//! And a count can go through a vector register, which is how AArch64 counts the set bits: the
//! base architecture has `cnt` only for the bytes of a vector register, so the count is a move
//! across, `cnt`, an `addv` that adds the bytes and a move back. A function built with
//! `-mgeneral-regs-only` may not touch those registers, so the code generator leaves such a count
//! to its rule only where the function may, and the loop deletion pass does not count it as one
//! instruction either.
//!
//! It names the counts without the IR, since this crate sits below it, and each reader says which
//! of its instructions is which count.
use crate;
/// Which bits a count counts. All three answer the width for a zero.
/// A bit count that is one instruction on a processor with the extension that has it, or a short
/// sequence where it is guarded or goes through a vector register.