rucc_opt/scev.rs
1//! Scalar evolution: how a value changes across the iterations of a loop, and how many
2//! iterations there are.
3//!
4//! Design: `spec/optimizer/07-loops-and-scev.md` sections 7.4 through 7.7. This is the second
5//! half of document 07 and it answers the last two of the four questions section 7.6 says loop
6//! analysis exists for. The first two are in [`crate::loops`].
7//!
8//! # Chains of recurrences, and how much of one
9//!
10//! GCC writes how a value changes as a chain of recurrences, `{base, +, step}`, meaning a value
11//! that is `base` on the first iteration and `step` more on each one after. The representation is
12//! good because it is closed under the operations anyone wants: adding two chrecs of the same
13//! loop adds componentwise, multiplying by something invariant scales both parts, and evaluating
14//! one at a given iteration is arithmetic rather than a special case. That closure is why
15//! `j = 2 * i + 3` is as easy as `i = i + 1`, and pattern matching the second would run out of
16//! road on the first.
17//!
18//! Section 7.4 says what rucc builds and it is a subset: affine chrecs only. A value is
19//! invariant, or `{base, +, step}` with both parts invariant, or unknown. Addition, subtraction,
20//! multiplication by an invariant, shifting by a constant, and extension where the extension
21//! provably does not wrap. Nothing polynomial and nothing mutually recursive. That covers every
22//! induction variable a C programmer writes and every array subscript document 31 could use, and
23//! what it leaves out of GCC's four thousand lines is the part serving Fortran and the polyhedral
24//! framework.
25//!
26//! The one extension past affine is pointer chrecs, because C loops walk pointers and `p = p + 1`
27//! is `i = i + 1` with a scale. A `ptr_add` is addition with the byte offset as the step, which
28//! is the difference between analysing half of real C loops and analysing nearly all of them.
29//!
30//! # Trip counts, and the part that is uncomfortable
31//!
32//! Given an exit that compares an affine chrec against something invariant, solving for the
33//! iteration at which the comparison first fails is arithmetic. What makes it hard is that the
34//! answer is almost always conditional: on the loop being entered at all, and on the induction
35//! variable not wrapping before it gets there. Section 7.5 says a trip count returned without its
36//! assumptions is a miscompilation generator, and that the temptation to return one is strong
37//! because the assumptions are usually true.
38//!
39//! So [`Bound`] carries them and there is no way to read the count without seeing them.
40//! [`Bound::parts`] hands back both, and [`Bound::proven`] hands back the count only when there
41//! is nothing left to prove. A caller that means to emit a runtime check reads the assumptions
42//! and emits it, and a caller that forgets cannot get at the number.
43//!
44//! [`Bound`] and [`Estimate`] are different types on purpose. A bound is used for correctness, an
45//! estimate is used to decide whether a transformation is worth doing, and section 7.5 calls
46//! conflating them a category error that costs correctness. GCC keeps them apart as
47//! `max_loop_iterations` and `estimate_numbers_of_iterations` and the names do not stop anyone.
48//! Different structs do.
49
50use std::collections::HashMap;
51
52use rucc_ir::{Block, Def, Extra, Flags, Func, Imm, Inst, IntPred, Opcode, Type, Value};
53
54use crate::cfg::Cfg;
55use crate::loops::{LoopId, Loops};
56
57/// How deep the search for a step walks back through arithmetic.
58///
59/// The chain from a header parameter to the value fed back to it is two or three instructions in
60/// anything a person writes, and the walk terminates on its own because SSA has no cycles except
61/// through block parameters. The limit is here so a generated function with a thousand additions
62/// in the increment costs a bounded amount rather than a stack.
63const STEP_LIMIT: u32 = 16;
64
65/// How many blocks that do nothing but pass a value on the walk reads through.
66///
67/// One is what a canonicalized loop has. The limit is here for the same reason the one above is,
68/// which is that a generated function can have a chain of them and the cost of following it should
69/// not depend on how long somebody made it.
70const FORWARD_LIMIT: u32 = 8;
71
72/// How many times a loop is assumed to run when nothing better is known.
73///
74/// GCC's `--param avg-loop-niter`, whose default is the same number. It is a guess, so nothing may
75/// rest on it. It reaches [`Estimate`], which is only ever used to decide whether something is
76/// worth doing, and [`crate::split`], which spends it on how far to ask the runtime to look and is
77/// answered with a true count of bytes whatever it asked for.
78pub(crate) const ASSUMED_ITERATIONS: u64 = 10;
79
80/// A value that does not change inside the loop, read as `scale * value + offset`.
81///
82/// The `value` is a value defined outside the loop, or `None` when the expression is a plain
83/// number. Keeping the shape rather than a bare [`Value`] is what lets `j = 2 * i + 3` come out
84/// as `{3, +, 2}` instead of unknown: the base and the step of that chrec are expressions nothing
85/// in the function computes, so a representation that could only name existing values would have
86/// to give up.
87///
88/// Arithmetic on two of these is refused when both are symbolic and the symbols differ, because
89/// `x + y` is not of this shape. That is the boundary of the subset and it is where the answer
90/// becomes unknown rather than wrong.
91#[derive(Clone, Copy, Debug, PartialEq, Eq)]
92pub struct Invariant {
93 /// What it is built on, or `None` for a plain number.
94 pub value: Option<Value>,
95 /// How many of it.
96 pub scale: i128,
97 /// What is added to it.
98 pub offset: i128,
99}
100
101impl Invariant {
102 /// A plain number.
103 #[must_use]
104 pub fn number(offset: i128) -> Self {
105 Self { value: None, scale: 0, offset }
106 }
107
108 /// One of a value.
109 #[must_use]
110 pub fn of(value: Value) -> Self {
111 Self { value: Some(value), scale: 1, offset: 0 }
112 }
113
114 /// The number this is, when it is one.
115 #[must_use]
116 pub fn as_number(self) -> Option<i128> {
117 (self.value.is_none() || self.scale == 0).then_some(self.offset)
118 }
119
120 /// Whether this is the number zero.
121 #[must_use]
122 pub fn is_zero(self) -> bool {
123 self.as_number() == Some(0)
124 }
125
126 /// The symbol both expressions are built on, when they agree on one or one has none.
127 fn shared(self, other: Self) -> Option<Option<Value>> {
128 match (self.as_number().is_some(), other.as_number().is_some()) {
129 (true, _) => Some(other.value),
130 (_, true) => Some(self.value),
131 _ => (self.value == other.value).then_some(self.value),
132 }
133 }
134
135 /// The two added, when the sum is of this shape.
136 #[must_use]
137 pub fn plus(self, other: Self) -> Option<Self> {
138 let value = self.shared(other)?;
139 Some(Self {
140 value,
141 scale: self.scale.checked_add(other.scale)?,
142 offset: self.offset.checked_add(other.offset)?,
143 })
144 }
145
146 /// The second subtracted from the first, when the difference is of this shape.
147 #[must_use]
148 pub fn minus(self, other: Self) -> Option<Self> {
149 self.plus(other.negated()?)
150 }
151
152 /// This with its sign flipped.
153 #[must_use]
154 pub fn negated(self) -> Option<Self> {
155 Some(Self {
156 value: self.value,
157 scale: self.scale.checked_neg()?,
158 offset: self.offset.checked_neg()?,
159 })
160 }
161
162 /// The two multiplied, which needs one of them to be a plain number.
163 #[must_use]
164 pub fn times(self, other: Self) -> Option<Self> {
165 let (symbol, by) = match (self.as_number(), other.as_number()) {
166 (Some(by), _) => (other, by),
167 (_, Some(by)) => (self, by),
168 _ => return None,
169 };
170 Some(Self {
171 value: symbol.value,
172 scale: symbol.scale.checked_mul(by)?,
173 offset: symbol.offset.checked_mul(by)?,
174 })
175 }
176}
177
178/// How a value changes from one iteration of a loop to the next.
179#[derive(Clone, Copy, Debug, PartialEq, Eq)]
180pub enum Evolution {
181 /// The same on every iteration.
182 Invariant(Invariant),
183 /// `{base, +, step}`: `base` the first time round and `step` more each time after.
184 Affine(Chrec),
185 /// Not something this analysis describes. Never a claim that the value does not evolve.
186 Unknown,
187}
188
189impl Evolution {
190 /// The chrec, when this is one.
191 #[must_use]
192 pub fn chrec(self) -> Option<Chrec> {
193 match self {
194 Self::Affine(chrec) => Some(chrec),
195 _ => None,
196 }
197 }
198
199 /// The invariant expression, when this is one.
200 #[must_use]
201 pub fn invariant(self) -> Option<Invariant> {
202 match self {
203 Self::Invariant(inv) => Some(inv),
204 _ => None,
205 }
206 }
207}
208
209/// An affine chain of recurrences, `{base, +, step}`, evolving in a named type.
210///
211/// The type is not decoration. `{0, +, 1}` in `unsigned char` is not the sequence `0, 1, 2, ...`,
212/// it is that sequence modulo two hundred and fifty six, and section 7.7 says this is where a
213/// naive implementation is wrong constantly and in ways that pass every test written by someone
214/// thinking in `int`. Every operation here checks the type and every one that cannot stay right
215/// in it answers unknown.
216#[derive(Clone, Copy, Debug, PartialEq, Eq)]
217pub struct Chrec {
218 /// What the value is on the first iteration.
219 pub base: Invariant,
220 /// What is added each time round.
221 pub step: Invariant,
222 /// The type it evolves in, which is what says when it wraps.
223 pub ty: Type,
224 /// What the instruction that increments it promised. `nsw` means the sequence does not wrap
225 /// when read as signed and `nuw` means it does not when read as unsigned, and both come from
226 /// the increment rather than from anything this analysis proved.
227 pub flags: Flags,
228}
229
230impl Chrec {
231 /// Whether the sequence is known not to wrap under the reading this predicate takes.
232 #[must_use]
233 pub fn does_not_wrap(self, signed: bool) -> bool {
234 self.flags.contains(if signed { Flags::NSW } else { Flags::NUW })
235 }
236}
237
238/// Something that has to be true for a trip count to be the right answer.
239///
240/// Section 7.5 asks for exactly this: not a trip count but a trip count plus a predicate under
241/// which it holds, so the consumer either proves the predicate, emits a runtime check for it, or
242/// gives up. These are the predicates.
243#[derive(Clone, Copy, Debug, PartialEq, Eq)]
244pub enum Assumption {
245 /// The counter starts on the near side of its limit, so the distance between them is a
246 /// number that is not negative.
247 ///
248 /// For a loop ending on an ordering this is the loop being entered at all.
249 /// `for (i = 0; i < n; i++)` with `n` of zero runs no times and the distance is zero, but `n`
250 /// of minus one also runs no times and the distance is minus one, so a count taken from the
251 /// distance has to be told which case it is in. For a loop ending on `!=` it is the limit
252 /// being somewhere the counter is heading, because one stepping away from its limit never
253 /// arrives.
254 ///
255 /// Only ever present on a symbolic count. When the distance is a number the sign of it is
256 /// there to be read, so this is settled rather than assumed.
257 Approaching,
258 /// The induction variable does not wrap in its own type before the exit is taken.
259 ///
260 /// Present whenever the increment did not carry the matching `nsw` or `nuw` flag. With the
261 /// flag there is nothing to assume, because the flag is the promise.
262 NoWrap(Chrec),
263 /// Signed overflow is undefined here, which is what makes `for (int i = 0; i <= n; i++)`
264 /// finite.
265 ///
266 /// GCC infers loop bounds from this in `infer_loop_bounds_from_signedness`, and it is the
267 /// single most common source of a report that the compiler broke a working program. It is
268 /// recorded rather than assumed silently so that `-fwrapv` can withdraw the count and so that
269 /// a dump can name it.
270 StrictOverflow,
271}
272
273impl Assumption {
274 /// What it says, in a line, for a dump to print.
275 ///
276 /// Section 7.5 asks that every inference of this kind be dumpable and say what it rests on,
277 /// because a user who has been bitten by one deserves a command that tells them which line
278 /// the compiler used against them. This is the sentence that command prints.
279 #[must_use]
280 pub fn describe(&self) -> String {
281 match self {
282 Self::Approaching => "the counter starts on the near side of its limit".to_string(),
283 Self::NoWrap(chrec) => {
284 format!("the induction variable does not wrap in i{}", chrec.ty.bits())
285 }
286 Self::StrictOverflow => {
287 "signed overflow is undefined, so -fwrapv withdraws this count".to_string()
288 }
289 }
290 }
291}
292
293/// How many iterations, as a number or as an expression.
294#[derive(Clone, Copy, Debug, PartialEq, Eq)]
295pub enum Count {
296 /// Exactly this many.
297 Exact(u128),
298 /// This many, worked out from something the loop does not change.
299 Symbolic(Invariant),
300}
301
302/// Which reading of its operands the test the count came from took.
303///
304/// It matters to anybody widening the value a symbolic count is built out of. The count is the
305/// distance to the limit of the exit test, the limit is a value of the counter's own type, and
306/// what that value means is the reading its test took. A limit past the middle of a thirty two bit
307/// type is a large number to an unsigned test and a negative one to a signed test, and a consumer
308/// that sign extends what an unsigned test compared has turned a loop over three billion elements
309/// into a loop that runs no times.
310#[derive(Clone, Copy, Debug, PartialEq, Eq)]
311pub enum Reading {
312 /// The test read its operands as signed, so widening the count means sign extending it.
313 Signed,
314 /// The test read them as unsigned, so widening the count means zero extending it.
315 Unsigned,
316}
317
318/// How many times a loop runs at most, and what that rests on.
319///
320/// For correctness. A pass that deletes an iteration, peels one off, or decides a memory access
321/// is in bounds needs one of these. The count cannot be read without the assumptions, which is
322/// section 7.7's defence against a caller proving two of three and forgetting the third.
323#[derive(Clone, Debug, PartialEq, Eq)]
324pub struct Bound {
325 count: Count,
326 assumptions: Vec<Assumption>,
327 reading: Reading,
328}
329
330impl Bound {
331 /// The count and everything it rests on, together, because they cannot be asked for apart.
332 #[must_use]
333 pub fn parts(&self) -> (Count, &[Assumption]) {
334 (self.count, &self.assumptions)
335 }
336
337 /// How the value a symbolic count is built out of has to be read.
338 ///
339 /// Meaningless on a count that is a number, since a number has already been read.
340 #[must_use]
341 pub fn reading(&self) -> Reading {
342 self.reading
343 }
344
345 /// What has to be proved before the count means anything.
346 #[must_use]
347 pub fn assumptions(&self) -> &[Assumption] {
348 &self.assumptions
349 }
350
351 /// The count, for a caller with nothing left to prove.
352 ///
353 /// `None` does not mean the count is unknown. It means there are assumptions and this is not
354 /// the accessor for reading a count that has them.
355 #[must_use]
356 pub fn proven(&self) -> Option<Count> {
357 self.assumptions.is_empty().then_some(self.count)
358 }
359
360 /// The count, for a caller compiling a language where signed overflow is undefined.
361 ///
362 /// [`Bound::proven`] answers nothing for any `for (int i = 0; i < n; i++)` in any C program,
363 /// because `solve` puts [`Assumption::StrictOverflow`] on every count taken from a signed
364 /// test, and a pass built on `proven` alone is a pass that never fires. What that assumption
365 /// says is that the count rests on signed overflow being undefined, and `-fwrapv` is
366 /// implemented in `rucc-lower` by not setting `nsw` rather than by a flag anything down here
367 /// reads. So an increment that still carries `nsw` under `-fwrapv` does not exist, and a bound
368 /// with `StrictOverflow` and nothing else on it is a bound whose counter the front end
369 /// promised does not wrap. That promise is exactly what the assumption wanted.
370 ///
371 /// [`Assumption::NoWrap`] is the case where there is no such promise, and it is refused here.
372 /// So is [`Assumption::Approaching`], though only in passing, because it never appears on a
373 /// count that is a number.
374 #[must_use]
375 pub fn under_undefined_overflow(&self) -> Option<Count> {
376 self.assumptions
377 .iter()
378 .all(|rests_on| matches!(rests_on, Assumption::StrictOverflow))
379 .then_some(self.count)
380 }
381}
382
383/// How many times a loop probably runs.
384///
385/// For cost decisions and never for correctness. A pass asking whether unrolling pays for itself
386/// wants one of these, and it is fine for the answer to be a guess, because being wrong makes the
387/// code slower rather than wrong. Nothing here can be turned into a [`Bound`].
388#[derive(Clone, Copy, Debug, PartialEq, Eq)]
389pub struct Estimate {
390 iterations: u64,
391 guessed: bool,
392}
393
394impl Estimate {
395 /// The number to do arithmetic with.
396 #[must_use]
397 pub fn iterations(self) -> u64 {
398 self.iterations
399 }
400
401 /// Whether nothing was known and this is the default.
402 #[must_use]
403 pub fn is_guess(self) -> bool {
404 self.guessed
405 }
406}
407
408/// An exit test, read so that the loop keeps going while it holds.
409///
410/// Not public. It is the shape [`Scev::bound_at`] and [`Scev::holds`] both want out of the same
411/// branch, and what either of them says about it is what the outside sees.
412#[derive(Clone, Copy, Debug)]
413struct Test {
414 /// The side that moves, with the predicate already turned round to put it on the left.
415 chrec: Chrec,
416 /// The side that does not.
417 limit: Invariant,
418 /// The comparison that has to hold for the loop to go round again.
419 pred: IntPred,
420 /// Whether every iteration that goes round asks it.
421 each: bool,
422}
423
424/// The analysis, which works out an answer when asked and remembers it.
425///
426/// Demand driven and memoized, per section 7.8, because the cost of scalar evolution is a
427/// function of how many distinct values get asked about rather than of the size of the function.
428/// The cache holds one loop's worth of answers per loop and the whole thing is thrown away when
429/// anything about the loops changes, which per document 04.4 is any pass that touches one.
430#[derive(Debug)]
431pub struct Scev<'a> {
432 func: &'a Func,
433 cfg: &'a Cfg,
434 loops: &'a Loops,
435 known: HashMap<(LoopId, Value), Evolution>,
436 held: HashMap<LoopId, Option<Chrec>>,
437}
438
439impl<'a> Scev<'a> {
440 /// A fresh analysis over these loops, knowing nothing yet.
441 #[must_use]
442 pub fn new(func: &'a Func, cfg: &'a Cfg, loops: &'a Loops) -> Self {
443 Self { func, cfg, loops, known: HashMap::new(), held: HashMap::new() }
444 }
445
446 /// How this value changes across the iterations of this loop.
447 ///
448 /// The way in, and what it does before answering is settle `Scev::holds` for the loop. That has
449 /// to happen out here rather than at the point `Scev::extend` wants it, because settling it
450 /// means asking about other values and `Scev::at` parks a marker on the value it is working on.
451 /// Asked from in there, the answer would depend on what was already in flight.
452 pub fn evolution(&mut self, id: LoopId, value: Value) -> Evolution {
453 self.holds(id);
454 self.at(id, value)
455 }
456
457 /// How this value changes, with the loop's own facts already settled.
458 fn at(&mut self, id: LoopId, value: Value) -> Evolution {
459 if let Some(&known) = self.known.get(&(id, value)) {
460 return known;
461 }
462 // Unknown while the answer is being worked out, so the cycle from a header parameter back
463 // to itself terminates instead of asking the same question forever. Anything that reaches
464 // the parameter again gets unknown and the shape it was matching fails, which is the
465 // right answer for a value defined in terms of itself through arithmetic this does not
466 // describe.
467 self.known.insert((id, value), Evolution::Unknown);
468 let found = self.compute(id, value);
469 self.known.insert((id, value), found);
470 found
471 }
472
473 /// How many times this loop runs at most, and what that rests on.
474 ///
475 /// Any one exit gives a valid upper bound, because a loop cannot run more times than the
476 /// first exit that fires, so this takes the first exit it can solve rather than the smallest.
477 /// That is `max_loop_iterations` and not `estimate_numbers_of_iterations`, which is why the
478 /// answer is a [`Bound`].
479 pub fn bound(&mut self, id: LoopId) -> Option<Bound> {
480 self.holds(id);
481 let exits: Vec<Block> = self.loops.exits(id).iter().map(|exit| exit.from).collect();
482 exits.into_iter().find_map(|from| self.bound_at(id, from))
483 }
484
485 /// The counter an exit test of this loop keeps inside its own type, when there is one.
486 ///
487 /// [`bounded_by_its_test`] is the argument and this is where its answer is written down as a
488 /// fact about the loop rather than spent on one trip count. What it buys is [`Scev::extend`]:
489 /// an unsigned counter carries no `nuw`, so widening anything built out of one used to be
490 /// refused, and the test that holds the counter holds everything walking beside it.
491 ///
492 /// Settled once per loop and then read. It is settled from [`Scev::evolution`] and
493 /// [`Scev::bound`], which are the two ways in, so that it is worked out with nothing in flight.
494 /// The cache for the loop is emptied afterwards, because the answers already in it were worked
495 /// out while this was still unknown and a conservative answer that stayed would make what the
496 /// analysis says depend on which question was asked first.
497 fn holds(&mut self, id: LoopId) -> Option<Chrec> {
498 if let Some(&known) = self.held.get(&id) {
499 return known;
500 }
501 // Unknown while it is being worked out, which is what stops the recursion below from
502 // asking the same question forever, and which is why the cache is emptied after.
503 self.held.insert(id, None);
504 let exits: Vec<Block> = self.loops.exits(id).iter().map(|exit| exit.from).collect();
505 let found = exits.into_iter().find_map(|from| {
506 let test = self.test_at(id, from)?;
507 let step = test.chrec.step.as_number()?;
508 (test.each && bounded_by_its_test(test.pred, step)).then_some(test.chrec)
509 });
510 self.held.insert(id, found);
511 self.known.retain(|&(of, _), _| of != id);
512 found
513 }
514
515 /// How many times this loop probably runs.
516 pub fn estimate(&mut self, id: LoopId) -> Estimate {
517 match self.bound(id).map(|bound| bound.count) {
518 Some(Count::Exact(exact)) => {
519 Estimate { iterations: u64::try_from(exact).unwrap_or(u64::MAX), guessed: false }
520 }
521 _ => Estimate { iterations: ASSUMED_ITERATIONS, guessed: true },
522 }
523 }
524
525 /// The evolution of a value nothing is known about yet.
526 fn compute(&mut self, id: LoopId, value: Value) -> Evolution {
527 if let Some(invariant) = self.invariant(id, value) {
528 return Evolution::Invariant(invariant);
529 }
530 match self.func[value].def {
531 Def::Param { block, index } if block == self.loops.header(id) => {
532 self.at_header(id, value, index as usize)
533 }
534 // A parameter of a block inside the loop that is not the header takes a different
535 // value depending on which way control came, and describing that is a job for the
536 // value range work of document 10 rather than for a chrec. Unless there is only one
537 // way in, in which case it does not.
538 Def::Param { .. } => match self.forwarded(value) {
539 same if same == value => Evolution::Unknown,
540 through => self.at(id, through),
541 },
542 Def::Result { inst, .. } => self.at_inst(id, inst, value),
543 }
544 }
545
546 /// The value as an expression that does not change inside the loop, if it is one.
547 fn invariant(&self, id: LoopId, value: Value) -> Option<Invariant> {
548 if let Some((imm, ty)) = constant(self.func, value) {
549 return Some(Invariant::number(imm.signed(ty)));
550 }
551 // A constant is invariant wherever it sits, which is why it is asked about first. Anything
552 // else has to be defined outside the loop.
553 self.loops.is_invariant(self.func, id, value).then(|| Invariant::of(value))
554 }
555
556 /// The evolution of a parameter of the loop header, which is where an induction variable is.
557 ///
558 /// The parameter takes one value on the way in and another on the way round, which is what
559 /// other IRs spell as a phi node. If the way round is the parameter plus something invariant,
560 /// the parameter is an affine chrec and that something is its step.
561 fn at_header(&mut self, id: LoopId, value: Value, index: usize) -> Evolution {
562 let (func, cfg, loops) = (self.func, self.cfg, self.loops);
563 let header = loops.header(id);
564 // Section 7.3 wants exactly one latch and the canonicalizer makes one. Two of them means
565 // two ways round with two different increments, and picking one would be a guess.
566 let [latch] = loops.latches(id) else { return Evolution::Unknown };
567 let mut entering = None;
568 let mut around = None;
569 for &pred in cfg.predecessors(header) {
570 let Some(arg) = argument(func, pred, header, index) else { return Evolution::Unknown };
571 let arg = self.forwarded(arg);
572 let slot = if pred == *latch { &mut around } else { &mut entering };
573 if slot.replace(arg).is_some_and(|old| old != arg) {
574 return Evolution::Unknown;
575 }
576 }
577 let (Some(entering), Some(around)) = (entering, around) else { return Evolution::Unknown };
578 let Some(base) = self.invariant(id, entering) else { return Evolution::Unknown };
579 let Some((step, flags)) = self.step(id, around, value, 0) else {
580 return Evolution::Unknown;
581 };
582 affine(base, step, func[value].ty, flags)
583 }
584
585 /// The value a block parameter stands for, when there is only one way into its block.
586 ///
587 /// This is not an analysis, it is undoing a rename. A block with one predecessor has one value
588 /// for each of its parameters and it is the argument that predecessor passes, so reading
589 /// through it loses nothing and assumes nothing.
590 ///
591 /// It is here because of what canonicalization does. `crate::canon` splits the back edge of a
592 /// loop to give it a latch of its own, and after that the value going round the loop is not the
593 /// increment the loop computed, it is a parameter of a block that does nothing but pass the
594 /// increment on. Without this, every counted loop the pipeline actually produces looks like a
595 /// loop whose counter comes from somewhere unknown, and the trip count of a `for` loop in a
596 /// real function comes back as nothing.
597 fn forwarded(&self, value: Value) -> Value {
598 let mut value = value;
599 for _ in 0..FORWARD_LIMIT {
600 let Def::Param { block, index } = self.func[value].def else { return value };
601 let [pred] = self.cfg.predecessors(block) else { return value };
602 let Some(arg) = argument(self.func, *pred, block, index as usize) else { return value };
603 if arg == value {
604 return value;
605 }
606 value = arg;
607 }
608 value
609 }
610
611 /// What is added to `of` to get `value`, and what the additions promised.
612 ///
613 /// Written as its own walk rather than as the general combination below, because at the point
614 /// this runs the parameter's own evolution is not known yet and the general walk would ask
615 /// for it and get unknown.
616 fn step(&self, id: LoopId, value: Value, of: Value, depth: u32) -> Option<(Invariant, Flags)> {
617 let value = self.forwarded(value);
618 if value == of {
619 // Nothing added yet, and nothing has had a chance to overflow either.
620 return Some((Invariant::number(0), Flags::NSW.union(Flags::NUW)));
621 }
622 if depth >= STEP_LIMIT {
623 return None;
624 }
625 let Def::Result { inst, .. } = self.func[value].def else { return None };
626 let data = &self.func[inst];
627 let args = &self.func[data.args];
628 let (&lhs, &rhs) = (args.first()?, args.get(1)?);
629 let combine = |carried: (Invariant, Flags), other: Invariant, subtract: bool| {
630 let (delta, flags) = carried;
631 let moved = if subtract { delta.minus(other)? } else { delta.plus(other)? };
632 Some((moved, flags.intersection(data.flags)))
633 };
634 match data.opcode {
635 Opcode::Add => {
636 if let Some(carried) = self.step(id, lhs, of, depth + 1) {
637 return combine(carried, self.invariant(id, rhs)?, false);
638 }
639 combine(self.step(id, rhs, of, depth + 1)?, self.invariant(id, lhs)?, false)
640 }
641 Opcode::Sub => {
642 combine(self.step(id, lhs, of, depth + 1)?, self.invariant(id, rhs)?, true)
643 }
644 // A pointer walks by bytes, and only the pointer side can be the one carrying the
645 // induction variable. The offset is the step, which is the element size the front end
646 // already multiplied in.
647 Opcode::PtrAdd => {
648 combine(self.step(id, lhs, of, depth + 1)?, self.invariant(id, rhs)?, false)
649 }
650 _ => None,
651 }
652 }
653
654 /// The evolution of an instruction's result, from the evolutions of its operands.
655 fn at_inst(&mut self, id: LoopId, inst: Inst, value: Value) -> Evolution {
656 let func = self.func;
657 let data = &func[inst];
658 let (opcode, flags) = (data.opcode, data.flags);
659 let args = &func[data.args];
660 let ty = func[value].ty;
661 let Some(&lhs) = args.first() else { return Evolution::Unknown };
662 match opcode {
663 Opcode::Add | Opcode::PtrAdd => {
664 let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
665 let (left, right) = (self.at(id, lhs), self.at(id, rhs));
666 combine(left, right, ty, flags, false)
667 }
668 Opcode::Sub => {
669 let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
670 let (left, right) = (self.at(id, lhs), self.at(id, rhs));
671 combine(left, right, ty, flags, true)
672 }
673 Opcode::Mul => {
674 let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
675 let (left, right) = (self.at(id, lhs), self.at(id, rhs));
676 scale(left, right, ty, flags)
677 }
678 // A shift by a constant is a multiplication by a power of two, and only by a constant:
679 // a variable count is invariant in the loop and still not a number this can multiply
680 // by. A count at or above the width is poison rather than a shift to zero, so the
681 // range is checked here rather than assumed.
682 Opcode::Shl => {
683 let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
684 let Some((count, count_ty)) = constant(func, rhs) else {
685 return Evolution::Unknown;
686 };
687 let count = count.unsigned();
688 if count >= u128::from(ty.bits()) || !count_ty.is_int() {
689 return Evolution::Unknown;
690 }
691 let by = Evolution::Invariant(Invariant::number(1i128 << count));
692 scale(self.at(id, lhs), by, ty, flags)
693 }
694 Opcode::SExt | Opcode::ZExt => self.extend(id, opcode, lhs, ty),
695 // A truncation is a wrap by construction, so a chrec through one describes a sequence
696 // that restarts, and this does not have a representation for that.
697 _ => Evolution::Unknown,
698 }
699 }
700
701 /// A chrec widened, which needs the sequence not to wrap at the narrow width.
702 ///
703 /// Section 7.4 allows extension only where the extension provably does not wrap, and the first
704 /// proof here is the flag the increment carries. `nsw` on the increment is the promise that the
705 /// signed sequence does not wrap, which is exactly what makes the wide sequence the same
706 /// numbers as the narrow one.
707 ///
708 /// The second proof is the loop's own exit test, through [`Scev::holds`] and [`trails`], and it
709 /// is here because of what an unsigned counter looks like. `for (unsigned i = 0; i < n; i++)`
710 /// carries no `nuw`, because C says unsigned arithmetic wraps, so `a[i]` on that counter used
711 /// to come back unwidened and every bounds check in the loop stayed where it was. The test that
712 /// keeps the counter inside its type keeps everything walking beside it inside too.
713 ///
714 /// Both parts have to be plain numbers. A symbolic base or step is a value of the narrow type
715 /// and the widened chrec would need it widened too, which is an expression nothing computes
716 /// and which [`Invariant`] has no room to describe. Saying so is the honest answer, the case
717 /// that matters most is a counter from a constant by a constant, and lifting the restriction
718 /// is work for whoever needs a symbolic one.
719 fn extend(&mut self, id: LoopId, opcode: Opcode, from: Value, to: Type) -> Evolution {
720 let narrow = self.func[from].ty;
721 let signed = opcode == Opcode::SExt;
722 let held = self.held.get(&id).copied().flatten();
723 let settled = |chrec: Chrec| {
724 chrec.does_not_wrap(signed) || (!signed && held.is_some_and(|held| trails(chrec, held)))
725 };
726 match self.at(id, from) {
727 Evolution::Invariant(inv) => match inv.as_number() {
728 // A number read at the narrow width means the same thing at the wide one under
729 // sign extension, and under zero extension once it is not negative.
730 Some(number) if signed || number >= 0 => Evolution::Invariant(inv),
731 _ => Evolution::Unknown,
732 },
733 Evolution::Affine(chrec) if chrec.ty == narrow && settled(chrec) => {
734 let (Some(base), Some(step)) = (chrec.base.as_number(), chrec.step.as_number())
735 else {
736 return Evolution::Unknown;
737 };
738 Evolution::Affine(Chrec {
739 base: Invariant::number(base),
740 step: Invariant::number(step),
741 ty: to,
742 flags: chrec.flags,
743 })
744 }
745 _ => Evolution::Unknown,
746 }
747 }
748
749 /// The trip count from the exit leaving this block, if this exit can be solved.
750 fn bound_at(&mut self, id: LoopId, from: Block) -> Option<Bound> {
751 let test = self.test_at(id, from)?;
752 solve(test.chrec, test.limit, test.pred, test.each)
753 }
754
755 /// The exit test leaving this block, read into the pieces its two readers want.
756 ///
757 /// [`Scev::bound_at`] spends it on a trip count and [`Scev::holds`] spends it on whether the
758 /// counter can wrap, and both want the same reading of the same branch, so the reading is
759 /// written once.
760 fn test_at(&mut self, id: LoopId, from: Block) -> Option<Test> {
761 let func = self.func;
762 let term = func.terminator(from)?;
763 if func[term].opcode != Opcode::BrIf {
764 return None;
765 }
766 let args = &func[func[term].args];
767 let &cond = args.first()?;
768 let calls = &func[func.target_list(term)];
769 let (&taken, ¬_taken) = (calls.first()?, calls.get(1)?);
770 // Which arm keeps going. If both stay in or both leave, the branch is not the test that
771 // ends the loop and there is nothing here to solve.
772 let stays = match (
773 self.loops.contains(id, taken.block),
774 self.loops.contains(id, not_taken.block),
775 ) {
776 (true, false) => true,
777 (false, true) => false,
778 _ => return None,
779 };
780
781 let Def::Result { inst, .. } = func[cond].def else { return None };
782 if func[inst].opcode != Opcode::ICmp {
783 return None;
784 }
785 let Extra::IntPred(pred) = func[inst].extra else { return None };
786 // The loop keeps going while the test says so, so an exit taken when the test is true is
787 // an exit whose continuing condition is the opposite one.
788 let pred = if stays { pred } else { invert(pred) };
789 let operands = &func[func[inst].args];
790 let (&lhs, &rhs) = (operands.first()?, operands.get(1)?);
791
792 // One side evolves and the other does not. Swapping puts the one that evolves on the left
793 // and turns the predicate round with it, so only one direction has to be solved.
794 let (chrec, limit, pred) = match (self.at(id, lhs), self.at(id, rhs)) {
795 (Evolution::Affine(chrec), other) => (chrec, other.invariant()?, pred),
796 (other, Evolution::Affine(chrec)) => (chrec, other.invariant()?, swap(pred)),
797 _ => return None,
798 };
799
800 // Whether every iteration that goes round asks this test. The header runs on all of them by
801 // being the header. A latch runs on all of them only when it is the loop's one latch, since
802 // with two of them an iteration can go round the other and never reach the test. Anywhere
803 // else is a test under a condition, which [`bounded_by_its_test`] must not be given.
804 //
805 // The one latch is written out rather than taken for granted. `at_header` refuses a loop
806 // with two of them already, so nothing reaching here has two, but the two conditions are
807 // about different things and a later loosening of that one should not quietly loosen this.
808 let each = from == self.loops.header(id) || self.loops.latches(id) == [from];
809 Some(Test { chrec, limit, pred, each })
810 }
811}
812
813/// Two evolutions added, or subtracted when asked.
814fn combine(left: Evolution, right: Evolution, ty: Type, flags: Flags, subtract: bool) -> Evolution {
815 let apply = |a: Invariant, b: Invariant| if subtract { a.minus(b) } else { a.plus(b) };
816 match (left, right) {
817 (Evolution::Invariant(a), Evolution::Invariant(b)) => {
818 apply(a, b).map_or(Evolution::Unknown, Evolution::Invariant)
819 }
820 (Evolution::Affine(chrec), Evolution::Invariant(b)) => {
821 // Adding something that does not move only moves the base.
822 let Some(base) = apply(chrec.base, b) else { return Evolution::Unknown };
823 affine(base, chrec.step, ty, flags.intersection(chrec.flags))
824 }
825 (Evolution::Invariant(a), Evolution::Affine(chrec)) => {
826 let (Some(base), Some(step)) = (
827 apply(a, chrec.base),
828 if subtract { chrec.step.negated() } else { Some(chrec.step) },
829 ) else {
830 return Evolution::Unknown;
831 };
832 affine(base, step, ty, flags.intersection(chrec.flags))
833 }
834 (Evolution::Affine(a), Evolution::Affine(b)) => {
835 // Two chrecs of the same loop add componentwise, which is the closure property that
836 // makes the representation worth having. Of different types they do not, because the
837 // two sequences wrap at different widths.
838 if a.ty != b.ty {
839 return Evolution::Unknown;
840 }
841 let (Some(base), Some(step)) = (apply(a.base, b.base), apply(a.step, b.step)) else {
842 return Evolution::Unknown;
843 };
844 affine(base, step, ty, flags.intersection(a.flags).intersection(b.flags))
845 }
846 _ => Evolution::Unknown,
847 }
848}
849
850/// One evolution multiplied by another, which needs one of them to stand still.
851fn scale(left: Evolution, right: Evolution, ty: Type, flags: Flags) -> Evolution {
852 let (chrec, by) = match (left, right) {
853 (Evolution::Invariant(a), Evolution::Invariant(b)) => {
854 return a.times(b).map_or(Evolution::Unknown, Evolution::Invariant);
855 }
856 (Evolution::Affine(chrec), Evolution::Invariant(by))
857 | (Evolution::Invariant(by), Evolution::Affine(chrec)) => (chrec, by),
858 // Two chrecs multiplied give a quadratic, which is a chain of recurrences with a second
859 // step and is outside the subset section 7.4 chose.
860 _ => return Evolution::Unknown,
861 };
862 let (Some(base), Some(step)) = (chrec.base.times(by), chrec.step.times(by)) else {
863 return Evolution::Unknown;
864 };
865 affine(base, step, ty, flags.intersection(chrec.flags))
866}
867
868/// A chrec, or invariant when the step turns out to be nothing.
869///
870/// A step of zero is a valid affine chrec describing a value that does not move, and section 7.7
871/// warns that code dividing by the step to get a trip count divides by zero. Reporting it as
872/// invariant here means the shape is right for every reader rather than only for the careful
873/// ones, and the trip count solver still checks, because a step can also come out zero from a
874/// header parameter incremented by an invariant that happens to be zero.
875fn affine(base: Invariant, step: Invariant, ty: Type, flags: Flags) -> Evolution {
876 if step.is_zero() {
877 return Evolution::Invariant(base);
878 }
879 Evolution::Affine(Chrec { base, step, ty, flags })
880}
881
882/// The iteration at which `chrec pred limit` first fails, with what that rests on.
883///
884/// `each` says the test runs on every iteration that goes round, which is what lets the test itself
885/// stand in for a promise the counter does not carry. See [`bounded_by_its_test`].
886fn solve(chrec: Chrec, limit: Invariant, pred: IntPred, each: bool) -> Option<Bound> {
887 // Section 7.7's first way of being wrong. A step of zero is a loop that never leaves through
888 // this exit, and dividing the distance by it is a crash rather than an answer.
889 let step = chrec.step.as_number()?;
890 if step == 0 {
891 return None;
892 }
893 let signed = matches!(pred, IntPred::Slt | IntPred::Sle | IntPred::Sgt | IntPred::Sge);
894
895 let mut assumptions = Vec::new();
896 if !chrec.does_not_wrap(signed) && !(each && bounded_by_its_test(pred, step)) {
897 assumptions.push(Assumption::NoWrap(chrec));
898 }
899 if signed {
900 assumptions.push(Assumption::StrictOverflow);
901 }
902
903 // A test that does not read its operands as signed does not read the constants in them that
904 // way either, and every constant reaching here was read as signed on the way in.
905 let (base, limit) = if signed {
906 (chrec.base, limit)
907 } else {
908 (as_unsigned(chrec.base, chrec.ty)?, as_unsigned(limit, chrec.ty)?)
909 };
910
911 // The distance the counter has to travel, always counting up. A loop going down is the same
912 // problem with the ends swapped, which is why the step is used by size below and its sign is
913 // spent here.
914 let apart = step.unsigned_abs();
915 let found = match (pred, step > 0) {
916 (IntPred::Slt | IntPred::Ult, true) => {
917 ordered(limit.minus(base)?, apart, false, assumptions)
918 }
919 (IntPred::Sle | IntPred::Ule, true) => {
920 ordered(limit.minus(base)?, apart, true, assumptions)
921 }
922 (IntPred::Sgt | IntPred::Ugt, false) => {
923 ordered(base.minus(limit)?, apart, false, assumptions)
924 }
925 (IntPred::Sge | IntPred::Uge, false) => {
926 ordered(base.minus(limit)?, apart, true, assumptions)
927 }
928 (IntPred::Ne, _) => {
929 let distance = if step > 0 { limit.minus(base)? } else { base.minus(limit)? };
930 landing(distance, apart, assumptions)
931 }
932 // Either the counter steps away from the limit, in which case the loop is endless rather
933 // than long, or the test is one this does not solve. Silence is the answer to both.
934 _ => None,
935 };
936 // Written once here rather than threaded through the two solvers, because it is a fact about
937 // the test and neither of them looks at the test. A count taken from a test with no sign to it,
938 // which is `!=`, is read unsigned, because that is the reading `as_unsigned` above already put
939 // its operands through.
940 let reading = if signed { Reading::Signed } else { Reading::Unsigned };
941 found.map(|(count, assumptions)| Bound { count, assumptions, reading })
942}
943
944/// Whether the exit test by itself rules out the counter wrapping before the loop ends.
945///
946/// An unsigned counter carries no `nuw`, because C says unsigned arithmetic wraps, so without this
947/// every `for (unsigned i = 0; i < n; i++)` comes back resting on an assumption nothing downstream
948/// can discharge. What discharges it is the test. A counter stepping up by exactly one is at the
949/// limit before it is anywhere past it, and the test ends the loop there, so it never reaches the
950/// top of its type. GCC works the same thing out in `scev_probably_wraps_p`.
951///
952/// Every part of that is load bearing. The step has to be one: `i += 2` can go from one below the
953/// limit to one above the top of the type and come back round at the bottom, which is a loop that
954/// runs forever rather than one that runs twice as fast. The test has to be the strict one: `<=`
955/// lets the counter reach the limit and step once more, and a limit that is the largest number of
956/// its type makes that last step the one that wraps. And the test has to run on every iteration
957/// that goes round, or the counter can be stepped by a path that never asks it anything.
958///
959/// Nothing is claimed here about a signed counter, which needs no help: a signed counter that would
960/// wrap is a program with undefined behaviour in it and [`Assumption::StrictOverflow`] is where
961/// that is recorded.
962fn bounded_by_its_test(pred: IntPred, step: i128) -> bool {
963 matches!((pred, step), (IntPred::Ult, 1) | (IntPred::Ugt, -1))
964}
965
966/// Whether this sequence stays behind one the exit test already keeps inside its type.
967///
968/// [`bounded_by_its_test`] says the counter the test compares never reaches the top of its type.
969/// Everything else the loop counts with is that counter plus a fixed distance, because two affine
970/// chrecs of the same loop with the same step differ by a constant, so a sequence starting no
971/// further along than the counter is a sequence that gets to the top no sooner than the counter
972/// does, which is never.
973///
974/// Same base is the case that matters most and the easiest to see: the test compares `i + 1` and
975/// the subscript reads `i`, which is one loop written two ways, and the two chrecs differ only in
976/// where they start.
977///
978/// Going up only. A counter going down wraps at the bottom rather than the top, so the sequence
979/// that is safe is the one that starts further along rather than the one that starts behind, and
980/// nothing measured so far walks an array downwards. Doing it would be turning the comparison
981/// round, and it should come with the program that wants it.
982fn trails(chrec: Chrec, held: Chrec) -> bool {
983 if chrec.ty != held.ty || chrec.step != held.step {
984 return false;
985 }
986 if chrec.base == held.base {
987 return true;
988 }
989 let (Some(step), Some(mine), Some(theirs)) =
990 (chrec.step.as_number(), chrec.base.as_number(), held.base.as_number())
991 else {
992 return false;
993 };
994 // Read as unsigned, which is the reading the test took, so a base that came in negative is a
995 // large number rather than a small one and starting behind is not what it is doing.
996 step > 0 && mine >= 0 && theirs >= 0 && mine <= theirs
997}
998
999/// The same expression, read the way a test without a sign reads it.
1000///
1001/// Constants arrive here as the number their bits are when the sign bit is taken seriously,
1002/// because that is the only reading available before anybody knows what will be done with them.
1003/// An unsigned test disagrees about half of them. `for (unsigned char i = 0; i < 200; i++)` holds
1004/// its limit as minus fifty six, and a distance worked out from that is negative, which reads as
1005/// a loop that runs no times rather than one that runs two hundred.
1006///
1007/// The step is not put through this, because a step is a difference rather than a value and its
1008/// signed reading is the one that says which way the counter goes.
1009fn as_unsigned(inv: Invariant, ty: Type) -> Option<Invariant> {
1010 match inv.as_number() {
1011 Some(number) if number >= 0 => Some(inv),
1012 Some(number) => {
1013 // Only an integer constant was read as signed in the first place. A pointer never
1014 // was, so a negative number sitting in one is an expression this cannot reinterpret.
1015 let bits = ty.is_int().then(|| ty.bits()).filter(|&bits| bits < 127)?;
1016 Some(Invariant::number(number & ((1i128 << bits) - 1)))
1017 }
1018 // A symbolic operand is whatever it is at run time, and the subtraction below cancels it
1019 // rather than reading it, so long as nothing signed has been folded in beside it.
1020 None => (inv.scale == 1 && inv.offset == 0).then_some(inv),
1021 }
1022}
1023
1024/// The count for an exit tested with an ordering, where overshooting the limit still ends it.
1025fn ordered(
1026 distance: Invariant,
1027 step: u128,
1028 inclusive: bool,
1029 mut assumptions: Vec<Assumption>,
1030) -> Option<(Count, Vec<Assumption>)> {
1031 match distance.as_number() {
1032 Some(exact) => {
1033 if exact < 0 {
1034 // The counter starts past the limit, so the test fails the first time it runs.
1035 // That is a count of zero and it rests on nothing at all, not even on the counter
1036 // behaving, because the counter never moves.
1037 return Some((Count::Exact(0), Vec::new()));
1038 }
1039 // Rounding up, because a step that overshoots still took the iteration that overshot.
1040 let count = (exact.unsigned_abs() + u128::from(inclusive)).div_ceil(step);
1041 Some((Count::Exact(count), assumptions))
1042 }
1043 // Symbolic, and only for a step of one, because dividing an expression by anything else
1044 // needs a representation for a division and there is not one here.
1045 None if step == 1 => {
1046 assumptions.push(Assumption::Approaching);
1047 let count = distance.plus(Invariant::number(i128::from(inclusive)))?;
1048 Some((Count::Symbolic(count), assumptions))
1049 }
1050 None => None,
1051 }
1052}
1053
1054/// The count for an exit tested with `!=`, where the counter has to land on the limit exactly.
1055///
1056/// This is a different problem from the one above and not a special case of it. An ordering test
1057/// ends the loop the moment the counter is past the limit, so a step that overshoots still stops.
1058/// `!=` only ends the loop on the one iteration where the counter is the limit, so a counter that
1059/// steps over the limit, or that starts on the far side of it, keeps going until it wraps. Both
1060/// of those are endless loops rather than short ones, and answering zero for either was the bug
1061/// this function exists to not have.
1062fn landing(
1063 distance: Invariant,
1064 step: u128,
1065 mut assumptions: Vec<Assumption>,
1066) -> Option<(Count, Vec<Assumption>)> {
1067 match distance.as_number() {
1068 Some(exact) => {
1069 let travel = u128::try_from(exact).ok()?;
1070 // Checked outright rather than assumed, which is why nothing here needs an assumption
1071 // about the step dividing anything.
1072 (travel % step == 0).then(|| (Count::Exact(travel / step), assumptions))
1073 }
1074 // A step of one lands on everything ahead of it, so the only thing left to establish is
1075 // that the limit is ahead. `while (p != end)` is this case, and a step of anything else
1076 // would need the division a symbolic distance has no room for.
1077 None if step == 1 => {
1078 assumptions.push(Assumption::Approaching);
1079 Some((Count::Symbolic(distance), assumptions))
1080 }
1081 None => None,
1082 }
1083}
1084
1085/// The predicate that is true exactly when this one is not.
1086fn invert(pred: IntPred) -> IntPred {
1087 match pred {
1088 IntPred::Eq => IntPred::Ne,
1089 IntPred::Ne => IntPred::Eq,
1090 IntPred::Slt => IntPred::Sge,
1091 IntPred::Sle => IntPred::Sgt,
1092 IntPred::Sgt => IntPred::Sle,
1093 IntPred::Sge => IntPred::Slt,
1094 IntPred::Ult => IntPred::Uge,
1095 IntPred::Ule => IntPred::Ugt,
1096 IntPred::Ugt => IntPred::Ule,
1097 IntPred::Uge => IntPred::Ult,
1098 }
1099}
1100
1101/// The predicate that says the same thing with the operands the other way round.
1102fn swap(pred: IntPred) -> IntPred {
1103 match pred {
1104 IntPred::Eq => IntPred::Eq,
1105 IntPred::Ne => IntPred::Ne,
1106 IntPred::Slt => IntPred::Sgt,
1107 IntPred::Sle => IntPred::Sge,
1108 IntPred::Sgt => IntPred::Slt,
1109 IntPred::Sge => IntPred::Sle,
1110 IntPred::Ult => IntPred::Ugt,
1111 IntPred::Ule => IntPred::Uge,
1112 IntPred::Ugt => IntPred::Ult,
1113 IntPred::Uge => IntPred::Ule,
1114 }
1115}
1116
1117/// The constant a value is, if it is one.
1118fn constant(func: &Func, value: Value) -> Option<(Imm, Type)> {
1119 let Def::Result { inst, .. } = func[value].def else { return None };
1120 if func[inst].opcode != Opcode::IConst {
1121 return None;
1122 }
1123 let Extra::Imm(at) = func[inst].extra else { return None };
1124 let ty = func[value].ty;
1125 ty.is_int().then(|| (func[at], ty))
1126}
1127
1128/// What this predecessor passes to the block's parameter at this position.
1129///
1130/// `None` when the predecessor branches to the block more than once with different arguments,
1131/// which a `br_if` with both arms on the same block can do and which means the parameter takes a
1132/// value that depends on the test rather than on the edge.
1133fn argument(func: &Func, pred: Block, block: Block, index: usize) -> Option<Value> {
1134 let term = func.terminator(pred)?;
1135 let mut found = None;
1136 for call in func.successors(term) {
1137 if call.block != block {
1138 continue;
1139 }
1140 let arg = *func[call.args].get(index)?;
1141 if found.replace(arg).is_some_and(|old| old != arg) {
1142 return None;
1143 }
1144 }
1145 found
1146}
1147
1148#[cfg(test)]
1149mod tests {
1150 use rucc_base::Interner;
1151 use rucc_ir::{Builder, Flags, Func, IntPred, Opcode, Signature, Type, Value};
1152
1153 use crate::cfg::Cfg;
1154 use crate::dom::Dominators;
1155 use crate::loops::{LoopId, Loops};
1156 use crate::scev::{Assumption, Bound, Count, Evolution, Invariant, Reading, Scev};
1157
1158 /// A loop counting in `ty` from `from` by `step` while the counter is below `to`.
1159 ///
1160 /// ```text
1161 /// entry: jump header(from)
1162 /// header(i): test = icmp pred i, to ; br_if test, body, exit
1163 /// body: next = add i, step ; jump header(next)
1164 /// exit: ret
1165 /// ```
1166 ///
1167 /// The counter is the header's only parameter, which is what the tests ask about.
1168 struct Counted {
1169 func: Func,
1170 counter: Value,
1171 next: Value,
1172 }
1173
1174 fn counted(ty: Type, from: i128, to: i128, step: i128, pred: IntPred, flags: Flags) -> Counted {
1175 let (it, ()) = counted_with(ty, from, to, step, pred, flags, |_, _| ());
1176 it
1177 }
1178
1179 /// The same loop, with `extra` run in the body on the counter before the counter steps.
1180 ///
1181 /// The builder appends, and the body's `jump` back to the header has to stay the last
1182 /// instruction in it or the block has no terminator and the loop stops being one. So anything
1183 /// a test wants derived from the counter goes in here rather than being tacked on afterwards.
1184 fn counted_with<T>(
1185 ty: Type,
1186 from: i128,
1187 to: i128,
1188 step: i128,
1189 pred: IntPred,
1190 flags: Flags,
1191 extra: impl FnOnce(&mut Builder<'_>, Value) -> T,
1192 ) -> (Counted, T) {
1193 let mut names = Interner::new();
1194 let mut func = Func::new(names.intern("f"), Signature::new());
1195 let entry = func.create_block();
1196 let header = func.create_block();
1197 let body = func.create_block();
1198 let exit = func.create_block();
1199 let counter = func.append_param(header, ty);
1200
1201 let mut build = Builder::new(&mut func, entry);
1202 let start = build.iconst(ty, from);
1203 build.jump(header, &[start]);
1204
1205 let mut build = Builder::new(&mut func, header);
1206 let limit = build.iconst(ty, to);
1207 let test = build.icmp(pred, counter, limit);
1208 build.br_if(test, body, &[], exit, &[]);
1209
1210 let mut build = Builder::new(&mut func, body);
1211 let derived = extra(&mut build, counter);
1212 let by = build.iconst(ty, step);
1213 let next = build.binary(Opcode::Add, counter, by, flags);
1214 build.jump(header, &[next]);
1215
1216 let mut build = Builder::new(&mut func, exit);
1217 build.ret(&[]);
1218
1219 (Counted { func, counter, next }, derived)
1220 }
1221
1222 /// The analysis over a function, along with the one loop it has.
1223 fn analyse(func: &Func) -> (Cfg, Loops) {
1224 let cfg = Cfg::new(func);
1225 let doms = Dominators::new(&cfg);
1226 let loops = Loops::new(&cfg, &doms);
1227 (cfg, loops)
1228 }
1229
1230 /// The chrec of a value in the one loop of a function.
1231 fn evolution(func: &Func, value: Value) -> Evolution {
1232 let (cfg, loops) = analyse(func);
1233 let id = loops.roots()[0];
1234 Scev::new(func, &cfg, &loops).evolution(id, value)
1235 }
1236
1237 /// The trip count of the one loop of a function.
1238 fn bound(func: &Func) -> Option<Bound> {
1239 let (cfg, loops) = analyse(func);
1240 let id: LoopId = loops.roots()[0];
1241 Scev::new(func, &cfg, &loops).bound(id)
1242 }
1243
1244 #[test]
1245 fn a_counter_from_zero_by_one_is_the_chrec_everyone_expects() {
1246 let it = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
1247 let chrec = evolution(&it.func, it.counter).chrec().expect("the counter evolves");
1248 assert_eq!(chrec.base, Invariant::number(0));
1249 assert_eq!(chrec.step, Invariant::number(1));
1250 assert_eq!(chrec.ty, Type::int(32));
1251 assert!(chrec.does_not_wrap(true));
1252 }
1253
1254 #[test]
1255 fn the_value_fed_back_is_the_chrec_one_step_along() {
1256 let it = counted(Type::int(32), 5, 100, 3, IntPred::Slt, Flags::NSW);
1257 let chrec = evolution(&it.func, it.next).chrec().expect("the increment evolves");
1258 assert_eq!(chrec.base, Invariant::number(8));
1259 assert_eq!(chrec.step, Invariant::number(3));
1260 }
1261
1262 #[test]
1263 fn a_multiple_of_the_counter_plus_a_number_is_a_chrec_of_its_own() {
1264 // `j = 2 * i + 3` where `i = {0, +, 1}`, which is the shape section 7.4 says pattern
1265 // matching runs out of road on and chains of recurrences do not.
1266 let (it, shifted) =
1267 counted_with(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1268 let two = build.iconst(Type::int(32), 2);
1269 let three = build.iconst(Type::int(32), 3);
1270 let doubled = build.binary(Opcode::Mul, counter, two, Flags::NSW);
1271 build.binary(Opcode::Add, doubled, three, Flags::NSW)
1272 });
1273
1274 let chrec = evolution(&it.func, shifted).chrec().expect("it evolves");
1275 assert_eq!(chrec.base, Invariant::number(3));
1276 assert_eq!(chrec.step, Invariant::number(2));
1277 }
1278
1279 #[test]
1280 fn a_shift_by_a_constant_scales_the_chrec_and_a_shift_past_the_width_does_not() {
1281 let (it, (scaled, poison)) =
1282 counted_with(Type::int(32), 1, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1283 let three = build.iconst(Type::int(32), 3);
1284 let wide = build.iconst(Type::int(32), 32);
1285 (
1286 build.binary(Opcode::Shl, counter, three, Flags::NSW),
1287 build.binary(Opcode::Shl, counter, wide, Flags::NSW),
1288 )
1289 });
1290
1291 let chrec = evolution(&it.func, scaled).chrec().expect("it evolves");
1292 assert_eq!(chrec.base, Invariant::number(8));
1293 assert_eq!(chrec.step, Invariant::number(8));
1294 // A count at the width is poison rather than a shift to zero, so there is no sequence to
1295 // describe.
1296 assert_eq!(evolution(&it.func, poison), Evolution::Unknown);
1297 }
1298
1299 #[test]
1300 fn a_pointer_walked_by_the_element_size_is_a_chrec_in_bytes() {
1301 // What `for (p = a; p != end; p++)` lowers to on an array of four byte elements. Section
1302 // 7.4 calls this the one deliberate extension past affine and the difference between
1303 // analysing half of real C loops and nearly all of them.
1304 let mut names = Interner::new();
1305 let mut func = Func::new(names.intern("f"), Signature::new());
1306 let entry = func.create_block();
1307 let header = func.create_block();
1308 let body = func.create_block();
1309 let exit = func.create_block();
1310 let start = func.append_param(entry, Type::PTR);
1311 let cursor = func.append_param(header, Type::PTR);
1312
1313 let mut build = Builder::new(&mut func, entry);
1314 build.jump(header, &[start]);
1315 let mut build = Builder::new(&mut func, header);
1316 let done = build.icmp(IntPred::Eq, cursor, start);
1317 build.br_if(done, exit, &[], body, &[]);
1318 let mut build = Builder::new(&mut func, body);
1319 let four = build.iconst(Type::int(64), 4);
1320 let next = build.binary(Opcode::PtrAdd, cursor, four, Flags::NONE);
1321 build.jump(header, &[next]);
1322 let mut build = Builder::new(&mut func, exit);
1323 build.ret(&[]);
1324
1325 let chrec = evolution(&func, cursor).chrec().expect("the cursor evolves");
1326 assert_eq!(chrec.base, Invariant::of(start));
1327 assert_eq!(chrec.step, Invariant::number(4));
1328 assert_eq!(chrec.ty, Type::PTR);
1329 }
1330
1331 #[test]
1332 fn a_counter_in_unsigned_char_wraps_and_does_not_widen_without_a_promise() {
1333 // Section 7.7's second way of being wrong. `{0, +, 1}` in `unsigned char` is not
1334 // `0, 1, 2, ...`, it is that modulo two hundred and fifty six, and widening it is only
1335 // the same sequence if it does not get that far.
1336 //
1337 // An inclusive test, because a strict one is a proof of its own and the case below is
1338 // about what happens when there is no proof at all. This loop does not in fact wrap, and
1339 // the point is that nothing here can say so.
1340 let (it, wide) =
1341 counted_with(Type::int(8), 0, 100, 1, IntPred::Ule, Flags::NONE, |build, counter| {
1342 build.unary(Opcode::ZExt, counter, Type::int(32))
1343 });
1344 let chrec = evolution(&it.func, it.counter).chrec().expect("the counter evolves");
1345 assert_eq!(chrec.ty, Type::int(8));
1346 assert!(!chrec.does_not_wrap(false));
1347 assert_eq!(evolution(&it.func, wide), Evolution::Unknown);
1348 }
1349
1350 #[test]
1351 fn a_counter_its_own_test_holds_widens_without_a_promise() {
1352 // The same counter under the strict test, which is the shape `for (unsigned i = 0; i < n;
1353 // i++)` has. Nothing promised anything, and the test is the proof: the counter is at the
1354 // limit before it is anywhere past it, and the loop ends there.
1355 let (it, wide) =
1356 counted_with(Type::int(8), 0, 100, 1, IntPred::Ult, Flags::NONE, |build, counter| {
1357 build.unary(Opcode::ZExt, counter, Type::int(32))
1358 });
1359 let narrow = evolution(&it.func, it.counter).chrec().expect("the counter evolves");
1360 assert!(!narrow.does_not_wrap(false), "nothing was promised, so nothing carries a flag");
1361 let chrec = evolution(&it.func, wide).chrec().expect("its own test holds it");
1362 assert_eq!(chrec.ty, Type::int(32));
1363 assert_eq!(chrec.base, Invariant::number(0));
1364 assert_eq!(chrec.step, Invariant::number(1));
1365 }
1366
1367 #[test]
1368 fn a_sequence_that_starts_further_along_than_the_counter_does_not_widen() {
1369 // `trails` in the direction it refuses. The test holds `i`, which starts at zero, and this
1370 // asks about `i + 1`, which starts one further along. One further along is where the
1371 // counter would be if it had gone round once more, and going round once more is the step
1372 // nothing here rules out.
1373 let (it, wide) =
1374 counted_with(Type::int(8), 0, 100, 1, IntPred::Ult, Flags::NONE, |build, counter| {
1375 let one = build.iconst(Type::int(8), 1);
1376 let ahead = build.binary(Opcode::Add, counter, one, Flags::NONE);
1377 build.unary(Opcode::ZExt, ahead, Type::int(32))
1378 });
1379 assert_eq!(evolution(&it.func, wide), Evolution::Unknown);
1380 }
1381
1382 #[test]
1383 fn a_counter_in_short_widens_when_the_increment_promised_it_would_not_wrap() {
1384 let (it, (wide, zero_extended)) =
1385 counted_with(Type::int(16), 0, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1386 (
1387 build.unary(Opcode::SExt, counter, Type::int(32)),
1388 build.unary(Opcode::ZExt, counter, Type::int(32)),
1389 )
1390 });
1391
1392 let chrec = evolution(&it.func, wide).chrec().expect("it widens");
1393 assert_eq!(chrec.ty, Type::int(32));
1394 assert_eq!(chrec.base, Invariant::number(0));
1395 assert_eq!(chrec.step, Invariant::number(1));
1396 // `nsw` is a promise about the signed reading and says nothing about the unsigned one.
1397 assert_eq!(evolution(&it.func, zero_extended), Evolution::Unknown);
1398 }
1399
1400 #[test]
1401 fn a_step_of_zero_is_invariant_and_has_no_trip_count() {
1402 // Section 7.7's first way of being wrong. `i += k` with `k` of zero is a valid affine
1403 // chrec of a loop that never leaves through this exit, and code dividing the distance by
1404 // the step divides by zero.
1405 let it = counted(Type::int(32), 0, 100, 0, IntPred::Slt, Flags::NSW);
1406 assert!(matches!(evolution(&it.func, it.counter), Evolution::Invariant(_)));
1407 assert_eq!(bound(&it.func), None);
1408 }
1409
1410 #[test]
1411 fn a_counted_loop_has_the_count_anyone_would_work_out_by_hand() {
1412 let it = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
1413 let found = bound(&it.func).expect("it is counted");
1414 let (count, assumptions) = found.parts();
1415 assert_eq!(count, Count::Exact(100));
1416 // The distance is a number and it is not negative, so being entered is not in question.
1417 // Signed overflow being undefined still is, which is what `-fwrapv` would withdraw.
1418 assert_eq!(assumptions, [Assumption::StrictOverflow]);
1419 assert_eq!(found.proven(), None);
1420 }
1421
1422 #[test]
1423 fn a_step_that_overshoots_still_takes_the_iteration_that_overshot() {
1424 // Zero, three, six, nine, and the test fails at twelve, so four iterations rather than
1425 // three and a third. Rounding the other way is an off by one in every unroller.
1426 let it = counted(Type::int(32), 0, 10, 3, IntPred::Slt, Flags::NSW);
1427 let (count, _) = bound(&it.func).expect("it is counted").parts();
1428 assert_eq!(count, Count::Exact(4));
1429 }
1430
1431 #[test]
1432 fn an_inclusive_test_runs_one_more_time() {
1433 let it = counted(Type::int(32), 0, 10, 1, IntPred::Sle, Flags::NSW);
1434 let (count, _) = bound(&it.func).expect("it is counted").parts();
1435 assert_eq!(count, Count::Exact(11));
1436 }
1437
1438 #[test]
1439 fn a_loop_whose_test_fails_first_time_runs_no_times_and_rests_on_nothing() {
1440 let it = counted(Type::int(32), 10, 0, 1, IntPred::Slt, Flags::NSW);
1441 let found = bound(&it.func).expect("it is counted");
1442 assert_eq!(found.proven(), Some(Count::Exact(0)));
1443 assert!(found.assumptions().is_empty());
1444 }
1445
1446 #[test]
1447 fn counting_down_is_the_same_problem_with_the_ends_swapped() {
1448 let it = counted(Type::int(32), 10, 0, -1, IntPred::Sgt, Flags::NSW);
1449 let (count, _) = bound(&it.func).expect("it is counted").parts();
1450 assert_eq!(count, Count::Exact(10));
1451 }
1452
1453 #[test]
1454 fn an_unsigned_test_does_not_drag_in_the_signed_overflow_assumption() {
1455 let it = counted(Type::int(32), 0, 100, 1, IntPred::Ult, Flags::NUW);
1456 let found = bound(&it.func).expect("it is counted");
1457 assert_eq!(found.proven(), Some(Count::Exact(100)));
1458 }
1459
1460 #[test]
1461 fn a_test_against_something_the_loop_does_not_change_gives_a_symbolic_count() {
1462 // `for (i = 0; i < n; i++)`, where the answer is `n` and is only `n` if the loop is
1463 // entered, because `n` of minus one runs no times and the distance is minus one.
1464 let mut names = Interner::new();
1465 let mut func = Func::new(names.intern("f"), Signature::new());
1466 let entry = func.create_block();
1467 let header = func.create_block();
1468 let body = func.create_block();
1469 let exit = func.create_block();
1470 let limit = func.append_param(entry, Type::int(32));
1471 let counter = func.append_param(header, Type::int(32));
1472
1473 let mut build = Builder::new(&mut func, entry);
1474 let zero = build.iconst(Type::int(32), 0);
1475 build.jump(header, &[zero]);
1476 let mut build = Builder::new(&mut func, header);
1477 let test = build.icmp(IntPred::Slt, counter, limit);
1478 build.br_if(test, body, &[], exit, &[]);
1479 let mut build = Builder::new(&mut func, body);
1480 let one = build.iconst(Type::int(32), 1);
1481 let next = build.binary(Opcode::Add, counter, one, Flags::NSW);
1482 build.jump(header, &[next]);
1483 let mut build = Builder::new(&mut func, exit);
1484 build.ret(&[]);
1485
1486 let found = bound(&func).expect("it is counted");
1487 let (count, assumptions) = found.parts();
1488 assert_eq!(count, Count::Symbolic(Invariant::of(limit)));
1489 assert!(assumptions.contains(&Assumption::Approaching), "{assumptions:?}");
1490 assert!(assumptions.contains(&Assumption::StrictOverflow), "{assumptions:?}");
1491 assert_eq!(found.proven(), None);
1492 }
1493
1494 #[test]
1495 fn the_count_records_which_reading_its_test_took() {
1496 // What a consumer widening a symbolic count has to know. The limit is a value of the
1497 // counter's type and which number that value is depends on how its test read it.
1498 let signed = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
1499 assert_eq!(bound(&signed.func).expect("it is counted").reading(), Reading::Signed);
1500 let unsigned = counted(Type::int(32), 0, 100, 1, IntPred::Ult, Flags::NUW);
1501 assert_eq!(bound(&unsigned.func).expect("it is counted").reading(), Reading::Unsigned);
1502 }
1503
1504 #[test]
1505 fn a_counter_without_a_no_wrap_promise_carries_the_assumption_instead() {
1506 // An inclusive test, because the strict one is the case the test itself answers. Under
1507 // `<=` the counter reaches the limit and is stepped once more, so a limit at the top of
1508 // the type makes that last step the one that wraps and nothing here rules it out.
1509 let it = counted(Type::int(32), 0, 100, 1, IntPred::Ule, Flags::NONE);
1510 let found = bound(&it.func).expect("it is counted");
1511 let (_, assumptions) = found.parts();
1512 assert!(assumptions.iter().any(|a| matches!(a, Assumption::NoWrap(_))), "{assumptions:?}");
1513 }
1514
1515 #[test]
1516 fn an_unsigned_counter_stepping_by_one_is_held_by_its_own_test() {
1517 // `for (unsigned i = 0; i < n; i++)` written out. Unsigned arithmetic wraps in C so the
1518 // increment carries no `nuw`, and without reading the test this would rest on an
1519 // assumption nothing downstream can discharge.
1520 let it = counted(Type::int(32), 0, 100, 1, IntPred::Ult, Flags::NONE);
1521 let found = bound(&it.func).expect("it is counted");
1522 assert_eq!(found.assumptions(), &[]);
1523 assert_eq!(found.proven(), Some(Count::Exact(100)));
1524 }
1525
1526 #[test]
1527 fn counting_down_by_one_is_held_the_same_way() {
1528 let it = counted(Type::int(32), 100, 0, -1, IntPred::Ugt, Flags::NONE);
1529 let found = bound(&it.func).expect("it is counted");
1530 assert_eq!(found.assumptions(), &[]);
1531 assert_eq!(found.proven(), Some(Count::Exact(100)));
1532 }
1533
1534 #[test]
1535 fn a_step_of_two_can_jump_the_limit_so_the_test_holds_nothing() {
1536 // The counter is never at the limit, so the loop can be left by a step that goes from one
1537 // below the limit to one past the top of the type and comes back round at the bottom.
1538 let it = counted(Type::int(32), 0, 100, 2, IntPred::Ult, Flags::NONE);
1539 let found = bound(&it.func).expect("it is counted");
1540 let (_, assumptions) = found.parts();
1541 assert!(assumptions.iter().any(|a| matches!(a, Assumption::NoWrap(_))), "{assumptions:?}");
1542 }
1543
1544 #[test]
1545 fn a_test_the_counter_can_be_stepped_without_being_asked_holds_nothing_either() {
1546 // ```text
1547 // header(i): br_if flag, check, latch
1548 // check: br_if i <u 100, latch, exit
1549 // latch: jump header(i + 1)
1550 // ```
1551 // The counter goes round by a path that never reaches the test, so the test says nothing
1552 // about how far the counter got.
1553 let mut names = Interner::new();
1554 let mut func = Func::new(names.intern("f"), Signature::new());
1555 let entry = func.create_block();
1556 let header = func.create_block();
1557 let check = func.create_block();
1558 let latch = func.create_block();
1559 let exit = func.create_block();
1560 let flag = func.append_param(entry, Type::int(1));
1561 let counter = func.append_param(header, Type::int(32));
1562
1563 let mut build = Builder::new(&mut func, entry);
1564 let zero = build.iconst(Type::int(32), 0);
1565 build.jump(header, &[zero]);
1566 let mut build = Builder::new(&mut func, header);
1567 build.br_if(flag, check, &[], latch, &[]);
1568 let mut build = Builder::new(&mut func, check);
1569 let limit = build.iconst(Type::int(32), 100);
1570 let test = build.icmp(IntPred::Ult, counter, limit);
1571 build.br_if(test, latch, &[], exit, &[]);
1572 let mut build = Builder::new(&mut func, latch);
1573 let one = build.iconst(Type::int(32), 1);
1574 let next = build.binary(Opcode::Add, counter, one, Flags::NONE);
1575 build.jump(header, &[next]);
1576 let mut build = Builder::new(&mut func, exit);
1577 build.ret(&[]);
1578
1579 let found = bound(&func).expect("it is counted");
1580 let (_, assumptions) = found.parts();
1581 assert!(assumptions.iter().any(|a| matches!(a, Assumption::NoWrap(_))), "{assumptions:?}");
1582 }
1583
1584 #[test]
1585 fn a_test_that_ends_the_loop_when_it_succeeds_is_read_the_other_way_round() {
1586 // `for (i = 0; ; i++) if (i >= 100) break;`, which is the same loop with the arms of the
1587 // branch swapped. The test that keeps the loop going is the opposite of the one written.
1588 let mut names = Interner::new();
1589 let mut func = Func::new(names.intern("f"), Signature::new());
1590 let entry = func.create_block();
1591 let header = func.create_block();
1592 let body = func.create_block();
1593 let exit = func.create_block();
1594 let counter = func.append_param(header, Type::int(32));
1595
1596 let mut build = Builder::new(&mut func, entry);
1597 let zero = build.iconst(Type::int(32), 0);
1598 build.jump(header, &[zero]);
1599 let mut build = Builder::new(&mut func, header);
1600 let limit = build.iconst(Type::int(32), 100);
1601 let done = build.icmp(IntPred::Sge, counter, limit);
1602 build.br_if(done, exit, &[], body, &[]);
1603 let mut build = Builder::new(&mut func, body);
1604 let one = build.iconst(Type::int(32), 1);
1605 let next = build.binary(Opcode::Add, counter, one, Flags::NSW);
1606 build.jump(header, &[next]);
1607 let mut build = Builder::new(&mut func, exit);
1608 build.ret(&[]);
1609
1610 let (count, _) = bound(&func).expect("it is counted").parts();
1611 assert_eq!(count, Count::Exact(100));
1612 }
1613
1614 #[test]
1615 fn an_unsigned_limit_past_the_middle_of_its_type_is_not_a_negative_one() {
1616 // `for (unsigned char i = 0; i < 200; i++)`. Two hundred does not fit in a signed byte
1617 // and the constant is held as minus fifty six, so a distance taken at face value is
1618 // negative and reads as a loop that runs no times.
1619 let it = counted(Type::int(8), 0, 200, 1, IntPred::Ult, Flags::NUW);
1620 let found = bound(&it.func).expect("it is counted");
1621 assert_eq!(found.proven(), Some(Count::Exact(200)));
1622 }
1623
1624 #[test]
1625 fn a_walk_that_lands_on_a_not_equal_limit_exactly_is_counted() {
1626 // `while (i != 10)` counting by one, which is `while (p != end)` over an array once the
1627 // element size has been divided out. `!=` says nothing about how its operands are read,
1628 // so the promise it wants is the unsigned one and an `nsw` on its own is not enough.
1629 let it = counted(Type::int(32), 0, 10, 1, IntPred::Ne, Flags::NSW.union(Flags::NUW));
1630 let found = bound(&it.func).expect("it lands on its limit");
1631 // The step divides the distance and both are numbers, so it was checked rather than
1632 // assumed and there is nothing left over.
1633 assert_eq!(found.proven(), Some(Count::Exact(10)));
1634 }
1635
1636 #[test]
1637 fn a_counter_stepping_away_from_a_not_equal_limit_is_not_a_loop_that_runs_no_times() {
1638 // The distance is negative and an ordering test would read that as the loop never being
1639 // entered. `!=` reads it as the counter never arriving, which is an endless loop, and
1640 // answering zero for it was a real bug that the property test in `tests/scev.rs` found.
1641 let it = counted(Type::int(32), 48, 15, 1, IntPred::Ne, Flags::NSW);
1642 assert_eq!(bound(&it.func), None);
1643 }
1644
1645 #[test]
1646 fn a_counter_stepping_over_a_not_equal_limit_never_arrives_either() {
1647 // Zero, three, six, nine, twelve, and ten is never one of them. An ordering test would
1648 // have stopped at twelve.
1649 let it = counted(Type::int(32), 0, 10, 3, IntPred::Ne, Flags::NSW);
1650 assert_eq!(bound(&it.func), None);
1651 }
1652
1653 #[test]
1654 fn an_estimate_is_the_count_when_there_is_one_and_a_guess_when_there_is_not() {
1655 let counted_loop = counted(Type::int(32), 0, 7, 1, IntPred::Slt, Flags::NSW);
1656 let (cfg, loops) = analyse(&counted_loop.func);
1657 let id = loops.roots()[0];
1658 let estimate = Scev::new(&counted_loop.func, &cfg, &loops).estimate(id);
1659 assert_eq!(estimate.iterations(), 7);
1660 assert!(!estimate.is_guess());
1661
1662 // A loop this cannot count still has to answer, because the caller is deciding whether
1663 // something is worth doing rather than whether it is legal.
1664 let uncounted = counted(Type::int(32), 0, 100, 0, IntPred::Slt, Flags::NSW);
1665 let (cfg, loops) = analyse(&uncounted.func);
1666 let id = loops.roots()[0];
1667 let estimate = Scev::new(&uncounted.func, &cfg, &loops).estimate(id);
1668 assert!(estimate.is_guess());
1669 assert_eq!(estimate.iterations(), super::ASSUMED_ITERATIONS);
1670 }
1671
1672 #[test]
1673 fn a_value_the_loop_does_not_touch_is_invariant_rather_than_unknown() {
1674 let it = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
1675 let (cfg, loops) = analyse(&it.func);
1676 let id = loops.roots()[0];
1677 let mut scev = Scev::new(&it.func, &cfg, &loops);
1678 // The counter's start is an `iconst` in the entry block, which is both.
1679 assert_eq!(
1680 scev.evolution(id, it.counter).chrec().expect("it evolves").base,
1681 Invariant::number(0)
1682 );
1683 }
1684
1685 #[test]
1686 fn a_back_edge_of_its_own_does_not_hide_the_counter() {
1687 // What canonicalization leaves behind. The back edge goes through a block that does nothing
1688 // but pass the increment on, so the value arriving at the header is a parameter of that
1689 // block rather than the increment itself. Reading through it is undoing a rename and not an
1690 // analysis, and without it the trip count of every loop the pipeline produces is nothing.
1691 let mut names = Interner::new();
1692 let mut func = Func::new(names.intern("f"), Signature::new());
1693 let entry = func.create_block();
1694 let header = func.create_block();
1695 let body = func.create_block();
1696 let latch = func.create_block();
1697 let exit = func.create_block();
1698 let counter = func.append_param(header, Type::int(32));
1699 let carried = func.append_param(latch, Type::int(32));
1700
1701 let start = Builder::new(&mut func, entry).iconst(Type::int(32), 0);
1702 Builder::new(&mut func, entry).jump(header, &[start]);
1703
1704 let mut build = Builder::new(&mut func, header);
1705 let limit = build.iconst(Type::int(32), 100);
1706 let test = build.icmp(IntPred::Slt, counter, limit);
1707 build.br_if(test, body, &[], exit, &[]);
1708
1709 let mut build = Builder::new(&mut func, body);
1710 let by = build.iconst(Type::int(32), 1);
1711 let next = build.binary(Opcode::Add, counter, by, Flags::NSW);
1712 build.jump(latch, &[next]);
1713
1714 Builder::new(&mut func, latch).jump(header, &[carried]);
1715 Builder::new(&mut func, exit).ret(&[]);
1716
1717 let chrec = evolution(&func, counter).chrec().expect("the counter still evolves");
1718 assert_eq!(chrec.base, Invariant::number(0));
1719 assert_eq!(chrec.step, Invariant::number(1));
1720 let (count, _) = bound(&func).expect("it is still counted").parts();
1721 assert_eq!(count, Count::Exact(100));
1722 }
1723
1724 #[test]
1725 fn every_assumption_says_what_it_is_in_a_line() {
1726 let it = counted(Type::int(8), 0, 100, 1, IntPred::Ult, Flags::NONE);
1727 let found = bound(&it.func).expect("it is counted");
1728 for assumption in found.assumptions() {
1729 let line = assumption.describe();
1730 assert!(!line.is_empty());
1731 assert!(!line.contains('\n'), "an assumption is one line: {line}");
1732 }
1733 }
1734}