Skip to main content

rucc_opt/
scev.rs

1//! Scalar evolution: how a value changes across the iterations of a loop, and how many
2//! iterations there are.
3//!
4//! Design: `spec/optimizer/07-loops-and-scev.md` sections 7.4 through 7.7. This is the second
5//! half of document 07 and it answers the last two of the four questions section 7.6 says loop
6//! analysis exists for. The first two are in [`crate::loops`].
7//!
8//! # Chains of recurrences, and how much of one
9//!
10//! GCC writes how a value changes as a chain of recurrences, `{base, +, step}`, meaning a value
11//! that is `base` on the first iteration and `step` more on each one after. The representation is
12//! good because it is closed under the operations anyone wants: adding two chrecs of the same
13//! loop adds componentwise, multiplying by something invariant scales both parts, and evaluating
14//! one at a given iteration is arithmetic rather than a special case. That closure is why
15//! `j = 2 * i + 3` is as easy as `i = i + 1`, and pattern matching the second would run out of
16//! road on the first.
17//!
18//! Section 7.4 says what rucc builds and it is a subset: affine chrecs only. A value is
19//! invariant, or `{base, +, step}` with both parts invariant, or unknown. Addition, subtraction,
20//! multiplication by an invariant, shifting by a constant, and extension where the extension
21//! provably does not wrap. Nothing polynomial and nothing mutually recursive. That covers every
22//! induction variable a C programmer writes and every array subscript document 31 could use, and
23//! what it leaves out of GCC's four thousand lines is the part serving Fortran and the polyhedral
24//! framework.
25//!
26//! The one extension past affine is pointer chrecs, because C loops walk pointers and `p = p + 1`
27//! is `i = i + 1` with a scale. A `ptr_add` is addition with the byte offset as the step, which
28//! is the difference between analysing half of real C loops and analysing nearly all of them.
29//!
30//! # Trip counts, and the part that is uncomfortable
31//!
32//! Given an exit that compares an affine chrec against something invariant, solving for the
33//! iteration at which the comparison first fails is arithmetic. What makes it hard is that the
34//! answer is almost always conditional: on the loop being entered at all, and on the induction
35//! variable not wrapping before it gets there. Section 7.5 says a trip count returned without its
36//! assumptions is a miscompilation generator, and that the temptation to return one is strong
37//! because the assumptions are usually true.
38//!
39//! So [`Bound`] carries them and there is no way to read the count without seeing them.
40//! [`Bound::parts`] hands back both, and [`Bound::proven`] hands back the count only when there
41//! is nothing left to prove. A caller that means to emit a runtime check reads the assumptions
42//! and emits it, and a caller that forgets cannot get at the number.
43//!
44//! [`Bound`] and [`Estimate`] are different types on purpose. A bound is used for correctness, an
45//! estimate is used to decide whether a transformation is worth doing, and section 7.5 calls
46//! conflating them a category error that costs correctness. GCC keeps them apart as
47//! `max_loop_iterations` and `estimate_numbers_of_iterations` and the names do not stop anyone.
48//! Different structs do.
49
50use std::collections::HashMap;
51
52use rucc_base::Symbol;
53use rucc_ir::{Block, Def, Extra, Flags, Func, Imm, Inst, IntPred, Opcode, Type, Value};
54
55use crate::cfg::Cfg;
56use crate::loops::{LoopId, Loops};
57
58/// How deep the search for a step walks back through arithmetic.
59///
60/// The chain from a header parameter to the value fed back to it is two or three instructions in
61/// anything a person writes, and the walk terminates on its own because SSA has no cycles except
62/// through block parameters. The limit is here so a generated function with a thousand additions
63/// in the increment costs a bounded amount rather than a stack.
64const STEP_LIMIT: u32 = 16;
65
66/// How many blocks that do nothing but pass a value on the walk reads through.
67///
68/// One is what a canonicalized loop has. The limit is here for the same reason the one above is,
69/// which is that a generated function can have a chain of them and the cost of following it should
70/// not depend on how long somebody made it.
71const FORWARD_LIMIT: u32 = 8;
72
73/// How many times a loop is assumed to run when nothing better is known.
74///
75/// GCC's `--param avg-loop-niter`, whose default is the same number. It is a guess, so nothing may
76/// rest on it. It reaches [`Estimate`], which is only ever used to decide whether something is
77/// worth doing, and [`crate::split`], which spends it on how far to ask the runtime to look and is
78/// answered with a true count of bytes whatever it asked for.
79pub(crate) const ASSUMED_ITERATIONS: u64 = 10;
80
81/// A value that does not change inside the loop, read as `on + scale * value + offset`.
82///
83/// The `value` is a value defined outside the loop, or `None` when the linear part is a plain
84/// number. Keeping the shape rather than a bare [`Value`] is what lets `j = 2 * i + 3` come out
85/// as `{3, +, 2}` instead of unknown: the base and the step of that chrec are expressions nothing
86/// in the function computes, so a representation that could only name existing values would have
87/// to give up.
88///
89/// The `on` is a second symbol, and it is there for one shape: a pointer plus an index the loop
90/// did not start at zero. `a[i]` with `i` starting at a parameter has a first address of
91/// `a + start * 4`, which is two symbols, and a representation with room for one has to answer
92/// unknown to it. Nothing scales `on` and nothing negates it, because the thing it was added for
93/// is a pointer and a pointer is not something a loop multiplies. It is an [`Anchor`] rather than
94/// a value so that the address of a global can be one of them.
95///
96/// The `read` is the other half of the same shape, since in C that index is an `int` and what
97/// reaches the address is `sext(start)`. It is described rather than named, for the reason on
98/// [`Widening`]. The two together are what let `a[start + i]` be followed, and on the SQLite
99/// amalgamation they take 158 checks and 12 sites off the largest row of loop splitting's census.
100/// See tamnd/rucc#810.
101///
102/// Arithmetic on two of these is refused once the sum would need a third symbol, because
103/// `x + y + z` is not of this shape. That is the boundary of the subset and it is where the answer
104/// becomes unknown rather than wrong.
105///
106/// The fields are private on purpose. Every reader has to go through [`Invariant::plain`], which
107/// hands back the one symbol reading and refuses when there is a pointer in it, or through
108/// [`Invariant::on`], which hands back both halves. A reader that helped itself to `value` and
109/// `scale` would quietly drop the `on` and build an address off the wrong object.
110#[derive(Clone, Copy, Debug, PartialEq, Eq)]
111pub struct Invariant {
112    /// A second symbol the whole expression is measured from, or `None`. Always one of it.
113    on: Option<Anchor>,
114    /// What the linear part is built on, or `None` for a plain number.
115    value: Option<Value>,
116    /// How that value is read, when it is read at a width that is not its own.
117    read: Option<Widening>,
118    /// How many of it.
119    scale: i128,
120    /// What is added.
121    offset: i128,
122}
123
124/// What an expression is measured from.
125///
126/// Usually a value the function computed somewhere outside the loop, which whoever reads the
127/// invariant can name. Sometimes the address of a global, which nothing has to compute because it
128/// is settled at link time and is the same number everywhere in the program.
129///
130/// The second one is here because of where a `global_addr` sits. [`crate::licm`] gives it a cost of
131/// zero and so never moves it out of a loop, which is the right call: working the address out again
132/// is one instruction and holding it in a register across a loop is a register. But that leaves the
133/// instruction inside the loop, and [`Loops::is_invariant`] answers by where a value is defined, so
134/// `a[i]` on a file scope `a` came out unknown. Describing the address rather than naming a value
135/// is the same move [`Widening`] makes, and it means a reader that wants the address in front of
136/// the loop writes another `global_addr` there for the one instruction it costs. On the SQLite
137/// amalgamation that is 178 checks at 47 sites of loop splitting's largest census row.
138/// See tamnd/rucc#810.
139#[derive(Clone, Copy, Debug, PartialEq, Eq)]
140pub enum Anchor {
141    /// A value, which is defined outside the loop and so can be named where it is wanted.
142    Value(Value),
143    /// The address of a global, which is written again wherever it is wanted.
144    Address(Symbol),
145}
146
147impl Anchor {
148    /// The value, when it is one. `None` for an address, which no value names.
149    #[must_use]
150    pub fn value(self) -> Option<Value> {
151        match self {
152            Self::Value(value) => Some(value),
153            Self::Address(_) => None,
154        }
155    }
156}
157
158/// A value read at a type wider than its own.
159///
160/// Widening `{start, +, 1}` in `int` gives `{sext(start), +, 1}` in `long`, and `sext(start)` is an
161/// expression nothing in the function computes. A representation that could only name values had
162/// to refuse the whole widening on that account, which is what shut the door on a walk from an
163/// index the caller handed in, because in C that index is an `int`. So the extension is described
164/// rather than named and whoever builds code from the invariant emits it.
165///
166/// The value stays the narrow one. Reading it at a third width later is an extension of an
167/// extension, and the two collapse into one wherever they mean the same thing, which is everywhere
168/// except a zero extension read as signed afterwards.
169#[derive(Clone, Copy, Debug, PartialEq, Eq)]
170pub struct Widening {
171    /// Sign extended or zero extended.
172    pub reading: Reading,
173    /// The type it is read at, which is wider than the value's own.
174    pub to: Type,
175}
176
177/// An invariant with no second symbol in it, read as `scale * value + offset`.
178///
179/// What every reader but [`crate::split`] wants, and what every reader wanted before there was an
180/// `on` at all. [`Invariant::plain`] is the only way to one, so a reader that does not know about
181/// the second symbol cannot get an expression that has one.
182#[derive(Clone, Copy, Debug, PartialEq, Eq)]
183pub struct Plain {
184    /// What it is built on, or `None` for a plain number.
185    pub value: Option<Value>,
186    /// How that value is read, when it is read at a width that is not its own.
187    pub read: Option<Widening>,
188    /// How many of it.
189    pub scale: i128,
190    /// What is added to it.
191    pub offset: i128,
192}
193
194impl Invariant {
195    /// A plain number.
196    #[must_use]
197    pub fn number(offset: i128) -> Self {
198        Self { on: None, value: None, read: None, scale: 0, offset }
199    }
200
201    /// One of a value.
202    #[must_use]
203    pub fn of(value: Value) -> Self {
204        Self { on: None, value: Some(value), read: None, scale: 1, offset: 0 }
205    }
206
207    /// So many of a value, plus a number.
208    #[must_use]
209    pub fn scaled(value: Value, scale: i128, offset: i128) -> Self {
210        Self { on: None, value: Some(value), read: None, scale, offset }
211    }
212
213    /// The address of a global.
214    ///
215    /// It goes straight into the `on` slot rather than into `value`, because that slot is the one
216    /// for the thing an address is measured from and an address is the only thing this ever is.
217    /// Nothing scales it and nothing negates it, which the rest of the arithmetic here already
218    /// refuses for whatever is in that slot.
219    #[must_use]
220    pub fn address(symbol: Symbol) -> Self {
221        Self { on: Some(Anchor::Address(symbol)), value: None, read: None, scale: 0, offset: 0 }
222    }
223
224    /// The one symbol reading, and `None` when there is a second symbol in it.
225    #[must_use]
226    pub fn plain(self) -> Option<Plain> {
227        self.on.is_none().then_some(Plain {
228            value: self.value,
229            read: self.read,
230            scale: self.scale,
231            offset: self.offset,
232        })
233    }
234
235    /// What it is measured from and how far past that, when there is a second symbol in it.
236    ///
237    /// Exactly one of this and [`Invariant::plain`] answers, so a reader that handles both has
238    /// handled every invariant there is.
239    #[must_use]
240    pub fn on(self) -> Option<(Anchor, Plain)> {
241        let on = self.on?;
242        Some((
243            on,
244            Plain { value: self.value, read: self.read, scale: self.scale, offset: self.offset },
245        ))
246    }
247
248    /// What it is measured from, when anything, and the rest of it with that taken off.
249    ///
250    /// For a reader that has to cancel the thing two expressions are measured from before it can
251    /// do arithmetic on what is left, which [`Invariant::minus`] will not do on its own because
252    /// it has no way to know the two are the same object.
253    #[must_use]
254    pub fn loose(self) -> (Option<Anchor>, Self) {
255        (self.on, Self { on: None, ..self })
256    }
257
258    /// Whether the two are the same expression apart from the number added to them.
259    #[must_use]
260    pub fn alike(self, other: Self) -> bool {
261        self.on == other.on
262            && self.value == other.value
263            && self.read == other.read
264            && self.scale == other.scale
265    }
266
267    /// The number added to it, whatever else it has in it.
268    #[must_use]
269    pub fn offset(self) -> i128 {
270        self.offset
271    }
272
273    /// The number this is, when it is one.
274    #[must_use]
275    pub fn as_number(self) -> Option<i128> {
276        (self.on.is_none() && self.symbol().is_none()).then_some(self.offset)
277    }
278
279    /// Whether this is the number zero.
280    #[must_use]
281    pub fn is_zero(self) -> bool {
282        self.as_number() == Some(0)
283    }
284
285    /// The value the linear part is built on, when the linear part has one.
286    fn symbol(self) -> Option<Value> {
287        if self.scale == 0 { None } else { self.value }
288    }
289
290    /// This as something to measure from, when it is one of a value and a number.
291    ///
292    /// Never a widened one. What an expression is measured from is a pointer, and a pointer is not
293    /// something anything here extends.
294    fn measure(self) -> Option<Anchor> {
295        (self.on.is_none() && self.read.is_none() && self.scale == 1)
296            .then_some(self.value)
297            .flatten()
298            .map(Anchor::Value)
299    }
300
301    /// The symbol both linear parts are built on and how it is read, when they agree on one or one
302    /// of them has none.
303    ///
304    /// The same value read two ways is two different numbers, so agreeing on the value is not
305    /// enough. `sext(x)` and `zext(x)` are the same bits and not the same quantity.
306    fn shared(self, other: Self) -> Option<(Option<Value>, Option<Widening>)> {
307        match (self.symbol(), other.symbol()) {
308            (None, _) => Some((other.value, other.read)),
309            (_, None) => Some((self.value, self.read)),
310            (left, right) => {
311                (left == right && self.read == other.read).then_some((left, self.read))
312            }
313        }
314    }
315
316    /// This same value read at a wider type, when the widening has a form here.
317    ///
318    /// A number means the same thing at both widths under a sign extension, and under a zero
319    /// extension once it is not negative. One of a value becomes that value read through the
320    /// extension. Anything else is refused, because the narrow arithmetic may already have wrapped
321    /// and `sext(2 * x + 3)` is not `2 * sext(x) + 3`.
322    fn widened(self, reading: Reading, to: Type) -> Option<Self> {
323        if let Some(number) = self.as_number() {
324            return (reading == Reading::Signed || number >= 0).then_some(Self::number(number));
325        }
326        if self.on.is_some() || self.scale != 1 || self.offset != 0 {
327            return None;
328        }
329        let value = self.value?;
330        // An extension of an extension. A zero extension is never negative, so reading its result
331        // as signed afterwards is the same numbers and the pair collapses into the zero extension
332        // at the outer width. The other way round it does not: a sign extension of a negative
333        // number read as unsigned afterwards is a different number entirely.
334        let reading = match (self.read.map(|read| read.reading), reading) {
335            (None, outer) => outer,
336            (Some(Reading::Unsigned), _) => Reading::Unsigned,
337            (Some(Reading::Signed), Reading::Signed) => Reading::Signed,
338            (Some(Reading::Signed), Reading::Unsigned) => return None,
339        };
340        Some(Self {
341            on: None,
342            value: Some(value),
343            read: Some(Widening { reading, to }),
344            scale: 1,
345            offset: 0,
346        })
347    }
348
349    /// The two added, when the sum is of this shape.
350    #[must_use]
351    pub fn plus(self, other: Self) -> Option<Self> {
352        let offset = self.offset.checked_add(other.offset)?;
353        // At most one of the two brought something to measure from, since a sum measured from two
354        // pointers is not an address.
355        let on = match (self.on, other.on) {
356            (None, on) | (on, None) => on,
357            (Some(_), Some(_)) => return None,
358        };
359        // The linear parts are about the same symbol, or one of them is a number, so they add.
360        if let Some((value, read)) = self.shared(other) {
361            let scale = self.scale.checked_add(other.scale)?;
362            return Some(Self { on, value, read, scale, offset });
363        }
364        // Two different symbols, which is what a pointer plus an index the loop did not start at
365        // zero is. Nothing may already be measured from anything, and one of the two has to be one
366        // of a value and a number, and that one becomes what the sum is measured from.
367        if on.is_some() {
368            return None;
369        }
370        let (on, rest) = match (self.measure(), other.measure()) {
371            (Some(on), _) => (on, other),
372            (_, Some(on)) => (on, self),
373            _ => return None,
374        };
375        Some(Self { on: Some(on), value: rest.value, read: rest.read, scale: rest.scale, offset })
376    }
377
378    /// The second subtracted from the first, when the difference is of this shape.
379    #[must_use]
380    pub fn minus(self, other: Self) -> Option<Self> {
381        self.plus(other.negated()?)
382    }
383
384    /// This with its sign flipped, which needs nothing to measure from.
385    ///
386    /// A pointer is not a thing to negate, and the second symbol is only ever there because a
387    /// pointer put it there.
388    #[must_use]
389    pub fn negated(self) -> Option<Self> {
390        if self.on.is_some() {
391            return None;
392        }
393        Some(Self {
394            on: None,
395            value: self.value,
396            read: self.read,
397            scale: self.scale.checked_neg()?,
398            offset: self.offset.checked_neg()?,
399        })
400    }
401
402    /// The two multiplied, which needs one of them to be a plain number and neither to be measured
403    /// from anything.
404    #[must_use]
405    pub fn times(self, other: Self) -> Option<Self> {
406        if self.on.is_some() || other.on.is_some() {
407            return None;
408        }
409        let (symbol, by) = match (self.as_number(), other.as_number()) {
410            (Some(by), _) => (other, by),
411            (_, Some(by)) => (self, by),
412            _ => return None,
413        };
414        Some(Self {
415            on: None,
416            value: symbol.value,
417            read: symbol.read,
418            scale: symbol.scale.checked_mul(by)?,
419            offset: symbol.offset.checked_mul(by)?,
420        })
421    }
422}
423
424/// How a value changes from one iteration of a loop to the next.
425#[derive(Clone, Copy, Debug, PartialEq, Eq)]
426pub enum Evolution {
427    /// The same on every iteration.
428    Invariant(Invariant),
429    /// `{base, +, step}`: `base` the first time round and `step` more each time after.
430    Affine(Chrec),
431    /// Not something this analysis describes. Never a claim that the value does not evolve.
432    Unknown,
433}
434
435impl Evolution {
436    /// The chrec, when this is one.
437    #[must_use]
438    pub fn chrec(self) -> Option<Chrec> {
439        match self {
440            Self::Affine(chrec) => Some(chrec),
441            _ => None,
442        }
443    }
444
445    /// The invariant expression, when this is one.
446    #[must_use]
447    pub fn invariant(self) -> Option<Invariant> {
448        match self {
449            Self::Invariant(inv) => Some(inv),
450            _ => None,
451        }
452    }
453}
454
455/// An affine chain of recurrences, `{base, +, step}`, evolving in a named type.
456///
457/// The type is not decoration. `{0, +, 1}` in `unsigned char` is not the sequence `0, 1, 2, ...`,
458/// it is that sequence modulo two hundred and fifty six, and section 7.7 says this is where a
459/// naive implementation is wrong constantly and in ways that pass every test written by someone
460/// thinking in `int`. Every operation here checks the type and every one that cannot stay right
461/// in it answers unknown.
462#[derive(Clone, Copy, Debug, PartialEq, Eq)]
463pub struct Chrec {
464    /// What the value is on the first iteration.
465    pub base: Invariant,
466    /// What is added each time round.
467    pub step: Invariant,
468    /// The type it evolves in, which is what says when it wraps.
469    pub ty: Type,
470    /// What the instruction that increments it promised. `nsw` means the sequence does not wrap
471    /// when read as signed and `nuw` means it does not when read as unsigned, and both come from
472    /// the increment rather than from anything this analysis proved.
473    pub flags: Flags,
474}
475
476impl Chrec {
477    /// Whether the sequence is known not to wrap under the reading this predicate takes.
478    #[must_use]
479    pub fn does_not_wrap(self, signed: bool) -> bool {
480        self.flags.contains(if signed { Flags::NSW } else { Flags::NUW })
481    }
482}
483
484/// Something that has to be true for a trip count to be the right answer.
485///
486/// Section 7.5 asks for exactly this: not a trip count but a trip count plus a predicate under
487/// which it holds, so the consumer either proves the predicate, emits a runtime check for it, or
488/// gives up. These are the predicates.
489#[derive(Clone, Copy, Debug, PartialEq, Eq)]
490pub enum Assumption {
491    /// The loop is entered at all, so the distance from the counter to its limit is a number that
492    /// is not negative.
493    ///
494    /// `for (i = 0; i < n; i++)` with `n` of zero runs no times and the distance is zero, but `n`
495    /// of minus one also runs no times and the distance is minus one, so a count taken from the
496    /// distance has to be told which case it is in.
497    ///
498    /// What it does not say anything about is whether the loop comes back. A counter stepping
499    /// toward a limit under an ordering test either reaches it or is already past it, and either
500    /// way that is a finite number of steps, so a caller whose question is whether the loop ends
501    /// may have this one for nothing. [`crate::hoist`] discharges it by clamping the count at zero
502    /// and [`crate::loop_delete`] by never reading the count. That is the whole of why this is a
503    /// separate assumption from [`Assumption::Approaching`] rather than the same one worded to
504    /// cover both.
505    ///
506    /// Only ever present on a symbolic count. When the distance is a number the sign of it is
507    /// there to be read, so this is settled rather than assumed.
508    Entered,
509    /// The limit is somewhere the counter is heading, which for a loop ending on `!=` is what
510    /// makes it end at all.
511    ///
512    /// A counter stepping away from its limit never arrives, and one stepping past it keeps going
513    /// until it wraps, so what is unproven here is termination rather than which number the count
514    /// is. Nothing discharges it by clamping, because there is no number to clamp when the loop
515    /// does not come back. Document 17.2 says rucc does not take out a loop that might not end,
516    /// so a pass that deletes loops refuses this one outright.
517    ///
518    /// Only ever present on a symbolic count, for the same reason [`Assumption::Entered`] is.
519    Approaching,
520    /// The induction variable does not wrap in its own type before the exit is taken.
521    ///
522    /// Present whenever the increment did not carry the matching `nsw` or `nuw` flag. With the
523    /// flag there is nothing to assume, because the flag is the promise.
524    NoWrap(Chrec),
525    /// Signed overflow is undefined here, which is what makes `for (int i = 0; i <= n; i++)`
526    /// finite.
527    ///
528    /// GCC infers loop bounds from this in `infer_loop_bounds_from_signedness`, and it is the
529    /// single most common source of a report that the compiler broke a working program. It is
530    /// recorded rather than assumed silently so that `-fwrapv` can withdraw the count and so that
531    /// a dump can name it.
532    StrictOverflow,
533}
534
535impl Assumption {
536    /// What it says, in a line, for a dump to print.
537    ///
538    /// Section 7.5 asks that every inference of this kind be dumpable and say what it rests on,
539    /// because a user who has been bitten by one deserves a command that tells them which line
540    /// the compiler used against them. This is the sentence that command prints.
541    #[must_use]
542    pub fn describe(&self) -> String {
543        match self {
544            Self::Entered => "the loop is entered at all".to_string(),
545            Self::Approaching => "the counter is heading towards its limit".to_string(),
546            Self::NoWrap(chrec) => {
547                format!("the induction variable does not wrap in i{}", chrec.ty.bits())
548            }
549            Self::StrictOverflow => {
550                "signed overflow is undefined, so -fwrapv withdraws this count".to_string()
551            }
552        }
553    }
554}
555
556/// How many iterations, as a number or as an expression.
557#[derive(Clone, Copy, Debug, PartialEq, Eq)]
558pub enum Count {
559    /// Exactly this many.
560    Exact(u128),
561    /// This many, worked out from something the loop does not change.
562    Symbolic(Invariant),
563}
564
565/// Which reading of its operands the test the count came from took.
566///
567/// It matters to anybody widening the value a symbolic count is built out of. The count is the
568/// distance to the limit of the exit test, the limit is a value of the counter's own type, and
569/// what that value means is the reading its test took. A limit past the middle of a thirty two bit
570/// type is a large number to an unsigned test and a negative one to a signed test, and a consumer
571/// that sign extends what an unsigned test compared has turned a loop over three billion elements
572/// into a loop that runs no times.
573#[derive(Clone, Copy, Debug, PartialEq, Eq)]
574pub enum Reading {
575    /// The test read its operands as signed, so widening the count means sign extending it.
576    Signed,
577    /// The test read them as unsigned, so widening the count means zero extending it.
578    Unsigned,
579}
580
581/// How many times a loop runs at most, and what that rests on.
582///
583/// For correctness. A pass that deletes an iteration, peels one off, or decides a memory access
584/// is in bounds needs one of these. The count cannot be read without the assumptions, which is
585/// section 7.7's defence against a caller proving two of three and forgetting the third.
586#[derive(Clone, Debug, PartialEq, Eq)]
587pub struct Bound {
588    count: Count,
589    assumptions: Vec<Assumption>,
590    reading: Reading,
591}
592
593impl Bound {
594    /// The count and everything it rests on, together, because they cannot be asked for apart.
595    #[must_use]
596    pub fn parts(&self) -> (Count, &[Assumption]) {
597        (self.count, &self.assumptions)
598    }
599
600    /// How the value a symbolic count is built out of has to be read.
601    ///
602    /// Meaningless on a count that is a number, since a number has already been read.
603    #[must_use]
604    pub fn reading(&self) -> Reading {
605        self.reading
606    }
607
608    /// What has to be proved before the count means anything.
609    #[must_use]
610    pub fn assumptions(&self) -> &[Assumption] {
611        &self.assumptions
612    }
613
614    /// The count, for a caller with nothing left to prove.
615    ///
616    /// `None` does not mean the count is unknown. It means there are assumptions and this is not
617    /// the accessor for reading a count that has them.
618    #[must_use]
619    pub fn proven(&self) -> Option<Count> {
620        self.assumptions.is_empty().then_some(self.count)
621    }
622
623    /// The count, for a caller compiling a language where signed overflow is undefined.
624    ///
625    /// [`Bound::proven`] answers nothing for any `for (int i = 0; i < n; i++)` in any C program,
626    /// because `solve` puts [`Assumption::StrictOverflow`] on every count taken from a signed
627    /// test, and a pass built on `proven` alone is a pass that never fires. What that assumption
628    /// says is that the count rests on signed overflow being undefined, and `-fwrapv` is
629    /// implemented in `rucc-lower` by not setting `nsw` rather than by a flag anything down here
630    /// reads. So an increment that still carries `nsw` under `-fwrapv` does not exist, and a bound
631    /// with `StrictOverflow` and nothing else on it is a bound whose counter the front end
632    /// promised does not wrap. That promise is exactly what the assumption wanted.
633    ///
634    /// [`Assumption::NoWrap`] is the case where there is no such promise, and it is refused here.
635    /// So are [`Assumption::Entered`] and [`Assumption::Approaching`], though only in passing,
636    /// because neither ever appears on a count that is a number.
637    #[must_use]
638    pub fn under_undefined_overflow(&self) -> Option<Count> {
639        self.assumptions
640            .iter()
641            .all(|rests_on| matches!(rests_on, Assumption::StrictOverflow))
642            .then_some(self.count)
643    }
644
645    /// The count, for a caller that needs the loop to come back and not how many times.
646    ///
647    /// One assumption wider than [`Bound::under_undefined_overflow`], and the one it adds is
648    /// [`Assumption::Entered`]. What that says is whether the count is the distance to the limit
649    /// or zero, and both of those are numbers of iterations the loop has, so a pass asking whether
650    /// the loop ends has already been answered whichever way it goes. A pass multiplying by the
651    /// count has not, which is why this is a second accessor and not a loosening of the first.
652    ///
653    /// [`Assumption::Approaching`] is refused, and telling those two apart is the reason they are
654    /// two assumptions. A loop ending on `!=` whose counter steps past its limit runs until it
655    /// wraps, document 17.2 is explicit that rucc does not take out a loop that might not end, and
656    /// a count that comes back from here is one no caller has to check that against.
657    #[must_use]
658    pub fn comes_back(&self) -> Option<Count> {
659        self.assumptions
660            .iter()
661            .all(|rests_on| matches!(rests_on, Assumption::StrictOverflow | Assumption::Entered))
662            .then_some(self.count)
663    }
664}
665
666/// How many times a loop probably runs.
667///
668/// For cost decisions and never for correctness. A pass asking whether unrolling pays for itself
669/// wants one of these, and it is fine for the answer to be a guess, because being wrong makes the
670/// code slower rather than wrong. Nothing here can be turned into a [`Bound`].
671#[derive(Clone, Copy, Debug, PartialEq, Eq)]
672pub struct Estimate {
673    iterations: u64,
674    guessed: bool,
675}
676
677impl Estimate {
678    /// The number to do arithmetic with.
679    #[must_use]
680    pub fn iterations(self) -> u64 {
681        self.iterations
682    }
683
684    /// Whether nothing was known and this is the default.
685    #[must_use]
686    pub fn is_guess(self) -> bool {
687        self.guessed
688    }
689}
690
691/// An exit test, read so that the loop keeps going while it holds.
692///
693/// Not public. It is the shape [`Scev::bound_at`] and [`Scev::holds`] both want out of the same
694/// branch, and what either of them says about it is what the outside sees.
695#[derive(Clone, Copy, Debug)]
696struct Test {
697    /// The side that moves, with the predicate already turned round to put it on the left.
698    chrec: Chrec,
699    /// The side that does not.
700    limit: Invariant,
701    /// The comparison that has to hold for the loop to go round again.
702    pred: IntPred,
703    /// Whether every iteration that goes round asks it.
704    each: bool,
705}
706
707/// The analysis, which works out an answer when asked and remembers it.
708///
709/// Demand driven and memoized, per section 7.8, because the cost of scalar evolution is a
710/// function of how many distinct values get asked about rather than of the size of the function.
711/// The cache holds one loop's worth of answers per loop and the whole thing is thrown away when
712/// anything about the loops changes, which per document 04.4 is any pass that touches one.
713#[derive(Debug)]
714pub struct Scev<'a> {
715    func: &'a Func,
716    cfg: &'a Cfg,
717    loops: &'a Loops,
718    known: HashMap<(LoopId, Value), Evolution>,
719    held: HashMap<LoopId, Option<Chrec>>,
720}
721
722impl<'a> Scev<'a> {
723    /// A fresh analysis over these loops, knowing nothing yet.
724    #[must_use]
725    pub fn new(func: &'a Func, cfg: &'a Cfg, loops: &'a Loops) -> Self {
726        Self { func, cfg, loops, known: HashMap::new(), held: HashMap::new() }
727    }
728
729    /// How this value changes across the iterations of this loop.
730    ///
731    /// The way in, and what it does before answering is settle `Scev::holds` for the loop. That has
732    /// to happen out here rather than at the point `Scev::extend` wants it, because settling it
733    /// means asking about other values and `Scev::at` parks a marker on the value it is working on.
734    /// Asked from in there, the answer would depend on what was already in flight.
735    pub fn evolution(&mut self, id: LoopId, value: Value) -> Evolution {
736        self.holds(id);
737        self.at(id, value)
738    }
739
740    /// How this value changes, with the loop's own facts already settled.
741    fn at(&mut self, id: LoopId, value: Value) -> Evolution {
742        if let Some(&known) = self.known.get(&(id, value)) {
743            return known;
744        }
745        // Unknown while the answer is being worked out, so the cycle from a header parameter back
746        // to itself terminates instead of asking the same question forever. Anything that reaches
747        // the parameter again gets unknown and the shape it was matching fails, which is the
748        // right answer for a value defined in terms of itself through arithmetic this does not
749        // describe.
750        self.known.insert((id, value), Evolution::Unknown);
751        let found = self.compute(id, value);
752        self.known.insert((id, value), found);
753        found
754    }
755
756    /// How many times this loop runs at most, and what that rests on.
757    ///
758    /// Any one exit gives a valid upper bound, because a loop cannot run more times than the
759    /// first exit that fires, so this takes the first exit it can solve rather than the smallest.
760    /// That is `max_loop_iterations` and not `estimate_numbers_of_iterations`, which is why the
761    /// answer is a [`Bound`].
762    pub fn bound(&mut self, id: LoopId) -> Option<Bound> {
763        self.holds(id);
764        let exits: Vec<Block> = self.loops.exits(id).iter().map(|exit| exit.from).collect();
765        exits.into_iter().find_map(|from| self.bound_at(id, from))
766    }
767
768    /// The counter an exit test of this loop keeps inside its own type, when there is one.
769    ///
770    /// [`bounded_by_its_test`] is the argument and this is where its answer is written down as a
771    /// fact about the loop rather than spent on one trip count. What it buys is [`Scev::extend`]:
772    /// an unsigned counter carries no `nuw`, so widening anything built out of one used to be
773    /// refused, and the test that holds the counter holds everything walking beside it.
774    ///
775    /// Settled once per loop and then read. It is settled from [`Scev::evolution`] and
776    /// [`Scev::bound`], which are the two ways in, so that it is worked out with nothing in flight.
777    /// The cache for the loop is emptied afterwards, because the answers already in it were worked
778    /// out while this was still unknown and a conservative answer that stayed would make what the
779    /// analysis says depend on which question was asked first.
780    fn holds(&mut self, id: LoopId) -> Option<Chrec> {
781        if let Some(&known) = self.held.get(&id) {
782            return known;
783        }
784        // Unknown while it is being worked out, which is what stops the recursion below from
785        // asking the same question forever, and which is why the cache is emptied after.
786        self.held.insert(id, None);
787        let exits: Vec<Block> = self.loops.exits(id).iter().map(|exit| exit.from).collect();
788        let found = exits.into_iter().find_map(|from| {
789            let test = self.test_at(id, from)?;
790            let step = test.chrec.step.as_number()?;
791            (test.each && bounded_by_its_test(test.pred, step)).then_some(test.chrec)
792        });
793        self.held.insert(id, found);
794        self.known.retain(|&(of, _), _| of != id);
795        found
796    }
797
798    /// How many times this loop probably runs.
799    pub fn estimate(&mut self, id: LoopId) -> Estimate {
800        match self.bound(id).map(|bound| bound.count) {
801            Some(Count::Exact(exact)) => {
802                Estimate { iterations: u64::try_from(exact).unwrap_or(u64::MAX), guessed: false }
803            }
804            _ => Estimate { iterations: ASSUMED_ITERATIONS, guessed: true },
805        }
806    }
807
808    /// The evolution of a value nothing is known about yet.
809    fn compute(&mut self, id: LoopId, value: Value) -> Evolution {
810        if let Some(invariant) = self.invariant(id, value) {
811            return Evolution::Invariant(invariant);
812        }
813        match self.func[value].def {
814            Def::Param { block, index } if block == self.loops.header(id) => {
815                self.at_header(id, value, index as usize)
816            }
817            // A parameter of a block inside the loop that is not the header takes a different
818            // value depending on which way control came, and describing that is a job for the
819            // value range work of document 10 rather than for a chrec. Unless there is only one
820            // way in, in which case it does not.
821            Def::Param { .. } => match self.forwarded(value) {
822                same if same == value => Evolution::Unknown,
823                through => self.at(id, through),
824            },
825            Def::Result { inst, .. } => self.at_inst(id, inst, value),
826        }
827    }
828
829    /// The value as an expression that does not change inside the loop, if it is one.
830    fn invariant(&self, id: LoopId, value: Value) -> Option<Invariant> {
831        if let Some((imm, ty)) = constant(self.func, value) {
832            return Some(Invariant::number(imm.signed(ty)));
833        }
834        // A constant is invariant wherever it sits, which is why it is asked about first. Anything
835        // else has to be defined outside the loop.
836        if self.loops.is_invariant(self.func, id, value) {
837            return Some(Invariant::of(value));
838        }
839        // Except the address of a global, which is a link time constant and so does not change
840        // inside a loop wherever it is written. Asked after the question above and not instead of
841        // it, so that a `global_addr` already sitting outside the loop stays a value every reader
842        // can name, and this arm is only the case that used to come out unknown. See [`Anchor`].
843        symbol(self.func, value).map(Invariant::address)
844    }
845
846    /// The evolution of a parameter of the loop header, which is where an induction variable is.
847    ///
848    /// The parameter takes one value on the way in and another on the way round, which is what
849    /// other IRs spell as a phi node. If the way round is the parameter plus something invariant,
850    /// the parameter is an affine chrec and that something is its step.
851    fn at_header(&mut self, id: LoopId, value: Value, index: usize) -> Evolution {
852        let (func, cfg, loops) = (self.func, self.cfg, self.loops);
853        let header = loops.header(id);
854        // Section 7.3 wants exactly one latch and the canonicalizer makes one. Two of them means
855        // two ways round with two different increments, and picking one would be a guess.
856        let [latch] = loops.latches(id) else { return Evolution::Unknown };
857        let mut entering = None;
858        let mut around = None;
859        for &pred in cfg.predecessors(header) {
860            let Some(arg) = argument(func, pred, header, index) else { return Evolution::Unknown };
861            let arg = self.forwarded(arg);
862            let slot = if pred == *latch { &mut around } else { &mut entering };
863            if slot.replace(arg).is_some_and(|old| old != arg) {
864                return Evolution::Unknown;
865            }
866        }
867        let (Some(entering), Some(around)) = (entering, around) else { return Evolution::Unknown };
868        let Some(base) = self.invariant(id, entering) else { return Evolution::Unknown };
869        let Some((step, flags)) = self.step(id, around, value, 0) else {
870            return Evolution::Unknown;
871        };
872        affine(base, step, func[value].ty, flags)
873    }
874
875    /// The value a block parameter stands for, when there is only one way into its block.
876    ///
877    /// This is not an analysis, it is undoing a rename. A block with one predecessor has one value
878    /// for each of its parameters and it is the argument that predecessor passes, so reading
879    /// through it loses nothing and assumes nothing.
880    ///
881    /// It is here because of what canonicalization does. `crate::canon` splits the back edge of a
882    /// loop to give it a latch of its own, and after that the value going round the loop is not the
883    /// increment the loop computed, it is a parameter of a block that does nothing but pass the
884    /// increment on. Without this, every counted loop the pipeline actually produces looks like a
885    /// loop whose counter comes from somewhere unknown, and the trip count of a `for` loop in a
886    /// real function comes back as nothing.
887    fn forwarded(&self, value: Value) -> Value {
888        let mut value = value;
889        for _ in 0..FORWARD_LIMIT {
890            let Def::Param { block, index } = self.func[value].def else { return value };
891            let [pred] = self.cfg.predecessors(block) else { return value };
892            let Some(arg) = argument(self.func, *pred, block, index as usize) else { return value };
893            if arg == value {
894                return value;
895            }
896            value = arg;
897        }
898        value
899    }
900
901    /// What is added to `of` to get `value`, and what the additions promised.
902    ///
903    /// Written as its own walk rather than as the general combination below, because at the point
904    /// this runs the parameter's own evolution is not known yet and the general walk would ask
905    /// for it and get unknown.
906    fn step(&self, id: LoopId, value: Value, of: Value, depth: u32) -> Option<(Invariant, Flags)> {
907        let value = self.forwarded(value);
908        if value == of {
909            // Nothing added yet, and nothing has had a chance to overflow either.
910            return Some((Invariant::number(0), Flags::NSW.union(Flags::NUW)));
911        }
912        if depth >= STEP_LIMIT {
913            return None;
914        }
915        let Def::Result { inst, .. } = self.func[value].def else { return None };
916        let data = &self.func[inst];
917        let args = &self.func[data.args];
918        let (&lhs, &rhs) = (args.first()?, args.get(1)?);
919        let combine = |carried: (Invariant, Flags), other: Invariant, subtract: bool| {
920            let (delta, flags) = carried;
921            let moved = if subtract { delta.minus(other)? } else { delta.plus(other)? };
922            Some((moved, flags.intersection(data.flags)))
923        };
924        match data.opcode {
925            Opcode::Add => {
926                if let Some(carried) = self.step(id, lhs, of, depth + 1) {
927                    return combine(carried, self.invariant(id, rhs)?, false);
928                }
929                combine(self.step(id, rhs, of, depth + 1)?, self.invariant(id, lhs)?, false)
930            }
931            Opcode::Sub => {
932                combine(self.step(id, lhs, of, depth + 1)?, self.invariant(id, rhs)?, true)
933            }
934            // A pointer walks by bytes, and only the pointer side can be the one carrying the
935            // induction variable. The offset is the step, which is the element size the front end
936            // already multiplied in.
937            Opcode::PtrAdd => {
938                combine(self.step(id, lhs, of, depth + 1)?, self.invariant(id, rhs)?, false)
939            }
940            _ => None,
941        }
942    }
943
944    /// The evolution of an instruction's result, from the evolutions of its operands.
945    fn at_inst(&mut self, id: LoopId, inst: Inst, value: Value) -> Evolution {
946        let func = self.func;
947        let data = &func[inst];
948        let (opcode, flags) = (data.opcode, data.flags);
949        let args = &func[data.args];
950        let ty = func[value].ty;
951        let Some(&lhs) = args.first() else { return Evolution::Unknown };
952        match opcode {
953            Opcode::Add | Opcode::PtrAdd => {
954                let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
955                let (left, right) = (self.at(id, lhs), self.at(id, rhs));
956                combine(left, right, ty, flags, false)
957            }
958            Opcode::Sub => {
959                let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
960                let (left, right) = (self.at(id, lhs), self.at(id, rhs));
961                combine(left, right, ty, flags, true)
962            }
963            Opcode::Mul => {
964                let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
965                let (left, right) = (self.at(id, lhs), self.at(id, rhs));
966                scale(left, right, ty, flags)
967            }
968            // A shift by a constant is a multiplication by a power of two, and only by a constant:
969            // a variable count is invariant in the loop and still not a number this can multiply
970            // by. A count at or above the width is poison rather than a shift to zero, so the
971            // range is checked here rather than assumed.
972            Opcode::Shl => {
973                let Some(&rhs) = args.get(1) else { return Evolution::Unknown };
974                let Some((count, count_ty)) = constant(func, rhs) else {
975                    return Evolution::Unknown;
976                };
977                let count = count.unsigned();
978                if count >= u128::from(ty.bits()) || !count_ty.is_int() {
979                    return Evolution::Unknown;
980                }
981                let by = Evolution::Invariant(Invariant::number(1i128 << count));
982                scale(self.at(id, lhs), by, ty, flags)
983            }
984            Opcode::SExt | Opcode::ZExt => self.extend(id, opcode, lhs, ty),
985            // A truncation is a wrap by construction, so a chrec through one describes a sequence
986            // that restarts, and this does not have a representation for that.
987            _ => Evolution::Unknown,
988        }
989    }
990
991    /// A chrec widened, which needs the sequence not to wrap at the narrow width.
992    ///
993    /// Section 7.4 allows extension only where the extension provably does not wrap, and the first
994    /// proof here is the flag the increment carries. `nsw` on the increment is the promise that the
995    /// signed sequence does not wrap, which is exactly what makes the wide sequence the same
996    /// numbers as the narrow one.
997    ///
998    /// The second proof is the loop's own exit test, through [`Scev::holds`] and [`trails`], and it
999    /// is here because of what an unsigned counter looks like. `for (unsigned i = 0; i < n; i++)`
1000    /// carries no `nuw`, because C says unsigned arithmetic wraps, so `a[i]` on that counter used
1001    /// to come back unwidened and every bounds check in the loop stayed where it was. The test that
1002    /// keeps the counter inside its type keeps everything walking beside it inside too.
1003    ///
1004    /// Each part is either a plain number or one of a value, and nothing else. A number means the
1005    /// same thing at both widths, and one of a value becomes that value read through the extension,
1006    /// which is what [`Widening`] is for. Anything with arithmetic in it is refused, because the
1007    /// narrow arithmetic may already have wrapped and `sext(2 * x + 3)` is not `2 * sext(x) + 3`.
1008    /// What that leaves out is a base like `start + 1`, and what it lets in is `start`, which is
1009    /// the shape a walk from an index the caller handed in is in. See #810.
1010    ///
1011    /// A value the loop does not change is widened by the same rule. It used to be widened only
1012    /// when it was a number, and everything else came back unknown, which is a sequence that does
1013    /// not move being harder to widen than one that does. What it cost is the row of a two
1014    /// dimensional array: `a[row * N + k]` round `k` has `(long)row * N` in it, that is invariant
1015    /// and is not a number, so the address of the whole subscript came back unknown and every pass
1016    /// reading it had nothing to work with. There is no wrapping question to answer here, because
1017    /// there is no sequence and so nothing to wrap, and the shapes that get through are the same
1018    /// ones [`Invariant::widened`] lets through for a chrec's base.
1019    fn extend(&mut self, id: LoopId, opcode: Opcode, from: Value, to: Type) -> Evolution {
1020        let narrow = self.func[from].ty;
1021        let signed = opcode == Opcode::SExt;
1022        let held = self.held.get(&id).copied().flatten();
1023        let settled = |chrec: Chrec| {
1024            chrec.does_not_wrap(signed) || (!signed && held.is_some_and(|held| trails(chrec, held)))
1025        };
1026        let reading = if signed { Reading::Signed } else { Reading::Unsigned };
1027        match self.at(id, from) {
1028            Evolution::Invariant(inv) => match inv.as_number() {
1029                // A number read at the narrow width means the same thing at the wide one under
1030                // sign extension, and under zero extension once it is not negative.
1031                Some(number) if signed || number >= 0 => Evolution::Invariant(inv),
1032                Some(_) => Evolution::Unknown,
1033                // Not a number, and still the same value read wider. This is the widening a chrec
1034                // gets, asked about something that does not move: `(long)row * 64` inside a loop
1035                // over `k` is an expression the loop does not change, and it used to come back
1036                // unknown, which made the whole of `a[row * N + k]` unknown. [`Invariant::widened`]
1037                // is the one that decides, and it refuses anything with arithmetic in it for the
1038                // reason written on it, so what gets through is one of a value and nothing else.
1039                None => match inv.widened(reading, to) {
1040                    Some(wide) => Evolution::Invariant(wide),
1041                    None => Evolution::Unknown,
1042                },
1043            },
1044            Evolution::Affine(chrec) if chrec.ty == narrow && settled(chrec) => {
1045                let (Some(base), Some(step)) =
1046                    (chrec.base.widened(reading, to), chrec.step.widened(reading, to))
1047                else {
1048                    return Evolution::Unknown;
1049                };
1050                Evolution::Affine(Chrec { base, step, ty: to, flags: chrec.flags })
1051            }
1052            _ => Evolution::Unknown,
1053        }
1054    }
1055
1056    /// The trip count from the exit leaving this block, if this exit can be solved.
1057    fn bound_at(&mut self, id: LoopId, from: Block) -> Option<Bound> {
1058        let test = self.test_at(id, from)?;
1059        solve(test.chrec, test.limit, test.pred, test.each)
1060    }
1061
1062    /// The exit test leaving this block, read into the pieces its two readers want.
1063    ///
1064    /// [`Scev::bound_at`] spends it on a trip count and [`Scev::holds`] spends it on whether the
1065    /// counter can wrap, and both want the same reading of the same branch, so the reading is
1066    /// written once.
1067    fn test_at(&mut self, id: LoopId, from: Block) -> Option<Test> {
1068        let func = self.func;
1069        let term = func.terminator(from)?;
1070        if func[term].opcode != Opcode::BrIf {
1071            return None;
1072        }
1073        let args = &func[func[term].args];
1074        let &cond = args.first()?;
1075        let calls = &func[func.target_list(term)];
1076        let (&taken, &not_taken) = (calls.first()?, calls.get(1)?);
1077        // Which arm keeps going. If both stay in or both leave, the branch is not the test that
1078        // ends the loop and there is nothing here to solve.
1079        let stays = match (
1080            self.loops.contains(id, taken.block),
1081            self.loops.contains(id, not_taken.block),
1082        ) {
1083            (true, false) => true,
1084            (false, true) => false,
1085            _ => return None,
1086        };
1087
1088        let Def::Result { inst, .. } = func[cond].def else { return None };
1089        if func[inst].opcode != Opcode::ICmp {
1090            return None;
1091        }
1092        let Extra::IntPred(pred) = func[inst].extra else { return None };
1093        // The loop keeps going while the test says so, so an exit taken when the test is true is
1094        // an exit whose continuing condition is the opposite one.
1095        let pred = if stays { pred } else { invert(pred) };
1096        let operands = &func[func[inst].args];
1097        let (&lhs, &rhs) = (operands.first()?, operands.get(1)?);
1098
1099        // One side evolves and the other does not. Swapping puts the one that evolves on the left
1100        // and turns the predicate round with it, so only one direction has to be solved.
1101        let (chrec, limit, pred) = match (self.at(id, lhs), self.at(id, rhs)) {
1102            (Evolution::Affine(chrec), other) => (chrec, other.invariant()?, pred),
1103            (other, Evolution::Affine(chrec)) => (chrec, other.invariant()?, swap(pred)),
1104            _ => return None,
1105        };
1106
1107        // Whether every iteration that goes round asks this test. The header runs on all of them by
1108        // being the header. A latch runs on all of them only when it is the loop's one latch, since
1109        // with two of them an iteration can go round the other and never reach the test. Anywhere
1110        // else is a test under a condition, which [`bounded_by_its_test`] must not be given.
1111        //
1112        // The one latch is written out rather than taken for granted. `at_header` refuses a loop
1113        // with two of them already, so nothing reaching here has two, but the two conditions are
1114        // about different things and a later loosening of that one should not quietly loosen this.
1115        let each = from == self.loops.header(id) || self.loops.latches(id) == [from];
1116        Some(Test { chrec, limit, pred, each })
1117    }
1118}
1119
1120/// Two evolutions added, or subtracted when asked.
1121fn combine(left: Evolution, right: Evolution, ty: Type, flags: Flags, subtract: bool) -> Evolution {
1122    let apply = |a: Invariant, b: Invariant| if subtract { a.minus(b) } else { a.plus(b) };
1123    match (left, right) {
1124        (Evolution::Invariant(a), Evolution::Invariant(b)) => {
1125            apply(a, b).map_or(Evolution::Unknown, Evolution::Invariant)
1126        }
1127        (Evolution::Affine(chrec), Evolution::Invariant(b)) => {
1128            // Adding something that does not move only moves the base.
1129            let Some(base) = apply(chrec.base, b) else { return Evolution::Unknown };
1130            affine(base, chrec.step, ty, flags.intersection(chrec.flags))
1131        }
1132        (Evolution::Invariant(a), Evolution::Affine(chrec)) => {
1133            let (Some(base), Some(step)) = (
1134                apply(a, chrec.base),
1135                if subtract { chrec.step.negated() } else { Some(chrec.step) },
1136            ) else {
1137                return Evolution::Unknown;
1138            };
1139            affine(base, step, ty, flags.intersection(chrec.flags))
1140        }
1141        (Evolution::Affine(a), Evolution::Affine(b)) => {
1142            // Two chrecs of the same loop add componentwise, which is the closure property that
1143            // makes the representation worth having. Of different types they do not, because the
1144            // two sequences wrap at different widths.
1145            if a.ty != b.ty {
1146                return Evolution::Unknown;
1147            }
1148            let (Some(base), Some(step)) = (apply(a.base, b.base), apply(a.step, b.step)) else {
1149                return Evolution::Unknown;
1150            };
1151            affine(base, step, ty, flags.intersection(a.flags).intersection(b.flags))
1152        }
1153        _ => Evolution::Unknown,
1154    }
1155}
1156
1157/// One evolution multiplied by another, which needs one of them to stand still.
1158fn scale(left: Evolution, right: Evolution, ty: Type, flags: Flags) -> Evolution {
1159    let (chrec, by) = match (left, right) {
1160        (Evolution::Invariant(a), Evolution::Invariant(b)) => {
1161            return a.times(b).map_or(Evolution::Unknown, Evolution::Invariant);
1162        }
1163        (Evolution::Affine(chrec), Evolution::Invariant(by))
1164        | (Evolution::Invariant(by), Evolution::Affine(chrec)) => (chrec, by),
1165        // Two chrecs multiplied give a quadratic, which is a chain of recurrences with a second
1166        // step and is outside the subset section 7.4 chose.
1167        _ => return Evolution::Unknown,
1168    };
1169    let (Some(base), Some(step)) = (chrec.base.times(by), chrec.step.times(by)) else {
1170        return Evolution::Unknown;
1171    };
1172    affine(base, step, ty, flags.intersection(chrec.flags))
1173}
1174
1175/// A chrec, or invariant when the step turns out to be nothing.
1176///
1177/// A step of zero is a valid affine chrec describing a value that does not move, and section 7.7
1178/// warns that code dividing by the step to get a trip count divides by zero. Reporting it as
1179/// invariant here means the shape is right for every reader rather than only for the careful
1180/// ones, and the trip count solver still checks, because a step can also come out zero from a
1181/// header parameter incremented by an invariant that happens to be zero.
1182fn affine(base: Invariant, step: Invariant, ty: Type, flags: Flags) -> Evolution {
1183    if step.is_zero() {
1184        return Evolution::Invariant(base);
1185    }
1186    Evolution::Affine(Chrec { base, step, ty, flags })
1187}
1188
1189/// The iteration at which `chrec pred limit` first fails, with what that rests on.
1190///
1191/// `each` says the test runs on every iteration that goes round, which is what lets the test itself
1192/// stand in for a promise the counter does not carry. See [`bounded_by_its_test`].
1193fn solve(chrec: Chrec, limit: Invariant, pred: IntPred, each: bool) -> Option<Bound> {
1194    // Section 7.7's first way of being wrong. A step of zero is a loop that never leaves through
1195    // this exit, and dividing the distance by it is a crash rather than an answer.
1196    let step = chrec.step.as_number()?;
1197    if step == 0 {
1198        return None;
1199    }
1200    let signed = matches!(pred, IntPred::Slt | IntPred::Sle | IntPred::Sgt | IntPred::Sge);
1201
1202    let mut assumptions = Vec::new();
1203    if !chrec.does_not_wrap(signed) && !(each && bounded_by_its_test(pred, step)) {
1204        assumptions.push(Assumption::NoWrap(chrec));
1205    }
1206    if signed {
1207        assumptions.push(Assumption::StrictOverflow);
1208    }
1209
1210    // A test that does not read its operands as signed does not read the constants in them that
1211    // way either, and every constant reaching here was read as signed on the way in.
1212    let (base, limit) = if signed {
1213        (chrec.base, limit)
1214    } else {
1215        (unsigned_base(chrec)?, as_unsigned(limit, chrec.ty)?)
1216    };
1217
1218    // The distance the counter has to travel, always counting up. A loop going down is the same
1219    // problem with the ends swapped, which is why the step is used by size below and its sign is
1220    // spent here.
1221    let apart = step.unsigned_abs();
1222    let found = match (pred, step > 0) {
1223        (IntPred::Slt | IntPred::Ult, true) => {
1224            ordered(limit.minus(base)?, apart, false, assumptions)
1225        }
1226        (IntPred::Sle | IntPred::Ule, true) => {
1227            ordered(limit.minus(base)?, apart, true, assumptions)
1228        }
1229        (IntPred::Sgt | IntPred::Ugt, false) => {
1230            ordered(base.minus(limit)?, apart, false, assumptions)
1231        }
1232        (IntPred::Sge | IntPred::Uge, false) => {
1233            ordered(base.minus(limit)?, apart, true, assumptions)
1234        }
1235        (IntPred::Ne, _) => {
1236            let distance = if step > 0 { limit.minus(base)? } else { base.minus(limit)? };
1237            landing(distance, apart, step < 0 && limit.is_zero(), assumptions)
1238        }
1239        // Either the counter steps away from the limit, in which case the loop is endless rather
1240        // than long, or the test is one this does not solve. Silence is the answer to both.
1241        _ => None,
1242    };
1243    // Written once here rather than threaded through the two solvers, because it is a fact about
1244    // the test and neither of them looks at the test. A count taken from a test with no sign to it,
1245    // which is `!=`, is read unsigned, because that is the reading `as_unsigned` above already put
1246    // its operands through.
1247    let reading = if signed { Reading::Signed } else { Reading::Unsigned };
1248    found.map(|(count, assumptions)| Bound { count, assumptions, reading })
1249}
1250
1251/// Whether the exit test by itself rules out the counter wrapping before the loop ends.
1252///
1253/// An unsigned counter carries no `nuw`, because C says unsigned arithmetic wraps, so without this
1254/// every `for (unsigned i = 0; i < n; i++)` comes back resting on an assumption nothing downstream
1255/// can discharge. What discharges it is the test. A counter stepping up by exactly one is at the
1256/// limit before it is anywhere past it, and the test ends the loop there, so it never reaches the
1257/// top of its type. GCC works the same thing out in `scev_probably_wraps_p`.
1258///
1259/// Every part of that is load bearing. The step has to be one: `i += 2` can go from one below the
1260/// limit to one above the top of the type and come back round at the bottom, which is a loop that
1261/// runs forever rather than one that runs twice as fast. The test has to be the strict one: `<=`
1262/// lets the counter reach the limit and step once more, and a limit that is the largest number of
1263/// its type makes that last step the one that wraps. And the test has to run on every iteration
1264/// that goes round, or the counter can be stepped by a path that never asks it anything.
1265///
1266/// Nothing is claimed here about a signed counter, which needs no help: a signed counter that would
1267/// wrap is a program with undefined behaviour in it and [`Assumption::StrictOverflow`] is where
1268/// that is recorded.
1269fn bounded_by_its_test(pred: IntPred, step: i128) -> bool {
1270    matches!((pred, step), (IntPred::Ult, 1) | (IntPred::Ugt, -1))
1271}
1272
1273/// Whether this sequence stays behind one the exit test already keeps inside its type.
1274///
1275/// [`bounded_by_its_test`] says the counter the test compares never reaches the top of its type.
1276/// Everything else the loop counts with is that counter plus a fixed distance, because two affine
1277/// chrecs of the same loop with the same step differ by a constant, so a sequence starting no
1278/// further along than the counter is a sequence that gets to the top no sooner than the counter
1279/// does, which is never.
1280///
1281/// Same base is the case that matters most and the easiest to see: the test compares `i + 1` and
1282/// the subscript reads `i`, which is one loop written two ways, and the two chrecs differ only in
1283/// where they start.
1284///
1285/// Going up only. A counter going down wraps at the bottom rather than the top, so the sequence
1286/// that is safe is the one that starts further along rather than the one that starts behind, and
1287/// nothing measured so far walks an array downwards. Doing it would be turning the comparison
1288/// round, and it should come with the program that wants it.
1289fn trails(chrec: Chrec, held: Chrec) -> bool {
1290    if chrec.ty != held.ty || chrec.step != held.step {
1291        return false;
1292    }
1293    if chrec.base == held.base {
1294        return true;
1295    }
1296    let (Some(step), Some(mine), Some(theirs)) =
1297        (chrec.step.as_number(), chrec.base.as_number(), held.base.as_number())
1298    else {
1299        return false;
1300    };
1301    // Read as unsigned, which is the reading the test took, so a base that came in negative is a
1302    // large number rather than a small one and starting behind is not what it is doing.
1303    step > 0 && mine >= 0 && theirs >= 0 && mine <= theirs
1304}
1305
1306/// The same expression, read the way a test without a sign reads it.
1307///
1308/// Constants arrive here as the number their bits are when the sign bit is taken seriously,
1309/// because that is the only reading available before anybody knows what will be done with them.
1310/// An unsigned test disagrees about half of them. `for (unsigned char i = 0; i < 200; i++)` holds
1311/// its limit as minus fifty six, and a distance worked out from that is negative, which reads as
1312/// a loop that runs no times rather than one that runs two hundred.
1313///
1314/// The step is not put through this, because a step is a difference rather than a value and its
1315/// signed reading is the one that says which way the counter goes.
1316/// Where a counter starts, read unsigned.
1317///
1318/// What [`as_unsigned`] says, and one more case it has to refuse without the counter to ask. A base
1319/// with a number folded in beside its symbol is safe to read unsigned when the counter promises not
1320/// to wrap that way, because the base is the first value the counter took and it took it without
1321/// wrapping, so the sum is the number it looks like. The countdown ivopts writes tests its variable
1322/// after taking one off, and this is what its base looks like.
1323fn unsigned_base(chrec: Chrec) -> Option<Invariant> {
1324    let base = chrec.base;
1325    let plain = base.on.is_none() && base.read.is_none() && base.scale == 1;
1326    as_unsigned(base, chrec.ty).or_else(|| (plain && chrec.does_not_wrap(false)).then_some(base))
1327}
1328
1329fn as_unsigned(inv: Invariant, ty: Type) -> Option<Invariant> {
1330    match inv.as_number() {
1331        Some(number) if number >= 0 => Some(inv),
1332        Some(number) => {
1333            // Only an integer constant was read as signed in the first place. A pointer never
1334            // was, so a negative number sitting in one is an expression this cannot reinterpret.
1335            let bits = ty.is_int().then(|| ty.bits()).filter(|&bits| bits < 127)?;
1336            Some(Invariant::number(number & ((1i128 << bits) - 1)))
1337        }
1338        // A symbolic operand is whatever it is at run time, and the subtraction below cancels it
1339        // rather than reading it, so long as nothing signed has been folded in beside it. Two
1340        // symbols is two things to cancel and the subtraction only ever cancels one.
1341        None => (inv.on.is_none() && inv.scale == 1 && inv.offset == 0).then_some(inv),
1342    }
1343}
1344
1345/// The count for an exit tested with an ordering, where overshooting the limit still ends it.
1346fn ordered(
1347    distance: Invariant,
1348    step: u128,
1349    inclusive: bool,
1350    mut assumptions: Vec<Assumption>,
1351) -> Option<(Count, Vec<Assumption>)> {
1352    match distance.as_number() {
1353        Some(exact) => {
1354            if exact < 0 {
1355                // The counter starts past the limit, so the test fails the first time it runs.
1356                // That is a count of zero and it rests on nothing at all, not even on the counter
1357                // behaving, because the counter never moves.
1358                return Some((Count::Exact(0), Vec::new()));
1359            }
1360            // Rounding up, because a step that overshoots still took the iteration that overshot.
1361            let count = (exact.unsigned_abs() + u128::from(inclusive)).div_ceil(step);
1362            Some((Count::Exact(count), assumptions))
1363        }
1364        // Symbolic, and only for a step of one, because dividing an expression by anything else
1365        // needs a representation for a division and there is not one here.
1366        None if step == 1 => {
1367            assumptions.push(Assumption::Entered);
1368            let count = distance.plus(Invariant::number(i128::from(inclusive)))?;
1369            Some((Count::Symbolic(count), assumptions))
1370        }
1371        None => None,
1372    }
1373}
1374
1375/// The count for an exit tested with `!=`, where the counter has to land on the limit exactly.
1376///
1377/// This is a different problem from the one above and not a special case of it. An ordering test
1378/// ends the loop the moment the counter is past the limit, so a step that overshoots still stops.
1379/// `!=` only ends the loop on the one iteration where the counter is the limit, so a counter that
1380/// steps over the limit, or that starts on the far side of it, keeps going until it wraps. Both
1381/// of those are endless loops rather than short ones, and answering zero for either was the bug
1382/// this function exists to not have.
1383fn landing(
1384    distance: Invariant,
1385    step: u128,
1386    bottom: bool,
1387    mut assumptions: Vec<Assumption>,
1388) -> Option<(Count, Vec<Assumption>)> {
1389    match distance.as_number() {
1390        Some(exact) => {
1391            let travel = u128::try_from(exact).ok()?;
1392            // Checked outright rather than assumed, which is why nothing here needs an assumption
1393            // about the step dividing anything.
1394            (travel % step == 0).then(|| (Count::Exact(travel / step), assumptions))
1395        }
1396        // A step of one lands on everything ahead of it, so the only thing left to establish is
1397        // that the limit is ahead. `while (p != end)` is this case, and a step of anything else
1398        // would need the division a symbolic distance has no room for.
1399        //
1400        // A counter going down to zero has it established already, because `!=` reads it unsigned
1401        // and nothing unsigned is below zero, so zero is ahead of wherever it starts. The `!=`
1402        // that ivopts writes for a countdown is this case. A promise not to wrap would not do
1403        // instead, since a loop with a limit behind its counter can stop on something else, a
1404        // bounds check for one, long before the counter comes round to break the promise.
1405        None if step == 1 => {
1406            if !bottom {
1407                assumptions.push(Assumption::Approaching);
1408            }
1409            Some((Count::Symbolic(distance), assumptions))
1410        }
1411        None => None,
1412    }
1413}
1414
1415/// The predicate that is true exactly when this one is not.
1416fn invert(pred: IntPred) -> IntPred {
1417    match pred {
1418        IntPred::Eq => IntPred::Ne,
1419        IntPred::Ne => IntPred::Eq,
1420        IntPred::Slt => IntPred::Sge,
1421        IntPred::Sle => IntPred::Sgt,
1422        IntPred::Sgt => IntPred::Sle,
1423        IntPred::Sge => IntPred::Slt,
1424        IntPred::Ult => IntPred::Uge,
1425        IntPred::Ule => IntPred::Ugt,
1426        IntPred::Ugt => IntPred::Ule,
1427        IntPred::Uge => IntPred::Ult,
1428    }
1429}
1430
1431/// The predicate that says the same thing with the operands the other way round.
1432fn swap(pred: IntPred) -> IntPred {
1433    match pred {
1434        IntPred::Eq => IntPred::Eq,
1435        IntPred::Ne => IntPred::Ne,
1436        IntPred::Slt => IntPred::Sgt,
1437        IntPred::Sle => IntPred::Sge,
1438        IntPred::Sgt => IntPred::Slt,
1439        IntPred::Sge => IntPred::Sle,
1440        IntPred::Ult => IntPred::Ugt,
1441        IntPred::Ule => IntPred::Uge,
1442        IntPred::Ugt => IntPred::Ult,
1443        IntPred::Uge => IntPred::Ule,
1444    }
1445}
1446
1447/// The constant a value is, if it is one.
1448fn constant(func: &Func, value: Value) -> Option<(Imm, Type)> {
1449    let Def::Result { inst, .. } = func[value].def else { return None };
1450    if func[inst].opcode != Opcode::IConst {
1451        return None;
1452    }
1453    let Extra::Imm(at) = func[inst].extra else { return None };
1454    let ty = func[value].ty;
1455    ty.is_int().then(|| (func[at], ty))
1456}
1457
1458/// The global whose address a value is, if it is one.
1459fn symbol(func: &Func, value: Value) -> Option<Symbol> {
1460    let Def::Result { inst, .. } = func[value].def else { return None };
1461    if func[inst].opcode != Opcode::GlobalAddr {
1462        return None;
1463    }
1464    let Extra::Symbol(symbol) = func[inst].extra else { return None };
1465    Some(symbol)
1466}
1467
1468/// What this predecessor passes to the block's parameter at this position.
1469///
1470/// `None` when the predecessor branches to the block more than once with different arguments,
1471/// which a `br_if` with both arms on the same block can do and which means the parameter takes a
1472/// value that depends on the test rather than on the edge.
1473fn argument(func: &Func, pred: Block, block: Block, index: usize) -> Option<Value> {
1474    let term = func.terminator(pred)?;
1475    let mut found = None;
1476    for call in func.successors(term) {
1477        if call.block != block {
1478            continue;
1479        }
1480        let arg = *func[call.args].get(index)?;
1481        if found.replace(arg).is_some_and(|old| old != arg) {
1482            return None;
1483        }
1484    }
1485    found
1486}
1487
1488#[cfg(test)]
1489mod tests {
1490    use rucc_base::Interner;
1491    use rucc_ir::{Builder, Extra, Flags, Func, InstData, IntPred, Opcode, Signature, Type, Value};
1492
1493    use crate::cfg::Cfg;
1494    use crate::dom::Dominators;
1495    use crate::loops::{LoopId, Loops};
1496    use crate::scev::{
1497        Anchor, Assumption, Bound, Count, Evolution, Invariant, Reading, Scev, Widening,
1498    };
1499
1500    /// A loop counting in `ty` from `from` by `step` while the counter is below `to`.
1501    ///
1502    /// ```text
1503    /// entry:  jump header(from)
1504    /// header(i): test = icmp pred i, to ; br_if test, body, exit
1505    /// body:   next = add i, step ; jump header(next)
1506    /// exit:   ret
1507    /// ```
1508    ///
1509    /// The counter is the header's only parameter, which is what the tests ask about.
1510    struct Counted {
1511        func: Func,
1512        counter: Value,
1513        next: Value,
1514    }
1515
1516    fn counted(ty: Type, from: i128, to: i128, step: i128, pred: IntPred, flags: Flags) -> Counted {
1517        let (it, ()) = counted_with(ty, from, to, step, pred, flags, |_, _| ());
1518        it
1519    }
1520
1521    /// The same loop, with `extra` run in the body on the counter before the counter steps.
1522    ///
1523    /// The builder appends, and the body's `jump` back to the header has to stay the last
1524    /// instruction in it or the block has no terminator and the loop stops being one. So anything
1525    /// a test wants derived from the counter goes in here rather than being tacked on afterwards.
1526    fn counted_with<T>(
1527        ty: Type,
1528        from: i128,
1529        to: i128,
1530        step: i128,
1531        pred: IntPred,
1532        flags: Flags,
1533        extra: impl FnOnce(&mut Builder<'_>, Value) -> T,
1534    ) -> (Counted, T) {
1535        let mut names = Interner::new();
1536        let mut func = Func::new(names.intern("f"), Signature::new());
1537        let entry = func.create_block();
1538        let header = func.create_block();
1539        let body = func.create_block();
1540        let exit = func.create_block();
1541        let counter = func.append_param(header, ty);
1542
1543        let mut build = Builder::new(&mut func, entry);
1544        let start = build.iconst(ty, from);
1545        build.jump(header, &[start]);
1546
1547        let mut build = Builder::new(&mut func, header);
1548        let limit = build.iconst(ty, to);
1549        let test = build.icmp(pred, counter, limit);
1550        build.br_if(test, body, &[], exit, &[]);
1551
1552        let mut build = Builder::new(&mut func, body);
1553        let derived = extra(&mut build, counter);
1554        let by = build.iconst(ty, step);
1555        let next = build.binary(Opcode::Add, counter, by, flags);
1556        build.jump(header, &[next]);
1557
1558        let mut build = Builder::new(&mut func, exit);
1559        build.ret(&[]);
1560
1561        (Counted { func, counter, next }, derived)
1562    }
1563
1564    /// The analysis over a function, along with the one loop it has.
1565    fn analyse(func: &Func) -> (Cfg, Loops) {
1566        let cfg = Cfg::new(func);
1567        let doms = Dominators::new(&cfg);
1568        let loops = Loops::new(&cfg, &doms);
1569        (cfg, loops)
1570    }
1571
1572    /// The chrec of a value in the one loop of a function.
1573    fn evolution(func: &Func, value: Value) -> Evolution {
1574        let (cfg, loops) = analyse(func);
1575        let id = loops.roots()[0];
1576        Scev::new(func, &cfg, &loops).evolution(id, value)
1577    }
1578
1579    /// The trip count of the one loop of a function.
1580    fn bound(func: &Func) -> Option<Bound> {
1581        let (cfg, loops) = analyse(func);
1582        let id: LoopId = loops.roots()[0];
1583        Scev::new(func, &cfg, &loops).bound(id)
1584    }
1585
1586    #[test]
1587    fn a_counter_from_zero_by_one_is_the_chrec_everyone_expects() {
1588        let it = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
1589        let chrec = evolution(&it.func, it.counter).chrec().expect("the counter evolves");
1590        assert_eq!(chrec.base, Invariant::number(0));
1591        assert_eq!(chrec.step, Invariant::number(1));
1592        assert_eq!(chrec.ty, Type::int(32));
1593        assert!(chrec.does_not_wrap(true));
1594    }
1595
1596    #[test]
1597    fn a_walk_over_a_file_scope_array_is_a_chrec_measured_from_the_symbol() {
1598        // The `global_addr` is inside the loop, which is where the compiler leaves one: working
1599        // the address out again is a single instruction and `crate::licm` would rather do that
1600        // than hold it in a register the whole way round. Answering by where a value is defined
1601        // meant `a[i]` on a file scope `a` was an address with nothing to say about it.
1602        let mut names = Interner::new();
1603        let tab = names.intern("tab");
1604        let (it, address) =
1605            counted_with(Type::int(64), 0, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1606                let four = build.iconst(Type::int(64), 4);
1607                let by = build.binary(Opcode::Mul, counter, four, Flags::NSW);
1608                let extra = Extra::Symbol(tab);
1609                let at =
1610                    build.value(InstData { extra, ..InstData::new(Opcode::GlobalAddr) }, Type::PTR);
1611                let args = build.func().push_values(&[at, by]);
1612                build.value(InstData { args, ..InstData::new(Opcode::PtrAdd) }, Type::PTR)
1613            });
1614
1615        let chrec = evolution(&it.func, address).chrec().expect("the address evolves");
1616        assert_eq!(chrec.step, Invariant::number(4));
1617        // Described rather than named, so there is nothing for `plain` to hand back and a reader
1618        // of the base has to go through `on` and see what it is measured from.
1619        assert!(chrec.base.plain().is_none());
1620        let (base, rest) = chrec.base.on().expect("the base is measured from the symbol");
1621        assert_eq!(base, Anchor::Address(tab));
1622        assert_eq!(base.value(), None);
1623        assert_eq!(rest.value, None);
1624        assert_eq!(rest.offset, 0);
1625    }
1626
1627    #[test]
1628    fn the_value_fed_back_is_the_chrec_one_step_along() {
1629        let it = counted(Type::int(32), 5, 100, 3, IntPred::Slt, Flags::NSW);
1630        let chrec = evolution(&it.func, it.next).chrec().expect("the increment evolves");
1631        assert_eq!(chrec.base, Invariant::number(8));
1632        assert_eq!(chrec.step, Invariant::number(3));
1633    }
1634
1635    #[test]
1636    fn a_multiple_of_the_counter_plus_a_number_is_a_chrec_of_its_own() {
1637        // `j = 2 * i + 3` where `i = {0, +, 1}`, which is the shape section 7.4 says pattern
1638        // matching runs out of road on and chains of recurrences do not.
1639        let (it, shifted) =
1640            counted_with(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1641                let two = build.iconst(Type::int(32), 2);
1642                let three = build.iconst(Type::int(32), 3);
1643                let doubled = build.binary(Opcode::Mul, counter, two, Flags::NSW);
1644                build.binary(Opcode::Add, doubled, three, Flags::NSW)
1645            });
1646
1647        let chrec = evolution(&it.func, shifted).chrec().expect("it evolves");
1648        assert_eq!(chrec.base, Invariant::number(3));
1649        assert_eq!(chrec.step, Invariant::number(2));
1650    }
1651
1652    #[test]
1653    fn a_shift_by_a_constant_scales_the_chrec_and_a_shift_past_the_width_does_not() {
1654        let (it, (scaled, poison)) =
1655            counted_with(Type::int(32), 1, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1656                let three = build.iconst(Type::int(32), 3);
1657                let wide = build.iconst(Type::int(32), 32);
1658                (
1659                    build.binary(Opcode::Shl, counter, three, Flags::NSW),
1660                    build.binary(Opcode::Shl, counter, wide, Flags::NSW),
1661                )
1662            });
1663
1664        let chrec = evolution(&it.func, scaled).chrec().expect("it evolves");
1665        assert_eq!(chrec.base, Invariant::number(8));
1666        assert_eq!(chrec.step, Invariant::number(8));
1667        // A count at the width is poison rather than a shift to zero, so there is no sequence to
1668        // describe.
1669        assert_eq!(evolution(&it.func, poison), Evolution::Unknown);
1670    }
1671
1672    #[test]
1673    fn a_pointer_walked_by_the_element_size_is_a_chrec_in_bytes() {
1674        // What `for (p = a; p != end; p++)` lowers to on an array of four byte elements. Section
1675        // 7.4 calls this the one deliberate extension past affine and the difference between
1676        // analysing half of real C loops and nearly all of them.
1677        let mut names = Interner::new();
1678        let mut func = Func::new(names.intern("f"), Signature::new());
1679        let entry = func.create_block();
1680        let header = func.create_block();
1681        let body = func.create_block();
1682        let exit = func.create_block();
1683        let start = func.append_param(entry, Type::PTR);
1684        let cursor = func.append_param(header, Type::PTR);
1685
1686        let mut build = Builder::new(&mut func, entry);
1687        build.jump(header, &[start]);
1688        let mut build = Builder::new(&mut func, header);
1689        let done = build.icmp(IntPred::Eq, cursor, start);
1690        build.br_if(done, exit, &[], body, &[]);
1691        let mut build = Builder::new(&mut func, body);
1692        let four = build.iconst(Type::int(64), 4);
1693        let next = build.binary(Opcode::PtrAdd, cursor, four, Flags::NONE);
1694        build.jump(header, &[next]);
1695        let mut build = Builder::new(&mut func, exit);
1696        build.ret(&[]);
1697
1698        let chrec = evolution(&func, cursor).chrec().expect("the cursor evolves");
1699        assert_eq!(chrec.base, Invariant::of(start));
1700        assert_eq!(chrec.step, Invariant::number(4));
1701        assert_eq!(chrec.ty, Type::PTR);
1702    }
1703
1704    #[test]
1705    fn a_value_the_loop_does_not_change_widens_the_way_a_sequence_does() {
1706        // `(long)row` inside a loop over something else. There is no sequence here and so nothing
1707        // that could wrap, and the answer was unknown all the same, which made a sequence that does
1708        // not move harder to widen than one that does. What it cost is `a[row * N + k]` round `k`,
1709        // whose address came back unknown on account of the widening in the middle of it.
1710        let mut names = Interner::new();
1711        let mut func = Func::new(names.intern("f"), Signature::new());
1712        let entry = func.create_block();
1713        let header = func.create_block();
1714        let body = func.create_block();
1715        let exit = func.create_block();
1716        let row = func.append_param(entry, Type::int(32));
1717        let counter = func.append_param(header, Type::int(64));
1718
1719        let mut build = Builder::new(&mut func, entry);
1720        let zero = build.iconst(Type::int(64), 0);
1721        build.jump(header, &[zero]);
1722
1723        let mut build = Builder::new(&mut func, header);
1724        let limit = build.iconst(Type::int(64), 100);
1725        let test = build.icmp(IntPred::Slt, counter, limit);
1726        build.br_if(test, body, &[], exit, &[]);
1727
1728        let mut build = Builder::new(&mut func, body);
1729        let wide = build.unary(Opcode::SExt, row, Type::int(64));
1730        let one = build.iconst(Type::int(64), 1);
1731        let next = build.binary(Opcode::Add, counter, one, Flags::NSW);
1732        build.jump(header, &[next]);
1733        Builder::new(&mut func, exit).ret(&[]);
1734
1735        let word = Type::int(64);
1736        let widened =
1737            Invariant::of(row).widened(Reading::Signed, word).expect("one of a value widens");
1738        assert_eq!(evolution(&func, wide), Evolution::Invariant(widened));
1739    }
1740
1741    /// A value to hang an invariant on, which these never look inside.
1742    fn some_value() -> Value {
1743        let mut names = Interner::new();
1744        let mut func = Func::new(names.intern("f"), Signature::new());
1745        let entry = func.create_block();
1746        func.append_param(entry, Type::int(8))
1747    }
1748
1749    #[test]
1750    fn one_of_a_value_widens_and_arithmetic_on_it_does_not() {
1751        // What `Scev::extend` may take. A value is widened by describing the extension rather than
1752        // by naming a value nothing computes, which is what lets `for (i = start; i < n; i++)`
1753        // have a chrec at pointer width. `2 * x + 3` is refused, because the narrow arithmetic may
1754        // already have wrapped and `sext(2 * x + 3)` is not `2 * sext(x) + 3`.
1755        let value = some_value();
1756        let word = Type::int(64);
1757        assert_eq!(
1758            Invariant::of(value).widened(Reading::Signed, word),
1759            Some(Invariant {
1760                on: None,
1761                value: Some(value),
1762                read: Some(Widening { reading: Reading::Signed, to: word }),
1763                scale: 1,
1764                offset: 0,
1765            }),
1766        );
1767        assert_eq!(Invariant::scaled(value, 2, 3).widened(Reading::Signed, word), None);
1768        assert_eq!(Invariant::scaled(value, 1, 3).widened(Reading::Signed, word), None);
1769        // A number is the same number at both widths under a sign extension, and under a zero
1770        // extension once it is not negative.
1771        assert_eq!(
1772            Invariant::number(-1).widened(Reading::Signed, word),
1773            Some(Invariant::number(-1)),
1774        );
1775        assert_eq!(Invariant::number(-1).widened(Reading::Unsigned, word), None);
1776    }
1777
1778    #[test]
1779    fn an_extension_of_an_extension_collapses_only_where_it_means_the_same_thing() {
1780        // A zero extension is never negative, so reading its result as signed afterwards is the
1781        // same numbers and the pair is one zero extension at the outer width. The other way round
1782        // it is not: a sign extended negative number read as unsigned is a different quantity, and
1783        // there is nothing to collapse to.
1784        let value = some_value();
1785        let (half, word) = (Type::int(32), Type::int(64));
1786        let read = |inv: Invariant| inv.read.expect("a widened value carries how it is read");
1787
1788        let zeroed = Invariant::of(value).widened(Reading::Unsigned, half).expect("it widens");
1789        let again = zeroed.widened(Reading::Signed, word).expect("and it widens again");
1790        assert_eq!(read(again), Widening { reading: Reading::Unsigned, to: word });
1791
1792        let signed = Invariant::of(value).widened(Reading::Signed, half).expect("it widens");
1793        assert_eq!(signed.widened(Reading::Unsigned, word), None);
1794        let again = signed.widened(Reading::Signed, word).expect("and it widens again");
1795        assert_eq!(read(again), Widening { reading: Reading::Signed, to: word });
1796    }
1797
1798    #[test]
1799    fn two_invariants_on_the_same_value_read_two_ways_do_not_add() {
1800        // `sext(x)` and `zext(x)` are the same bits and not the same quantity, so a sum of them is
1801        // not two of anything and there is no shape here for it.
1802        let value = some_value();
1803        let word = Type::int(64);
1804        let signed = Invariant::of(value).widened(Reading::Signed, word).expect("it widens");
1805        let zeroed = Invariant::of(value).widened(Reading::Unsigned, word).expect("it widens");
1806        assert_eq!(signed.plus(zeroed), None);
1807        assert_eq!(
1808            signed.plus(signed),
1809            Some(Invariant {
1810                on: None,
1811                value: Some(value),
1812                read: Some(Widening { reading: Reading::Signed, to: word }),
1813                scale: 2,
1814                offset: 0,
1815            }),
1816            "the same value read the same way adds to two of it",
1817        );
1818    }
1819
1820    #[test]
1821    fn a_counter_in_unsigned_char_wraps_and_does_not_widen_without_a_promise() {
1822        // Section 7.7's second way of being wrong. `{0, +, 1}` in `unsigned char` is not
1823        // `0, 1, 2, ...`, it is that modulo two hundred and fifty six, and widening it is only
1824        // the same sequence if it does not get that far.
1825        //
1826        // An inclusive test, because a strict one is a proof of its own and the case below is
1827        // about what happens when there is no proof at all. This loop does not in fact wrap, and
1828        // the point is that nothing here can say so.
1829        let (it, wide) =
1830            counted_with(Type::int(8), 0, 100, 1, IntPred::Ule, Flags::NONE, |build, counter| {
1831                build.unary(Opcode::ZExt, counter, Type::int(32))
1832            });
1833        let chrec = evolution(&it.func, it.counter).chrec().expect("the counter evolves");
1834        assert_eq!(chrec.ty, Type::int(8));
1835        assert!(!chrec.does_not_wrap(false));
1836        assert_eq!(evolution(&it.func, wide), Evolution::Unknown);
1837    }
1838
1839    #[test]
1840    fn a_counter_its_own_test_holds_widens_without_a_promise() {
1841        // The same counter under the strict test, which is the shape `for (unsigned i = 0; i < n;
1842        // i++)` has. Nothing promised anything, and the test is the proof: the counter is at the
1843        // limit before it is anywhere past it, and the loop ends there.
1844        let (it, wide) =
1845            counted_with(Type::int(8), 0, 100, 1, IntPred::Ult, Flags::NONE, |build, counter| {
1846                build.unary(Opcode::ZExt, counter, Type::int(32))
1847            });
1848        let narrow = evolution(&it.func, it.counter).chrec().expect("the counter evolves");
1849        assert!(!narrow.does_not_wrap(false), "nothing was promised, so nothing carries a flag");
1850        let chrec = evolution(&it.func, wide).chrec().expect("its own test holds it");
1851        assert_eq!(chrec.ty, Type::int(32));
1852        assert_eq!(chrec.base, Invariant::number(0));
1853        assert_eq!(chrec.step, Invariant::number(1));
1854    }
1855
1856    #[test]
1857    fn a_sequence_that_starts_further_along_than_the_counter_does_not_widen() {
1858        // `trails` in the direction it refuses. The test holds `i`, which starts at zero, and this
1859        // asks about `i + 1`, which starts one further along. One further along is where the
1860        // counter would be if it had gone round once more, and going round once more is the step
1861        // nothing here rules out.
1862        let (it, wide) =
1863            counted_with(Type::int(8), 0, 100, 1, IntPred::Ult, Flags::NONE, |build, counter| {
1864                let one = build.iconst(Type::int(8), 1);
1865                let ahead = build.binary(Opcode::Add, counter, one, Flags::NONE);
1866                build.unary(Opcode::ZExt, ahead, Type::int(32))
1867            });
1868        assert_eq!(evolution(&it.func, wide), Evolution::Unknown);
1869    }
1870
1871    #[test]
1872    fn a_counter_in_short_widens_when_the_increment_promised_it_would_not_wrap() {
1873        let (it, (wide, zero_extended)) =
1874            counted_with(Type::int(16), 0, 100, 1, IntPred::Slt, Flags::NSW, |build, counter| {
1875                (
1876                    build.unary(Opcode::SExt, counter, Type::int(32)),
1877                    build.unary(Opcode::ZExt, counter, Type::int(32)),
1878                )
1879            });
1880
1881        let chrec = evolution(&it.func, wide).chrec().expect("it widens");
1882        assert_eq!(chrec.ty, Type::int(32));
1883        assert_eq!(chrec.base, Invariant::number(0));
1884        assert_eq!(chrec.step, Invariant::number(1));
1885        // `nsw` is a promise about the signed reading and says nothing about the unsigned one.
1886        assert_eq!(evolution(&it.func, zero_extended), Evolution::Unknown);
1887    }
1888
1889    #[test]
1890    fn a_step_of_zero_is_invariant_and_has_no_trip_count() {
1891        // Section 7.7's first way of being wrong. `i += k` with `k` of zero is a valid affine
1892        // chrec of a loop that never leaves through this exit, and code dividing the distance by
1893        // the step divides by zero.
1894        let it = counted(Type::int(32), 0, 100, 0, IntPred::Slt, Flags::NSW);
1895        assert!(matches!(evolution(&it.func, it.counter), Evolution::Invariant(_)));
1896        assert_eq!(bound(&it.func), None);
1897    }
1898
1899    #[test]
1900    fn a_counted_loop_has_the_count_anyone_would_work_out_by_hand() {
1901        let it = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
1902        let found = bound(&it.func).expect("it is counted");
1903        let (count, assumptions) = found.parts();
1904        assert_eq!(count, Count::Exact(100));
1905        // The distance is a number and it is not negative, so being entered is not in question.
1906        // Signed overflow being undefined still is, which is what `-fwrapv` would withdraw.
1907        assert_eq!(assumptions, [Assumption::StrictOverflow]);
1908        assert_eq!(found.proven(), None);
1909    }
1910
1911    #[test]
1912    fn a_step_that_overshoots_still_takes_the_iteration_that_overshot() {
1913        // Zero, three, six, nine, and the test fails at twelve, so four iterations rather than
1914        // three and a third. Rounding the other way is an off by one in every unroller.
1915        let it = counted(Type::int(32), 0, 10, 3, IntPred::Slt, Flags::NSW);
1916        let (count, _) = bound(&it.func).expect("it is counted").parts();
1917        assert_eq!(count, Count::Exact(4));
1918    }
1919
1920    #[test]
1921    fn an_inclusive_test_runs_one_more_time() {
1922        let it = counted(Type::int(32), 0, 10, 1, IntPred::Sle, Flags::NSW);
1923        let (count, _) = bound(&it.func).expect("it is counted").parts();
1924        assert_eq!(count, Count::Exact(11));
1925    }
1926
1927    #[test]
1928    fn a_loop_whose_test_fails_first_time_runs_no_times_and_rests_on_nothing() {
1929        let it = counted(Type::int(32), 10, 0, 1, IntPred::Slt, Flags::NSW);
1930        let found = bound(&it.func).expect("it is counted");
1931        assert_eq!(found.proven(), Some(Count::Exact(0)));
1932        assert!(found.assumptions().is_empty());
1933    }
1934
1935    #[test]
1936    fn counting_down_is_the_same_problem_with_the_ends_swapped() {
1937        let it = counted(Type::int(32), 10, 0, -1, IntPred::Sgt, Flags::NSW);
1938        let (count, _) = bound(&it.func).expect("it is counted").parts();
1939        assert_eq!(count, Count::Exact(10));
1940    }
1941
1942    #[test]
1943    fn an_unsigned_test_does_not_drag_in_the_signed_overflow_assumption() {
1944        let it = counted(Type::int(32), 0, 100, 1, IntPred::Ult, Flags::NUW);
1945        let found = bound(&it.func).expect("it is counted");
1946        assert_eq!(found.proven(), Some(Count::Exact(100)));
1947    }
1948
1949    #[test]
1950    fn a_test_against_something_the_loop_does_not_change_gives_a_symbolic_count() {
1951        // `for (i = 0; i < n; i++)`, where the answer is `n` and is only `n` if the loop is
1952        // entered, because `n` of minus one runs no times and the distance is minus one.
1953        let mut names = Interner::new();
1954        let mut func = Func::new(names.intern("f"), Signature::new());
1955        let entry = func.create_block();
1956        let header = func.create_block();
1957        let body = func.create_block();
1958        let exit = func.create_block();
1959        let limit = func.append_param(entry, Type::int(32));
1960        let counter = func.append_param(header, Type::int(32));
1961
1962        let mut build = Builder::new(&mut func, entry);
1963        let zero = build.iconst(Type::int(32), 0);
1964        build.jump(header, &[zero]);
1965        let mut build = Builder::new(&mut func, header);
1966        let test = build.icmp(IntPred::Slt, counter, limit);
1967        build.br_if(test, body, &[], exit, &[]);
1968        let mut build = Builder::new(&mut func, body);
1969        let one = build.iconst(Type::int(32), 1);
1970        let next = build.binary(Opcode::Add, counter, one, Flags::NSW);
1971        build.jump(header, &[next]);
1972        let mut build = Builder::new(&mut func, exit);
1973        build.ret(&[]);
1974
1975        let found = bound(&func).expect("it is counted");
1976        let (count, assumptions) = found.parts();
1977        assert_eq!(count, Count::Symbolic(Invariant::of(limit)));
1978        assert!(assumptions.contains(&Assumption::Entered), "{assumptions:?}");
1979        assert!(assumptions.contains(&Assumption::StrictOverflow), "{assumptions:?}");
1980        assert_eq!(found.proven(), None);
1981    }
1982
1983    /// `for (c = n; c != limit; c--)`, with the counter in sixty four bits and no flags on it.
1984    fn down_to(limit: i128) -> (Func, Value) {
1985        let mut names = Interner::new();
1986        let mut func = Func::new(names.intern("f"), Signature::new());
1987        let entry = func.create_block();
1988        let header = func.create_block();
1989        let body = func.create_block();
1990        let exit = func.create_block();
1991        let start = func.append_param(entry, Type::int(64));
1992        let counter = func.append_param(header, Type::int(64));
1993
1994        let mut build = Builder::new(&mut func, entry);
1995        build.jump(header, &[start]);
1996        let mut build = Builder::new(&mut func, header);
1997        let limit = build.iconst(Type::int(64), limit);
1998        let test = build.icmp(IntPred::Ne, counter, limit);
1999        build.br_if(test, body, &[], exit, &[]);
2000        let mut build = Builder::new(&mut func, body);
2001        let one = build.iconst(Type::int(64), 1);
2002        let next = build.binary(Opcode::Sub, counter, one, Flags::NONE);
2003        build.jump(header, &[next]);
2004        let mut build = Builder::new(&mut func, exit);
2005        build.ret(&[]);
2006        (func, start)
2007    }
2008
2009    #[test]
2010    fn a_countdown_to_zero_is_always_heading_for_it() {
2011        let (func, start) = down_to(0);
2012        let found = bound(&func).expect("it is counted");
2013        let (count, assumptions) = found.parts();
2014        assert_eq!(count, Count::Symbolic(Invariant::of(start)));
2015        assert!(!assumptions.contains(&Assumption::Approaching), "{assumptions:?}");
2016    }
2017
2018    #[test]
2019    fn a_countdown_tested_after_its_step_is_counted_when_it_cannot_wrap() {
2020        // The shape ivopts writes: the variable starts one above the count and the header takes
2021        // one off before the test, so what the test sees starts at the start less one.
2022        for (flags, counted) in [(Flags::NSW | Flags::NUW, true), (Flags::NSW, false)] {
2023            let mut names = Interner::new();
2024            let mut func = Func::new(names.intern("f"), Signature::new());
2025            let entry = func.create_block();
2026            let header = func.create_block();
2027            let body = func.create_block();
2028            let exit = func.create_block();
2029            let start = func.append_param(entry, Type::int(64));
2030            let counter = func.append_param(header, Type::int(64));
2031
2032            Builder::new(&mut func, entry).jump(header, &[start]);
2033            let mut build = Builder::new(&mut func, header);
2034            let one = build.iconst(Type::int(64), 1);
2035            let next = build.binary(Opcode::Sub, counter, one, flags);
2036            let zero = build.iconst(Type::int(64), 0);
2037            let test = build.icmp(IntPred::Ne, next, zero);
2038            build.br_if(test, body, &[], exit, &[]);
2039            Builder::new(&mut func, body).jump(header, &[next]);
2040            Builder::new(&mut func, exit).ret(&[]);
2041
2042            let found = bound(&func);
2043            assert_eq!(found.is_some(), counted, "{flags:?}");
2044            if let Some(found) = found {
2045                let at = Invariant::of(start).plus(Invariant::number(-1)).expect("it adds");
2046                assert_eq!(found.comes_back(), Some(Count::Symbolic(at)));
2047            }
2048        }
2049    }
2050
2051    #[test]
2052    fn a_countdown_to_anything_else_may_have_started_below_it() {
2053        // Started at zero, this one goes all the way round before it gets to one.
2054        let (func, _) = down_to(1);
2055        let found = bound(&func).expect("it is counted");
2056        let (_, assumptions) = found.parts();
2057        assert!(assumptions.contains(&Assumption::Approaching), "{assumptions:?}");
2058    }
2059
2060    #[test]
2061    fn the_count_records_which_reading_its_test_took() {
2062        // What a consumer widening a symbolic count has to know. The limit is a value of the
2063        // counter's type and which number that value is depends on how its test read it.
2064        let signed = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
2065        assert_eq!(bound(&signed.func).expect("it is counted").reading(), Reading::Signed);
2066        let unsigned = counted(Type::int(32), 0, 100, 1, IntPred::Ult, Flags::NUW);
2067        assert_eq!(bound(&unsigned.func).expect("it is counted").reading(), Reading::Unsigned);
2068    }
2069
2070    #[test]
2071    fn a_counter_without_a_no_wrap_promise_carries_the_assumption_instead() {
2072        // An inclusive test, because the strict one is the case the test itself answers. Under
2073        // `<=` the counter reaches the limit and is stepped once more, so a limit at the top of
2074        // the type makes that last step the one that wraps and nothing here rules it out.
2075        let it = counted(Type::int(32), 0, 100, 1, IntPred::Ule, Flags::NONE);
2076        let found = bound(&it.func).expect("it is counted");
2077        let (_, assumptions) = found.parts();
2078        assert!(assumptions.iter().any(|a| matches!(a, Assumption::NoWrap(_))), "{assumptions:?}");
2079    }
2080
2081    #[test]
2082    fn an_unsigned_counter_stepping_by_one_is_held_by_its_own_test() {
2083        // `for (unsigned i = 0; i < n; i++)` written out. Unsigned arithmetic wraps in C so the
2084        // increment carries no `nuw`, and without reading the test this would rest on an
2085        // assumption nothing downstream can discharge.
2086        let it = counted(Type::int(32), 0, 100, 1, IntPred::Ult, Flags::NONE);
2087        let found = bound(&it.func).expect("it is counted");
2088        assert_eq!(found.assumptions(), &[]);
2089        assert_eq!(found.proven(), Some(Count::Exact(100)));
2090    }
2091
2092    #[test]
2093    fn counting_down_by_one_is_held_the_same_way() {
2094        let it = counted(Type::int(32), 100, 0, -1, IntPred::Ugt, Flags::NONE);
2095        let found = bound(&it.func).expect("it is counted");
2096        assert_eq!(found.assumptions(), &[]);
2097        assert_eq!(found.proven(), Some(Count::Exact(100)));
2098    }
2099
2100    #[test]
2101    fn a_step_of_two_can_jump_the_limit_so_the_test_holds_nothing() {
2102        // The counter is never at the limit, so the loop can be left by a step that goes from one
2103        // below the limit to one past the top of the type and comes back round at the bottom.
2104        let it = counted(Type::int(32), 0, 100, 2, IntPred::Ult, Flags::NONE);
2105        let found = bound(&it.func).expect("it is counted");
2106        let (_, assumptions) = found.parts();
2107        assert!(assumptions.iter().any(|a| matches!(a, Assumption::NoWrap(_))), "{assumptions:?}");
2108    }
2109
2110    #[test]
2111    fn a_test_the_counter_can_be_stepped_without_being_asked_holds_nothing_either() {
2112        // ```text
2113        // header(i): br_if flag, check, latch
2114        // check:     br_if i <u 100, latch, exit
2115        // latch:     jump header(i + 1)
2116        // ```
2117        // The counter goes round by a path that never reaches the test, so the test says nothing
2118        // about how far the counter got.
2119        let mut names = Interner::new();
2120        let mut func = Func::new(names.intern("f"), Signature::new());
2121        let entry = func.create_block();
2122        let header = func.create_block();
2123        let check = func.create_block();
2124        let latch = func.create_block();
2125        let exit = func.create_block();
2126        let flag = func.append_param(entry, Type::int(1));
2127        let counter = func.append_param(header, Type::int(32));
2128
2129        let mut build = Builder::new(&mut func, entry);
2130        let zero = build.iconst(Type::int(32), 0);
2131        build.jump(header, &[zero]);
2132        let mut build = Builder::new(&mut func, header);
2133        build.br_if(flag, check, &[], latch, &[]);
2134        let mut build = Builder::new(&mut func, check);
2135        let limit = build.iconst(Type::int(32), 100);
2136        let test = build.icmp(IntPred::Ult, counter, limit);
2137        build.br_if(test, latch, &[], exit, &[]);
2138        let mut build = Builder::new(&mut func, latch);
2139        let one = build.iconst(Type::int(32), 1);
2140        let next = build.binary(Opcode::Add, counter, one, Flags::NONE);
2141        build.jump(header, &[next]);
2142        let mut build = Builder::new(&mut func, exit);
2143        build.ret(&[]);
2144
2145        let found = bound(&func).expect("it is counted");
2146        let (_, assumptions) = found.parts();
2147        assert!(assumptions.iter().any(|a| matches!(a, Assumption::NoWrap(_))), "{assumptions:?}");
2148    }
2149
2150    #[test]
2151    fn a_test_that_ends_the_loop_when_it_succeeds_is_read_the_other_way_round() {
2152        // `for (i = 0; ; i++) if (i >= 100) break;`, which is the same loop with the arms of the
2153        // branch swapped. The test that keeps the loop going is the opposite of the one written.
2154        let mut names = Interner::new();
2155        let mut func = Func::new(names.intern("f"), Signature::new());
2156        let entry = func.create_block();
2157        let header = func.create_block();
2158        let body = func.create_block();
2159        let exit = func.create_block();
2160        let counter = func.append_param(header, Type::int(32));
2161
2162        let mut build = Builder::new(&mut func, entry);
2163        let zero = build.iconst(Type::int(32), 0);
2164        build.jump(header, &[zero]);
2165        let mut build = Builder::new(&mut func, header);
2166        let limit = build.iconst(Type::int(32), 100);
2167        let done = build.icmp(IntPred::Sge, counter, limit);
2168        build.br_if(done, exit, &[], body, &[]);
2169        let mut build = Builder::new(&mut func, body);
2170        let one = build.iconst(Type::int(32), 1);
2171        let next = build.binary(Opcode::Add, counter, one, Flags::NSW);
2172        build.jump(header, &[next]);
2173        let mut build = Builder::new(&mut func, exit);
2174        build.ret(&[]);
2175
2176        let (count, _) = bound(&func).expect("it is counted").parts();
2177        assert_eq!(count, Count::Exact(100));
2178    }
2179
2180    #[test]
2181    fn an_unsigned_limit_past_the_middle_of_its_type_is_not_a_negative_one() {
2182        // `for (unsigned char i = 0; i < 200; i++)`. Two hundred does not fit in a signed byte
2183        // and the constant is held as minus fifty six, so a distance taken at face value is
2184        // negative and reads as a loop that runs no times.
2185        let it = counted(Type::int(8), 0, 200, 1, IntPred::Ult, Flags::NUW);
2186        let found = bound(&it.func).expect("it is counted");
2187        assert_eq!(found.proven(), Some(Count::Exact(200)));
2188    }
2189
2190    #[test]
2191    fn a_walk_that_lands_on_a_not_equal_limit_exactly_is_counted() {
2192        // `while (i != 10)` counting by one, which is `while (p != end)` over an array once the
2193        // element size has been divided out. `!=` says nothing about how its operands are read,
2194        // so the promise it wants is the unsigned one and an `nsw` on its own is not enough.
2195        let it = counted(Type::int(32), 0, 10, 1, IntPred::Ne, Flags::NSW.union(Flags::NUW));
2196        let found = bound(&it.func).expect("it lands on its limit");
2197        // The step divides the distance and both are numbers, so it was checked rather than
2198        // assumed and there is nothing left over.
2199        assert_eq!(found.proven(), Some(Count::Exact(10)));
2200    }
2201
2202    #[test]
2203    fn a_counter_stepping_away_from_a_not_equal_limit_is_not_a_loop_that_runs_no_times() {
2204        // The distance is negative and an ordering test would read that as the loop never being
2205        // entered. `!=` reads it as the counter never arriving, which is an endless loop, and
2206        // answering zero for it was a real bug that the property test in `tests/scev.rs` found.
2207        let it = counted(Type::int(32), 48, 15, 1, IntPred::Ne, Flags::NSW);
2208        assert_eq!(bound(&it.func), None);
2209    }
2210
2211    #[test]
2212    fn a_counter_stepping_over_a_not_equal_limit_never_arrives_either() {
2213        // Zero, three, six, nine, twelve, and ten is never one of them. An ordering test would
2214        // have stopped at twelve.
2215        let it = counted(Type::int(32), 0, 10, 3, IntPred::Ne, Flags::NSW);
2216        assert_eq!(bound(&it.func), None);
2217    }
2218
2219    #[test]
2220    fn an_estimate_is_the_count_when_there_is_one_and_a_guess_when_there_is_not() {
2221        let counted_loop = counted(Type::int(32), 0, 7, 1, IntPred::Slt, Flags::NSW);
2222        let (cfg, loops) = analyse(&counted_loop.func);
2223        let id = loops.roots()[0];
2224        let estimate = Scev::new(&counted_loop.func, &cfg, &loops).estimate(id);
2225        assert_eq!(estimate.iterations(), 7);
2226        assert!(!estimate.is_guess());
2227
2228        // A loop this cannot count still has to answer, because the caller is deciding whether
2229        // something is worth doing rather than whether it is legal.
2230        let uncounted = counted(Type::int(32), 0, 100, 0, IntPred::Slt, Flags::NSW);
2231        let (cfg, loops) = analyse(&uncounted.func);
2232        let id = loops.roots()[0];
2233        let estimate = Scev::new(&uncounted.func, &cfg, &loops).estimate(id);
2234        assert!(estimate.is_guess());
2235        assert_eq!(estimate.iterations(), super::ASSUMED_ITERATIONS);
2236    }
2237
2238    #[test]
2239    fn a_value_the_loop_does_not_touch_is_invariant_rather_than_unknown() {
2240        let it = counted(Type::int(32), 0, 100, 1, IntPred::Slt, Flags::NSW);
2241        let (cfg, loops) = analyse(&it.func);
2242        let id = loops.roots()[0];
2243        let mut scev = Scev::new(&it.func, &cfg, &loops);
2244        // The counter's start is an `iconst` in the entry block, which is both.
2245        assert_eq!(
2246            scev.evolution(id, it.counter).chrec().expect("it evolves").base,
2247            Invariant::number(0)
2248        );
2249    }
2250
2251    #[test]
2252    fn a_back_edge_of_its_own_does_not_hide_the_counter() {
2253        // What canonicalization leaves behind. The back edge goes through a block that does nothing
2254        // but pass the increment on, so the value arriving at the header is a parameter of that
2255        // block rather than the increment itself. Reading through it is undoing a rename and not an
2256        // analysis, and without it the trip count of every loop the pipeline produces is nothing.
2257        let mut names = Interner::new();
2258        let mut func = Func::new(names.intern("f"), Signature::new());
2259        let entry = func.create_block();
2260        let header = func.create_block();
2261        let body = func.create_block();
2262        let latch = func.create_block();
2263        let exit = func.create_block();
2264        let counter = func.append_param(header, Type::int(32));
2265        let carried = func.append_param(latch, Type::int(32));
2266
2267        let start = Builder::new(&mut func, entry).iconst(Type::int(32), 0);
2268        Builder::new(&mut func, entry).jump(header, &[start]);
2269
2270        let mut build = Builder::new(&mut func, header);
2271        let limit = build.iconst(Type::int(32), 100);
2272        let test = build.icmp(IntPred::Slt, counter, limit);
2273        build.br_if(test, body, &[], exit, &[]);
2274
2275        let mut build = Builder::new(&mut func, body);
2276        let by = build.iconst(Type::int(32), 1);
2277        let next = build.binary(Opcode::Add, counter, by, Flags::NSW);
2278        build.jump(latch, &[next]);
2279
2280        Builder::new(&mut func, latch).jump(header, &[carried]);
2281        Builder::new(&mut func, exit).ret(&[]);
2282
2283        let chrec = evolution(&func, counter).chrec().expect("the counter still evolves");
2284        assert_eq!(chrec.base, Invariant::number(0));
2285        assert_eq!(chrec.step, Invariant::number(1));
2286        let (count, _) = bound(&func).expect("it is still counted").parts();
2287        assert_eq!(count, Count::Exact(100));
2288    }
2289
2290    #[test]
2291    fn every_assumption_says_what_it_is_in_a_line() {
2292        let it = counted(Type::int(8), 0, 100, 1, IntPred::Ult, Flags::NONE);
2293        let found = bound(&it.func).expect("it is counted");
2294        for assumption in found.assumptions() {
2295            let line = assumption.describe();
2296            assert!(!line.is_empty());
2297            assert!(!line.contains('\n'), "an assumption is one line: {line}");
2298        }
2299    }
2300}