Skip to main content

rucc_regalloc/
check.rs

1//! The allocation checker: whether an assignment is one the machine can actually run.
2//!
3//! Design: `spec/10-backend.md` section 10.4, which asks for this in debug and CI builds.
4//!
5//! A register allocator is the pass whose bugs are hardest to find from the outside. It does not
6//! change what a program means, so a wrong allocation compiles, links and runs, and then produces
7//! the wrong number in one function of one program under one register pressure. The stack trace
8//! points at the arithmetic, the arithmetic is right, and the value it read was overwritten four
9//! instructions earlier by something unrelated. A checker turns all of that into an assertion at
10//! the point the mistake was made, naming the two values and the register they were both put in.
11//!
12//! # What it asks
13//!
14//! Five questions, and they are the whole of what an assignment has to get right.
15//!
16//! Every value the function reads is written first, on every path that reaches the read. Every
17//! value the function uses has somewhere to live. Two values that are both wanted at the same
18//! point are not in the same register or the same slot. Nothing is sitting in a register that an
19//! instruction insists on for itself, because that register belongs to the instruction for as long
20//! as it runs. A value an instruction can only read from memory is in memory.
21//!
22//! The first of those is not about the allocation at all, since the value would be read before it
23//! was written whatever register it went to. It is asked here because this is where the answer is
24//! already computed: a value read before it is written is a value live on the way into the entry
25//! block, and the liveness the allocator needs anyway says which those are. A function that gets
26//! this wrong is one the allocator will happily place, and what comes out reads a stack slot
27//! nothing ever stored to.
28//!
29//! # What it does not ask
30//!
31//! Whether the allocation is any good. A function with every value on the stack passes, and so it
32//! should: it is slow and it is correct, and this is the thing that says which of the two a
33//! problem is. Quality is what the numbers in `spec/14-target-ladder.md` are for.
34//!
35//! It also does not read the rewrite. It runs on the assignment, before [`crate::rewrite`] has
36//! touched the function, because the assignment is the decision and the rewrite is a
37//! transcription of it. A rewrite that transcribes a good decision badly is a different bug, and
38//! [`crate::trace`] is the checker that catches it by following each value from the instruction
39//! that wrote it to the instructions that read it.
40//!
41//! # Why it repeats work
42//!
43//! The two address instructions are worked out again here rather than borrowed from
44//! [`crate::assign`], and that is deliberate. A checker that shares its reasoning with the thing
45//! it checks agrees with it about everything, including the mistakes, and the one bug it can never
46//! find is the one in the code they share. Fifteen lines is a cheap price for a second opinion.
47//!
48//! It is allowed to be slower than the allocator, but not by a power of the function's size. Both
49//! questions that compare values with something are asked one place at a time: two values can
50//! only be wrong together if they are in the same place, and a value can only be in the way of an
51//! instruction if it is in the register the instruction wants. Asking them over every value
52//! instead was the plain product of the values and the constrained operands, which is nothing on
53//! a function a person wrote and minutes on one the safety instrumentation grew. libjpeg-turbo's
54//! Huffman encoder took over five minutes to build with the safety checks under `-Zverify-each`
55//! that way, against under three seconds without it.
56
57use std::collections::{HashMap, HashSet};
58use std::fmt;
59
60use rucc_mir::{Constraint, Func, Inst, Reg, Role};
61use rucc_target::{PhysReg, RegClass};
62
63use crate::assign::{Assignment, Place};
64use crate::live::{Area, Live, Range};
65use crate::order::{Order, Point};
66
67/// One thing wrong with an allocation.
68#[derive(Debug, Clone, Copy, PartialEq, Eq)]
69pub enum Problem {
70    /// A value the function reads or writes was given no place at all.
71    Nowhere {
72        /// The value with nowhere to be.
73        reg: Reg,
74    },
75    /// Two values that are both live somewhere were put in the same place, so whichever is written
76    /// second destroys the other.
77    Shared {
78        /// The value that was there first.
79        first: Reg,
80        /// The value that was put on top of it.
81        second: Reg,
82        /// The place they were both given.
83        place: Place,
84    },
85    /// A value was left in a register an instruction claims for itself, over the instruction that
86    /// claims it, so the moves around that instruction overwrite the value.
87    InTheWay {
88        /// The value in the way.
89        reg: Reg,
90        /// The register the instruction insists on.
91        at: PhysReg,
92        /// The instruction that insists on it.
93        inst: Inst,
94    },
95    /// A value an instruction can only read from memory was put in a register.
96    NotOnTheStack {
97        /// The value that has to be in memory.
98        reg: Reg,
99        /// The instruction that says so.
100        inst: Inst,
101    },
102    /// A value is read on some path from the entry block without anything on that path having
103    /// written it, so what the instruction reading it gets is whatever was left there.
104    NeverWritten {
105        /// The value nothing writes.
106        reg: Reg,
107    },
108}
109
110impl fmt::Display for Problem {
111    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
112        match self {
113            Problem::Nowhere { reg } => write!(f, "{} has nowhere to live", name(*reg)),
114            Problem::Shared { first, second, place } => {
115                let (first, second) = (name(*first), name(*second));
116                write!(f, "{first} and {second} are both live and both in {}", place_name(*place))
117            }
118            Problem::InTheWay { reg, at, inst } => {
119                let reg = name(*reg);
120                let inst = inst.index();
121                write!(f, "{reg} is in register {}, which instruction {inst} wants", at.number())
122            }
123            Problem::NotOnTheStack { reg, inst } => {
124                let reg = name(*reg);
125                write!(f, "{reg} is not on the stack, and instruction {} needs it", inst.index())
126            }
127            Problem::NeverWritten { reg } => {
128                write!(f, "{} is read before anything writes it", name(*reg))
129            }
130        }
131    }
132}
133
134/// Everything wrong with an allocation, in an order a person can read.
135///
136/// An empty answer is the one every allocation is supposed to give. Anything else is a compiler
137/// bug rather than a program the compiler cannot handle, which is why [`crate::run`] asserts on it
138/// instead of reporting it as a diagnostic.
139///
140/// # Panics
141///
142/// Panics on a function with two billion virtual registers in it, which is a function no machine
143/// has the memory to hold.
144#[must_use]
145pub fn check(func: &Func, order: &Order, live: &Live, assignment: &Assignment) -> Vec<Problem> {
146    let mut problems = Vec::new();
147    // What arrives live in the entry block is what the function reads without writing, since
148    // nothing runs in front of the entry block to have written it.
149    if let Some(entry) = func.entry() {
150        for reg in live.live_in(entry) {
151            problems.push(Problem::NeverWritten { reg });
152        }
153    }
154    let reuses = reuses(func, order);
155    let mut values = Vec::new();
156    for (number, reuse) in reuses.iter().enumerate() {
157        let reg = Reg::virtual_reg(u32::try_from(number).expect("a register number"));
158        let (Some(mut area), Some(class)) = (live.area(reg), func.class_of(reg)) else {
159            continue;
160        };
161        let Some(place) = assignment.place(reg) else {
162            problems.push(Problem::Nowhere { reg });
163            continue;
164        };
165        // A two address instruction writes its answer into a register it read, so the answer is
166        // really in that register from the moment the instruction starts and not from the moment
167        // it ends. Reading its area any other way lets it share the register with something the
168        // same instruction is still reading.
169        if let Some(reuse) = reuse {
170            area = area.with(reuse.at);
171        }
172        values.push(Value { reg, class, range: area.hull(), area, place });
173    }
174    overlaps(&values, &reuses, live, &mut problems);
175    instructions(func, order, assignment, &values, &reuses, &mut problems);
176    problems
177}
178
179/// Everything wrong with an allocation, as an assertion message.
180#[must_use]
181pub fn report(problems: &[Problem]) -> String {
182    let places = if problems.len() == 1 { "place" } else { "places" };
183    let mut report = format!("the allocation is wrong in {} {places}", problems.len());
184    for problem in problems {
185        report.push_str("\n  ");
186        report.push_str(&problem.to_string());
187    }
188    report
189}
190
191/// One value, where it is wanted and where it was put.
192#[derive(Debug, Clone, Copy)]
193struct Value<'a> {
194    reg: Reg,
195    class: RegClass,
196    /// The interval around the area, which is what the sweep below reads.
197    range: Range,
198    /// Everywhere the value is really live, which is what says whether sharing a place with
199    /// another value is a mistake.
200    area: Area<'a>,
201    place: Place,
202}
203
204/// A value written into the register another operand of the same instruction was read from.
205#[derive(Debug, Clone, Copy)]
206struct Reuse {
207    source: Reg,
208    at: Point,
209}
210
211/// Looks for two values that are both live somewhere and were put in the same place.
212///
213/// A sweep in the order the values start, holding the ones whose interval still reaches this one,
214/// so the pairs it compares are the pairs that can be wrong rather than all of them. The interval
215/// is generous, so a pair that survives the sweep is then asked whether the areas inside those
216/// intervals really meet.
217///
218/// The sweep runs once for each place rather than once over everything. Two values in different
219/// places are never wrong together however much they overlap, and in a long function nearly every
220/// pair the single sweep held was such a pair, so the list it carried grew with the function and
221/// every value was compared with all of it. What is found is put back in the order the single
222/// sweep would have found it, so a report reads the same either way.
223fn overlaps(
224    values: &[Value<'_>],
225    reuses: &[Option<Reuse>],
226    live: &Live,
227    problems: &mut Vec<Problem>,
228) {
229    let mut sorted = values.to_vec();
230    sorted.sort_by_key(|value| (value.range.start, value.reg));
231    let mut places: HashMap<(Place, Option<RegClass>), Vec<usize>> = HashMap::new();
232    for (position, value) in sorted.iter().enumerate() {
233        places.entry(place_of(*value)).or_default().push(position);
234    }
235    let mut found = Vec::new();
236    for positions in places.values() {
237        let mut active: Vec<usize> = Vec::new();
238        for &position in positions {
239            let value = sorted[position];
240            active.retain(|&held| sorted[held].range.end >= value.range.start);
241            for &held in &active {
242                let held_value = sorted[held];
243                if !held_value.area.overlaps(value.area)
244                    || coalesced(held_value, value, reuses, live)
245                {
246                    continue;
247                }
248                let problem = Problem::Shared {
249                    first: held_value.reg,
250                    second: value.reg,
251                    place: value.place,
252                };
253                found.push((position, held, problem));
254            }
255            active.push(position);
256        }
257    }
258    // The single sweep found problems in the order the later value started, and for one value in
259    // the order the earlier ones did, which is the order of the two positions.
260    found.sort_by_key(|&(position, held, _)| (position, held));
261    problems.extend(found.into_iter().map(|(_, _, problem)| problem));
262}
263
264/// The place a value was put in, as the thing two values have to share to be in each other's way.
265///
266/// Two registers of different classes are different registers even when they are the same number,
267/// which is what a class is. Two slots are the same slot whatever is in them, because a frame is
268/// one piece of memory, so a slot's class is left out.
269fn place_of(value: Value<'_>) -> (Place, Option<RegClass>) {
270    match value.place {
271        Place::Reg(_) => (value.place, Some(value.class)),
272        Place::Slot(_) => (value.place, None),
273    }
274}
275
276/// Whether one of the two is the answer a two address instruction wrote into the register it read
277/// the other from, which is the one overlap that is not a mistake.
278///
279/// It only holds when the two are not live at the same time anywhere, asked of the areas liveness
280/// worked out, without the extra point the reuse adds. A value read again afterwards needs its
281/// register afterwards, so writing over it is the plain bug this whole file exists to find. And a
282/// value that covers the reuse point on its own is one that was already live on the way in. That
283/// is what a loop carrying its own answer round looks like: written at the bottom and read by the
284/// next turn. Such a value is wanted where the instruction reads as well as after it, so it is
285/// genuinely on top of the one it reuses and no excuse at the one instruction they share makes
286/// them fit in a single register.
287fn coalesced(first: Value<'_>, second: Value<'_>, reuses: &[Option<Reuse>], live: &Live) -> bool {
288    let pair = |source: Value<'_>, dest: Value<'_>| {
289        let Some(reuse) = reuses[index(dest.reg)] else { return false };
290        reuse.source == source.reg && crate::assign::apart(live, source.reg, dest.reg)
291    };
292    pair(first, second) || pair(second, first)
293}
294
295/// Looks for a value in a register an instruction wants, and for a value that had to be in memory
296/// and is not.
297fn instructions(
298    func: &Func,
299    order: &Order,
300    assignment: &Assignment,
301    values: &[Value<'_>],
302    reuses: &[Option<Reuse>],
303    problems: &mut Vec<Problem>,
304) {
305    // The values in each register, in the order they were given, since a value anywhere else can
306    // never be in the way of an instruction that wants that register.
307    let mut held: HashMap<PhysReg, Vec<Value<'_>>> = HashMap::new();
308    // The instructions each value is put away around, which may destroy its register and nothing
309    // else of what it insists on.
310    let saved: HashSet<(Reg, Inst)> =
311        assignment.saves().iter().map(|save| (save.reg, save.inst)).collect();
312    for value in values {
313        if let Place::Reg(at) = value.place {
314            held.entry(at).or_default().push(*value);
315        }
316    }
317    for block in func.blocks() {
318        for inst in func.insts(block) {
319            for operand in &func[func[inst].operands] {
320                if operand.constraint == Constraint::Stack
321                    && matches!(assignment.place(operand.reg), Some(Place::Reg(_)))
322                {
323                    problems.push(Problem::NotOnTheStack { reg: operand.reg, inst });
324                }
325                // A physical register an operand names outright is claimed exactly as firmly as
326                // one a constraint asks for, since nothing before allocation writes one except an
327                // instruction that has no choice.
328                let at = match operand.constraint {
329                    Constraint::Fixed(at) => Some(at),
330                    _ => operand.reg.phys(),
331                };
332                let Some(at) = at else { continue };
333                let Some(here) = held.get(&at) else { continue };
334                let early = order.early(inst);
335                let point = if operand.role == Role::Def { order.late(inst) } else { early };
336                for value in here {
337                    let mine = value.reg == operand.reg
338                        || reuses[index(value.reg)].is_some_and(|reuse| {
339                            reuse.source == operand.reg
340                                && reuse.at == early
341                                && value.place == Place::Reg(at)
342                        });
343                    if mine || value.class != operand.class {
344                        continue;
345                    }
346                    // A write of only the top of the register, which a value that fits under it
347                    // survives. See [`Constraint::Above`].
348                    let width = func.width(value.reg);
349                    let under = |above| width.is_some_and(|width| width <= above);
350                    if matches!(operand.constraint, Constraint::Above(above) if under(above)) {
351                        continue;
352                    }
353                    // A register the instruction destroys and hands to no value, with the value in
354                    // it put away in front of the instruction and brought back behind it. Where the
355                    // instruction reads is still the value's, so this is only the write.
356                    let destroyed = operand.role == Role::Def && operand.reg.phys().is_some();
357                    if destroyed && saved.contains(&(value.reg, inst)) {
358                        continue;
359                    }
360                    // The interval around a value covers blocks the value never reaches, so what
361                    // decides this is the area inside it, which says whether the value is live at
362                    // this point rather than whether the point is between its ends.
363                    // tamnd/rucc#982.
364                    if value.place == Place::Reg(at) && value.area.covers(point) {
365                        problems.push(Problem::InTheWay { reg: value.reg, at, inst });
366                    }
367                }
368            }
369        }
370    }
371}
372
373/// The value each two address instruction reuses, by the virtual register it writes.
374fn reuses(func: &Func, order: &Order) -> Vec<Option<Reuse>> {
375    let mut reuses = vec![None; func.vregs()];
376    for block in func.blocks() {
377        for inst in func.insts(block) {
378            let operands = &func[func[inst].operands];
379            for operand in operands {
380                let Constraint::Reuse(other) = operand.constraint else { continue };
381                let number = operand.reg.number().and_then(|number| usize::try_from(number).ok());
382                let Some(number) = number else { continue };
383                let source = operands[usize::from(other)].reg;
384                reuses[number] = Some(Reuse { source, at: order.early(inst) });
385            }
386        }
387    }
388    reuses
389}
390
391/// A virtual register's number as a table index, and zero for a physical one, which never has an
392/// entry of its own and is never what a reuse writes.
393fn index(reg: Reg) -> usize {
394    reg.number().and_then(|number| usize::try_from(number).ok()).unwrap_or(0)
395}
396
397/// What a value is called in a report.
398fn name(reg: Reg) -> String {
399    match reg.number() {
400        Some(number) => format!("%{number}"),
401        None => format!("register {}", reg.phys().expect("a physical register").number()),
402    }
403}
404
405/// What a place is called in a report, without the target's name for it, since this crate holds
406/// nothing of any target.
407fn place_name(place: Place) -> String {
408    match place {
409        Place::Reg(at) => format!("register {}", at.number()),
410        Place::Slot(slot) => format!("slot {slot}"),
411    }
412}
413
414#[cfg(test)]
415mod tests {
416    use rucc_base::Interner;
417    use rucc_mir::{BlockCall, Opcode, Operand};
418    use rucc_target::x86_64::{GPR, RAX, RCX, RDX, SYSV};
419
420    use super::*;
421    use crate::assign::{Env, assign};
422
423    /// The x86-64 environment, with the last three of the allocation order held back as scratch.
424    fn env() -> Env {
425        let (order, scratch) = SYSV.int_order.split_at(SYSV.int_order.len() - 3);
426        Env::new().with(GPR, order, scratch)
427    }
428
429    /// What the checker says about the allocation the single pass allocator works out, which is
430    /// supposed to be nothing at all.
431    fn allocated(func: &Func) -> Vec<String> {
432        let order = Order::of(func);
433        let live = Live::of(func, &order);
434        let assignment = assign(func, &order, &live, &env());
435        said(func, &order, &live, &assignment)
436    }
437
438    /// What the checker says about an allocation somebody wrote by hand.
439    fn said(func: &Func, order: &Order, live: &Live, assignment: &Assignment) -> Vec<String> {
440        check(func, order, live, assignment).iter().map(ToString::to_string).collect()
441    }
442
443    /// The order and the liveness of a function, which every hand written case needs both of.
444    fn read(func: &Func) -> (Order, Live) {
445        let order = Order::of(func);
446        let live = Live::of(func, &order);
447        (order, live)
448    }
449
450    /// A value in `rcx` over an instruction that writes `rcx` from its eighth byte up.
451    fn under_the_top(width: u32) -> Vec<String> {
452        let mut names = Interner::new();
453        let mut func = Func::new(names.intern("f"));
454        let opcode = Opcode::new(names.intern("x64.nop"));
455        let block = func.create_block();
456        let held = func.new_vreg(GPR);
457        func.set_width(held, width);
458        func.build(block, opcode).def(held, GPR).finish();
459        func.build(block, opcode)
460            .operand(Operand::write(Reg::physical(RCX), GPR).with(Constraint::Above(8)))
461            .finish();
462        func.build(block, opcode).uses(held, GPR).finish();
463        let (order, live) = read(&func);
464        let mut assignment = Assignment::empty(func.vregs());
465        assignment.put(held, Place::Reg(RCX));
466        said(&func, &order, &live, &assignment)
467    }
468
469    #[test]
470    fn a_value_under_the_part_an_instruction_writes_is_not_in_its_way() {
471        assert!(under_the_top(8).is_empty(), "{:?}", under_the_top(8));
472        let wide = under_the_top(16);
473        assert!(wide.len() == 1 && wide[0].contains("%0"), "{wide:?}");
474        assert_eq!(under_the_top(0), wide);
475    }
476
477    #[test]
478    fn an_allocation_the_allocator_worked_out_has_nothing_wrong_with_it() {
479        let mut names = Interner::new();
480        let mut func = Func::new(names.intern("f"));
481        let opcode = Opcode::new(names.intern("x64.nop"));
482        let block = func.create_block();
483        let first = func.new_vreg(GPR);
484        let second = func.new_vreg(GPR);
485        func.build(block, opcode).def(first, GPR).finish();
486        func.build(block, opcode).def(second, GPR).finish();
487        func.build(block, opcode).uses(first, GPR).uses(second, GPR).finish();
488
489        assert_eq!(allocated(&func), Vec::<String>::new());
490    }
491
492    #[test]
493    fn a_value_with_nowhere_to_live_is_found() {
494        let mut names = Interner::new();
495        let mut func = Func::new(names.intern("f"));
496        let opcode = Opcode::new(names.intern("x64.nop"));
497        let block = func.create_block();
498        let only = func.new_vreg(GPR);
499        func.build(block, opcode).def(only, GPR).finish();
500        func.build(block, opcode).uses(only, GPR).finish();
501
502        let (order, live) = read(&func);
503        let assignment = Assignment::empty(func.vregs());
504
505        assert_eq!(said(&func, &order, &live, &assignment), ["%0 has nowhere to live"]);
506    }
507
508    #[test]
509    fn a_value_read_before_anything_writes_it_is_found() {
510        let mut names = Interner::new();
511        let mut func = Func::new(names.intern("f"));
512        let opcode = Opcode::new(names.intern("x64.nop"));
513        let block = func.create_block();
514        let never = func.new_vreg(GPR);
515        func.build(block, opcode).uses(never, GPR).finish();
516
517        let (order, live) = read(&func);
518        let mut assignment = Assignment::empty(func.vregs());
519        assignment.put(never, Place::Reg(RAX));
520
521        assert_eq!(
522            said(&func, &order, &live, &assignment),
523            ["%0 is read before anything writes it"]
524        );
525    }
526
527    #[test]
528    fn two_values_that_are_both_wanted_and_share_a_register_are_found() {
529        let mut names = Interner::new();
530        let mut func = Func::new(names.intern("f"));
531        let opcode = Opcode::new(names.intern("x64.nop"));
532        let block = func.create_block();
533        let first = func.new_vreg(GPR);
534        let second = func.new_vreg(GPR);
535        func.build(block, opcode).def(first, GPR).finish();
536        func.build(block, opcode).def(second, GPR).finish();
537        func.build(block, opcode).uses(first, GPR).uses(second, GPR).finish();
538
539        let (order, live) = read(&func);
540        let mut assignment = Assignment::empty(func.vregs());
541        assignment.put(first, Place::Reg(RAX));
542        assignment.put(second, Place::Reg(RAX));
543
544        let said = said(&func, &order, &live, &assignment);
545        assert_eq!(said, ["%0 and %1 are both live and both in register 0"]);
546    }
547
548    #[test]
549    fn values_sharing_different_places_are_reported_in_the_order_they_start() {
550        // Three pairs, each pair in a place of its own, defined so that the order the later value
551        // of each pair starts in is not the order the places would come out of a table in. The
552        // report has to read the same as it did when one sweep went over every value at once.
553        let mut names = Interner::new();
554        let mut func = Func::new(names.intern("f"));
555        let opcode = Opcode::new(names.intern("x64.nop"));
556        let block = func.create_block();
557        let regs: Vec<Reg> = (0..6).map(|_| func.new_vreg(GPR)).collect();
558        for reg in &regs {
559            func.build(block, opcode).def(*reg, GPR).finish();
560        }
561        let mut last = func.build(block, opcode);
562        for reg in &regs {
563            last = last.uses(*reg, GPR);
564        }
565        last.finish();
566
567        let (order, live) = read(&func);
568        let mut assignment = Assignment::empty(func.vregs());
569        let places = [Place::Slot(0), Place::Reg(RCX), Place::Reg(RAX)];
570        for (number, reg) in regs.iter().enumerate() {
571            assignment.put(*reg, places[number % 3]);
572        }
573
574        let rcx = RCX.number();
575        let rax = RAX.number();
576        assert_eq!(
577            said(&func, &order, &live, &assignment),
578            [
579                "%0 and %3 are both live and both in slot 0".to_string(),
580                format!("%1 and %4 are both live and both in register {rcx}"),
581                format!("%2 and %5 are both live and both in register {rax}"),
582            ]
583        );
584    }
585
586    #[test]
587    fn a_value_that_lives_in_a_hole_of_another_may_share_its_register() {
588        let mut names = Interner::new();
589        let mut func = Func::new(names.intern("f"));
590        let opcode = Opcode::new(names.intern("x64.nop"));
591        let entry = func.create_block();
592        let arm = func.create_block();
593        let tail = func.create_block();
594        let across = func.new_vreg(GPR);
595        let inside = func.new_vreg(GPR);
596        func.build(entry, opcode).def(across, GPR).finish();
597        *func.succs_mut(entry) = vec![BlockCall::to(arm), BlockCall::to(tail)];
598        func.build(arm, opcode).def(inside, GPR).finish();
599        func.build(arm, opcode).uses(inside, GPR).finish();
600        func.build(tail, opcode).uses(across, GPR).finish();
601
602        let (order, live) = read(&func);
603        let mut assignment = Assignment::empty(func.vregs());
604        assignment.put(across, Place::Reg(RAX));
605        assignment.put(inside, Place::Reg(RAX));
606
607        // The arm is written between the two blocks the first value is live in and is a block that
608        // value's own path never goes through, so the second is not sitting on top of it and the
609        // interval around the first saying so is not what decides this. A checker that read the
610        // intervals would call every register the allocator has learned to share a value written
611        // over another. tamnd/rucc#982.
612        assert_eq!(said(&func, &order, &live, &assignment), Vec::<String>::new());
613    }
614
615    #[test]
616    fn two_values_that_are_both_wanted_and_share_a_slot_are_found() {
617        let mut names = Interner::new();
618        let mut func = Func::new(names.intern("f"));
619        let opcode = Opcode::new(names.intern("x64.nop"));
620        let block = func.create_block();
621        let first = func.new_vreg(GPR);
622        let second = func.new_vreg(GPR);
623        func.build(block, opcode).def(first, GPR).finish();
624        func.build(block, opcode).def(second, GPR).finish();
625        func.build(block, opcode).uses(first, GPR).uses(second, GPR).finish();
626
627        let (order, live) = read(&func);
628        let mut assignment = Assignment::empty(func.vregs());
629        let slot = assignment.take_slot(GPR);
630        assignment.put(first, Place::Slot(slot));
631        assignment.put(second, Place::Slot(slot));
632
633        let said = said(&func, &order, &live, &assignment);
634        assert_eq!(said, ["%0 and %1 are both live and both in slot 0"]);
635    }
636
637    #[test]
638    fn two_values_that_are_never_both_wanted_may_share_anything() {
639        let mut names = Interner::new();
640        let mut func = Func::new(names.intern("f"));
641        let opcode = Opcode::new(names.intern("x64.nop"));
642        let block = func.create_block();
643        let first = func.new_vreg(GPR);
644        let second = func.new_vreg(GPR);
645        func.build(block, opcode).def(first, GPR).finish();
646        func.build(block, opcode).uses(first, GPR).finish();
647        func.build(block, opcode).def(second, GPR).finish();
648        func.build(block, opcode).uses(second, GPR).finish();
649
650        let (order, live) = read(&func);
651        let mut assignment = Assignment::empty(func.vregs());
652        assignment.put(first, Place::Reg(RAX));
653        assignment.put(second, Place::Reg(RAX));
654
655        assert_eq!(said(&func, &order, &live, &assignment), Vec::<String>::new());
656    }
657
658    #[test]
659    fn a_value_left_in_a_register_an_instruction_wants_is_found() {
660        let mut names = Interner::new();
661        let mut func = Func::new(names.intern("f"));
662        let nop = Opcode::new(names.intern("x64.nop"));
663        let divide = Opcode::new(names.intern("x64.idiv"));
664        let block = func.create_block();
665        let held = func.new_vreg(GPR);
666        let dividend = func.new_vreg(GPR);
667        func.build(block, nop).def(held, GPR).finish();
668        func.build(block, nop).def(dividend, GPR).finish();
669        // The division reads its dividend out of one register and no other, so anything still
670        // wanted afterwards has to be somewhere else while it runs.
671        func.build(block, divide)
672            .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
673            .finish();
674        func.build(block, nop).uses(held, GPR).finish();
675
676        let (order, live) = read(&func);
677        let mut assignment = Assignment::empty(func.vregs());
678        assignment.put(held, Place::Reg(RAX));
679        assignment.put(dividend, Place::Reg(RCX));
680
681        let said = said(&func, &order, &live, &assignment);
682        assert_eq!(said, ["%0 is in register 0, which instruction 2 wants"]);
683    }
684
685    #[test]
686    fn the_value_an_instruction_wants_a_register_for_may_be_in_it_already() {
687        let mut names = Interner::new();
688        let mut func = Func::new(names.intern("f"));
689        let nop = Opcode::new(names.intern("x64.nop"));
690        let divide = Opcode::new(names.intern("x64.idiv"));
691        let block = func.create_block();
692        let dividend = func.new_vreg(GPR);
693        func.build(block, nop).def(dividend, GPR).finish();
694        func.build(block, divide)
695            .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
696            .finish();
697
698        let (order, live) = read(&func);
699        let mut assignment = Assignment::empty(func.vregs());
700        assignment.put(dividend, Place::Reg(RAX));
701
702        // Being in the register the instruction wanted is the best answer, not a problem, and the
703        // rewrite writes no move at all for it.
704        assert_eq!(said(&func, &order, &live, &assignment), Vec::<String>::new());
705    }
706
707    #[test]
708    fn a_value_that_can_only_be_read_from_memory_and_is_in_a_register_is_found() {
709        let mut names = Interner::new();
710        let mut func = Func::new(names.intern("f"));
711        let nop = Opcode::new(names.intern("x64.nop"));
712        let wide = Opcode::new(names.intern("x64.wide"));
713        let block = func.create_block();
714        let only = func.new_vreg(GPR);
715        func.build(block, nop).def(only, GPR).finish();
716        func.build(block, wide).operand(Operand::read(only, GPR).with(Constraint::Stack)).finish();
717
718        let (order, live) = read(&func);
719        let mut assignment = Assignment::empty(func.vregs());
720        assignment.put(only, Place::Reg(RAX));
721
722        let said = said(&func, &order, &live, &assignment);
723        assert_eq!(said, ["%0 is not on the stack, and instruction 1 needs it"]);
724    }
725
726    #[test]
727    fn a_two_address_instruction_may_write_the_register_it_read_a_finished_value_from() {
728        let mut names = Interner::new();
729        let mut func = Func::new(names.intern("f"));
730        let nop = Opcode::new(names.intern("x64.nop"));
731        let add = Opcode::new(names.intern("x64.add"));
732        let block = func.create_block();
733        let left = func.new_vreg(GPR);
734        let right = func.new_vreg(GPR);
735        let sum = func.new_vreg(GPR);
736        func.build(block, nop).def(left, GPR).finish();
737        func.build(block, nop).def(right, GPR).finish();
738        func.build(block, add)
739            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
740            .uses(left, GPR)
741            .uses(right, GPR)
742            .finish();
743        func.build(block, nop).uses(sum, GPR).finish();
744
745        let (order, live) = read(&func);
746        let mut assignment = Assignment::empty(func.vregs());
747        assignment.put(left, Place::Reg(RAX));
748        assignment.put(right, Place::Reg(RCX));
749        assignment.put(sum, Place::Reg(RAX));
750
751        // The left operand is finished with at the addition, so the sum takes its register and
752        // the addition is the one instruction rather than a move and an instruction.
753        assert_eq!(said(&func, &order, &live, &assignment), Vec::<String>::new());
754    }
755
756    #[test]
757    fn a_two_address_instruction_may_not_write_over_a_value_wanted_afterwards() {
758        let mut names = Interner::new();
759        let mut func = Func::new(names.intern("f"));
760        let nop = Opcode::new(names.intern("x64.nop"));
761        let add = Opcode::new(names.intern("x64.add"));
762        let block = func.create_block();
763        let left = func.new_vreg(GPR);
764        let right = func.new_vreg(GPR);
765        let sum = func.new_vreg(GPR);
766        func.build(block, nop).def(left, GPR).finish();
767        func.build(block, nop).def(right, GPR).finish();
768        func.build(block, add)
769            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
770            .uses(left, GPR)
771            .uses(right, GPR)
772            .finish();
773        func.build(block, nop).uses(sum, GPR).uses(left, GPR).finish();
774
775        let (order, live) = read(&func);
776        let mut assignment = Assignment::empty(func.vregs());
777        assignment.put(left, Place::Reg(RAX));
778        assignment.put(right, Place::Reg(RCX));
779        assignment.put(sum, Place::Reg(RAX));
780
781        // The left operand is read again after the addition, so the addition may not have its
782        // register even though it is the one the addition reads.
783        let said = said(&func, &order, &live, &assignment);
784        assert_eq!(said, ["%0 and %2 are both live and both in register 0"]);
785    }
786
787    #[test]
788    fn a_two_address_instruction_may_not_write_the_register_it_reads_its_other_operand_from() {
789        let mut names = Interner::new();
790        let mut func = Func::new(names.intern("f"));
791        let nop = Opcode::new(names.intern("x64.nop"));
792        let add = Opcode::new(names.intern("x64.add"));
793        let block = func.create_block();
794        let left = func.new_vreg(GPR);
795        let right = func.new_vreg(GPR);
796        let sum = func.new_vreg(GPR);
797        func.build(block, nop).def(left, GPR).finish();
798        func.build(block, nop).def(right, GPR).finish();
799        func.build(block, add)
800            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
801            .uses(left, GPR)
802            .uses(right, GPR)
803            .finish();
804        func.build(block, nop).uses(sum, GPR).finish();
805
806        let (order, live) = read(&func);
807        let mut assignment = Assignment::empty(func.vregs());
808        assignment.put(left, Place::Reg(RAX));
809        assignment.put(right, Place::Reg(RCX));
810        assignment.put(sum, Place::Reg(RCX));
811
812        // Copying the left operand into the sum's register would destroy the right operand before
813        // the addition has read it, even though both are finished with at the addition.
814        let said = said(&func, &order, &live, &assignment);
815        assert_eq!(said, ["%1 and %2 are both live and both in register 1"]);
816    }
817
818    #[test]
819    fn a_two_address_instruction_may_not_write_the_register_it_read_over_its_own_last_answer() {
820        let mut names = Interner::new();
821        let mut func = Func::new(names.intern("f"));
822        let nop = Opcode::new(names.intern("x64.nop"));
823        let add = Opcode::new(names.intern("x64.add"));
824        let head = func.create_block();
825        let latch = func.create_block();
826        let out = func.create_block();
827        let source = func.new_vreg(GPR);
828        let carried = func.new_vreg(GPR);
829        func.build(head, nop).def(source, GPR).finish();
830        func.build(head, nop).def(carried, GPR).finish();
831        *func.succs_mut(head) = vec![BlockCall::to(latch)];
832        func.build(latch, add)
833            .operand(Operand::write(carried, GPR).with(Constraint::Reuse(1)))
834            .uses(source, GPR)
835            .uses(carried, GPR)
836            .finish();
837        *func.succs_mut(latch) = vec![BlockCall::to(head), BlockCall::to(out)];
838        func.build(out, nop).uses(carried, GPR).finish();
839
840        let (order, live) = read(&func);
841        let mut assignment = Assignment::empty(func.vregs());
842        assignment.put(source, Place::Reg(RAX));
843        assignment.put(carried, Place::Reg(RAX));
844
845        // The source is finished with at the addition, which is what would normally let the answer
846        // have its register. It does not here, because the answer is the one the last turn round
847        // the loop wrote and the addition reads it too, so both are wanted where it reads.
848        let said = said(&func, &order, &live, &assignment);
849        assert_eq!(said, ["%0 and %1 are both live and both in register 0"]);
850    }
851
852    #[test]
853    fn a_two_address_answer_with_a_hole_in_front_of_it_still_may_not_take_the_other_operand() {
854        let mut names = Interner::new();
855        let mut func = Func::new(names.intern("f"));
856        let nop = Opcode::new(names.intern("x64.nop"));
857        let add = Opcode::new(names.intern("x64.add"));
858        let entry = func.create_block();
859        let head = func.create_block();
860        let arm = func.create_block();
861        let latch = func.create_block();
862        let out = func.create_block();
863        let seed = func.new_vreg(GPR);
864        let sum = func.new_vreg(GPR);
865        let inside = func.new_vreg(GPR);
866        let loaded = func.new_vreg(GPR);
867        func.build(entry, nop).def(seed, GPR).finish();
868        func.build(entry, nop).def(sum, GPR).finish();
869        *func.succs_mut(entry) = vec![BlockCall::to(head)];
870        func.build(head, nop).uses(sum, GPR).finish();
871        *func.succs_mut(head) = vec![BlockCall::to(arm), BlockCall::to(latch)];
872        func.build(arm, nop).def(inside, GPR).finish();
873        func.build(arm, nop).uses(inside, GPR).finish();
874        *func.succs_mut(arm) = vec![BlockCall::to(out)];
875        func.build(latch, nop).def(loaded, GPR).finish();
876        func.build(latch, add)
877            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
878            .uses(seed, GPR)
879            .uses(loaded, GPR)
880            .finish();
881        *func.succs_mut(latch) = vec![BlockCall::to(head), BlockCall::to(out)];
882
883        let (order, live) = read(&func);
884        let mut assignment = Assignment::empty(func.vregs());
885        assignment.put(seed, Place::Reg(RCX));
886        assignment.put(sum, Place::Reg(RAX));
887        assignment.put(inside, Place::Reg(RDX));
888        assignment.put(loaded, Place::Reg(RAX));
889
890        // The answer is live in the entry and the head too, so the piece the addition writes is not
891        // the first one and the arm in between is a hole. Copying the left operand into the answer's
892        // register still destroys the right operand before the addition reads it, and a checker that
893        // added the extra point to the first piece rather than the piece the addition writes saw
894        // nothing wrong with any of it. tamnd/rucc#982.
895        let said = said(&func, &order, &live, &assignment);
896        assert_eq!(said, ["%1 and %3 are both live and both in register 0"]);
897    }
898
899    #[test]
900    fn a_report_names_every_problem() {
901        let mut names = Interner::new();
902        let mut func = Func::new(names.intern("f"));
903        let opcode = Opcode::new(names.intern("x64.nop"));
904        let block = func.create_block();
905        let first = func.new_vreg(GPR);
906        let second = func.new_vreg(GPR);
907        func.build(block, opcode).def(first, GPR).finish();
908        func.build(block, opcode).def(second, GPR).finish();
909        func.build(block, opcode).uses(first, GPR).uses(second, GPR).finish();
910
911        let (order, live) = read(&func);
912        let mut assignment = Assignment::empty(func.vregs());
913        assignment.put(first, Place::Reg(RAX));
914        assignment.put(second, Place::Reg(RAX));
915
916        let problems = check(&func, &order, &live, &assignment);
917        assert_eq!(
918            report(&problems),
919            "the allocation is wrong in 1 place\n  %0 and %1 are both live and both in register 0"
920        );
921        assert_eq!(report(&[]), "the allocation is wrong in 0 places");
922    }
923}