Skip to main content

rucc_regalloc/
assign.rs

1//! Which register each value lives in, and which values live on the stack instead.
2//!
3//! Design: `spec/10-backend.md` section 10.4.
4//!
5//! This is the `-O0` allocator's decision and nothing else. It is linear scan over the line
6//! [`crate::order`] lays the function out in: the values are taken in the order they are written,
7//! each is given a register that nothing else live at the same time is in, and when there is no
8//! such register one of the values in flight goes to the stack instead. There is no splitting and
9//! no coalescing, so a value gets one place for the whole of its range and keeps it. That produces
10//! mediocre code quickly, which is what `-O0` is for, and the allocator that produces good code
11//! slowly is a separate one, in M4.
12//!
13//! Which value is sent to the stack is the one whose range ends last, counting the value being
14//! placed among the candidates. A value wanted for a long time is the cheapest to spill per
15//! instruction it frees a register over, and it is the only heuristic here. What is picked is
16//! really a register and not a value, since two values that are never both wanted share one, and
17//! then every value in that register which is in this one's way goes.
18//!
19//! # Where the line is not the function
20//!
21//! The line is the order the blocks arrived in, and `crate::layout` puts them in a different one
22//! afterwards, so being between two blocks on the line says nothing about being between them in
23//! the code. A value live in one loop and live again in a later one is written down with
24//! everything in between inside the interval around it, and it is not live in any of it.
25//!
26//! Which is why what decides anything here is the area from `crate::live`, and the interval is
27//! only the sweep's bookkeeping: it says which values to compare and the areas say which of them
28//! actually collide. Three loops one after another in a function put a dozen values in flight at
29//! the same instant of the line and never at the same instant of the program, and asking the
30//! interval would spill the one this loop is walking for the sake of eleven values in the other
31//! two. tamnd/rucc#982.
32//!
33//! The same holds for a register an instruction insists on. A call destroys seven registers on
34//! x86-64, and a function whose blocks happen to arrive with a call written between the blocks of
35//! a loop would otherwise lose all seven for every value in that loop, for a call the loop never
36//! reaches, so that question is asked of the area and not of the interval either.
37//!
38//! Allowed is not the same as free, though, so the registers are offered in two passes. First the
39//! ones nothing insists on anywhere the range reaches, then the ones something insists on somewhere
40//! the value never goes. The second kind costs: the instruction that insists has to be handed the
41//! register in the end, and what hands it over is a move. A function that gives a value back has an
42//! operand fixed to `rax` at the end of it, and putting the busiest value in the function in `rax`
43//! because no path reaches the return with it live buys one register and pays a move at every
44//! return. Ordering the two passes is what keeps the register and drops the moves.
45//!
46//! The hint below is asked the first question rather than the second for the same reason. A value
47//! taking the register its own operand asked for saves a move, and taking one somebody else's
48//! operand asked for somewhere it never goes costs one, so a hint is worth following when the
49//! register is clear and not worth following when it is merely allowed.
50//!
51//! # What it does with a register an instruction insists on
52//!
53//! Two things. It stays out of that register for everybody else, and it tries that register first
54//! for the value the operand names. A division wants its dividend in `rax`, so `rax` is
55//! unavailable to every other value that is live where the division reads, and it is the first
56//! register offered to the dividend itself. When the dividend gets it there is no move on the way
57//! in, and when it does not the rewrite writes one and nothing else changes.
58//!
59//! That second half is the hint, and without it the register an instruction insists on is the one
60//! register the value in it can never have, since the value's own operand is what makes the
61//! register look busy. The effect is largest on returns, because a function that gives a value
62//! back has an operand fixed to `rax` at the end of it and most functions give a value back.
63//!
64//! What makes the hint safe is asking about the register at each of the instruction's two points
65//! rather than across the whole of it. An instruction reads at the first and writes at the second,
66//! so a register it insists on is one value's at the first, another value's at the second, and
67//! nobody else's at either. A division reads its dividend from `rax` and writes its quotient to
68//! `rax`, and those are different values that can both live there. A value passed to a call in
69//! `rdi` and wanted again afterwards cannot, because nothing writes `rdi` at the second point and
70//! a register the call does not write is a register the call is assumed to destroy.
71//!
72//! An operand that has to be in memory is the other way round. The value it names goes on the
73//! stack whatever else is true of it, because that is the only place the instruction could read it
74//! from.
75//!
76//! # What it does with a two address instruction
77//!
78//! An `add` on x86-64 writes one of the registers it reads, which the operand says as a reuse of
79//! another operand. The rewrite can always make that true by copying the source into the
80//! destination first, but only if the destination is a register the instruction does not otherwise
81//! read, so a value written by a reuse is treated here as live from where the instruction reads
82//! rather than from where it writes. Then the copy is always safe.
83//!
84//! The copy is also usually unnecessary, and the one place this looks past the interval it is
85//! placing is to see that: if the value being reused is read here for the last time and the value
86//! being written starts here, the second may have the first's register, and the instruction is
87//! already two address without anything being moved anywhere. That is the whole of the coalescing
88//! this allocator does, and it is worth the dozen lines, because otherwise every piece of
89//! arithmetic in the output carries a move in front of it.
90//!
91//! Both halves of that are needed. The second is the one a loop breaks: an instruction at the
92//! bottom of a loop can write a value the top of the loop reads on the next turn, and such a value
93//! is live on the way into the instruction that writes it as well as after. It is then wanted at
94//! the same time as the value it reuses, whatever is true of the reuse, and giving it the same
95//! register makes an addition read the answer to the last one instead of its own operand.
96//!
97//! # What it does not do
98//!
99//! It does not touch the function. What comes out is a table saying where each value went, and the
100//! pass that rewrites the operands and writes the moves reads it. Keeping the decision and the
101//! rewrite apart is what lets the decision be checked by looking at it, and it is the shape
102//! `spec/10-backend.md` section 10.4 asks for: an allocator is a function from a program to an
103//! assignment and the moves that make it true.
104
105use std::cmp::Reverse;
106
107use rucc_mir::{Constraint, Func, Operand, Reg, Role};
108use rucc_target::{PhysReg, RegClass};
109
110use crate::live::{Area, Live, Range};
111use crate::order::{Order, Point};
112
113/// Where a value lives.
114#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
115pub enum Place {
116    /// In a register, for the whole of its range.
117    Reg(PhysReg),
118    /// In a slot of the frame, which is what a value the allocator ran out of registers for gets,
119    /// and what a value an instruction can only read from memory gets.
120    Slot(u32),
121}
122
123/// What the allocator is allowed to use.
124///
125/// The order is the calling convention's, because which register to hand out first follows from
126/// which ones a call destroys, and `rucc-target` is where a convention says so. The scratch
127/// registers are held back out of the order and are what a spilled value is read into at each
128/// instruction that wants it, so a class needs as many of them as one of its instructions has
129/// register operands. Nothing here uses them, since a spilled value is only read once the rewrite
130/// is writing the instruction that reads it, but they are held back here because this is what
131/// decides what everything else may have.
132#[derive(Debug, Clone, Default)]
133pub struct Env {
134    classes: Vec<Class>,
135}
136
137/// What one class of registers offers.
138#[derive(Debug, Clone, Default)]
139struct Class {
140    order: Vec<PhysReg>,
141    scratch: Vec<PhysReg>,
142}
143
144impl Env {
145    /// An environment offering nothing, which is what a target that has said nothing offers.
146    #[must_use]
147    pub fn new() -> Self {
148        Self::default()
149    }
150
151    /// The same environment, with that class described.
152    #[must_use]
153    pub fn with(mut self, class: RegClass, order: &[PhysReg], scratch: &[PhysReg]) -> Self {
154        let index = usize::from(class.number());
155        if self.classes.len() <= index {
156            self.classes.resize(index + 1, Class::default());
157        }
158        self.classes[index] = Class { order: order.to_vec(), scratch: scratch.to_vec() };
159        self
160    }
161
162    /// The registers it may hand out in a class, in the order it prefers them.
163    #[must_use]
164    pub fn order(&self, class: RegClass) -> &[PhysReg] {
165        self.classes.get(usize::from(class.number())).map_or(&[], |class| &class.order)
166    }
167
168    /// The registers held back in a class for reading a spilled value into.
169    #[must_use]
170    pub fn scratch(&self, class: RegClass) -> &[PhysReg] {
171        self.classes.get(usize::from(class.number())).map_or(&[], |class| &class.scratch)
172    }
173}
174
175/// Where every value in a function went.
176#[derive(Debug, Clone)]
177pub struct Assignment {
178    places: Vec<Option<Place>>,
179    slots: Vec<RegClass>,
180}
181
182impl Assignment {
183    /// An assignment that says nothing yet about a function with that many values.
184    ///
185    /// This and [`Assignment::put`] and [`Assignment::take_slot`] are how an allocator says what
186    /// it decided. There will be a second one in M4 and it will not reach its answer this way, so
187    /// what an assignment is has to be separable from how this file arrives at one, and the
188    /// checker in [`crate::check`] reads an assignment without caring which allocator wrote it.
189    #[must_use]
190    pub fn empty(vregs: usize) -> Self {
191        Self { places: vec![None; vregs], slots: Vec::new() }
192    }
193
194    /// Records where a value went.
195    ///
196    /// # Panics
197    ///
198    /// Panics on a physical register, which is somewhere already, and on a virtual one the
199    /// function never handed out.
200    pub fn put(&mut self, reg: Reg, place: Place) {
201        self.places[index(reg)] = Some(place);
202    }
203
204    /// Takes a slot of the frame, of that class, and gives back which one it is.
205    ///
206    /// # Panics
207    ///
208    /// Panics past four billion slots, which is a frame no machine has room for.
209    pub fn take_slot(&mut self, class: RegClass) -> u32 {
210        let slot = u32::try_from(self.slots.len()).expect("too many spilled values");
211        self.slots.push(class);
212        slot
213    }
214
215    /// Where a value lives, or `None` for a virtual register this function never mentions and for
216    /// a physical one, which is already where it is.
217    #[must_use]
218    pub fn place(&self, reg: Reg) -> Option<Place> {
219        self.places.get(usize::try_from(reg.number()?).ok()?).copied().flatten()
220    }
221
222    /// The class of each slot of the frame, which is what says how wide it has to be.
223    #[must_use]
224    pub fn slots(&self) -> &[RegClass] {
225        &self.slots
226    }
227
228    /// Every value that went somewhere, and where it went.
229    ///
230    /// The assignment read the other way round, which is what a caller wants when the question is
231    /// about the places rather than about the values. The stack slot allocator asks it that way,
232    /// since what it needs is which value is in each slot and the assignment is stored by value.
233    pub fn placed(&self) -> impl Iterator<Item = (Reg, Place)> + '_ {
234        self.places.iter().enumerate().filter_map(|(number, place)| {
235            let number = u32::try_from(number).ok()?;
236            Some((Reg::virtual_reg(number), (*place)?))
237        })
238    }
239
240    /// How many values went to the stack.
241    #[must_use]
242    pub fn spilled(&self) -> usize {
243        self.slots.len()
244    }
245
246    /// Puts a value on the stack, in a slot of its own.
247    fn spill(&mut self, reg: Reg, class: RegClass) {
248        let slot = self.take_slot(class);
249        self.put(reg, Place::Slot(slot));
250    }
251}
252
253/// One value waiting for a place.
254#[derive(Debug, Clone, Copy)]
255struct Interval<'a> {
256    reg: Reg,
257    class: RegClass,
258    /// The interval around the area, which is what the sweep below reads and what says which value
259    /// is wanted for longest when one of them has to go.
260    range: Range,
261    /// Everywhere the value is really live, which is what says whether two of them fit in one
262    /// register.
263    area: Area<'a>,
264}
265
266/// One value that has a register, for as long as it still wants it.
267#[derive(Debug, Clone, Copy)]
268struct Held<'a> {
269    reg: Reg,
270    class: RegClass,
271    range: Range,
272    area: Area<'a>,
273    at: PhysReg,
274}
275
276/// A register an instruction insists on, and where it insists on it.
277#[derive(Debug, Clone, Copy)]
278struct Blocked {
279    class: RegClass,
280    at: PhysReg,
281    /// One of the instruction's two points. Every register an instruction insists on has an entry
282    /// at each of them, because a register held at one of the two is a register nothing else may
283    /// be in across the instruction.
284    point: Point,
285    /// The one value that may be in it there, which is the value of an operand the instruction
286    /// reads at that point or writes at it. `None` means nothing may: an operand naming a physical
287    /// register outright claims it against everything, and a point no operand covers is a point
288    /// the instruction has the register to itself at.
289    by: Option<Reg>,
290}
291
292/// A value written into the register another operand of the same instruction was read from.
293#[derive(Debug, Clone, Copy)]
294struct Reuse {
295    /// The value being read, which is the one whose register would do.
296    source: Reg,
297    /// Where the instruction reads it.
298    at: Point,
299}
300
301/// Decides where every value in a function lives.
302///
303/// # Panics
304///
305/// Panics if a class has no registers to hand out and something in the function is in that class,
306/// since that is a target description that does not describe the target the function is for.
307#[must_use]
308pub fn assign(func: &Func, order: &Order, live: &Live, env: &Env) -> Assignment {
309    let blocked = blocked(func, order);
310    let forced = forced(func);
311    let reuses = reuses(func, order);
312    let hints = hints(func);
313
314    let mut intervals = Vec::with_capacity(func.vregs());
315    for (number, reuse) in reuses.iter().enumerate() {
316        let reg = Reg::virtual_reg(u32::try_from(number).expect("a register number"));
317        let (Some(mut area), Some(class)) = (live.area(reg), func.class_of(reg)) else {
318            continue;
319        };
320        if let Some(reuse) = reuse {
321            area = area.with(reuse.at);
322        }
323        intervals.push(Interval { reg, class, range: area.hull(), area });
324    }
325    intervals.sort_by_key(|interval| (interval.range.start, interval.reg));
326
327    let mut assignment = Assignment::empty(func.vregs());
328    let mut active: Vec<Held<'_>> = Vec::new();
329    for interval in intervals {
330        active.retain(|held| held.range.end >= interval.range.start);
331        if forced.contains(&interval.reg) {
332            assignment.spill(interval.reg, interval.class);
333            continue;
334        }
335        // A class with no order is one the target says nothing allocates from, which on x86-64 is
336        // the x87 stack. A value of such a class is a mistake at the point it was made rather than
337        // a value with nowhere to go: what the target means is that the value lives in memory and
338        // that whatever operates on it takes an address. See `ClassInfo::allocatable`.
339        assert!(
340            !env.order(interval.class).is_empty(),
341            "a value in class {}, which the target hands out no registers from",
342            interval.class.number()
343        );
344        let two_address = reuses[index(interval.reg)]
345            .and_then(|reuse| coalesce(&assignment, &active, &blocked, live, interval, reuse));
346        // The reuse comes first, because a two address instruction that has to copy its left
347        // operand in pays for the copy whatever the hint says, and taking the hint here would buy
348        // one move at the cost of another.
349        let hinted = hints[index(interval.reg)].iter().copied().find(|&at| {
350            env.order(interval.class).contains(&at)
351                && available(&active, &blocked, interval, at, None, Want::Clear)
352        });
353        // A register nobody else wants anywhere near this value first, and one somebody wants
354        // somewhere the value never goes only when there is no other. Both are correct and the
355        // second is the worse buy, since the instruction that wants it has to be handed it and
356        // whatever this value is doing there has to move out of the way first.
357        let scan = |want| {
358            env.order(interval.class)
359                .iter()
360                .copied()
361                .find(|&at| available(&active, &blocked, interval, at, None, want))
362        };
363        let chosen =
364            two_address.or(hinted).or_else(|| scan(Want::Clear)).or_else(|| scan(Want::Allowed));
365        match chosen {
366            Some(at) => {
367                assignment.places[index(interval.reg)] = Some(Place::Reg(at));
368                active.push(Held {
369                    reg: interval.reg,
370                    class: interval.class,
371                    range: interval.range,
372                    area: interval.area,
373                    at,
374                });
375            }
376            None => spill_one(&mut assignment, &mut active, &blocked, interval),
377        }
378    }
379    assignment
380}
381
382/// How much a register suits an interval.
383#[derive(Debug, Clone, Copy, PartialEq, Eq)]
384enum Want {
385    /// Nothing insists on it anywhere the range reaches, so taking it costs nobody anything.
386    Clear,
387    /// Something insists on it somewhere the range reaches and nowhere the value is live, so taking
388    /// it is allowed and may still cost: the instruction that insists wants the register for a
389    /// value of its own, and that value now has to be moved into it.
390    Allowed,
391}
392
393/// Every register every instruction in the function insists on, arranged to be asked about.
394///
395/// Built once and never changed afterwards, and there is only one question ever asked of it: of the
396/// constraints naming one register of one class, is there one at a point some interval covers. So
397/// the entries are ordered by the register they name and then by the point, and the question is a
398/// binary search for the start of the interval followed by a walk that stops at its end.
399///
400/// It used to be a flat list walked from one end for every candidate register of every interval,
401/// which is quadratic in the size of a function and is most of the compile on a large one. See
402/// tamnd/rucc#1003 for the profile that found it.
403struct Blocks {
404    /// The constraints, sorted by class, then by register, then by point.
405    all: Vec<Blocked>,
406}
407
408impl Blocks {
409    /// The constraints on one register of one class at the points an interval covers.
410    ///
411    /// Both ends of the walk come from the ordering rather than from a test, so what comes back is
412    /// exactly what the old `covers` call used to keep and in the same order.
413    fn over(
414        &self,
415        class: RegClass,
416        at: PhysReg,
417        range: Range,
418    ) -> impl Iterator<Item = &Blocked> + '_ {
419        let first = self
420            .all
421            .partition_point(|one| (one.class, one.at, one.point) < (class, at, range.start));
422        self.all[first..]
423            .iter()
424            .take_while(move |one| one.class == class && one.at == at && one.point <= range.end)
425    }
426}
427
428/// Whether a register is one this interval could have.
429///
430/// The exception is the value a reuse is coalescing with, which holds the register right up to the
431/// point the new value takes it over and is the one thing that may overlap.
432///
433/// The sweep only keeps a value in `active` while the interval around it reaches this one, so the
434/// areas still have to be compared: two values whose intervals cross can have holes that let them
435/// share a register anyway, which on a function with several loops in it is most of them.
436fn available(
437    active: &[Held<'_>],
438    blocked: &Blocks,
439    interval: Interval<'_>,
440    at: PhysReg,
441    except: Option<Reg>,
442    want: Want,
443) -> bool {
444    let taken = active.iter().any(|held| {
445        held.at == at
446            && held.class == interval.class
447            && Some(held.reg) != except
448            && held.area.overlaps(interval.area)
449    });
450    let insisted = blocked.over(interval.class, at, interval.range).any(|one| {
451        one.by != Some(interval.reg) && (want == Want::Clear || interval.area.covers(one.point))
452    });
453    !taken && !insisted
454}
455
456/// The register the value being reused is in, when this instruction is the last thing that reads
457/// it, the value being written starts here, and the register is otherwise free.
458fn coalesce(
459    assignment: &Assignment,
460    active: &[Held<'_>],
461    blocked: &Blocks,
462    live: &Live,
463    interval: Interval<'_>,
464    reuse: Reuse,
465) -> Option<PhysReg> {
466    let Some(Place::Reg(at)) = assignment.place(reuse.source) else { return None };
467    let source = active.iter().find(|held| held.reg == reuse.source)?;
468    // A value read again later needs its register after this instruction would have overwritten
469    // it, so the two really do have to be different and the rewrite really does have to copy.
470    let dies = source.range.end == reuse.at;
471    // And the value being written must not be live where the instruction reads already. The area
472    // asked here is the one liveness worked out, without the point the reuse adds, so a value that
473    // covers the reuse point on its own is one that was live on the way into this instruction. That
474    // is what a loop carrying its own result round looks like: the instruction writes it at the
475    // bottom and the top of the loop reads what the last turn wrote. Such a value overlaps the one
476    // it reuses over the whole loop, so the two cannot be the same register no matter that the read
477    // here is the last one.
478    let begins = live.area(interval.reg).is_some_and(|area| !area.covers(reuse.at));
479    let free = available(active, blocked, interval, at, Some(reuse.source), Want::Allowed);
480    (dies && begins && free).then_some(at)
481}
482
483/// Sends values to the stack to free a register: the ones wanted for longest, since a register
484/// held that long pays for itself over the most instructions.
485///
486/// What is chosen is a register rather than a value, because two values whose areas miss each
487/// other share one and taking it means every value in it this one is really on top of has to go.
488/// A register holding two of those costs twice as much to take as one holding a single value, so
489/// the cheap ones are looked at first and the reach only settles ties.
490fn spill_one<'a>(
491    assignment: &mut Assignment,
492    active: &mut Vec<Held<'a>>,
493    blocked: &Blocks,
494    interval: Interval<'a>,
495) {
496    // What each register would cost: how many values would go, and the furthest any of them
497    // reaches. The list is one entry per register of the class, so walking it for each value in
498    // flight is the same shape as everything else here.
499    let mut costs: Vec<(PhysReg, usize, Point)> = Vec::new();
500    for held in active.iter() {
501        if held.class != interval.class || !held.area.overlaps(interval.area) {
502            continue;
503        }
504        match costs.iter_mut().find(|(at, _, _)| *at == held.at) {
505            Some((_, count, reach)) => {
506                *count += 1;
507                *reach = (*reach).max(held.range.end);
508            }
509            None => costs.push((held.at, 1, held.range.end)),
510        }
511    }
512    // A register the instructions in the way insist on for themselves is no use, because taking it
513    // over would put this value in a register it may not have.
514    let chosen = costs
515        .iter()
516        .filter(|&&(at, _, reach)| {
517            reach > interval.range.end && available(&[], blocked, interval, at, None, Want::Allowed)
518        })
519        .min_by_key(|&&(_, count, reach)| (count, Reverse(reach)))
520        .map(|&(at, _, _)| at);
521    match chosen {
522        Some(at) => {
523            active.retain(|held| {
524                let goes = held.at == at
525                    && held.class == interval.class
526                    && held.area.overlaps(interval.area);
527                if goes {
528                    assignment.spill(held.reg, held.class);
529                }
530                !goes
531            });
532            assignment.places[index(interval.reg)] = Some(Place::Reg(at));
533            active.push(Held {
534                reg: interval.reg,
535                class: interval.class,
536                range: interval.range,
537                area: interval.area,
538                at,
539            });
540        }
541        None => assignment.spill(interval.reg, interval.class),
542    }
543}
544
545/// The registers the instructions insist on, and where.
546///
547/// A physical register an operand names outright counts the same way. Nothing before allocation
548/// writes one except an instruction that has to, and it has to for the length of that one
549/// instruction, which is the same statement a fixed constraint makes.
550fn blocked(func: &Func, order: &Order) -> Blocks {
551    let mut blocked = Vec::new();
552    let mut claimed: Vec<(RegClass, PhysReg)> = Vec::new();
553    for block in func.blocks() {
554        for inst in func.insts(block) {
555            let operands = &func[func[inst].operands];
556            claimed.clear();
557            for operand in operands {
558                if let Some(at) = insisted(operand) {
559                    let key = (operand.class, at);
560                    if !claimed.contains(&key) {
561                        claimed.push(key);
562                    }
563                }
564            }
565            for &(class, at) in &claimed {
566                // Both points, whether or not an operand is at them. A register an instruction
567                // reads and does not write is still gone by the time the instruction is done as far
568                // as anything here knows, which is what stops the value a call is passed in `rdi`
569                // from staying in `rdi` over the call.
570                for (point, role) in [(order.early(inst), Role::Use), (order.late(inst), Role::Def)]
571                {
572                    let mut named = false;
573                    for operand in operands {
574                        let mine = insisted(operand) == Some(at) && operand.class == class;
575                        if !mine || !(operand.role == role || operand.role == Role::EarlyDef) {
576                            continue;
577                        }
578                        named = true;
579                        let by = operand.reg.is_virtual().then_some(operand.reg);
580                        blocked.push(Blocked { class, at, point, by });
581                    }
582                    // A register no operand names where the operands are read is one the
583                    // instruction writes and does not read, which is what a clobber is, and the
584                    // seven registers a call destroys are the whole of why that case is worth
585                    // separating. Such a register is free right up to the point it is written, so a
586                    // value whose last read is this instruction may sit in one: it is read before
587                    // the instruction writes anything, the way any other operand is. Blocking it
588                    // where the operands are read as well would take every caller saved register
589                    // away from the value a call is passed, which is a value that dies at the call
590                    // and pays for a callee saved register it holds for two instructions. Anything
591                    // living past the instruction is still refused, by the block below.
592                    //
593                    // This is where a target's early definitions are paid for. An instruction that
594                    // fills a register before it has finished reading has to say so, because that
595                    // is the one thing a plain definition here no longer covers: a division on
596                    // x86-64 is a sign extension and then the division itself, so `rdx` is gone
597                    // before the divisor is read, and a divisor that went there would be read as
598                    // the dividend's own sign bits. `rucc_target::x86_64` writes both of them down
599                    // as early definitions for exactly that reason.
600                    if !named && role == Role::Def {
601                        blocked.push(Blocked { class, at, point, by: None });
602                    }
603                }
604            }
605        }
606    }
607    // Program order already has the points ascending, but the registers one instruction claims are
608    // walked outside the two points rather than inside them, so the list arrives in order by
609    // instruction and not by register. A sort by the key the lookup searches on is what makes it
610    // searchable, and it is stable so two constraints on one register at one point keep the order
611    // the instruction wrote them in.
612    blocked.sort_by_key(|one: &Blocked| (one.class, one.at, one.point));
613    Blocks { all: blocked }
614}
615
616/// The register an operand has to be in, which is the one a constraint asks for or the one the
617/// operand names outright.
618fn insisted(operand: &Operand) -> Option<PhysReg> {
619    match operand.constraint {
620        Constraint::Fixed(at) => Some(at),
621        _ => operand.reg.phys(),
622    }
623}
624
625/// The registers each value would rather be in, which are the ones the operands naming it insist on.
626///
627/// In the order the function writes them down, so the definition comes first where there is one,
628/// since a value written into a fixed register and then moved somewhere else pays for the move at
629/// the top of its life rather than at the bottom. The ones after it are worth keeping for the same
630/// reason the first one is, and the value a call is passed is where that shows: its definition may
631/// insist on the register a parameter arrived in, which the call it is handed to has usually taken
632/// back for an argument of its own by then, and behind that is the register the convention passes
633/// it in, which is free and is exactly where the value wants to end up.
634fn hints(func: &Func) -> Vec<Vec<PhysReg>> {
635    let mut hints = vec![Vec::new(); func.vregs()];
636    for block in func.blocks() {
637        for inst in func.insts(block) {
638            for operand in &func[func[inst].operands] {
639                let Constraint::Fixed(at) = operand.constraint else { continue };
640                let number = operand.reg.number().and_then(|number| usize::try_from(number).ok());
641                let Some(number) = number else { continue };
642                let wanted: &mut Vec<PhysReg> = &mut hints[number];
643                if func.class_of(operand.reg) == Some(operand.class) && !wanted.contains(&at) {
644                    wanted.push(at);
645                }
646            }
647        }
648    }
649    hints
650}
651
652/// The values that have to be on the stack whatever else is true of them.
653fn forced(func: &Func) -> Vec<Reg> {
654    let mut forced = Vec::new();
655    for block in func.blocks() {
656        for inst in func.insts(block) {
657            for operand in &func[func[inst].operands] {
658                if operand.constraint == Constraint::Stack
659                    && operand.reg.is_virtual()
660                    && !forced.contains(&operand.reg)
661                {
662                    forced.push(operand.reg);
663                }
664            }
665        }
666    }
667    forced
668}
669
670/// The value each two address instruction reuses, by the virtual register it writes.
671fn reuses(func: &Func, order: &Order) -> Vec<Option<Reuse>> {
672    let mut reuses = vec![None; func.vregs()];
673    for block in func.blocks() {
674        for inst in func.insts(block) {
675            let operands = &func[func[inst].operands];
676            for operand in operands {
677                let Constraint::Reuse(other) = operand.constraint else { continue };
678                let number = operand.reg.number().and_then(|number| usize::try_from(number).ok());
679                let Some(number) = number else { continue };
680                let source = operands[usize::from(other)].reg;
681                reuses[number] = Some(Reuse { source, at: order.early(inst) });
682            }
683        }
684    }
685    reuses
686}
687
688/// A virtual register's number as a table index.
689fn index(reg: Reg) -> usize {
690    usize::try_from(reg.number().expect("a virtual register")).expect("a register number")
691}
692
693#[cfg(test)]
694mod tests {
695    use rucc_base::Interner;
696    use rucc_mir::{BlockCall, Opcode, Operand};
697    use rucc_target::x86_64::{GPR, R13, R14, R15, RAX, RCX, RDX, REGS, SYSV};
698
699    use super::*;
700
701    /// The x86-64 environment, with the last three of the allocation order held back as scratch.
702    fn env() -> Env {
703        let (order, scratch) = SYSV.int_order.split_at(SYSV.int_order.len() - 3);
704        Env::new().with(GPR, order, scratch)
705    }
706
707    /// An environment with that many general purpose registers, for putting a function under
708    /// pressure without writing a hundred instructions.
709    fn narrow(count: usize) -> Env {
710        Env::new().with(GPR, &SYSV.int_order[..count], &SYSV.int_order[count..count + 1])
711    }
712
713    /// What a place is called, which is what an assertion reads.
714    fn named(place: Option<Place>) -> String {
715        match place {
716            Some(Place::Reg(reg)) => REGS.name(GPR, reg).expect("a register").to_string(),
717            Some(Place::Slot(slot)) => format!("slot {slot}"),
718            None => "nowhere".to_string(),
719        }
720    }
721
722    /// Where every value in a function went.
723    fn places(func: &Func, env: &Env) -> Vec<String> {
724        let order = Order::of(func);
725        let live = Live::of(func, &order);
726        let assignment = assign(func, &order, &live, env);
727        (0..func.vregs())
728            .map(|number| {
729                let reg = Reg::virtual_reg(u32::try_from(number).expect("a register number"));
730                named(assignment.place(reg))
731            })
732            .collect()
733    }
734
735    #[test]
736    fn two_values_that_are_never_both_wanted_share_a_register() {
737        let mut names = Interner::new();
738        let mut func = Func::new(names.intern("f"));
739        let opcode = Opcode::new(names.intern("x64.nop"));
740        let block = func.create_block();
741        let first = func.new_vreg(GPR);
742        let second = func.new_vreg(GPR);
743        func.build(block, opcode).def(first, GPR).finish();
744        func.build(block, opcode).uses(first, GPR).finish();
745        func.build(block, opcode).def(second, GPR).finish();
746        func.build(block, opcode).uses(second, GPR).finish();
747
748        // The first register in the order, twice, because the first value is finished with before
749        // the second one is written.
750        assert_eq!(places(&func, &env()), ["rax", "rax"]);
751    }
752
753    #[test]
754    fn two_values_that_are_both_wanted_do_not() {
755        let mut names = Interner::new();
756        let mut func = Func::new(names.intern("f"));
757        let opcode = Opcode::new(names.intern("x64.nop"));
758        let block = func.create_block();
759        let first = func.new_vreg(GPR);
760        let second = func.new_vreg(GPR);
761        func.build(block, opcode).def(first, GPR).finish();
762        func.build(block, opcode).def(second, GPR).finish();
763        func.build(block, opcode).uses(first, GPR).finish();
764        func.build(block, opcode).uses(second, GPR).finish();
765
766        assert_eq!(places(&func, &env()), ["rax", "rcx"]);
767    }
768
769    #[test]
770    fn a_value_written_early_that_nothing_reads_still_holds_its_register() {
771        let mut names = Interner::new();
772        let mut func = Func::new(names.intern("f"));
773        let opcode = Opcode::new(names.intern("x64.nop"));
774        let block = func.create_block();
775        let wanted = func.new_vreg(GPR);
776        let spare = func.new_vreg(GPR);
777        // A division: a remainder somebody wants, and a quotient nobody does. Both are written by
778        // the one instruction and the quotient is written before the operands have been read.
779        func.build(block, opcode)
780            .def(wanted, GPR)
781            .operand(Operand::write_early(spare, GPR))
782            .finish();
783        func.build(block, opcode).uses(wanted, GPR).finish();
784
785        // Two registers, not one. A value nothing reads is still somewhere, and the instruction
786        // that wrote it wrote the other one too, so the two cannot be the same place. Handing them
787        // the same register loses the remainder, because the copy that takes the quotient out of
788        // the register the machine insisted on goes on top of it. The quotient gets the first
789        // register because it is written first, which is the whole of what early means.
790        assert_eq!(places(&func, &env()), ["rcx", "rax"]);
791    }
792
793    #[test]
794    fn the_value_wanted_longest_is_the_one_that_goes_to_the_stack() {
795        let mut names = Interner::new();
796        let mut func = Func::new(names.intern("f"));
797        let opcode = Opcode::new(names.intern("x64.nop"));
798        let block = func.create_block();
799        let long = func.new_vreg(GPR);
800        let short = func.new_vreg(GPR);
801        let third = func.new_vreg(GPR);
802        func.build(block, opcode).def(long, GPR).finish();
803        func.build(block, opcode).def(short, GPR).finish();
804        func.build(block, opcode).def(third, GPR).finish();
805        func.build(block, opcode).uses(short, GPR).finish();
806        func.build(block, opcode).uses(third, GPR).finish();
807        func.build(block, opcode).uses(long, GPR).finish();
808
809        // Two registers between three values. The one still wanted at the end of the function is
810        // the one whose register is worth the most to everybody else, so it is the one that goes.
811        assert_eq!(places(&func, &narrow(2)), ["slot 0", "rcx", "rax"]);
812    }
813
814    #[test]
815    fn a_register_an_instruction_insists_on_goes_to_the_values_that_asked_for_it() {
816        let mut names = Interner::new();
817        let mut func = Func::new(names.intern("f"));
818        let opcode = Opcode::new(names.intern("x64.nop"));
819        let block = func.create_block();
820        let across = func.new_vreg(GPR);
821        let dividend = func.new_vreg(GPR);
822        let quotient = func.new_vreg(GPR);
823        let remainder = func.new_vreg(GPR);
824        func.build(block, opcode).def(across, GPR).finish();
825        func.build(block, opcode).def(dividend, GPR).finish();
826        func.build(block, opcode)
827            .operand(Operand::write(quotient, GPR).with(Constraint::Fixed(RAX)))
828            .operand(Operand::write_early(remainder, GPR).with(Constraint::Fixed(RDX)))
829            .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
830            .finish();
831        func.build(block, opcode).uses(across, GPR).finish();
832
833        // The value that has to be across the division is nowhere near `rax` or `rdx`, and each of
834        // the three the division names is in the register the division asked for it in. The
835        // dividend and the quotient share `rax` because the first is read where the second is
836        // written, which is what a division does.
837        assert_eq!(places(&func, &env()), ["rcx", "rax", "rax", "rdx"]);
838    }
839
840    /// A value read by an instruction that fills a register before it reads is kept out of that
841    /// register, even though the read is the last thing the value is wanted for.
842    ///
843    /// The divisor of a division is the case. What the machine runs is `cltd` and then `idivl`, so
844    /// `rdx` holds the top half of the dividend by the time the divisor is read, and a divisor
845    /// sitting in `rdx` is read as the dividend's own sign bits. An early definition is how the
846    /// target says a register goes before the operands are read, and this is where the allocator
847    /// has to hear it, since a value dying at an instruction is otherwise free to sit in a
848    /// register that instruction writes. tamnd/rucc#1232.
849    #[test]
850    fn a_value_that_dies_at_an_instruction_stays_out_of_what_it_fills_first() {
851        let mut names = Interner::new();
852        let mut func = Func::new(names.intern("f"));
853        let opcode = Opcode::new(names.intern("x64.nop"));
854        let block = func.create_block();
855        let across = func.new_vreg(GPR);
856        let dividend = func.new_vreg(GPR);
857        let divisor = func.new_vreg(GPR);
858        let remainder = func.new_vreg(GPR);
859        func.build(block, opcode).def(across, GPR).finish();
860        func.build(block, opcode).def(dividend, GPR).finish();
861        func.build(block, opcode).def(divisor, GPR).finish();
862        func.build(block, opcode)
863            .operand(Operand::write_early(remainder, GPR).with(Constraint::Fixed(RDX)))
864            .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
865            .operand(Operand::read(divisor, GPR))
866            .finish();
867        func.build(block, opcode).uses(across, GPR).uses(remainder, GPR).finish();
868
869        // Four registers for four values, and the divisor takes the fourth. `rdx` is free
870        // everywhere in this function except at the instruction that is about to fill it, which is
871        // the one instruction the divisor is wanted at.
872        assert_eq!(places(&func, &narrow(4)), ["rcx", "rax", "rsi", "rdx"]);
873    }
874
875    #[test]
876    fn a_value_wanted_after_the_instruction_that_insists_does_not_get_that_register() {
877        let mut names = Interner::new();
878        let mut func = Func::new(names.intern("f"));
879        let opcode = Opcode::new(names.intern("x64.nop"));
880        let block = func.create_block();
881        let dividend = func.new_vreg(GPR);
882        let quotient = func.new_vreg(GPR);
883        func.build(block, opcode).def(dividend, GPR).finish();
884        func.build(block, opcode)
885            .operand(Operand::write(quotient, GPR).with(Constraint::Fixed(RAX)))
886            .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
887            .finish();
888        func.build(block, opcode).uses(dividend, GPR).finish();
889
890        // The hint is a preference and not a claim. The dividend would rather be in `rax` and
891        // cannot be, because the division writes `rax` and the dividend is wanted afterwards, so
892        // it takes the next register and the quotient keeps the one it was promised.
893        assert_eq!(places(&func, &env()), ["rcx", "rax"]);
894    }
895
896    #[test]
897    fn a_value_an_instruction_can_only_read_from_memory_is_on_the_stack() {
898        let mut names = Interner::new();
899        let mut func = Func::new(names.intern("f"));
900        let opcode = Opcode::new(names.intern("x64.nop"));
901        let block = func.create_block();
902        let value = func.new_vreg(GPR);
903        func.build(block, opcode).def(value, GPR).finish();
904        func.build(block, opcode)
905            .operand(Operand::read(value, GPR).with(Constraint::Stack))
906            .finish();
907
908        assert_eq!(places(&func, &env()), ["slot 0"]);
909    }
910
911    #[test]
912    fn a_two_address_instruction_writes_the_register_it_read_when_it_can() {
913        let mut names = Interner::new();
914        let mut func = Func::new(names.intern("f"));
915        let opcode = Opcode::new(names.intern("x64.nop"));
916        let block = func.create_block();
917        let left = func.new_vreg(GPR);
918        let right = func.new_vreg(GPR);
919        let sum = func.new_vreg(GPR);
920        func.build(block, opcode).def(left, GPR).finish();
921        func.build(block, opcode).def(right, GPR).finish();
922        func.build(block, opcode)
923            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
924            .uses(left, GPR)
925            .uses(right, GPR)
926            .finish();
927        func.build(block, opcode).uses(right, GPR).finish();
928
929        // The addition reads the left value for the last time, so the answer goes where that was
930        // and the instruction is two address without a move in front of it.
931        assert_eq!(places(&func, &env()), ["rax", "rcx", "rax"]);
932    }
933
934    #[test]
935    fn a_two_address_instruction_that_cannot_gets_a_register_nothing_it_reads_is_in() {
936        let mut names = Interner::new();
937        let mut func = Func::new(names.intern("f"));
938        let opcode = Opcode::new(names.intern("x64.nop"));
939        let block = func.create_block();
940        let left = func.new_vreg(GPR);
941        let right = func.new_vreg(GPR);
942        let sum = func.new_vreg(GPR);
943        func.build(block, opcode).def(left, GPR).finish();
944        func.build(block, opcode).def(right, GPR).finish();
945        func.build(block, opcode)
946            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
947            .uses(left, GPR)
948            .uses(right, GPR)
949            .finish();
950        func.build(block, opcode).uses(left, GPR).finish();
951
952        // The left value is wanted afterwards, so the answer cannot have its register. It cannot
953        // have the right one's either, because the rewrite is about to write a move into it before
954        // the addition has read anything.
955        assert_eq!(places(&func, &env()), ["rax", "rcx", "rdx"]);
956    }
957
958    #[test]
959    fn a_value_live_across_a_whole_loop_holds_its_register_over_all_of_it() {
960        let mut names = Interner::new();
961        let mut func = Func::new(names.intern("f"));
962        let opcode = Opcode::new(names.intern("x64.nop"));
963        let head = func.create_block();
964        let body = func.create_block();
965        let carried = func.new_vreg(GPR);
966        let inside = func.new_vreg(GPR);
967        func.build(head, opcode).def(carried, GPR).finish();
968        *func.succs_mut(head) = vec![BlockCall::to(body)];
969        func.build(body, opcode).def(inside, GPR).finish();
970        func.build(body, opcode).uses(inside, GPR).uses(carried, GPR).finish();
971        *func.succs_mut(body) = vec![BlockCall::to(body)];
972
973        // The value inside the loop cannot have the carried one's register, even though nothing
974        // between the two definitions says so.
975        assert_eq!(places(&func, &env()), ["rax", "rcx"]);
976    }
977
978    #[test]
979    fn a_two_address_answer_already_live_does_not_take_the_register_it_read() {
980        let mut names = Interner::new();
981        let mut func = Func::new(names.intern("f"));
982        let opcode = Opcode::new(names.intern("x64.nop"));
983        let head = func.create_block();
984        let latch = func.create_block();
985        let out = func.create_block();
986        let source = func.new_vreg(GPR);
987        let carried = func.new_vreg(GPR);
988        func.build(head, opcode).def(source, GPR).finish();
989        func.build(head, opcode).def(carried, GPR).finish();
990        *func.succs_mut(head) = vec![BlockCall::to(latch)];
991        // The bottom of the loop adds the source to the carried value and writes the answer back
992        // over it, reusing the register the source is in. The next turn round redefines both.
993        func.build(latch, opcode)
994            .operand(Operand::write(carried, GPR).with(Constraint::Reuse(1)))
995            .uses(source, GPR)
996            .uses(carried, GPR)
997            .finish();
998        *func.succs_mut(latch) = vec![BlockCall::to(head), BlockCall::to(out)];
999        func.build(out, opcode).uses(carried, GPR).finish();
1000
1001        // The source is read here for the last time, which on its own is the shape the two address
1002        // shortcut is for, and taking it would be wrong. The carried value was written by the same
1003        // instruction on the last turn and is read by this one, so the two are both wanted where
1004        // the instruction reads and one register cannot hold both.
1005        assert_eq!(places(&func, &env()), ["rax", "rcx"]);
1006
1007        // And the checker has to agree, since it excused this pair on the same reasoning and so
1008        // would have let the answer through.
1009        let order = Order::of(&func);
1010        let live = Live::of(&func, &order);
1011        let assignment = assign(&func, &order, &live, &env());
1012        assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1013    }
1014
1015    #[test]
1016    fn a_two_address_answer_with_a_hole_in_front_of_it_does_not_take_its_other_operand() {
1017        let mut names = Interner::new();
1018        let mut func = Func::new(names.intern("f"));
1019        let nop = Opcode::new(names.intern("x64.nop"));
1020        let add = Opcode::new(names.intern("x64.add"));
1021        let entry = func.create_block();
1022        let head = func.create_block();
1023        let arm = func.create_block();
1024        let latch = func.create_block();
1025        let out = func.create_block();
1026        let seed = func.new_vreg(GPR);
1027        let sum = func.new_vreg(GPR);
1028        let inside = func.new_vreg(GPR);
1029        let loaded = func.new_vreg(GPR);
1030        func.build(entry, nop).def(seed, GPR).finish();
1031        func.build(entry, nop).def(sum, GPR).finish();
1032        *func.succs_mut(entry) = vec![BlockCall::to(head)];
1033        func.build(head, nop).uses(sum, GPR).finish();
1034        *func.succs_mut(head) = vec![BlockCall::to(arm), BlockCall::to(latch)];
1035        func.build(arm, nop).def(inside, GPR).finish();
1036        func.build(arm, nop).uses(inside, GPR).finish();
1037        *func.succs_mut(arm) = vec![BlockCall::to(out)];
1038        func.build(latch, nop).def(loaded, GPR).finish();
1039        func.build(latch, add)
1040            .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
1041            .uses(seed, GPR)
1042            .uses(loaded, GPR)
1043            .finish();
1044        *func.succs_mut(latch) = vec![BlockCall::to(head), BlockCall::to(out)];
1045
1046        // The answer is live in the entry and the head as well, and the arm between them is a hole
1047        // in it, so the piece the addition writes is not the first one. The value the addition reads
1048        // out of memory is still wanted where the addition reads, so it may not be in the register
1049        // the answer is about to be copied into, holes or no holes. tamnd/rucc#982.
1050        let places = places(&func, &env());
1051        assert_ne!(places[index(sum)], places[index(loaded)]);
1052
1053        let order = Order::of(&func);
1054        let live = Live::of(&func, &order);
1055        let assignment = assign(&func, &order, &live, &env());
1056        assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1057    }
1058
1059    /// Two blocks the entry chooses between, with the one the clobber is in written first. The two
1060    /// values written in the entry block are read in the other one, so their ranges cover the
1061    /// clobber whether or not either of them ever reaches it.
1062    fn arms(reaches: bool) -> Func {
1063        let mut names = Interner::new();
1064        let mut func = Func::new(names.intern("f"));
1065        let opcode = Opcode::new(names.intern("x64.nop"));
1066        let entry = func.create_block();
1067        let arm = func.create_block();
1068        let tail = func.create_block();
1069        let first = func.new_vreg(GPR);
1070        let second = func.new_vreg(GPR);
1071        func.build(entry, opcode).def(first, GPR).finish();
1072        func.build(entry, opcode).def(second, GPR).finish();
1073        *func.succs_mut(entry) = vec![BlockCall::to(arm), BlockCall::to(tail)];
1074        // What a call looks like here: an instruction writing the registers the convention says it
1075        // destroys, named outright so that nothing else may be in them.
1076        func.build(arm, opcode).operand(Operand::write(Reg::physical(RAX), GPR)).finish();
1077        *func.succs_mut(arm) = if reaches { vec![BlockCall::to(tail)] } else { Vec::new() };
1078        func.build(tail, opcode).uses(first, GPR).uses(second, GPR).finish();
1079        func
1080    }
1081
1082    #[test]
1083    fn a_register_a_clobber_takes_beats_the_stack_for_a_value_not_live_in_that_block() {
1084        let func = arms(false);
1085
1086        // Two registers between two values, and a clobber in the arm that takes the first of them.
1087        // The intervals around both values cover the clobber, since the arm is written between the
1088        // two blocks they are live in, and the arm is a hole in both of their areas. So the second
1089        // value has `rax` rather than a stack slot: the arm is a block its own path never goes
1090        // through. tamnd/rucc#982.
1091        assert_eq!(places(&func, &narrow(2)), ["rcx", "rax"]);
1092
1093        let order = Order::of(&func);
1094        let live = Live::of(&func, &order);
1095        let assignment = assign(&func, &order, &live, &narrow(2));
1096        assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1097    }
1098
1099    #[test]
1100    fn a_register_a_clobber_takes_is_not_free_to_a_value_that_is_live_there() {
1101        let func = arms(true);
1102
1103        // The same blocks with an edge from the arm to the tail, which is all it takes: both values
1104        // now arrive at the read either way, so the clobber is on a path they are live over and the
1105        // one register left has to do for both of them.
1106        assert_eq!(places(&func, &narrow(2)), ["rcx", "slot 0"]);
1107    }
1108
1109    /// A value and the instruction that destroys a register, written one after the other, with the
1110    /// value read by that instruction or by the one after it.
1111    fn dies_at_the_clobber(here: bool) -> Func {
1112        let mut names = Interner::new();
1113        let mut func = Func::new(names.intern("f"));
1114        let opcode = Opcode::new(names.intern("x64.nop"));
1115        let entry = func.create_block();
1116        let value = func.new_vreg(GPR);
1117        func.build(entry, opcode).def(value, GPR).finish();
1118        let call = func.build(entry, opcode).operand(Operand::write(Reg::physical(RAX), GPR));
1119        if here {
1120            call.uses(value, GPR).finish();
1121        } else {
1122            call.finish();
1123            func.build(entry, opcode).uses(value, GPR).finish();
1124        }
1125        func
1126    }
1127
1128    /// A value whose last read is the instruction that destroys a register may be in that register,
1129    /// because the instruction reads what it is handed before it writes anything.
1130    ///
1131    /// The call is what this is about, and the value a call is passed is the case: seven registers
1132    /// on this machine are destroyed by one, every argument dies at the call that reads it, and
1133    /// refusing all seven to those values left them taking a callee saved register for a life two
1134    /// instructions long and paying for it in the prologue and the epilogue. tamnd/rucc#1232.
1135    #[test]
1136    fn a_value_that_dies_where_a_register_is_destroyed_may_be_in_that_register() {
1137        let func = dies_at_the_clobber(true);
1138        assert_eq!(places(&func, &narrow(1)), ["rax"]);
1139
1140        let order = Order::of(&func);
1141        let live = Live::of(&func, &order);
1142        let assignment = assign(&func, &order, &live, &narrow(1));
1143        assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1144    }
1145
1146    /// And one read later than that is one the instruction really does destroy, which is the same
1147    /// function with the read moved down by one instruction.
1148    #[test]
1149    fn a_value_read_after_the_instruction_that_destroys_a_register_is_not_in_it() {
1150        let func = dies_at_the_clobber(false);
1151        assert_eq!(places(&func, &narrow(1)), ["slot 0"]);
1152    }
1153
1154    #[test]
1155    fn a_hint_is_followed_when_the_register_is_clear_and_not_when_it_is_merely_allowed() {
1156        let mut names = Interner::new();
1157        let mut func = Func::new(names.intern("f"));
1158        let opcode = Opcode::new(names.intern("x64.nop"));
1159        let entry = func.create_block();
1160        let mid = func.create_block();
1161        let tail = func.create_block();
1162        let first = func.new_vreg(GPR);
1163        let second = func.new_vreg(GPR);
1164        func.build(entry, opcode).def(first, GPR).finish();
1165        func.build(entry, opcode).def(second, GPR).finish();
1166        *func.succs_mut(entry) = vec![BlockCall::to(mid), BlockCall::to(tail)];
1167        // Two arms, each ending in an instruction that wants its own value in `rax`, which is what
1168        // a return out of either side of a branch looks like.
1169        func.build(mid, opcode)
1170            .operand(Operand::read(second, GPR).with(Constraint::Fixed(RAX)))
1171            .finish();
1172        func.build(tail, opcode)
1173            .operand(Operand::read(first, GPR).with(Constraint::Fixed(RAX)))
1174            .finish();
1175
1176        // The first value is hinted at `rax` and does not get it, because the other arm wants `rax`
1177        // for the other value and the first value's range reaches that far. Following the hint here
1178        // would save a move in the tail and cost one in the middle, and the second value gets `rax`
1179        // with nothing moved anywhere instead.
1180        assert_eq!(places(&func, &env()), ["rcx", "rax"]);
1181    }
1182
1183    #[test]
1184    fn a_value_living_in_a_hole_of_another_gets_the_same_register() {
1185        let mut names = Interner::new();
1186        let mut func = Func::new(names.intern("f"));
1187        let opcode = Opcode::new(names.intern("x64.nop"));
1188        let entry = func.create_block();
1189        let arm = func.create_block();
1190        let tail = func.create_block();
1191        let across = func.new_vreg(GPR);
1192        let inside = func.new_vreg(GPR);
1193        func.build(entry, opcode).def(across, GPR).finish();
1194        *func.succs_mut(entry) = vec![BlockCall::to(arm), BlockCall::to(tail)];
1195        func.build(arm, opcode).def(inside, GPR).finish();
1196        func.build(arm, opcode).uses(inside, GPR).finish();
1197        func.build(tail, opcode).uses(across, GPR).finish();
1198
1199        // One register between the two of them, and one register is enough. Nothing in the arm can
1200        // reach the read in the tail, so the value the arm makes is welcome to the register the
1201        // value crossing the function is in. The interval around that value covers the arm and the
1202        // value is nowhere near it, which is what used to send one of the two to the stack.
1203        // tamnd/rucc#982.
1204        assert_eq!(places(&func, &narrow(1)), ["rax", "rax"]);
1205
1206        let order = Order::of(&func);
1207        let live = Live::of(&func, &order);
1208        let assignment = assign(&func, &order, &live, &narrow(1));
1209        assert_eq!(assignment.spilled(), 0);
1210        assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1211    }
1212
1213    #[test]
1214    fn a_register_a_clobber_takes_is_the_last_one_offered_rather_than_the_first() {
1215        let func = arms(false);
1216
1217        // With a register to spare the value takes the spare one. Being allowed a register some
1218        // instruction insists on is not the same as it being free: the instruction has to be handed
1219        // it in the end, and what hands it over is a move.
1220        assert_eq!(places(&func, &narrow(3)), ["rcx", "rdx"]);
1221    }
1222
1223    #[test]
1224    fn a_frame_says_what_each_of_its_slots_is_for() {
1225        let mut names = Interner::new();
1226        let mut func = Func::new(names.intern("f"));
1227        let opcode = Opcode::new(names.intern("x64.nop"));
1228        let block = func.create_block();
1229        let first = func.new_vreg(GPR);
1230        let second = func.new_vreg(GPR);
1231        func.build(block, opcode).def(first, GPR).finish();
1232        func.build(block, opcode).def(second, GPR).finish();
1233        func.build(block, opcode).uses(first, GPR).uses(second, GPR).finish();
1234
1235        let order = Order::of(&func);
1236        let live = Live::of(&func, &order);
1237        let assignment = assign(&func, &order, &live, &narrow(1));
1238        assert_eq!(assignment.spilled(), 1);
1239        assert_eq!(assignment.slots(), [GPR]);
1240        // A register that is already a register is where it is, and this has nothing to say about
1241        // it.
1242        assert_eq!(assignment.place(Reg::physical(RCX)), None);
1243        assert_eq!(env().scratch(GPR), [R13, R14, R15]);
1244    }
1245}