rucc_regalloc/assign.rs
1//! Which register each value lives in, and which values live on the stack instead.
2//!
3//! Design: `spec/10-backend.md` section 10.4.
4//!
5//! This is the `-O0` allocator's decision and nothing else. It is linear scan over the line
6//! [`crate::order`] lays the function out in: the values are taken in the order they are written,
7//! each is given a register that nothing else live at the same time is in, and when there is no
8//! such register one of the values in flight goes to the stack instead. There is no splitting and
9//! no coalescing, so a value gets one place for the whole of its range and keeps it. That produces
10//! mediocre code quickly, which is what `-O0` is for, and the allocator that produces good code
11//! slowly is a separate one, in M4.
12//!
13//! Which value is sent to the stack is the one whose range ends last, counting the value being
14//! placed among the candidates. A value wanted for a long time is the cheapest to spill per
15//! instruction it frees a register over, and it is the only heuristic here. What is picked is
16//! really a register and not a value, since two values that are never both wanted share one, and
17//! then every value in that register which is in this one's way goes.
18//!
19//! # Where the line is not the function
20//!
21//! The line is the order the blocks arrived in, and `crate::layout` puts them in a different one
22//! afterwards, so being between two blocks on the line says nothing about being between them in
23//! the code. A value live in one loop and live again in a later one is written down with
24//! everything in between inside the interval around it, and it is not live in any of it.
25//!
26//! Which is why what decides anything here is the area from `crate::live`, and the interval is
27//! only the sweep's bookkeeping: it says which values to compare and the areas say which of them
28//! actually collide. Three loops one after another in a function put a dozen values in flight at
29//! the same instant of the line and never at the same instant of the program, and asking the
30//! interval would spill the one this loop is walking for the sake of eleven values in the other
31//! two. tamnd/rucc#982.
32//!
33//! The same holds for a register an instruction insists on. A call destroys seven registers on
34//! x86-64, and a function whose blocks happen to arrive with a call written between the blocks of
35//! a loop would otherwise lose all seven for every value in that loop, for a call the loop never
36//! reaches, so that question is asked of the area and not of the interval either.
37//!
38//! Allowed is not the same as free, though, so the registers are offered in two passes. First the
39//! ones nothing insists on anywhere the range reaches, then the ones something insists on somewhere
40//! the value never goes. The second kind costs: the instruction that insists has to be handed the
41//! register in the end, and what hands it over is a move. A function that gives a value back has an
42//! operand fixed to `rax` at the end of it, and putting the busiest value in the function in `rax`
43//! because no path reaches the return with it live buys one register and pays a move at every
44//! return. Ordering the two passes is what keeps the register and drops the moves.
45//!
46//! The hint below is asked the first question rather than the second for the same reason. A value
47//! taking the register its own operand asked for saves a move, and taking one somebody else's
48//! operand asked for somewhere it never goes costs one, so a hint is worth following when the
49//! register is clear and not worth following when it is merely allowed.
50//!
51//! # What it does with a register an instruction insists on
52//!
53//! Two things. It stays out of that register for everybody else, and it tries that register first
54//! for the value the operand names. A division wants its dividend in `rax`, so `rax` is
55//! unavailable to every other value that is live where the division reads, and it is the first
56//! register offered to the dividend itself. When the dividend gets it there is no move on the way
57//! in, and when it does not the rewrite writes one and nothing else changes.
58//!
59//! That second half is the hint, and without it the register an instruction insists on is the one
60//! register the value in it can never have, since the value's own operand is what makes the
61//! register look busy. The effect is largest on returns, because a function that gives a value
62//! back has an operand fixed to `rax` at the end of it and most functions give a value back.
63//!
64//! What makes the hint safe is asking about the register at each of the instruction's two points
65//! rather than across the whole of it. An instruction reads at the first and writes at the second,
66//! so a register it insists on is one value's at the first, another value's at the second, and
67//! nobody else's at either. A division reads its dividend from `rax` and writes its quotient to
68//! `rax`, and those are different values that can both live there. A value passed to a call in
69//! `rdi` and wanted again afterwards cannot, because nothing writes `rdi` at the second point and
70//! a register the call does not write is a register the call is assumed to destroy.
71//!
72//! An operand that has to be in memory is the other way round. The value it names goes on the
73//! stack whatever else is true of it, because that is the only place the instruction could read it
74//! from.
75//!
76//! # What it does with a two address instruction
77//!
78//! An `add` on x86-64 writes one of the registers it reads, which the operand says as a reuse of
79//! another operand. The rewrite can always make that true by copying the source into the
80//! destination first, but only if the destination is a register the instruction does not otherwise
81//! read, so a value written by a reuse is treated here as live from where the instruction reads
82//! rather than from where it writes. Then the copy is always safe.
83//!
84//! The copy is also usually unnecessary, and the one place this looks past the interval it is
85//! placing is to see that: if the value being reused is read here for the last time and the value
86//! being written starts here, the second may have the first's register, and the instruction is
87//! already two address without anything being moved anywhere. That is the whole of the coalescing
88//! this allocator does, and it is worth the dozen lines, because otherwise every piece of
89//! arithmetic in the output carries a move in front of it.
90//!
91//! Both halves of that are needed. The second is the one a loop breaks: an instruction at the
92//! bottom of a loop can write a value the top of the loop reads on the next turn, and such a value
93//! is live on the way into the instruction that writes it as well as after. It is then wanted at
94//! the same time as the value it reuses, whatever is true of the reuse, and giving it the same
95//! register makes an addition read the answer to the last one instead of its own operand.
96//!
97//! # What it does not do
98//!
99//! It does not touch the function. What comes out is a table saying where each value went, and the
100//! pass that rewrites the operands and writes the moves reads it. Keeping the decision and the
101//! rewrite apart is what lets the decision be checked by looking at it, and it is the shape
102//! `spec/10-backend.md` section 10.4 asks for: an allocator is a function from a program to an
103//! assignment and the moves that make it true.
104
105use std::cmp::Reverse;
106
107use rucc_mir::{Constraint, Func, Operand, Reg, Role};
108use rucc_target::{PhysReg, RegClass};
109
110use crate::live::{Area, Live, Range};
111use crate::order::{Order, Point};
112
113/// Where a value lives.
114#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
115pub enum Place {
116 /// In a register, for the whole of its range.
117 Reg(PhysReg),
118 /// In a slot of the frame, which is what a value the allocator ran out of registers for gets,
119 /// and what a value an instruction can only read from memory gets.
120 Slot(u32),
121}
122
123/// What the allocator is allowed to use.
124///
125/// The order is the calling convention's, because which register to hand out first follows from
126/// which ones a call destroys, and `rucc-target` is where a convention says so. The scratch
127/// registers are held back out of the order and are what a spilled value is read into at each
128/// instruction that wants it, so a class needs as many of them as one of its instructions has
129/// register operands. Nothing here uses them, since a spilled value is only read once the rewrite
130/// is writing the instruction that reads it, but they are held back here because this is what
131/// decides what everything else may have.
132#[derive(Debug, Clone, Default)]
133pub struct Env {
134 classes: Vec<Class>,
135}
136
137/// What one class of registers offers.
138#[derive(Debug, Clone, Default)]
139struct Class {
140 order: Vec<PhysReg>,
141 scratch: Vec<PhysReg>,
142}
143
144impl Env {
145 /// An environment offering nothing, which is what a target that has said nothing offers.
146 #[must_use]
147 pub fn new() -> Self {
148 Self::default()
149 }
150
151 /// The same environment, with that class described.
152 #[must_use]
153 pub fn with(mut self, class: RegClass, order: &[PhysReg], scratch: &[PhysReg]) -> Self {
154 let index = usize::from(class.number());
155 if self.classes.len() <= index {
156 self.classes.resize(index + 1, Class::default());
157 }
158 self.classes[index] = Class { order: order.to_vec(), scratch: scratch.to_vec() };
159 self
160 }
161
162 /// The registers it may hand out in a class, in the order it prefers them.
163 #[must_use]
164 pub fn order(&self, class: RegClass) -> &[PhysReg] {
165 self.classes.get(usize::from(class.number())).map_or(&[], |class| &class.order)
166 }
167
168 /// The registers held back in a class for reading a spilled value into.
169 #[must_use]
170 pub fn scratch(&self, class: RegClass) -> &[PhysReg] {
171 self.classes.get(usize::from(class.number())).map_or(&[], |class| &class.scratch)
172 }
173}
174
175/// Where every value in a function went.
176#[derive(Debug, Clone)]
177pub struct Assignment {
178 places: Vec<Option<Place>>,
179 slots: Vec<RegClass>,
180}
181
182impl Assignment {
183 /// An assignment that says nothing yet about a function with that many values.
184 ///
185 /// This and [`Assignment::put`] and [`Assignment::take_slot`] are how an allocator says what
186 /// it decided. There will be a second one in M4 and it will not reach its answer this way, so
187 /// what an assignment is has to be separable from how this file arrives at one, and the
188 /// checker in [`crate::check`] reads an assignment without caring which allocator wrote it.
189 #[must_use]
190 pub fn empty(vregs: usize) -> Self {
191 Self { places: vec![None; vregs], slots: Vec::new() }
192 }
193
194 /// Records where a value went.
195 ///
196 /// # Panics
197 ///
198 /// Panics on a physical register, which is somewhere already, and on a virtual one the
199 /// function never handed out.
200 pub fn put(&mut self, reg: Reg, place: Place) {
201 self.places[index(reg)] = Some(place);
202 }
203
204 /// Takes a slot of the frame, of that class, and gives back which one it is.
205 ///
206 /// # Panics
207 ///
208 /// Panics past four billion slots, which is a frame no machine has room for.
209 pub fn take_slot(&mut self, class: RegClass) -> u32 {
210 let slot = u32::try_from(self.slots.len()).expect("too many spilled values");
211 self.slots.push(class);
212 slot
213 }
214
215 /// Where a value lives, or `None` for a virtual register this function never mentions and for
216 /// a physical one, which is already where it is.
217 #[must_use]
218 pub fn place(&self, reg: Reg) -> Option<Place> {
219 self.places.get(usize::try_from(reg.number()?).ok()?).copied().flatten()
220 }
221
222 /// The class of each slot of the frame, which is what says how wide it has to be.
223 #[must_use]
224 pub fn slots(&self) -> &[RegClass] {
225 &self.slots
226 }
227
228 /// Every value that went somewhere, and where it went.
229 ///
230 /// The assignment read the other way round, which is what a caller wants when the question is
231 /// about the places rather than about the values. The stack slot allocator asks it that way,
232 /// since what it needs is which value is in each slot and the assignment is stored by value.
233 pub fn placed(&self) -> impl Iterator<Item = (Reg, Place)> + '_ {
234 self.places.iter().enumerate().filter_map(|(number, place)| {
235 let number = u32::try_from(number).ok()?;
236 Some((Reg::virtual_reg(number), (*place)?))
237 })
238 }
239
240 /// How many values went to the stack.
241 #[must_use]
242 pub fn spilled(&self) -> usize {
243 self.slots.len()
244 }
245
246 /// Puts a value on the stack, in a slot of its own.
247 fn spill(&mut self, reg: Reg, class: RegClass) {
248 let slot = self.take_slot(class);
249 self.put(reg, Place::Slot(slot));
250 }
251}
252
253/// One value waiting for a place.
254#[derive(Debug, Clone, Copy)]
255struct Interval<'a> {
256 reg: Reg,
257 class: RegClass,
258 /// The interval around the area, which is what the sweep below reads and what says which value
259 /// is wanted for longest when one of them has to go.
260 range: Range,
261 /// Everywhere the value is really live, which is what says whether two of them fit in one
262 /// register.
263 area: Area<'a>,
264}
265
266/// One value that has a register, for as long as it still wants it.
267#[derive(Debug, Clone, Copy)]
268struct Held<'a> {
269 reg: Reg,
270 class: RegClass,
271 range: Range,
272 area: Area<'a>,
273 at: PhysReg,
274}
275
276/// A register an instruction insists on, and where it insists on it.
277#[derive(Debug, Clone, Copy)]
278struct Blocked {
279 class: RegClass,
280 at: PhysReg,
281 /// One of the instruction's two points. Every register an instruction insists on has an entry
282 /// at each of them, because a register held at one of the two is a register nothing else may
283 /// be in across the instruction.
284 point: Point,
285 /// The one value that may be in it there, which is the value of an operand the instruction
286 /// reads at that point or writes at it. `None` means nothing may: an operand naming a physical
287 /// register outright claims it against everything, and a point no operand covers is a point
288 /// the instruction has the register to itself at.
289 by: Option<Reg>,
290}
291
292/// A value written into the register another operand of the same instruction was read from.
293#[derive(Debug, Clone, Copy)]
294struct Reuse {
295 /// The value being read, which is the one whose register would do.
296 source: Reg,
297 /// Where the instruction reads it.
298 at: Point,
299}
300
301/// Decides where every value in a function lives.
302///
303/// # Panics
304///
305/// Panics if a class has no registers to hand out and something in the function is in that class,
306/// since that is a target description that does not describe the target the function is for.
307#[must_use]
308pub fn assign(func: &Func, order: &Order, live: &Live, env: &Env) -> Assignment {
309 let blocked = blocked(func, order);
310 let forced = forced(func);
311 let reuses = reuses(func, order);
312 let hints = hints(func);
313
314 let mut intervals = Vec::with_capacity(func.vregs());
315 for (number, reuse) in reuses.iter().enumerate() {
316 let reg = Reg::virtual_reg(u32::try_from(number).expect("a register number"));
317 let (Some(mut area), Some(class)) = (live.area(reg), func.class_of(reg)) else {
318 continue;
319 };
320 if let Some(reuse) = reuse {
321 area = area.with(reuse.at);
322 }
323 intervals.push(Interval { reg, class, range: area.hull(), area });
324 }
325 intervals.sort_by_key(|interval| (interval.range.start, interval.reg));
326
327 let mut assignment = Assignment::empty(func.vregs());
328 let mut active: Vec<Held<'_>> = Vec::new();
329 for interval in intervals {
330 active.retain(|held| held.range.end >= interval.range.start);
331 if forced.contains(&interval.reg) {
332 assignment.spill(interval.reg, interval.class);
333 continue;
334 }
335 // A class with no order is one the target says nothing allocates from, which on x86-64 is
336 // the x87 stack. A value of such a class is a mistake at the point it was made rather than
337 // a value with nowhere to go: what the target means is that the value lives in memory and
338 // that whatever operates on it takes an address. See `ClassInfo::allocatable`.
339 assert!(
340 !env.order(interval.class).is_empty(),
341 "a value in class {}, which the target hands out no registers from",
342 interval.class.number()
343 );
344 let two_address = reuses[index(interval.reg)]
345 .and_then(|reuse| coalesce(&assignment, &active, &blocked, live, interval, reuse));
346 // The reuse comes first, because a two address instruction that has to copy its left
347 // operand in pays for the copy whatever the hint says, and taking the hint here would buy
348 // one move at the cost of another.
349 let hinted = hints[index(interval.reg)].iter().copied().find(|&at| {
350 env.order(interval.class).contains(&at)
351 && available(&active, &blocked, interval, at, None, Want::Clear)
352 });
353 // A register nobody else wants anywhere near this value first, and one somebody wants
354 // somewhere the value never goes only when there is no other. Both are correct and the
355 // second is the worse buy, since the instruction that wants it has to be handed it and
356 // whatever this value is doing there has to move out of the way first.
357 let scan = |want| {
358 env.order(interval.class)
359 .iter()
360 .copied()
361 .find(|&at| available(&active, &blocked, interval, at, None, want))
362 };
363 let chosen =
364 two_address.or(hinted).or_else(|| scan(Want::Clear)).or_else(|| scan(Want::Allowed));
365 match chosen {
366 Some(at) => {
367 assignment.places[index(interval.reg)] = Some(Place::Reg(at));
368 active.push(Held {
369 reg: interval.reg,
370 class: interval.class,
371 range: interval.range,
372 area: interval.area,
373 at,
374 });
375 }
376 None => spill_one(&mut assignment, &mut active, &blocked, interval),
377 }
378 }
379 assignment
380}
381
382/// How much a register suits an interval.
383#[derive(Debug, Clone, Copy, PartialEq, Eq)]
384enum Want {
385 /// Nothing insists on it anywhere the range reaches, so taking it costs nobody anything.
386 Clear,
387 /// Something insists on it somewhere the range reaches and nowhere the value is live, so taking
388 /// it is allowed and may still cost: the instruction that insists wants the register for a
389 /// value of its own, and that value now has to be moved into it.
390 Allowed,
391}
392
393/// Every register every instruction in the function insists on, arranged to be asked about.
394///
395/// Built once and never changed afterwards, and there is only one question ever asked of it: of the
396/// constraints naming one register of one class, is there one at a point some interval covers. So
397/// the entries are ordered by the register they name and then by the point, and the question is a
398/// binary search for the start of the interval followed by a walk that stops at its end.
399///
400/// It used to be a flat list walked from one end for every candidate register of every interval,
401/// which is quadratic in the size of a function and is most of the compile on a large one. See
402/// tamnd/rucc#1003 for the profile that found it.
403struct Blocks {
404 /// The constraints, sorted by class, then by register, then by point.
405 all: Vec<Blocked>,
406}
407
408impl Blocks {
409 /// The constraints on one register of one class at the points an interval covers.
410 ///
411 /// Both ends of the walk come from the ordering rather than from a test, so what comes back is
412 /// exactly what the old `covers` call used to keep and in the same order.
413 fn over(
414 &self,
415 class: RegClass,
416 at: PhysReg,
417 range: Range,
418 ) -> impl Iterator<Item = &Blocked> + '_ {
419 let first = self
420 .all
421 .partition_point(|one| (one.class, one.at, one.point) < (class, at, range.start));
422 self.all[first..]
423 .iter()
424 .take_while(move |one| one.class == class && one.at == at && one.point <= range.end)
425 }
426}
427
428/// Whether a register is one this interval could have.
429///
430/// The exception is the value a reuse is coalescing with, which holds the register right up to the
431/// point the new value takes it over and is the one thing that may overlap.
432///
433/// The sweep only keeps a value in `active` while the interval around it reaches this one, so the
434/// areas still have to be compared: two values whose intervals cross can have holes that let them
435/// share a register anyway, which on a function with several loops in it is most of them.
436fn available(
437 active: &[Held<'_>],
438 blocked: &Blocks,
439 interval: Interval<'_>,
440 at: PhysReg,
441 except: Option<Reg>,
442 want: Want,
443) -> bool {
444 let taken = active.iter().any(|held| {
445 held.at == at
446 && held.class == interval.class
447 && Some(held.reg) != except
448 && held.area.overlaps(interval.area)
449 });
450 let insisted = blocked.over(interval.class, at, interval.range).any(|one| {
451 one.by != Some(interval.reg) && (want == Want::Clear || interval.area.covers(one.point))
452 });
453 !taken && !insisted
454}
455
456/// The register the value being reused is in, when this instruction is the last thing that reads
457/// it, the value being written starts here, and the register is otherwise free.
458fn coalesce(
459 assignment: &Assignment,
460 active: &[Held<'_>],
461 blocked: &Blocks,
462 live: &Live,
463 interval: Interval<'_>,
464 reuse: Reuse,
465) -> Option<PhysReg> {
466 let Some(Place::Reg(at)) = assignment.place(reuse.source) else { return None };
467 let source = active.iter().find(|held| held.reg == reuse.source)?;
468 // A value read again later needs its register after this instruction would have overwritten
469 // it, so the two really do have to be different and the rewrite really does have to copy.
470 let dies = source.range.end == reuse.at;
471 // And the value being written must not be live where the instruction reads already. The area
472 // asked here is the one liveness worked out, without the point the reuse adds, so a value that
473 // covers the reuse point on its own is one that was live on the way into this instruction. That
474 // is what a loop carrying its own result round looks like: the instruction writes it at the
475 // bottom and the top of the loop reads what the last turn wrote. Such a value overlaps the one
476 // it reuses over the whole loop, so the two cannot be the same register no matter that the read
477 // here is the last one.
478 let begins = live.area(interval.reg).is_some_and(|area| !area.covers(reuse.at));
479 let free = available(active, blocked, interval, at, Some(reuse.source), Want::Allowed);
480 (dies && begins && free).then_some(at)
481}
482
483/// Sends values to the stack to free a register: the ones wanted for longest, since a register
484/// held that long pays for itself over the most instructions.
485///
486/// What is chosen is a register rather than a value, because two values whose areas miss each
487/// other share one and taking it means every value in it this one is really on top of has to go.
488/// A register holding two of those costs twice as much to take as one holding a single value, so
489/// the cheap ones are looked at first and the reach only settles ties.
490fn spill_one<'a>(
491 assignment: &mut Assignment,
492 active: &mut Vec<Held<'a>>,
493 blocked: &Blocks,
494 interval: Interval<'a>,
495) {
496 // What each register would cost: how many values would go, and the furthest any of them
497 // reaches. The list is one entry per register of the class, so walking it for each value in
498 // flight is the same shape as everything else here.
499 let mut costs: Vec<(PhysReg, usize, Point)> = Vec::new();
500 for held in active.iter() {
501 if held.class != interval.class || !held.area.overlaps(interval.area) {
502 continue;
503 }
504 match costs.iter_mut().find(|(at, _, _)| *at == held.at) {
505 Some((_, count, reach)) => {
506 *count += 1;
507 *reach = (*reach).max(held.range.end);
508 }
509 None => costs.push((held.at, 1, held.range.end)),
510 }
511 }
512 // A register the instructions in the way insist on for themselves is no use, because taking it
513 // over would put this value in a register it may not have.
514 let chosen = costs
515 .iter()
516 .filter(|&&(at, _, reach)| {
517 reach > interval.range.end && available(&[], blocked, interval, at, None, Want::Allowed)
518 })
519 .min_by_key(|&&(_, count, reach)| (count, Reverse(reach)))
520 .map(|&(at, _, _)| at);
521 match chosen {
522 Some(at) => {
523 active.retain(|held| {
524 let goes = held.at == at
525 && held.class == interval.class
526 && held.area.overlaps(interval.area);
527 if goes {
528 assignment.spill(held.reg, held.class);
529 }
530 !goes
531 });
532 assignment.places[index(interval.reg)] = Some(Place::Reg(at));
533 active.push(Held {
534 reg: interval.reg,
535 class: interval.class,
536 range: interval.range,
537 area: interval.area,
538 at,
539 });
540 }
541 None => assignment.spill(interval.reg, interval.class),
542 }
543}
544
545/// The registers the instructions insist on, and where.
546///
547/// A physical register an operand names outright counts the same way. Nothing before allocation
548/// writes one except an instruction that has to, and it has to for the length of that one
549/// instruction, which is the same statement a fixed constraint makes.
550fn blocked(func: &Func, order: &Order) -> Blocks {
551 let mut blocked = Vec::new();
552 let mut claimed: Vec<(RegClass, PhysReg)> = Vec::new();
553 for block in func.blocks() {
554 for inst in func.insts(block) {
555 let operands = &func[func[inst].operands];
556 claimed.clear();
557 for operand in operands {
558 if let Some(at) = insisted(operand) {
559 let key = (operand.class, at);
560 if !claimed.contains(&key) {
561 claimed.push(key);
562 }
563 }
564 }
565 for &(class, at) in &claimed {
566 // Both points, whether or not an operand is at them. A register an instruction
567 // reads and does not write is still gone by the time the instruction is done as far
568 // as anything here knows, which is what stops the value a call is passed in `rdi`
569 // from staying in `rdi` over the call.
570 for (point, role) in [(order.early(inst), Role::Use), (order.late(inst), Role::Def)]
571 {
572 let mut named = false;
573 for operand in operands {
574 let mine = insisted(operand) == Some(at) && operand.class == class;
575 if !mine || !(operand.role == role || operand.role == Role::EarlyDef) {
576 continue;
577 }
578 named = true;
579 let by = operand.reg.is_virtual().then_some(operand.reg);
580 blocked.push(Blocked { class, at, point, by });
581 }
582 // A register no operand names where the operands are read is one the
583 // instruction writes and does not read, which is what a clobber is, and the
584 // seven registers a call destroys are the whole of why that case is worth
585 // separating. Such a register is free right up to the point it is written, so a
586 // value whose last read is this instruction may sit in one: it is read before
587 // the instruction writes anything, the way any other operand is. Blocking it
588 // where the operands are read as well would take every caller saved register
589 // away from the value a call is passed, which is a value that dies at the call
590 // and pays for a callee saved register it holds for two instructions. Anything
591 // living past the instruction is still refused, by the block below.
592 //
593 // This is where a target's early definitions are paid for. An instruction that
594 // fills a register before it has finished reading has to say so, because that
595 // is the one thing a plain definition here no longer covers: a division on
596 // x86-64 is a sign extension and then the division itself, so `rdx` is gone
597 // before the divisor is read, and a divisor that went there would be read as
598 // the dividend's own sign bits. `rucc_target::x86_64` writes both of them down
599 // as early definitions for exactly that reason.
600 if !named && role == Role::Def {
601 blocked.push(Blocked { class, at, point, by: None });
602 }
603 }
604 }
605 }
606 }
607 // Program order already has the points ascending, but the registers one instruction claims are
608 // walked outside the two points rather than inside them, so the list arrives in order by
609 // instruction and not by register. A sort by the key the lookup searches on is what makes it
610 // searchable, and it is stable so two constraints on one register at one point keep the order
611 // the instruction wrote them in.
612 blocked.sort_by_key(|one: &Blocked| (one.class, one.at, one.point));
613 Blocks { all: blocked }
614}
615
616/// The register an operand has to be in, which is the one a constraint asks for or the one the
617/// operand names outright.
618fn insisted(operand: &Operand) -> Option<PhysReg> {
619 match operand.constraint {
620 Constraint::Fixed(at) => Some(at),
621 _ => operand.reg.phys(),
622 }
623}
624
625/// The registers each value would rather be in, which are the ones the operands naming it insist on.
626///
627/// In the order the function writes them down, so the definition comes first where there is one,
628/// since a value written into a fixed register and then moved somewhere else pays for the move at
629/// the top of its life rather than at the bottom. The ones after it are worth keeping for the same
630/// reason the first one is, and the value a call is passed is where that shows: its definition may
631/// insist on the register a parameter arrived in, which the call it is handed to has usually taken
632/// back for an argument of its own by then, and behind that is the register the convention passes
633/// it in, which is free and is exactly where the value wants to end up.
634fn hints(func: &Func) -> Vec<Vec<PhysReg>> {
635 let mut hints = vec![Vec::new(); func.vregs()];
636 for block in func.blocks() {
637 for inst in func.insts(block) {
638 for operand in &func[func[inst].operands] {
639 let Constraint::Fixed(at) = operand.constraint else { continue };
640 let number = operand.reg.number().and_then(|number| usize::try_from(number).ok());
641 let Some(number) = number else { continue };
642 let wanted: &mut Vec<PhysReg> = &mut hints[number];
643 if func.class_of(operand.reg) == Some(operand.class) && !wanted.contains(&at) {
644 wanted.push(at);
645 }
646 }
647 }
648 }
649 hints
650}
651
652/// The values that have to be on the stack whatever else is true of them.
653fn forced(func: &Func) -> Vec<Reg> {
654 let mut forced = Vec::new();
655 for block in func.blocks() {
656 for inst in func.insts(block) {
657 for operand in &func[func[inst].operands] {
658 if operand.constraint == Constraint::Stack
659 && operand.reg.is_virtual()
660 && !forced.contains(&operand.reg)
661 {
662 forced.push(operand.reg);
663 }
664 }
665 }
666 }
667 forced
668}
669
670/// The value each two address instruction reuses, by the virtual register it writes.
671fn reuses(func: &Func, order: &Order) -> Vec<Option<Reuse>> {
672 let mut reuses = vec![None; func.vregs()];
673 for block in func.blocks() {
674 for inst in func.insts(block) {
675 let operands = &func[func[inst].operands];
676 for operand in operands {
677 let Constraint::Reuse(other) = operand.constraint else { continue };
678 let number = operand.reg.number().and_then(|number| usize::try_from(number).ok());
679 let Some(number) = number else { continue };
680 let source = operands[usize::from(other)].reg;
681 reuses[number] = Some(Reuse { source, at: order.early(inst) });
682 }
683 }
684 }
685 reuses
686}
687
688/// A virtual register's number as a table index.
689fn index(reg: Reg) -> usize {
690 usize::try_from(reg.number().expect("a virtual register")).expect("a register number")
691}
692
693#[cfg(test)]
694mod tests {
695 use rucc_base::Interner;
696 use rucc_mir::{BlockCall, Opcode, Operand};
697 use rucc_target::x86_64::{GPR, R13, R14, R15, RAX, RCX, RDX, REGS, SYSV};
698
699 use super::*;
700
701 /// The x86-64 environment, with the last three of the allocation order held back as scratch.
702 fn env() -> Env {
703 let (order, scratch) = SYSV.int_order.split_at(SYSV.int_order.len() - 3);
704 Env::new().with(GPR, order, scratch)
705 }
706
707 /// An environment with that many general purpose registers, for putting a function under
708 /// pressure without writing a hundred instructions.
709 fn narrow(count: usize) -> Env {
710 Env::new().with(GPR, &SYSV.int_order[..count], &SYSV.int_order[count..count + 1])
711 }
712
713 /// What a place is called, which is what an assertion reads.
714 fn named(place: Option<Place>) -> String {
715 match place {
716 Some(Place::Reg(reg)) => REGS.name(GPR, reg).expect("a register").to_string(),
717 Some(Place::Slot(slot)) => format!("slot {slot}"),
718 None => "nowhere".to_string(),
719 }
720 }
721
722 /// Where every value in a function went.
723 fn places(func: &Func, env: &Env) -> Vec<String> {
724 let order = Order::of(func);
725 let live = Live::of(func, &order);
726 let assignment = assign(func, &order, &live, env);
727 (0..func.vregs())
728 .map(|number| {
729 let reg = Reg::virtual_reg(u32::try_from(number).expect("a register number"));
730 named(assignment.place(reg))
731 })
732 .collect()
733 }
734
735 #[test]
736 fn two_values_that_are_never_both_wanted_share_a_register() {
737 let mut names = Interner::new();
738 let mut func = Func::new(names.intern("f"));
739 let opcode = Opcode::new(names.intern("x64.nop"));
740 let block = func.create_block();
741 let first = func.new_vreg(GPR);
742 let second = func.new_vreg(GPR);
743 func.build(block, opcode).def(first, GPR).finish();
744 func.build(block, opcode).uses(first, GPR).finish();
745 func.build(block, opcode).def(second, GPR).finish();
746 func.build(block, opcode).uses(second, GPR).finish();
747
748 // The first register in the order, twice, because the first value is finished with before
749 // the second one is written.
750 assert_eq!(places(&func, &env()), ["rax", "rax"]);
751 }
752
753 #[test]
754 fn two_values_that_are_both_wanted_do_not() {
755 let mut names = Interner::new();
756 let mut func = Func::new(names.intern("f"));
757 let opcode = Opcode::new(names.intern("x64.nop"));
758 let block = func.create_block();
759 let first = func.new_vreg(GPR);
760 let second = func.new_vreg(GPR);
761 func.build(block, opcode).def(first, GPR).finish();
762 func.build(block, opcode).def(second, GPR).finish();
763 func.build(block, opcode).uses(first, GPR).finish();
764 func.build(block, opcode).uses(second, GPR).finish();
765
766 assert_eq!(places(&func, &env()), ["rax", "rcx"]);
767 }
768
769 #[test]
770 fn a_value_written_early_that_nothing_reads_still_holds_its_register() {
771 let mut names = Interner::new();
772 let mut func = Func::new(names.intern("f"));
773 let opcode = Opcode::new(names.intern("x64.nop"));
774 let block = func.create_block();
775 let wanted = func.new_vreg(GPR);
776 let spare = func.new_vreg(GPR);
777 // A division: a remainder somebody wants, and a quotient nobody does. Both are written by
778 // the one instruction and the quotient is written before the operands have been read.
779 func.build(block, opcode)
780 .def(wanted, GPR)
781 .operand(Operand::write_early(spare, GPR))
782 .finish();
783 func.build(block, opcode).uses(wanted, GPR).finish();
784
785 // Two registers, not one. A value nothing reads is still somewhere, and the instruction
786 // that wrote it wrote the other one too, so the two cannot be the same place. Handing them
787 // the same register loses the remainder, because the copy that takes the quotient out of
788 // the register the machine insisted on goes on top of it. The quotient gets the first
789 // register because it is written first, which is the whole of what early means.
790 assert_eq!(places(&func, &env()), ["rcx", "rax"]);
791 }
792
793 #[test]
794 fn the_value_wanted_longest_is_the_one_that_goes_to_the_stack() {
795 let mut names = Interner::new();
796 let mut func = Func::new(names.intern("f"));
797 let opcode = Opcode::new(names.intern("x64.nop"));
798 let block = func.create_block();
799 let long = func.new_vreg(GPR);
800 let short = func.new_vreg(GPR);
801 let third = func.new_vreg(GPR);
802 func.build(block, opcode).def(long, GPR).finish();
803 func.build(block, opcode).def(short, GPR).finish();
804 func.build(block, opcode).def(third, GPR).finish();
805 func.build(block, opcode).uses(short, GPR).finish();
806 func.build(block, opcode).uses(third, GPR).finish();
807 func.build(block, opcode).uses(long, GPR).finish();
808
809 // Two registers between three values. The one still wanted at the end of the function is
810 // the one whose register is worth the most to everybody else, so it is the one that goes.
811 assert_eq!(places(&func, &narrow(2)), ["slot 0", "rcx", "rax"]);
812 }
813
814 #[test]
815 fn a_register_an_instruction_insists_on_goes_to_the_values_that_asked_for_it() {
816 let mut names = Interner::new();
817 let mut func = Func::new(names.intern("f"));
818 let opcode = Opcode::new(names.intern("x64.nop"));
819 let block = func.create_block();
820 let across = func.new_vreg(GPR);
821 let dividend = func.new_vreg(GPR);
822 let quotient = func.new_vreg(GPR);
823 let remainder = func.new_vreg(GPR);
824 func.build(block, opcode).def(across, GPR).finish();
825 func.build(block, opcode).def(dividend, GPR).finish();
826 func.build(block, opcode)
827 .operand(Operand::write(quotient, GPR).with(Constraint::Fixed(RAX)))
828 .operand(Operand::write_early(remainder, GPR).with(Constraint::Fixed(RDX)))
829 .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
830 .finish();
831 func.build(block, opcode).uses(across, GPR).finish();
832
833 // The value that has to be across the division is nowhere near `rax` or `rdx`, and each of
834 // the three the division names is in the register the division asked for it in. The
835 // dividend and the quotient share `rax` because the first is read where the second is
836 // written, which is what a division does.
837 assert_eq!(places(&func, &env()), ["rcx", "rax", "rax", "rdx"]);
838 }
839
840 /// A value read by an instruction that fills a register before it reads is kept out of that
841 /// register, even though the read is the last thing the value is wanted for.
842 ///
843 /// The divisor of a division is the case. What the machine runs is `cltd` and then `idivl`, so
844 /// `rdx` holds the top half of the dividend by the time the divisor is read, and a divisor
845 /// sitting in `rdx` is read as the dividend's own sign bits. An early definition is how the
846 /// target says a register goes before the operands are read, and this is where the allocator
847 /// has to hear it, since a value dying at an instruction is otherwise free to sit in a
848 /// register that instruction writes. tamnd/rucc#1232.
849 #[test]
850 fn a_value_that_dies_at_an_instruction_stays_out_of_what_it_fills_first() {
851 let mut names = Interner::new();
852 let mut func = Func::new(names.intern("f"));
853 let opcode = Opcode::new(names.intern("x64.nop"));
854 let block = func.create_block();
855 let across = func.new_vreg(GPR);
856 let dividend = func.new_vreg(GPR);
857 let divisor = func.new_vreg(GPR);
858 let remainder = func.new_vreg(GPR);
859 func.build(block, opcode).def(across, GPR).finish();
860 func.build(block, opcode).def(dividend, GPR).finish();
861 func.build(block, opcode).def(divisor, GPR).finish();
862 func.build(block, opcode)
863 .operand(Operand::write_early(remainder, GPR).with(Constraint::Fixed(RDX)))
864 .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
865 .operand(Operand::read(divisor, GPR))
866 .finish();
867 func.build(block, opcode).uses(across, GPR).uses(remainder, GPR).finish();
868
869 // Four registers for four values, and the divisor takes the fourth. `rdx` is free
870 // everywhere in this function except at the instruction that is about to fill it, which is
871 // the one instruction the divisor is wanted at.
872 assert_eq!(places(&func, &narrow(4)), ["rcx", "rax", "rsi", "rdx"]);
873 }
874
875 #[test]
876 fn a_value_wanted_after_the_instruction_that_insists_does_not_get_that_register() {
877 let mut names = Interner::new();
878 let mut func = Func::new(names.intern("f"));
879 let opcode = Opcode::new(names.intern("x64.nop"));
880 let block = func.create_block();
881 let dividend = func.new_vreg(GPR);
882 let quotient = func.new_vreg(GPR);
883 func.build(block, opcode).def(dividend, GPR).finish();
884 func.build(block, opcode)
885 .operand(Operand::write(quotient, GPR).with(Constraint::Fixed(RAX)))
886 .operand(Operand::read(dividend, GPR).with(Constraint::Fixed(RAX)))
887 .finish();
888 func.build(block, opcode).uses(dividend, GPR).finish();
889
890 // The hint is a preference and not a claim. The dividend would rather be in `rax` and
891 // cannot be, because the division writes `rax` and the dividend is wanted afterwards, so
892 // it takes the next register and the quotient keeps the one it was promised.
893 assert_eq!(places(&func, &env()), ["rcx", "rax"]);
894 }
895
896 #[test]
897 fn a_value_an_instruction_can_only_read_from_memory_is_on_the_stack() {
898 let mut names = Interner::new();
899 let mut func = Func::new(names.intern("f"));
900 let opcode = Opcode::new(names.intern("x64.nop"));
901 let block = func.create_block();
902 let value = func.new_vreg(GPR);
903 func.build(block, opcode).def(value, GPR).finish();
904 func.build(block, opcode)
905 .operand(Operand::read(value, GPR).with(Constraint::Stack))
906 .finish();
907
908 assert_eq!(places(&func, &env()), ["slot 0"]);
909 }
910
911 #[test]
912 fn a_two_address_instruction_writes_the_register_it_read_when_it_can() {
913 let mut names = Interner::new();
914 let mut func = Func::new(names.intern("f"));
915 let opcode = Opcode::new(names.intern("x64.nop"));
916 let block = func.create_block();
917 let left = func.new_vreg(GPR);
918 let right = func.new_vreg(GPR);
919 let sum = func.new_vreg(GPR);
920 func.build(block, opcode).def(left, GPR).finish();
921 func.build(block, opcode).def(right, GPR).finish();
922 func.build(block, opcode)
923 .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
924 .uses(left, GPR)
925 .uses(right, GPR)
926 .finish();
927 func.build(block, opcode).uses(right, GPR).finish();
928
929 // The addition reads the left value for the last time, so the answer goes where that was
930 // and the instruction is two address without a move in front of it.
931 assert_eq!(places(&func, &env()), ["rax", "rcx", "rax"]);
932 }
933
934 #[test]
935 fn a_two_address_instruction_that_cannot_gets_a_register_nothing_it_reads_is_in() {
936 let mut names = Interner::new();
937 let mut func = Func::new(names.intern("f"));
938 let opcode = Opcode::new(names.intern("x64.nop"));
939 let block = func.create_block();
940 let left = func.new_vreg(GPR);
941 let right = func.new_vreg(GPR);
942 let sum = func.new_vreg(GPR);
943 func.build(block, opcode).def(left, GPR).finish();
944 func.build(block, opcode).def(right, GPR).finish();
945 func.build(block, opcode)
946 .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
947 .uses(left, GPR)
948 .uses(right, GPR)
949 .finish();
950 func.build(block, opcode).uses(left, GPR).finish();
951
952 // The left value is wanted afterwards, so the answer cannot have its register. It cannot
953 // have the right one's either, because the rewrite is about to write a move into it before
954 // the addition has read anything.
955 assert_eq!(places(&func, &env()), ["rax", "rcx", "rdx"]);
956 }
957
958 #[test]
959 fn a_value_live_across_a_whole_loop_holds_its_register_over_all_of_it() {
960 let mut names = Interner::new();
961 let mut func = Func::new(names.intern("f"));
962 let opcode = Opcode::new(names.intern("x64.nop"));
963 let head = func.create_block();
964 let body = func.create_block();
965 let carried = func.new_vreg(GPR);
966 let inside = func.new_vreg(GPR);
967 func.build(head, opcode).def(carried, GPR).finish();
968 *func.succs_mut(head) = vec![BlockCall::to(body)];
969 func.build(body, opcode).def(inside, GPR).finish();
970 func.build(body, opcode).uses(inside, GPR).uses(carried, GPR).finish();
971 *func.succs_mut(body) = vec![BlockCall::to(body)];
972
973 // The value inside the loop cannot have the carried one's register, even though nothing
974 // between the two definitions says so.
975 assert_eq!(places(&func, &env()), ["rax", "rcx"]);
976 }
977
978 #[test]
979 fn a_two_address_answer_already_live_does_not_take_the_register_it_read() {
980 let mut names = Interner::new();
981 let mut func = Func::new(names.intern("f"));
982 let opcode = Opcode::new(names.intern("x64.nop"));
983 let head = func.create_block();
984 let latch = func.create_block();
985 let out = func.create_block();
986 let source = func.new_vreg(GPR);
987 let carried = func.new_vreg(GPR);
988 func.build(head, opcode).def(source, GPR).finish();
989 func.build(head, opcode).def(carried, GPR).finish();
990 *func.succs_mut(head) = vec![BlockCall::to(latch)];
991 // The bottom of the loop adds the source to the carried value and writes the answer back
992 // over it, reusing the register the source is in. The next turn round redefines both.
993 func.build(latch, opcode)
994 .operand(Operand::write(carried, GPR).with(Constraint::Reuse(1)))
995 .uses(source, GPR)
996 .uses(carried, GPR)
997 .finish();
998 *func.succs_mut(latch) = vec![BlockCall::to(head), BlockCall::to(out)];
999 func.build(out, opcode).uses(carried, GPR).finish();
1000
1001 // The source is read here for the last time, which on its own is the shape the two address
1002 // shortcut is for, and taking it would be wrong. The carried value was written by the same
1003 // instruction on the last turn and is read by this one, so the two are both wanted where
1004 // the instruction reads and one register cannot hold both.
1005 assert_eq!(places(&func, &env()), ["rax", "rcx"]);
1006
1007 // And the checker has to agree, since it excused this pair on the same reasoning and so
1008 // would have let the answer through.
1009 let order = Order::of(&func);
1010 let live = Live::of(&func, &order);
1011 let assignment = assign(&func, &order, &live, &env());
1012 assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1013 }
1014
1015 #[test]
1016 fn a_two_address_answer_with_a_hole_in_front_of_it_does_not_take_its_other_operand() {
1017 let mut names = Interner::new();
1018 let mut func = Func::new(names.intern("f"));
1019 let nop = Opcode::new(names.intern("x64.nop"));
1020 let add = Opcode::new(names.intern("x64.add"));
1021 let entry = func.create_block();
1022 let head = func.create_block();
1023 let arm = func.create_block();
1024 let latch = func.create_block();
1025 let out = func.create_block();
1026 let seed = func.new_vreg(GPR);
1027 let sum = func.new_vreg(GPR);
1028 let inside = func.new_vreg(GPR);
1029 let loaded = func.new_vreg(GPR);
1030 func.build(entry, nop).def(seed, GPR).finish();
1031 func.build(entry, nop).def(sum, GPR).finish();
1032 *func.succs_mut(entry) = vec![BlockCall::to(head)];
1033 func.build(head, nop).uses(sum, GPR).finish();
1034 *func.succs_mut(head) = vec![BlockCall::to(arm), BlockCall::to(latch)];
1035 func.build(arm, nop).def(inside, GPR).finish();
1036 func.build(arm, nop).uses(inside, GPR).finish();
1037 *func.succs_mut(arm) = vec![BlockCall::to(out)];
1038 func.build(latch, nop).def(loaded, GPR).finish();
1039 func.build(latch, add)
1040 .operand(Operand::write(sum, GPR).with(Constraint::Reuse(1)))
1041 .uses(seed, GPR)
1042 .uses(loaded, GPR)
1043 .finish();
1044 *func.succs_mut(latch) = vec![BlockCall::to(head), BlockCall::to(out)];
1045
1046 // The answer is live in the entry and the head as well, and the arm between them is a hole
1047 // in it, so the piece the addition writes is not the first one. The value the addition reads
1048 // out of memory is still wanted where the addition reads, so it may not be in the register
1049 // the answer is about to be copied into, holes or no holes. tamnd/rucc#982.
1050 let places = places(&func, &env());
1051 assert_ne!(places[index(sum)], places[index(loaded)]);
1052
1053 let order = Order::of(&func);
1054 let live = Live::of(&func, &order);
1055 let assignment = assign(&func, &order, &live, &env());
1056 assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1057 }
1058
1059 /// Two blocks the entry chooses between, with the one the clobber is in written first. The two
1060 /// values written in the entry block are read in the other one, so their ranges cover the
1061 /// clobber whether or not either of them ever reaches it.
1062 fn arms(reaches: bool) -> Func {
1063 let mut names = Interner::new();
1064 let mut func = Func::new(names.intern("f"));
1065 let opcode = Opcode::new(names.intern("x64.nop"));
1066 let entry = func.create_block();
1067 let arm = func.create_block();
1068 let tail = func.create_block();
1069 let first = func.new_vreg(GPR);
1070 let second = func.new_vreg(GPR);
1071 func.build(entry, opcode).def(first, GPR).finish();
1072 func.build(entry, opcode).def(second, GPR).finish();
1073 *func.succs_mut(entry) = vec![BlockCall::to(arm), BlockCall::to(tail)];
1074 // What a call looks like here: an instruction writing the registers the convention says it
1075 // destroys, named outright so that nothing else may be in them.
1076 func.build(arm, opcode).operand(Operand::write(Reg::physical(RAX), GPR)).finish();
1077 *func.succs_mut(arm) = if reaches { vec![BlockCall::to(tail)] } else { Vec::new() };
1078 func.build(tail, opcode).uses(first, GPR).uses(second, GPR).finish();
1079 func
1080 }
1081
1082 #[test]
1083 fn a_register_a_clobber_takes_beats_the_stack_for_a_value_not_live_in_that_block() {
1084 let func = arms(false);
1085
1086 // Two registers between two values, and a clobber in the arm that takes the first of them.
1087 // The intervals around both values cover the clobber, since the arm is written between the
1088 // two blocks they are live in, and the arm is a hole in both of their areas. So the second
1089 // value has `rax` rather than a stack slot: the arm is a block its own path never goes
1090 // through. tamnd/rucc#982.
1091 assert_eq!(places(&func, &narrow(2)), ["rcx", "rax"]);
1092
1093 let order = Order::of(&func);
1094 let live = Live::of(&func, &order);
1095 let assignment = assign(&func, &order, &live, &narrow(2));
1096 assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1097 }
1098
1099 #[test]
1100 fn a_register_a_clobber_takes_is_not_free_to_a_value_that_is_live_there() {
1101 let func = arms(true);
1102
1103 // The same blocks with an edge from the arm to the tail, which is all it takes: both values
1104 // now arrive at the read either way, so the clobber is on a path they are live over and the
1105 // one register left has to do for both of them.
1106 assert_eq!(places(&func, &narrow(2)), ["rcx", "slot 0"]);
1107 }
1108
1109 /// A value and the instruction that destroys a register, written one after the other, with the
1110 /// value read by that instruction or by the one after it.
1111 fn dies_at_the_clobber(here: bool) -> Func {
1112 let mut names = Interner::new();
1113 let mut func = Func::new(names.intern("f"));
1114 let opcode = Opcode::new(names.intern("x64.nop"));
1115 let entry = func.create_block();
1116 let value = func.new_vreg(GPR);
1117 func.build(entry, opcode).def(value, GPR).finish();
1118 let call = func.build(entry, opcode).operand(Operand::write(Reg::physical(RAX), GPR));
1119 if here {
1120 call.uses(value, GPR).finish();
1121 } else {
1122 call.finish();
1123 func.build(entry, opcode).uses(value, GPR).finish();
1124 }
1125 func
1126 }
1127
1128 /// A value whose last read is the instruction that destroys a register may be in that register,
1129 /// because the instruction reads what it is handed before it writes anything.
1130 ///
1131 /// The call is what this is about, and the value a call is passed is the case: seven registers
1132 /// on this machine are destroyed by one, every argument dies at the call that reads it, and
1133 /// refusing all seven to those values left them taking a callee saved register for a life two
1134 /// instructions long and paying for it in the prologue and the epilogue. tamnd/rucc#1232.
1135 #[test]
1136 fn a_value_that_dies_where_a_register_is_destroyed_may_be_in_that_register() {
1137 let func = dies_at_the_clobber(true);
1138 assert_eq!(places(&func, &narrow(1)), ["rax"]);
1139
1140 let order = Order::of(&func);
1141 let live = Live::of(&func, &order);
1142 let assignment = assign(&func, &order, &live, &narrow(1));
1143 assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1144 }
1145
1146 /// And one read later than that is one the instruction really does destroy, which is the same
1147 /// function with the read moved down by one instruction.
1148 #[test]
1149 fn a_value_read_after_the_instruction_that_destroys_a_register_is_not_in_it() {
1150 let func = dies_at_the_clobber(false);
1151 assert_eq!(places(&func, &narrow(1)), ["slot 0"]);
1152 }
1153
1154 #[test]
1155 fn a_hint_is_followed_when_the_register_is_clear_and_not_when_it_is_merely_allowed() {
1156 let mut names = Interner::new();
1157 let mut func = Func::new(names.intern("f"));
1158 let opcode = Opcode::new(names.intern("x64.nop"));
1159 let entry = func.create_block();
1160 let mid = func.create_block();
1161 let tail = func.create_block();
1162 let first = func.new_vreg(GPR);
1163 let second = func.new_vreg(GPR);
1164 func.build(entry, opcode).def(first, GPR).finish();
1165 func.build(entry, opcode).def(second, GPR).finish();
1166 *func.succs_mut(entry) = vec![BlockCall::to(mid), BlockCall::to(tail)];
1167 // Two arms, each ending in an instruction that wants its own value in `rax`, which is what
1168 // a return out of either side of a branch looks like.
1169 func.build(mid, opcode)
1170 .operand(Operand::read(second, GPR).with(Constraint::Fixed(RAX)))
1171 .finish();
1172 func.build(tail, opcode)
1173 .operand(Operand::read(first, GPR).with(Constraint::Fixed(RAX)))
1174 .finish();
1175
1176 // The first value is hinted at `rax` and does not get it, because the other arm wants `rax`
1177 // for the other value and the first value's range reaches that far. Following the hint here
1178 // would save a move in the tail and cost one in the middle, and the second value gets `rax`
1179 // with nothing moved anywhere instead.
1180 assert_eq!(places(&func, &env()), ["rcx", "rax"]);
1181 }
1182
1183 #[test]
1184 fn a_value_living_in_a_hole_of_another_gets_the_same_register() {
1185 let mut names = Interner::new();
1186 let mut func = Func::new(names.intern("f"));
1187 let opcode = Opcode::new(names.intern("x64.nop"));
1188 let entry = func.create_block();
1189 let arm = func.create_block();
1190 let tail = func.create_block();
1191 let across = func.new_vreg(GPR);
1192 let inside = func.new_vreg(GPR);
1193 func.build(entry, opcode).def(across, GPR).finish();
1194 *func.succs_mut(entry) = vec![BlockCall::to(arm), BlockCall::to(tail)];
1195 func.build(arm, opcode).def(inside, GPR).finish();
1196 func.build(arm, opcode).uses(inside, GPR).finish();
1197 func.build(tail, opcode).uses(across, GPR).finish();
1198
1199 // One register between the two of them, and one register is enough. Nothing in the arm can
1200 // reach the read in the tail, so the value the arm makes is welcome to the register the
1201 // value crossing the function is in. The interval around that value covers the arm and the
1202 // value is nowhere near it, which is what used to send one of the two to the stack.
1203 // tamnd/rucc#982.
1204 assert_eq!(places(&func, &narrow(1)), ["rax", "rax"]);
1205
1206 let order = Order::of(&func);
1207 let live = Live::of(&func, &order);
1208 let assignment = assign(&func, &order, &live, &narrow(1));
1209 assert_eq!(assignment.spilled(), 0);
1210 assert!(crate::check::check(&func, &order, &live, &assignment).is_empty());
1211 }
1212
1213 #[test]
1214 fn a_register_a_clobber_takes_is_the_last_one_offered_rather_than_the_first() {
1215 let func = arms(false);
1216
1217 // With a register to spare the value takes the spare one. Being allowed a register some
1218 // instruction insists on is not the same as it being free: the instruction has to be handed
1219 // it in the end, and what hands it over is a move.
1220 assert_eq!(places(&func, &narrow(3)), ["rcx", "rdx"]);
1221 }
1222
1223 #[test]
1224 fn a_frame_says_what_each_of_its_slots_is_for() {
1225 let mut names = Interner::new();
1226 let mut func = Func::new(names.intern("f"));
1227 let opcode = Opcode::new(names.intern("x64.nop"));
1228 let block = func.create_block();
1229 let first = func.new_vreg(GPR);
1230 let second = func.new_vreg(GPR);
1231 func.build(block, opcode).def(first, GPR).finish();
1232 func.build(block, opcode).def(second, GPR).finish();
1233 func.build(block, opcode).uses(first, GPR).uses(second, GPR).finish();
1234
1235 let order = Order::of(&func);
1236 let live = Live::of(&func, &order);
1237 let assignment = assign(&func, &order, &live, &narrow(1));
1238 assert_eq!(assignment.spilled(), 1);
1239 assert_eq!(assignment.slots(), [GPR]);
1240 // A register that is already a register is where it is, and this has nothing to say about
1241 // it.
1242 assert_eq!(assignment.place(Reg::physical(RCX)), None);
1243 assert_eq!(env().scratch(GPR), [R13, R14, R15]);
1244 }
1245}