Skip to main content

rucc_codegen/
abi.rs

1//! Where a function's arguments already are when it starts running, and where a call puts its own.
2//!
3//! Design: `spec/12-abi-and-runtime.md`.
4//!
5//! This is the one part of the calling convention that is not a lowering rule, and it is worth
6//! saying why, because everything else in this crate is. A rule matches a term and rewrites it,
7//! and which register the third argument arrives in is not a fact about any term: it depends on
8//! the argument's position and on the classification of every argument before it. A pattern has
9//! nowhere to put that. So the arguments are built here, by hand, out of what the convention
10//! says, the same way [`crate::finish`] builds a prologue.
11//!
12//! The classification itself is not here either. `rucc-lower` has already run it by the time a
13//! function reaches this crate, which is why the parameters read here are nearly all plain
14//! scalars: an aggregate has been split into the pieces it travels in, and a return through memory
15//! is an ordinary pointer parameter in front of the rest. What is left for this is the step after
16//! classification, from how a value travels to which register it is actually in, which is
17//! [`rucc_target::Places`].
18//!
19//! The one parameter that is not a scalar is an aggregate the classification put in the argument
20//! area whole, which is [`rucc_ir::Abi::ByVal`]. The IR calls it a pointer, because a pointer is
21//! what an instruction reading it has to have, and the convention says the bytes travel and the
22//! pointer does not. So this is the one place that reads what the classification said rather than
23//! only the type, on both sides of the call, and the two sides are the two halves of one copy.
24//!
25//! # What it writes
26//!
27//! One `x64.arg_val_*` per parameter that arrived in a register, at the top of the entry block,
28//! each defining a fresh register constrained to the one the argument arrived in. They encode to
29//! nothing. The point of them is that a parameter has to be defined somewhere for the allocator to
30//! have anything to move, and the entry block cannot define it as a block parameter: there is no
31//! edge into the entry block for the move to go on, which is what `rucc_regalloc::rewrite` asserts.
32//!
33//! What the allocator does with them is the whole of the argument sequence. A parameter that is
34//! read where it arrived costs nothing, and one that is not gets a copy, which is the same
35//! bargain the return already makes and is decided by the same code.
36//!
37//! A parameter whose bytes travelled is the exception to all of that. Its bytes are already in
38//! this function, at a place in the caller's argument area the same walk gives, so nothing is
39//! brought in at all: what the parameter is is where they are, and that is one `lea`. It waits on
40//! the frame the way the loads below it do, and for the same reason.
41//!
42//! A parameter past the last register arrived in the caller's memory rather than in a register, so
43//! it is a load and not a pseudo, and it is a real instruction that encodes to real bytes. How far
44//! up the caller's argument area it is is a number [`rucc_target::Places`] answers here, but where
45//! that area is from inside this function is a distance into a frame, and no frame exists until
46//! after allocation. So the load is written with nothing in its displacement, which of the two
47//! registers it reads through is left to be settled too, and both are filled in by [`crate::finish`]
48//! out of [`crate::frame::Frame::incoming`]. That is the same bargain an `alloca` already makes,
49//! for the same reason and in the same two places.
50//!
51//! # A call
52//!
53//! The same reasoning the other way round, and one instruction rather than several. `x64.call`
54//! and `x64.call_reg` are the only opcodes in the description whose operand vector is empty
55//! there, because nothing about a call's operands is the same from one call to the next, so they
56//! are built here: one read per argument constrained to the register the convention passes it in,
57//! one definition for the value that comes back constrained to the register it comes back in, and
58//! one definition per register the convention does not preserve.
59//!
60//! A call through an address has one operand more, which is the address, and it is the one
61//! operand of a call that is a fact about the instruction rather than about the signature. It
62//! goes in front of the arguments, because the assembler has to find it and an index into a
63//! vector whose length depends on the convention is not a way of finding anything.
64//!
65//! Those last ones are the clobbers, and they are the whole of what the allocator has to know
66//! about a call besides where the values go. Each is a definition of the physical register itself
67//! rather than of a value, since there is no value: it says the register is written here, which
68//! is exactly what stops the allocator from leaving something in one across the call. A register
69//! an argument or the result already names is not repeated, because naming it once already blocks
70//! it for the length of the instruction, which is all a clobber does.
71//!
72//! An argument past the last register the convention has for it is a store into the outgoing area
73//! rather than an operand of the call, written in front of the call in the same block. Where that
74//! area is does not have to wait for the frame the way the incoming one does, because the outgoing
75//! area is at the bottom of the frame and the bottom of the frame is where the stack pointer is:
76//! that is the whole reason the frame puts it there, since it is where the callee will look. So the
77//! offset [`rucc_target::Places`] gives back is the offset the store is written with.
78//!
79//! An object passed by value in memory is the same thing again and a copy rather than a store. The
80//! caller owes the callee a copy it is free to write to, which is what makes a C call by value
81//! different from passing a pointer the callee must not keep, and the argument area is where the
82//! convention says that copy goes. So the bytes are read out of the object and written into the
83//! area a word at a time, in front of the call, with the words chosen by the same function that
84//! chooses them for a `memcpy`. An object with more words than that unrolls to is copied by a call
85//! to the runtime's `memcpy` instead, written in front of the outer call in the same block.
86//!
87//! That is one call built in the middle of building another, which sounds worse than it is.
88//! Nothing of the outer call is in a physical register when the copy is written: every argument
89//! that travels in one is still a virtual register, and the register it has to end up in is a
90//! constraint on an operand of the call instruction, which is not built until every copy in front
91//! of it has been. So what the inner call destroys is what any call destroys, and the allocator
92//! keeps the outer call's values out of those registers the same way it does across a call the
93//! program wrote. The arguments already stored into the outgoing area are safe for a plainer
94//! reason: the inner call's own frame is below the stack pointer and that area is above it.
95//!
96//! The call still reports how many bytes it needed, because the frame reserves as many as the
97//! widest call in the function asked for and cannot know that until every call has been seen.
98
99use rucc_base::{Interner, Symbol};
100use rucc_diag::Span;
101use rucc_ir::{Abi, Drains, Param, Type};
102use rucc_mir as mir;
103use rucc_target::{CallRegs, Constraint, PhysReg, Places, RegClass, Variadic, Where};
104
105use crate::capability;
106use crate::varargs::Area;
107
108pub mod aarch64;
109
110/// Why a parameter could not be brought in.
111#[derive(Debug, Clone, Copy, PartialEq, Eq)]
112pub enum Missing {
113    /// It travels on the x87 stack, which is a `long double` and nothing else. That stack is a
114    /// third register file, it is not one the allocator has, and no instruction in the
115    /// description touches it.
116    OnX87,
117    /// It is a width no pseudo covers, which is anything a machine register does not hold.
118    Width,
119    /// It is a value that comes back in more registers than the convention returns in. A structure
120    /// of at most sixteen bytes comes back in up to two, which is as many as SysV has, and a
121    /// convention with fewer of them returns such a structure through a hidden pointer instead. So
122    /// this is what a signature the classification did not produce would get.
123    NoRoom,
124    /// It is an object whose bytes travel in the argument area and there are more of them than
125    /// any count of them can be written down as. A copy too long to unroll is a call to the
126    /// runtime and not a refusal, so what is left here is an object of two gigabytes or more,
127    /// which is a size the immediate holding the byte count has nowhere to put and a structure no
128    /// program passes.
129    TooBig,
130}
131
132impl Missing {
133    /// What it says when a function could not be compiled because of it.
134    ///
135    /// Worded so that it reads the same about a value arriving and a value being passed, since
136    /// the two are the same fact seen from the two ends of one call.
137    #[must_use]
138    pub fn why(self) -> &'static str {
139        match self {
140            Missing::OnX87 => "is on the x87 stack",
141            Missing::Width => "is a width no argument register holds",
142            Missing::NoRoom => "takes more registers than this convention has for it",
143            Missing::TooBig => "is more bytes than a count of them can be written down as",
144        }
145    }
146}
147
148/// Which register file a value of that type travels in.
149///
150/// The whole of what the two files mean to this module. A float is in the vector one and
151/// everything else is in the general purpose one, which is what both of this machine's conventions
152/// say. A `long double` is in neither and what its register holds is an address, which is a general
153/// purpose value like every other address, so it answers with the other file rather than with the
154/// one its type would suggest.
155fn class_of(ty: Type, conv: &CallRegs) -> RegClass {
156    if ty.is_float() && !on_the_stack(ty) { conv.sse_class } else { conv.int_class }
157}
158
159/// How many bytes of the argument area a value in the vector file takes.
160///
161/// Its own width, lanes included, which is what [`Places::float`] wants and is a number that only
162/// matters to a value the registers ran out before. Everything the machine computes in is a word
163/// or narrower and takes a word either way. The `_Float128` is the one that is not: two words, and
164/// aligned to two words, which is the difference between the argument behind it being placed after
165/// it and being placed on top of half of it.
166pub(crate) fn float_bytes(ty: Type) -> u32 {
167    ty.bits().div_ceil(8).saturating_mul(ty.lanes())
168}
169
170/// Spends the registers an object in the argument area said nothing after it may have.
171pub(crate) fn drain(places: &mut Places<'_>, drains: Drains) {
172    match drains {
173        Drains::Nothing => {}
174        Drains::Integers => places.drain_integers(),
175        Drains::Floats => places.drain_floats(),
176    }
177}
178
179/// How many bytes of the argument area a value in the general purpose file takes.
180///
181/// Its own width, which only a convention that packs the argument area reads, since everywhere
182/// else anything this narrow takes a word. A pointer has no width of its own in the IR, so it is
183/// the word.
184pub(crate) fn int_bytes(ty: Type, conv: &CallRegs) -> u32 {
185    if ty.is_ptr() { conv.word } else { ty.bits().saturating_mul(ty.lanes()).div_ceil(8).max(1) }
186}
187
188/// Whether a value of that type travels as bytes in the argument area because of what it is.
189///
190/// One type does, and it is the `long double`. SysV classifies it X87 and X87UP, which is the
191/// classification that means memory, so it goes where a structure the classification put in memory
192/// goes and what the two ends pass is the address of the bytes. That is not a decision about
193/// registers running out: a `long double` travels in the argument area when it is the only argument
194/// there is.
195///
196/// Sixteen bytes aligned to sixteen, which is what the psABI says the type takes and is the same
197/// number [`crate::lower`] gives one in the frame, so a value being passed and a value being worked
198/// on are the same shape of object in two places.
199#[must_use]
200pub fn on_the_stack(ty: Type) -> bool {
201    ty.is_float() && ty.is_scalar() && ty.bits() == 80
202}
203
204/// Whether what a function gives back is left on the x87 stack, which is one `long double` in
205/// `st(0)` or a `_Complex long double` with its real half there and its imaginary half in `st(1)`.
206///
207/// Those two are the whole of what SysV returns on that stack, and a structure holding a `long
208/// double` is not one of them, since the lowering sends that back through memory.
209#[must_use]
210pub fn back_on_x87(returns: &[Type]) -> bool {
211    matches!(returns.len(), 1 | 2) && returns.iter().all(|&ty| on_the_stack(ty))
212}
213
214/// How much room one takes in the argument area, as a size and an alignment.
215pub(crate) const X87_AREA: (u32, u32) = (16, 16);
216
217/// Why a value of that type cannot travel at all, or nothing if it can.
218///
219/// The width question and the file question in one place, so that the two ends of a call give the
220/// same answer about the same type, and so that a `return` this cannot make says the same thing
221/// about a type as the call that would have received it.
222///
223/// A `long double` is not one of them any more when it is an argument, since an argument of that
224/// type travels as bytes and [`on_the_stack`] is what says so before this is asked. What is left
225/// here is the value that comes back, because coming back is the one direction where it is not
226/// bytes: it arrives in `st(0)`, which is a register file this cannot name.
227#[must_use]
228pub fn refuses(ty: Type, insts: &Insts) -> Option<Missing> {
229    if (insts.arg)(ty).is_some() {
230        return None;
231    }
232    // A `long double` is the one type here that is in neither of the two files. Saying so is worth
233    // more than calling it a width, because eighty bits is a width this machine computes in and
234    // the file it computes in is what actually stands in the way.
235    if on_the_stack(ty) {
236        return Some(Missing::OnX87);
237    }
238    Some(Missing::Width)
239}
240
241/// What a function's parameters came to.
242#[derive(Debug, Default, Clone, PartialEq, Eq)]
243pub struct Arrived {
244    /// The register each parameter is in, in the order the parameters were given, so the caller can
245    /// bind each IR parameter to the one at its position.
246    pub regs: Vec<mir::Reg>,
247    /// The loads that read a parameter out of the caller's argument area, and how far up that area
248    /// each of them reads.
249    ///
250    /// Empty for almost every function, because almost every function has few enough parameters to
251    /// have been handed all of them in registers. The distance is from the bottom of the caller's
252    /// argument area, which is somewhere [`crate::finish`] works out and this cannot.
253    pub stack: Vec<(mir::Inst, u32)>,
254    /// How many general purpose argument registers the parameters took, and how many vector ones.
255    ///
256    /// Nothing about an ordinary function needs this. A variadic one does: the first argument its
257    /// signature does not name is the one after the last it does, so where each of the two walks
258    /// stopped is where `va_start` has to say the next argument begins.
259    pub took: (usize, usize),
260    /// How far up the caller's argument area the first argument the signature does not name is.
261    ///
262    /// However much of that area the named parameters took, on a convention that counts the two
263    /// register files apart, because there the area holds only the arguments no register was left
264    /// for. On one that counts them as one run it is not the same number: that area begins with the
265    /// shadow space the caller reserved, every argument owns one word of it whether it also arrived
266    /// in a register or not, and the named ones own the first few of those words. So it is where
267    /// the walk over the registers stopped, which is a position rather than a size, until the named
268    /// parameters have used every register and the two agree again.
269    pub beyond: u32,
270    /// The argument registers left over for the arguments the signature does not name, as the
271    /// register each was bound into and how far up the save area its slot is.
272    ///
273    /// Empty unless a save area was asked for. The ones a named parameter took are not here,
274    /// because their slots are behind where `va_start` sets the two offsets and nothing ever reads
275    /// them, so writing them would be fourteen stores where six are wanted.
276    pub spare: Vec<(mir::Reg, RegClass, u32)>,
277}
278
279/// Binds a function's parameters to where the convention says they arrive.
280///
281/// # Errors
282///
283/// The first parameter this cannot bring in, and why. A function with one is reported rather
284/// than compiled, because the alternative is a function that reads an argument from wherever the
285/// last one happened to leave a register.
286pub fn entry(
287    out: &mut mir::Func,
288    block: mir::Block,
289    params: &[Param],
290    conv: &CallRegs,
291    insts: &Insts,
292    names: &mut Interner,
293    save: Option<Area>,
294) -> Result<Arrived, (usize, Missing)> {
295    // Where everything is, worked out before anything is written, both so that a parameter this
296    // cannot bring in stops the function before half of one is built and so that the two loops
297    // below can be two loops. Asking for the place of a parameter that cannot be brought in is
298    // still done, because every place after it depends on it and a reader stepping through this
299    // should see the same numbers a working version would.
300    let mut places = Places::new(conv);
301    let mut where_from = Vec::with_capacity(params.len());
302    for (index, &Param { ty, abi }) in params.iter().enumerate() {
303        // A structure the classification put in the argument area arrived as bytes, and the
304        // parameter the IR sees is a pointer to them. So there is nothing to bring in: the bytes
305        // are already in this function's frame, and what the pointer holds is where they are.
306        if let Abi::ByVal { size, align, drains } = abi {
307            let size = u32::try_from(size).map_err(|_| (index, Missing::TooBig))?;
308            where_from.push((ty, places.object(size, align), abi));
309            drain(&mut places, drains);
310            continue;
311        }
312        // And an eighty bit float, which arrives the same way for the same reason and is told
313        // apart only by the classification having said nothing about it: the front end passes it
314        // as a value of its own type, and it is this that knows the type is one that travels as
315        // bytes. What arrives is the address of those bytes, which is what an object in the
316        // argument area always hands over.
317        if on_the_stack(ty) {
318            let (size, align) = X87_AREA;
319            where_from.push((
320                ty,
321                places.on_stack(size, align),
322                Abi::ByVal { size: size.into(), align, drains: Drains::Nothing },
323            ));
324            continue;
325        }
326        let at = match (abi, conv.sret) {
327            // The address a result goes back through, on a convention with a register of its own
328            // for it, which takes no position from the arguments after it.
329            (Abi::Sret { .. }, Some(sret)) => Where::Reg(sret),
330            _ if ty.is_float() => places.float(float_bytes(ty)),
331            _ => places.integer(int_bytes(ty, conv)),
332        };
333        if let Some(missing) = refuses(ty, insts) {
334            return Err((index, missing));
335        }
336        where_from.push((ty, at, abi));
337    }
338    let took = (places.integers(), places.floats());
339    // Where the first argument the signature does not name is, which is the two sentences on
340    // [`Arrived::beyond`] written out. The run of words is contiguous from the bottom of the area on
341    // a convention that homes its register arguments, so a position multiplied by a word is the
342    // answer there until the positions run out and the named parameters start taking room of their
343    // own, at which point what they took is the answer again.
344    let reached = took.0 + took.1;
345    let beyond = match conv.shared_positions && reached < conv.int_args.len() {
346        true => conv.word * u32::try_from(reached).unwrap_or(0),
347        false => places.size(),
348    };
349    let mut arrived =
350        Arrived { regs: Vec::with_capacity(params.len()), took, beyond, ..Arrived::default() };
351
352    // Every pseudo first and everything else after, which is not a preference. A pseudo says a
353    // register holds an argument and defines nothing before it, so as far as the allocator can see
354    // the register was dead until then and is free to be used as a scratch. That is true of a
355    // register no pseudo has named yet, and it stops being true the moment one does. Anything that
356    // needs a scratch has to come after all of them, and a load out of the caller's stack needs one
357    // for the value it loads.
358    for (index, &(ty, at, _)) in where_from.iter().enumerate() {
359        let class = class_of(ty, conv);
360        let reg = out.new_vreg(class);
361        arrived.regs.push(reg);
362        let Where::Reg(arrived_in) = at else { continue };
363        let head = (insts.arg)(ty).ok_or((index, Missing::Width))?;
364        let opcode = mir::Opcode::new(names.intern(head));
365        let operand = mir::Operand::write(reg, class).with(Constraint::Fixed(arrived_in));
366        out.build(block, opcode).operand(operand).finish();
367    }
368    if let Some(area) = save {
369        arrived.spare = spare(out, block, conv, insts, names, area, arrived.took);
370    }
371
372    // The stack pointer is written down as the register to read through because it is the one that
373    // reaches the caller's stack in almost every function, and a realigned frame is the exception
374    // that [`crate::finish`] rewrites. Putting something here rather than nothing keeps the
375    // instruction printable and verifiable in between.
376    for (index, &(ty, at, abi)) in where_from.iter().enumerate() {
377        let Where::Stack(up) = at else { continue };
378        let class = class_of(ty, conv);
379        // The bytes of an object that travelled as bytes are read by whatever reads the parameter,
380        // and what the parameter is is their address, so this takes the address rather than a
381        // value out of it. Everything else about it is the load's, including the two fields
382        // [`crate::finish`] fills in, because where the caller's argument area is is the same
383        // question for both.
384        let name = match abi {
385            Abi::ByVal { .. } => insts.lea,
386            _ => (insts.load)(ty).ok_or((index, Missing::Width))?,
387        };
388        let opcode = mir::Opcode::new(names.intern(name));
389        let sp = mir::Operand::read(mir::Reg::physical(conv.stack_pointer), conv.int_class);
390        let made =
391            out.build(block, opcode).def(arrived.regs[index], class).mem(mir::Mem::at(sp)).finish();
392        arrived.stack.push((made, up));
393    }
394    Ok(arrived)
395}
396
397/// Binds the argument registers no parameter the signature names took, which are the ones the
398/// arguments it does not name arrived in.
399///
400/// One pseudo each and nothing else, for the reason the loop above them gives: what these do is say
401/// the register holds something, and the stores that put it in the save area are written by
402/// [`crate::lower`] once it has an address to store to, which is after every pseudo in the block.
403///
404/// Where each file's walk stopped is the file's own count on a convention that keeps two, and the
405/// sum of both on one that counts the files as a single run of positions, since there an argument
406/// of either kind steps the one counter. A file the area holds no slots of is skipped entirely,
407/// which is the vector file on the second kind: a variadic float travels in the general purpose
408/// register at its position as well, so the copy the walk reads is already the one being spilled.
409fn spare(
410    out: &mut mir::Func,
411    block: mir::Block,
412    conv: &CallRegs,
413    insts: &Insts,
414    names: &mut Interner,
415    area: Area,
416    took: (usize, usize),
417) -> Vec<(mir::Reg, RegClass, u32)> {
418    let word = Type::int(64);
419    let double = Type::float(rucc_ir::Float::F64);
420    let reached = |own: usize| if conv.shared_positions { took.0 + took.1 } else { own };
421    let files = [
422        (conv.int_args, reached(took.0), word, false),
423        (conv.sse_args, reached(took.1), double, true),
424    ];
425    let mut spare = Vec::new();
426    for (regs, taken, ty, float) in files {
427        let held = usize::try_from(area.holds(float)).unwrap_or(0);
428        let Some(head) = (insts.arg)(ty) else { continue };
429        let class = class_of(ty, conv);
430        for (index, &arrived_in) in regs.iter().enumerate().take(held).skip(taken) {
431            let reg = out.new_vreg(class);
432            let opcode = mir::Opcode::new(names.intern(head));
433            let operand = mir::Operand::write(reg, class).with(Constraint::Fixed(arrived_in));
434            out.build(block, opcode).operand(operand).finish();
435            let at = area.starts_at(float) + area.stride(float) * u32::try_from(index).unwrap_or(0);
436            spare.push((reg, class, at));
437        }
438    }
439    spare
440}
441
442/// The instructions this file writes, for one machine.
443///
444/// Every one of them is written here by hand rather than by a rule, for the reasons the module
445/// comment gives, so this file is where a machine's names for them have to come from. Each answer
446/// is the whole name, prefix and all, because each is interned as it is.
447///
448/// The four functions answer for the same set of types, and a type one of them has no name for is
449/// a type none of them should have: an argument that can arrive in a register but not be read off
450/// the stack is a function turned away for where its sixth argument happened to land.
451#[derive(Debug)]
452pub struct Insts {
453    /// The pseudo a parameter of that type arrives in a register as. See [`head_of`].
454    pub arg: fn(Type) -> Option<&'static str>,
455    /// The load that reads one of that type out of the argument area. See [`load_of`].
456    pub load: fn(Type) -> Option<&'static str>,
457    /// The store that writes one of that type into the argument area. See [`store_of`].
458    pub store: fn(Type) -> Option<&'static str>,
459    /// The pseudo a value of that type leaves in the register at that place. See [`ret_of`].
460    pub ret: fn(Type, usize) -> Option<&'static str>,
461    /// The call of a name. See [`CALL`].
462    pub call: &'static str,
463    /// The call of an address in a register. See [`CALL_REG`].
464    pub call_reg: &'static str,
465    /// The instruction that puts an address in a register without reading what is at it.
466    pub lea: &'static str,
467    /// The instruction that writes a small constant into a general purpose register, which is
468    /// what a byte count and a SysV vector count are.
469    pub small: &'static str,
470}
471
472/// The x86-64 ones.
473pub static X86_64: Insts = Insts {
474    arg: head_of,
475    load: load_of,
476    store: store_of,
477    ret: ret_of,
478    call: CALL,
479    call_reg: CALL_REG,
480    lea: "x64.lea_64",
481    small: "x64.mov_ri_32",
482};
483
484/// What the instruction that calls a name is called.
485///
486/// Here rather than in a rule for the same reason the arguments are: a rule pattern sees one term
487/// and a call's operands are whatever the signature made them, so no pattern could name them.
488pub const CALL: &str = "x64.call";
489
490/// What the instruction that calls an address in a register is called.
491///
492/// A different instruction rather than the same one with a different operand, which is what the
493/// machine says too: one carries the distance to somewhere in the program and takes a relocation,
494/// and the other carries the register the address is in and takes none. Sharing an opcode would
495/// mean an instruction whose bytes depend on whether a field beside it happens to be set.
496pub const CALL_REG: &str = "x64.call_reg";
497
498/// One value a call passes.
499#[derive(Debug, Clone, Copy, PartialEq, Eq)]
500pub struct Passing {
501    /// The type it travels as, which for an object travelling as bytes is the pointer's rather
502    /// than the object's, because the pointer is what the machine IR has.
503    pub ty: Type,
504    /// The register holding it, or holding its address when the bytes are what travel.
505    pub reg: mir::Reg,
506    /// What the classification asked of it. The one thing read here is whether the object behind
507    /// the pointer is the argument, since everything else it can say is about a value that is
508    /// already in a register in the form it travels in.
509    pub abi: Abi,
510}
511
512/// What one call came to.
513#[derive(Debug, Clone, PartialEq, Eq)]
514pub struct Made {
515    /// The registers the value came back in, in the order the signature returns them, which is
516    /// empty for a call that gives nothing back and holds two for a structure that comes back in a
517    /// pair. Which register each of them is is the classification's answer and is worked out here
518    /// rather than in a table, for the reason the second half of [`Calling::returns`] gives.
519    pub results: Vec<mir::Reg>,
520    /// How many bytes below the stack pointer this call needs for the arguments it passes there.
521    ///
522    /// Not always zero for a call that passes everything in registers: a Windows caller reserves
523    /// thirty two bytes for the callee to spill its register arguments into whether it uses them
524    /// or not, and that reservation is this.
525    pub outgoing: u32,
526}
527
528/// Which of a call's values could not be passed, and why.
529#[derive(Debug, Clone, Copy, PartialEq, Eq)]
530pub struct Refused {
531    /// Its position among the arguments, or `None` for the value that comes back.
532    pub argument: Option<usize>,
533    /// What is wrong with where it travels.
534    pub missing: Missing,
535}
536
537/// What a call goes to.
538///
539/// The whole of the difference between the two calls. Everything else about them, which is what
540/// they pass and what comes back and which registers they destroy, is the signature's answer and
541/// is the same answer either way.
542#[derive(Debug, Clone, Copy, PartialEq, Eq)]
543pub enum Callee {
544    /// A name, which the linker resolves.
545    Named(Symbol),
546    /// An address in a register, which nothing resolves because there is nothing to resolve: the
547    /// value is not known until the program runs.
548    ///
549    /// The register is unconstrained, and it has to be, because every register the convention
550    /// does not preserve is one this instruction writes and every register an argument travels in
551    /// is spoken for. What is left is the registers the callee has to put back, which is where
552    /// the allocator will put the address, and it is the right answer for the same reason it is
553    /// the only one.
554    Through(mir::Reg),
555}
556
557/// One call, as everything about it that is not the function it is being built into.
558#[derive(Debug, Clone, Copy)]
559pub struct Calling<'a> {
560    /// What it calls.
561    pub callee: Callee,
562    /// What it passes, in the order the signature holds them, which is the order the convention
563    /// places them in.
564    pub args: &'a [Passing],
565    /// What comes back, which is empty for a call that gives nothing back, one type for a value,
566    /// and two for a structure small enough to come back in a pair of registers.
567    ///
568    /// A pair is placed here rather than named by a rule for the reason the arguments are: which
569    /// register each half goes in depends on the halves before it, since the two files are walked
570    /// separately, and a pattern over a term cannot see them.
571    pub returns: &'a [Type],
572    /// Whether the callee takes arguments beyond the ones its signature names, which is what says
573    /// whether it reads the count of vector registers the call passed arguments in.
574    pub variadic: bool,
575    /// How many of the arguments the signature does name, so that the ones past it can be told
576    /// apart from the ones before it.
577    ///
578    /// A convention that passes a variadic float in both register files needs that, because which
579    /// arguments get the second copy is exactly the ones the callee has no prototype for. Every
580    /// other convention treats the two the same and never asks.
581    pub named: usize,
582    /// Where the call was written, which every instruction built for it is filed under.
583    ///
584    /// A call is one of the few places in the machine IR where a run of instructions comes from no
585    /// term in the IR at all: the stores into the outgoing area, the copies a structure passed by
586    /// value is made of and the call itself are the convention's answer rather than anything a rule
587    /// matched. So there is nothing for them to inherit a span from, and without this the bytes of
588    /// an entire call statement are covered by whichever row came before them, which is usually the
589    /// line above. gcc names the line the call is written on over all of it.
590    ///
591    /// [`Span::DUMMY`] in a call this crate builds for itself, which is the copy into the argument
592    /// area that the runtime does, since that one is under whatever the call it belongs to is under.
593    pub at: Span,
594}
595
596/// Builds one call: what it passes, what comes back, and what it destroys.
597///
598/// # Errors
599///
600/// The first value this cannot pass, and why, before anything is written. A call with one is
601/// reported rather than compiled, because the alternative is a call that leaves an argument
602/// wherever the last one happened to put a register.
603///
604/// # Panics
605///
606/// If a call passes two gigabytes of arguments on the stack, which is a distance no offset in a
607/// frame can hold and a call no program makes.
608pub fn call(
609    out: &mut mir::Func,
610    block: mir::Block,
611    made: &Calling<'_>,
612    conv: &CallRegs,
613    insts: &Insts,
614    names: &mut Interner,
615) -> Result<Made, Refused> {
616    let &Calling { callee, args, returns, variadic, named, at: span } = made;
617    // Where everything goes, worked out before anything is built, so that a call this cannot make
618    // leaves no half of one behind.
619    let mut places = Places::new(conv);
620    let mut passed = Vec::with_capacity(args.len());
621    // The ones with no register left for them, as the store each of them becomes and how far up
622    // the outgoing area it writes. Almost always empty.
623    let mut on_stack = Vec::new();
624    // How many of them went in vector registers, which is what a SysV variadic callee is told.
625    let mut vectors = 0u32;
626    // The ones whose bytes travel rather than their address, as the register that address is in,
627    // how far up the outgoing area they go and which words the copy is made of. Almost always
628    // empty too, and never at the same time as a register: an object in the argument area is in
629    // the argument area whatever is left of the register files.
630    let mut as_bytes = Vec::new();
631    // The arguments beyond the ones the signature names that travel in a vector register and have
632    // to travel in a general purpose one at the same time, as the register each is in and the
633    // general purpose register its copy belongs in. Empty on every convention but the one that says
634    // so, and on that one this is what makes `printf("%f", x)` read the right register.
635    let mut in_both = Vec::new();
636    // Which argument position the next one that gets a register is at, which is only the same as
637    // the index when nothing ahead of it went to memory. A convention that counts the two register
638    // files as one run is what needs it: the general purpose register a variadic float's second
639    // copy goes in is the one at that position.
640    let mut position = 0usize;
641    for (index, &Passing { ty, reg, abi }) in args.iter().enumerate() {
642        let refused = |missing| Refused { argument: Some(index), missing };
643        // An eighty bit float is bytes in the argument area whatever the classification said, for
644        // the reason [`on_the_stack`] gives, and the register holding it holds their address. So it
645        // joins the objects below rather than being a case of its own, and the copy it becomes is
646        // the copy any other sixteen byte object gets.
647        let abi = match abi {
648            _ if on_the_stack(ty) => {
649                let (size, align) = X87_AREA;
650                Abi::ByVal { size: size.into(), align, drains: Drains::Nothing }
651            }
652            abi => abi,
653        };
654        if let Abi::ByVal { size, align, drains } = abi {
655            let size = u32::try_from(size).map_err(|_| refused(Missing::TooBig))?;
656            // One past the `...` is never packed, so it is words wherever it is.
657            let at = if variadic && index >= named {
658                places.on_stack(size, align)
659            } else {
660                places.object(size, align)
661            };
662            drain(&mut places, drains);
663            let Where::Stack(up) = at else {
664                unreachable!("an object in the argument area is in the argument area")
665            };
666            // Nothing here refuses a plan it did not get, because a copy the words give up on is
667            // a call to the runtime below. What the byte count has to fit in is the immediate the
668            // call passes it as, and an object that large is what is left of the old refusal.
669            let plan = crate::expand::plan(u64::from(size), align, conv.word);
670            let count = i32::try_from(size).map_err(|_| refused(Missing::TooBig))?;
671            as_bytes.push((reg, up, count, plan));
672            continue;
673        }
674        // The address a result comes back through goes in its own register where the convention
675        // has one, and is not an argument position, the same as on the other side in [`entry`].
676        if let (Abi::Sret { .. }, Some(sret)) = (abi, conv.sret) {
677            passed.push((reg, sret, class_of(ty, conv)));
678            continue;
679        }
680        // Apple's AArch64 puts every argument past the named ones in memory, a word or more each,
681        // whatever registers are left.
682        let unnamed = variadic && index >= named && conv.abi.variadic == Variadic::AlwaysMemory;
683        let at = if unnamed {
684            let bytes = int_bytes(ty, conv);
685            places.on_stack(bytes, bytes)
686        } else if ty.is_float() {
687            places.float(float_bytes(ty))
688        } else {
689            places.integer(int_bytes(ty, conv))
690        };
691        if let Some(missing) = refuses(ty, insts) {
692            return Err(refused(missing));
693        }
694        let class = class_of(ty, conv);
695        match at {
696            Where::Reg(at) => {
697                // Only a register counts, because the count is of registers. An argument that went
698                // to memory is one the callee reads from memory whatever this says.
699                if class == conv.sse_class {
700                    vectors += 1;
701                    // Windows passes a float the callee has no prototype for in the vector register
702                    // and in the general purpose register at the same position, both at once,
703                    // because the callee has no way to know which file to look in and its walk over
704                    // the arguments reads the second one. An argument the signature does name needs
705                    // no second copy, since the callee's parameter says where it is.
706                    let both = conv.shared_positions && variadic && index >= named;
707                    if let Some(&also) = conv.int_args.get(position).filter(|_| both) {
708                        in_both.push((reg, also));
709                    }
710                }
711                passed.push((reg, at, class));
712                position += 1;
713            }
714            Where::Stack(up) => {
715                let store = (insts.store)(ty).ok_or(refused(Missing::Width))?;
716                on_stack.push((reg, class, names.intern(store), up));
717            }
718        }
719    }
720    // A `long double` comes back in `st(0)`, which is not a register in either file and not one
721    // this call can be said to write. So nothing is placed for it and nothing is constrained, and
722    // the call gives back no register at all: what takes the value off that stack is the `fstp`
723    // [`crate::lower`] writes straight after the call, which is the same shape every other use of
724    // the x87 stack is written in. A complex one is the same with its imaginary half in `st(1)`,
725    // and a second `fstp` takes that one off.
726    let comes_back =
727        if back_on_x87(returns) { Vec::new() } else { places_back(returns, conv, insts)? };
728
729    // A variadic callee on SysV reads how many vector registers the call passed arguments in and
730    // skips saving them when the answer is none, which is what makes `printf` with no floating
731    // point argument cheap. It is an obligation rather than an optimization: leaving whatever was
732    // in the register there makes the callee save a register file it was not given, and a count
733    // that is too low makes it read an argument out of a register nothing put one in.
734    let counted = if variadic { conv.vector_count } else { None };
735
736    // The arguments that go to memory go there now, in front of the call and after everything this
737    // could have refused, so that a call it cannot make leaves no store behind either. The offset
738    // is written straight in rather than left for [`crate::finish`]: the outgoing area is at the
739    // bottom of the frame because that is where the callee looks for it, and the bottom of the
740    // frame is where the stack pointer already is.
741    for (reg, class, store, up) in on_stack {
742        let sp = mir::Operand::read(mir::Reg::physical(conv.stack_pointer), conv.int_class);
743        let up = i32::try_from(up).expect("an argument area under two gigabytes");
744        let build = out.build(block, mir::Opcode::new(store)).at(span);
745        build.uses(reg, class).mem(mir::Mem::at(sp).plus(up)).finish();
746    }
747
748    // How many bytes the copies below needed for calls of their own, which is nothing on a
749    // convention that passes three pointers in registers and thirty two bytes on the one that
750    // reserves a place for them anyway. The frame has to hear about it, since a call built here is
751    // still a call this function makes.
752    let mut nested = 0u32;
753
754    // And the objects whose bytes go there, as a load and a store for each word of each of them.
755    // This is the copy the caller owes a callee that takes a structure by value: the callee is
756    // free to write to what it was handed, so what it was handed cannot be the caller's own copy,
757    // and the argument area is where the convention says the caller's copy goes. The words are the
758    // same words `crate::expand` would have chosen for a `memcpy` of the same block, because they
759    // are chosen by the same function.
760    for (from, up, count, plan) in as_bytes {
761        let up = i32::try_from(up).expect("an argument area under two gigabytes");
762        let Some(plan) = plan else {
763            let what = Copying { from, up, count, span };
764            nested = nested.max(by_runtime(out, block, conv, insts, names, what));
765            continue;
766        };
767        for (at, width) in plan {
768            let ty = Type::int(width * 8);
769            let at = i32::try_from(at).expect("an object under two gigabytes");
770            let word = out.new_vreg(conv.int_class);
771            let load = names.intern(
772                (insts.load)(ty).ok_or(Refused { argument: None, missing: Missing::Width })?,
773            );
774            let there = mir::Operand::read(from, conv.int_class);
775            let build = out.build(block, mir::Opcode::new(load)).at(span);
776            build.def(word, conv.int_class).mem(mir::Mem::at(there).plus(at)).finish();
777            let store = names.intern(
778                (insts.store)(ty).ok_or(Refused { argument: None, missing: Missing::Width })?,
779            );
780            let sp = mir::Operand::read(mir::Reg::physical(conv.stack_pointer), conv.int_class);
781            let build = out.build(block, mir::Opcode::new(store)).at(span);
782            build.uses(word, conv.int_class).mem(mir::Mem::at(sp).plus(up + at)).finish();
783        }
784    }
785
786    // And the second copy of each float the callee has no prototype for, which is one `movq` out of
787    // the vector register it is already in. What the callee reads out of the general purpose
788    // register is the sixty four bits and not a value of any type, so the bits are what move, and
789    // the copy joins the arguments rather than being a thing of its own: it is passed in a register
790    // the convention names, which is what every other argument here is.
791    for (from, into) in in_both {
792        let word = out.new_vreg(conv.int_class);
793        let movq = mir::Opcode::new(names.intern("x64.movq_from_xmm"));
794        out.build(block, movq)
795            .at(span)
796            .def(word, conv.int_class)
797            .uses(from, conv.sse_class)
798            .finish();
799        passed.push((word, into, conv.int_class));
800    }
801
802    // The definitions first and the reads after, which is the order every operand vector in the
803    // machine IR is in and the order `rucc_mir::defs` counts.
804    let mut operands = Vec::with_capacity(args.len() + conv.int_order.len() + 2);
805    let results: Vec<mir::Reg> = comes_back
806        .iter()
807        .map(|&(at, class)| {
808            let reg = out.new_vreg(class);
809            operands.push(mir::Operand::write(reg, class).with(Constraint::Fixed(at)));
810            reg
811        })
812        .collect();
813    // One list per file, because a physical register is a number and the class is what says which
814    // file it is a number in. One list would have `xmm0` blocking `rax`.
815    let spoken_for = |class: RegClass| -> Vec<PhysReg> {
816        comes_back
817            .iter()
818            .filter(|&&(_, at)| at == class)
819            .map(|&(reg, _)| reg)
820            .chain(counted.filter(|_| class == conv.int_class))
821            .chain(passed.iter().filter(|&&(_, _, at)| at == class).map(|&(_, reg, _)| reg))
822            .collect()
823    };
824    let named = spoken_for(conv.int_class);
825    for &reg in conv.int_order {
826        if !conv.preserves_int(reg) && !named.contains(&reg) {
827            operands.push(mir::Operand::write(mir::Reg::physical(reg), conv.int_class));
828        }
829    }
830    let named = spoken_for(conv.sse_class);
831    for &reg in conv.sse_order {
832        if !conv.preserves_sse(reg) && !named.contains(&reg) {
833            operands.push(mir::Operand::write(mir::Reg::physical(reg), conv.sse_class));
834        }
835    }
836    // The address in front of the arguments, because a call through one is written with the
837    // register it goes through and nothing in the operand vector is at a place a table could name.
838    // First read is a place that does not depend on the signature, which is what
839    // [`rucc_target::x86_64::Arg::Through`] is written against.
840    if let Callee::Through(reg) = callee {
841        operands.push(mir::Operand::read(reg, conv.int_class));
842    }
843    for (reg, at, class) in passed {
844        operands.push(mir::Operand::read(reg, class).with(Constraint::Fixed(at)));
845    }
846    if let Some(at) = counted {
847        let count = out.new_vreg(conv.int_class);
848        let zero = mir::Opcode::new(names.intern(insts.small));
849        out.build(block, zero).at(span).def(count, conv.int_class).imm(i64::from(vectors)).finish();
850        operands.push(mir::Operand::read(count, conv.int_class).with(Constraint::Fixed(at)));
851    }
852
853    let opcode = mir::Opcode::new(names.intern(match callee {
854        Callee::Named(_) => insts.call,
855        Callee::Through(_) => insts.call_reg,
856    }));
857    let mut build = out.build(block, opcode).at(span);
858    if let Callee::Named(symbol) = callee {
859        build = build.symbol(symbol);
860    }
861    for operand in operands {
862        build = build.operand(operand);
863    }
864    build.finish();
865    Ok(Made { results, outgoing: places.size().max(nested) })
866}
867
868/// One object whose bytes go into the argument area by a call to the runtime.
869#[derive(Debug, Clone, Copy)]
870struct Copying {
871    /// The register its address is in.
872    from: mir::Reg,
873    /// How far up the outgoing area its copy goes.
874    up: i32,
875    /// How many bytes it is.
876    count: i32,
877    /// Where the call it is an argument of was written.
878    span: Span,
879}
880
881/// One object with more words than a copy into the argument area unrolls to, copied there by a
882/// call to the runtime, and how many bytes that call itself needed below the stack pointer.
883///
884/// Which routine it is is [`crate::capability`]'s answer and not a name written here, because a
885/// call standing in for an operation the machine has no instruction for is what that table is a
886/// list of, and a copy too large to unroll is already on it: this is the same row `crate::expand`
887/// reads for a `memcpy` in the IR of the same size.
888fn by_runtime(
889    out: &mut mir::Func,
890    block: mir::Block,
891    conv: &CallRegs,
892    insts: &Insts,
893    names: &mut Interner,
894    what: Copying,
895) -> u32 {
896    let Copying { from, up, count, span } = what;
897    let routine = capability::libcall(rucc_ir::Opcode::Memcpy, "big")
898        .expect("the runtime copies a block too large to unroll");
899
900    // Where the copy goes, which is a distance up the outgoing area and so is the stack pointer
901    // plus that distance. A `lea` rather than an add, because the stack pointer is not this
902    // function's to move and what the call wants is the address in a register of the allocator's
903    // choosing.
904    let sp = mir::Operand::read(mir::Reg::physical(conv.stack_pointer), conv.int_class);
905    let into = out.new_vreg(conv.int_class);
906    let lea = mir::Opcode::new(names.intern(insts.lea));
907    out.build(block, lea)
908        .at(span)
909        .def(into, conv.int_class)
910        .mem(mir::Mem::at(sp).plus(up))
911        .finish();
912
913    // And how many bytes, which C takes as a `size_t` and this has as a number. A thirty two bit
914    // move carries it, because writing the low half of a general purpose register clears the high
915    // half, so the sixty four bit count it becomes is the count as long as the count fits in the
916    // immediate, which is what the caller checked before anything was built.
917    let bytes = out.new_vreg(conv.int_class);
918    let mov = mir::Opcode::new(names.intern(insts.small));
919    out.build(block, mov).at(span).def(bytes, conv.int_class).imm(i64::from(count)).finish();
920
921    let args = [
922        Passing { ty: Type::PTR, reg: into, abi: Abi::Plain },
923        Passing { ty: Type::PTR, reg: from, abi: Abi::Plain },
924        Passing { ty: Type::int(conv.word * 8), reg: bytes, abi: Abi::Plain },
925    ];
926    let made = Calling {
927        callee: Callee::Named(names.intern(routine)),
928        args: &args,
929        returns: &[],
930        variadic: false,
931        named: args.len(),
932        at: span,
933    };
934    // Three pointer sized arguments and nothing coming back is a call every convention here has
935    // registers for, so the only way this could refuse is a convention with fewer than three
936    // argument registers, and there is no such convention.
937    call(out, block, &made, conv, insts, names)
938        .expect("the runtime's copy passes three words and takes nothing back")
939        .outgoing
940}
941
942/// Which register each value comes back in, walked the way the arguments are.
943///
944/// The two files are counted separately, because a structure of a `double` and a `long` comes back
945/// with the `double` in the first vector register and the `long` in the first integer one, and a
946/// single count would put the second half one place further along a list it is not on.
947///
948/// # Errors
949///
950/// The first value that cannot come back at all, and why, so that a call this cannot make leaves
951/// nothing behind. Nothing here reports which value it was, because the caller has one answer for
952/// all of them: the value that comes back is not an argument and has no position among them.
953fn places_back(
954    returns: &[Type],
955    conv: &CallRegs,
956    insts: &Insts,
957) -> Result<Vec<(PhysReg, RegClass)>, Refused> {
958    let refused = |missing| Refused { argument: None, missing };
959    let mut back = Vec::with_capacity(returns.len());
960    let (mut ints, mut sses) = (0usize, 0usize);
961    for &ty in returns {
962        if let Some(missing) = refuses(ty, insts) {
963            return Err(refused(missing));
964        }
965        let class = class_of(ty, conv);
966        let (file, at) = if class == conv.sse_class {
967            (conv.sse_returns, &mut sses)
968        } else {
969            (conv.int_returns, &mut ints)
970        };
971        let reg = *file.get(*at).ok_or_else(|| refused(Missing::NoRoom))?;
972        *at += 1;
973        back.push((reg, class));
974    }
975    Ok(back)
976}
977
978/// Which of the four widths a value travels at, where a truth value travels as the byte it lives
979/// in.
980///
981/// [`crate::term::slot`] is the question the rule set asks and it answers nothing for one bit,
982/// because there is no register of that width and so no instruction written at it. A convention
983/// asks a different question. It has nothing narrower than a byte to put an argument in either,
984/// and what it says about the one type that is a bit is that the byte holding it is the argument,
985/// with the seven bits above unspecified. So the two lists differ by exactly this entry.
986///
987/// It is written here and not in [`crate::term::slot`] because moving it there would tell the rule
988/// set that one bit is a byte, and then every byte rule in the file would match a term that is not
989/// one. What the convention needs is narrower: a name for the register an argument arrives in, and
990/// that name says a width because a listing is easier to read when it does.
991fn place(ty: Type) -> Option<usize> {
992    if crate::term::is_bit(ty) { Some(0) } else { crate::term::slot(ty) }
993}
994
995/// What the pseudo for an argument of that type is called.
996///
997/// The width is in the name for the same reason it is in every other opcode here: it is what the
998/// instruction is about. Nothing encodes it, so nothing depends on it being right, but a listing
999/// that says an argument arrived and does not say how much of it did is a listing worth less.
1000///
1001/// Which widths there are is `place` above, and not a list of its own, because it has to be the
1002/// same list the three below use. An argument brought in at a width nothing downstream has a name
1003/// for is a register nothing could then read, and a width the others cover that this refuses is a
1004/// function turned away for no reason. Asking one question in one place is what keeps the four
1005/// answers from drifting, and an address is what they used to disagree about.
1006#[must_use]
1007pub fn head_of(ty: Type) -> Option<&'static str> {
1008    // The format that fills a whole vector register, which travels in one of them: the psABI
1009    // classifies it SSE and SSEUP, and those two eightbytes are the one register the pair names
1010    // rather than two registers.
1011    if crate::term::is_quad(ty) {
1012        return Some("x64.arg_val_f128");
1013    }
1014    // The half, which arrives in the low sixteen bits of a vector register the same way a `float`
1015    // arrives in the low thirty two. It is a name of its own rather than the `f32` one for the
1016    // reason every other width here has a name of its own: what arrived is two bytes, and a
1017    // listing that said four would be saying something the convention does not.
1018    if crate::term::is_half(ty) {
1019        return Some("x64.arg_val_f16");
1020    }
1021    if let Some(at) = crate::term::float_slot(ty) {
1022        return Some(["x64.arg_val_f32", "x64.arg_val_f64"][at]);
1023    }
1024    let names = ["x64.arg_val_8", "x64.arg_val_16", "x64.arg_val_32", "x64.arg_val_64"];
1025    Some(names[place(ty)?])
1026}
1027
1028/// What the instruction that reads an argument of that type out of memory is called.
1029///
1030/// Keyed off the same two questions [`head_of`] asks and answering for the same set of types, so
1031/// that a parameter this compiler can bring in from a register is one it can bring in from the
1032/// caller's stack as well. A width one of them covered and the other did not would be a function
1033/// turned away for where its sixth argument happened to land.
1034///
1035/// Reading a narrow argument at its own width and not at a word is deliberate. The caller wrote a
1036/// whole word, but what it put in the part above the value is not something the convention says, so
1037/// the bits this reads are exactly the bits that mean anything. That is the same thing an argument
1038/// arriving in a register gets: `x64.arg_val_8` says the low byte of that register is the argument
1039/// and says nothing at all about the rest of it.
1040#[must_use]
1041pub fn load_of(ty: Type) -> Option<&'static str> {
1042    // Sixteen bytes, which is the whole register and is also the whole value, so the instruction
1043    // a spill uses and the instruction an argument uses are the same one here. They are two
1044    // different instructions at the two narrower formats because there the value is part of the
1045    // register, and at this format there is no part of it to leave behind.
1046    if crate::term::is_quad(ty) {
1047        return Some("x64.movaps_rm");
1048    }
1049    // Two bytes out of the argument area and into the low lane of a vector register, which is one
1050    // instruction and is the same one a rule writes for a program's own read of a `_Float16`.
1051    if crate::term::is_half(ty) {
1052        return Some("x64.pinsrw_rm");
1053    }
1054    if let Some(at) = crate::term::float_slot(ty) {
1055        return Some(["x64.movss_rm", "x64.movsd_rm"][at]);
1056    }
1057    let names = ["x64.mov_rm_8", "x64.mov_rm_16", "x64.mov_rm_32", "x64.mov_rm_64"];
1058    Some(names[place(ty)?])
1059}
1060
1061/// What the instruction that writes an argument of that type into memory is called.
1062///
1063/// The mirror of [`load_of`], keyed off the same two questions and answering for the same set of
1064/// types, so that the two ends of one call agree about what travels. A type a callee can read out
1065/// of the argument area and a caller cannot write into it would be a call turned away for a reason
1066/// the function it calls does not have.
1067///
1068/// Writing a narrow argument at its own width leaves whatever was already in the rest of the word.
1069/// That is allowed, and it is what [`load_of`] is written against: the convention does not say what
1070/// is above the value, so the callee reads only the bits that mean anything and neither end has to
1071/// agree about the rest.
1072#[must_use]
1073pub fn store_of(ty: Type) -> Option<&'static str> {
1074    if crate::term::is_quad(ty) {
1075        return Some("x64.movaps_mr");
1076    }
1077    // The half goes out four bytes wide, which is the one place here where the instruction is not
1078    // the width of the value. SSE2 has no store of sixteen bits out of a vector register: the form
1079    // of `pextrw` that writes memory arrived with SSE4.1 and is above this target's baseline, so
1080    // the choices are a four byte store or a pair of instructions through a general purpose
1081    // register, and a pair is not something one name can be.
1082    //
1083    // Four bytes is safe here and would not be everywhere. An argument on the stack sits in an
1084    // eightbyte of its own, the paragraph above says the convention promises nothing about the
1085    // part of it above the value, and [`load_of`] reads back exactly the two bytes that mean
1086    // anything. What lands in the two bytes beside them is whatever was in the register, which is
1087    // no more than any other narrow argument leaves behind.
1088    if crate::term::is_half(ty) {
1089        return Some("x64.movss_mr");
1090    }
1091    if let Some(at) = crate::term::float_slot(ty) {
1092        return Some(["x64.movss_mr", "x64.movsd_mr"][at]);
1093    }
1094    let names = ["x64.mov_mr_8", "x64.mov_mr_16", "x64.mov_mr_32", "x64.mov_mr_64"];
1095    Some(names[place(ty)?])
1096}
1097
1098/// What the instruction that leaves a returned value in its register is called, for the value at
1099/// that place in its own register file.
1100///
1101/// Keyed off the same two questions [`head_of`] asks, so a type this can give back is a type it can
1102/// take in. The place is the one a call counted to when it laid the return out, which is per file
1103/// rather than over the whole list: a structure of a `double` and a `long` gives both of them back
1104/// at place zero.
1105///
1106/// The register itself is not here. It is in the operand table in `rucc_target::x86_64`, which is
1107/// where the first one has always been, and the two names below are how a value says which of the
1108/// two it is. A convention with more than two registers to come back in would need more names, and
1109/// there is none, which is what the `None` at the end is about.
1110#[must_use]
1111pub fn ret_of(ty: Type, at: usize) -> Option<&'static str> {
1112    if crate::term::is_quad(ty) {
1113        return Some(*["x64.ret_val_f128", "x64.ret_val2_f128"].get(at)?);
1114    }
1115    if crate::term::is_half(ty) {
1116        return Some(*["x64.ret_val_f16", "x64.ret_val2_f16"].get(at)?);
1117    }
1118    if let Some(width) = crate::term::float_slot(ty) {
1119        let names =
1120            [["x64.ret_val_f32", "x64.ret_val_f64"], ["x64.ret_val2_f32", "x64.ret_val2_f64"]];
1121        return Some(names.get(at)?[width]);
1122    }
1123    let names = [
1124        ["x64.ret_val_8", "x64.ret_val_16", "x64.ret_val_32", "x64.ret_val_64"],
1125        ["x64.ret_val2_8", "x64.ret_val2_16", "x64.ret_val2_32", "x64.ret_val2_64"],
1126    ];
1127    Some(names.get(at)?[place(ty)?])
1128}
1129
1130#[cfg(test)]
1131mod tests {
1132    use rucc_target::x86_64::{REGS, SYSV, WIN64};
1133
1134    use super::*;
1135
1136    /// Those types as parameters that travel as the values they are, which is every one of them
1137    /// that is not a structure the classification put in the argument area.
1138    fn plain(params: &[Type]) -> Vec<Param> {
1139        params.iter().copied().map(Param::new).collect()
1140    }
1141
1142    /// The parameters of a function under a convention, as machine IR text.
1143    fn bind(params: &[Type], conv: &CallRegs) -> String {
1144        let mut names = Interner::new();
1145        let mut out = mir::Func::new(names.intern("f"));
1146        let block = out.create_block();
1147        entry(&mut out, block, &plain(params), conv, &X86_64, &mut names, None)
1148            .expect("every parameter arrives");
1149        mir::print_func(&out, &names, &REGS)
1150    }
1151
1152    #[test]
1153    fn the_first_arguments_arrive_where_the_convention_puts_them() {
1154        let i32 = Type::int(32);
1155        assert_eq!(
1156            bind(&[i32, i32, Type::int(64)], &SYSV),
1157            "mfunc @f {\nblock0:\n    %0:gpr($rdi) = x64.arg_val_32\n    \
1158             %1:gpr($rsi) = x64.arg_val_32\n    %2:gpr($rdx) = x64.arg_val_64\n}\n"
1159        );
1160    }
1161
1162    #[test]
1163    fn the_other_convention_puts_the_same_arguments_somewhere_else() {
1164        // The first argument is in `rcx` here and in `rdi` above, which is the difference that
1165        // makes a SysV binary calling a Windows one read the wrong value rather than fail.
1166        let i64 = Type::int(64);
1167        assert_eq!(
1168            bind(&[i64, i64], &WIN64),
1169            "mfunc @f {\nblock0:\n    %0:gpr($rcx) = x64.arg_val_64\n    \
1170             %1:gpr($rdx) = x64.arg_val_64\n}\n"
1171        );
1172    }
1173
1174    /// The parameters of a function under a convention, and what each of the ones that arrived in
1175    /// memory is waiting on.
1176    fn arrive(params: &[Type], conv: &CallRegs) -> (String, Vec<u32>) {
1177        let mut names = Interner::new();
1178        let mut out = mir::Func::new(names.intern("f"));
1179        let block = out.create_block();
1180        let arrived = entry(&mut out, block, &plain(params), conv, &X86_64, &mut names, None)
1181            .expect("every parameter");
1182        let up = arrived.stack.iter().map(|&(_, up)| up).collect();
1183        (mir::print_func(&out, &names, &REGS), up)
1184    }
1185
1186    #[test]
1187    fn an_argument_past_the_last_register_is_read_out_of_the_caller_s_stack() {
1188        let (text, up) = arrive(&[Type::int(64); 7], &SYSV);
1189
1190        // Six of them got registers and the seventh did not, so the seventh is a load rather than
1191        // a pseudo. It reads through the stack pointer with nothing in its displacement, because
1192        // where the caller's argument area is from in here is a distance into a frame that does
1193        // not exist yet, and it is at the bottom of that area because it is the first one in it.
1194        assert_eq!(up, [0]);
1195        assert!(text.contains("%6:gpr = x64.mov_rm_64 [$rsp]"), "{text}");
1196        assert_eq!(text.matches("x64.arg_val_64").count(), 6, "{text}");
1197    }
1198
1199    #[test]
1200    fn the_other_convention_runs_out_of_registers_three_arguments_earlier() {
1201        let (text, up) = arrive(&[Type::int(64); 7], &WIN64);
1202
1203        // Windows passes four integers in registers and reserves thirty two bytes below the call
1204        // whether they are used or not, so the fifth argument is not at the bottom of the argument
1205        // area but above the shadow space, and the three after it follow it a word at a time.
1206        assert_eq!(up, [32, 40, 48]);
1207        assert_eq!(text.matches("x64.arg_val_64").count(), 4, "{text}");
1208        assert!(text.contains("%4:gpr = x64.mov_rm_64 [$rsp]"), "{text}");
1209    }
1210
1211    /// The parameters of a function under a convention, with one of them a structure whose bytes
1212    /// travel, and what each of the ones that arrived in memory is waiting on.
1213    fn arrive_with(params: &[Param], conv: &CallRegs) -> (String, Vec<u32>) {
1214        let mut names = Interner::new();
1215        let mut out = mir::Func::new(names.intern("f"));
1216        let block = out.create_block();
1217        let arrived = entry(&mut out, block, params, conv, &X86_64, &mut names, None)
1218            .expect("every parameter");
1219        let up = arrived.stack.iter().map(|&(_, up)| up).collect();
1220        (mir::print_func(&out, &names, &REGS), up)
1221    }
1222
1223    #[test]
1224    fn a_structure_that_arrived_as_bytes_is_an_address_and_not_a_load() {
1225        let byval =
1226            Param::with_abi(Type::PTR, Abi::ByVal { size: 32, align: 8, drains: Drains::Nothing });
1227        let (text, up) =
1228            arrive_with(&[Param::new(Type::int(32)), byval, Param::new(Type::int(32))], &SYSV);
1229
1230        // `int f(int a, struct Big b, int c)`. The bytes of `b` are already in this function, at
1231        // the bottom of the caller's argument area, so nothing is read out of them here: what the
1232        // parameter is is where they are, which is one address. The two integers still travel in
1233        // registers, because an object in the argument area takes no register and the arguments
1234        // behind it do not shift along.
1235        assert_eq!(up, [0]);
1236        assert_eq!(text.matches("x64.arg_val_32").count(), 2, "{text}");
1237        assert!(text.contains("%2:gpr = x64.lea_64 [$rsp]"), "{text}");
1238        assert!(!text.contains("mov_rm"), "nothing is read out of the bytes: {text}");
1239    }
1240
1241    #[test]
1242    fn the_argument_behind_a_structure_that_travelled_as_bytes_is_above_all_of_them() {
1243        let byval =
1244            Param::with_abi(Type::PTR, Abi::ByVal { size: 24, align: 16, drains: Drains::Nothing });
1245        let params: Vec<Param> = (0..7).map(|_| Param::new(Type::int(64))).collect();
1246        let (_, up) = arrive_with(&[&params[..], &[byval], &params[..1]].concat(), &SYSV);
1247
1248        // Six integers take the six registers, the seventh is at the bottom of the argument area,
1249        // and the structure is above it at the alignment its type asks for rather than at a word.
1250        // The one behind the structure is above all twenty four of its bytes, rounded up to a
1251        // whole number of words, because the area is a run of words.
1252        assert_eq!(up, [0, 16, 40]);
1253    }
1254
1255    /// A parameter narrower than a word is read at its own width rather than at a word, and one in
1256    /// the other register file is read with the other file's instruction. Both are the same list
1257    /// [`head_of`] answers from, which is what stops a function being turned away for the width of
1258    /// its seventh argument alone.
1259    #[test]
1260    fn what_a_stack_argument_is_read_with_is_its_own_width_and_its_own_file() {
1261        let f32 = Type::float(rucc_ir::Float::F32);
1262        let params = [Type::int(64), Type::int(64), Type::int(64), Type::int(64), Type::int(8)];
1263        let (text, up) = arrive(&params, &WIN64);
1264        assert_eq!(up, [32]);
1265        assert!(text.contains("x64.mov_rm_8 [$rsp]"), "{text}");
1266
1267        let floats = [f32; 5];
1268        let (text, up) = arrive(&floats, &WIN64);
1269        assert_eq!(up, [32]);
1270        assert!(text.contains("%4:xmm = x64.movss_rm [$rsp]"), "{text}");
1271    }
1272
1273    /// Every type a parameter can arrive in a register at is one it can be read from memory at.
1274    /// The two lists are keyed off the same two questions so that they cannot drift, and this is
1275    /// what says so: a width one covered and the other did not would be a function turned away for
1276    /// where its arguments happened to land rather than for anything about it.
1277    #[test]
1278    fn the_two_lists_of_widths_answer_for_the_same_types() {
1279        let types = [
1280            Type::int(1),
1281            Type::int(8),
1282            Type::int(16),
1283            Type::int(32),
1284            Type::int(64),
1285            Type::int(128),
1286            Type::PTR,
1287            Type::float(rucc_ir::Float::F32),
1288            Type::float(rucc_ir::Float::F64),
1289            Type::float(rucc_ir::Float::F80),
1290            Type::float(rucc_ir::Float::F128),
1291        ];
1292        for ty in types {
1293            assert_eq!(head_of(ty).is_some(), load_of(ty).is_some(), "{ty:?}");
1294            assert_eq!(head_of(ty).is_some(), store_of(ty).is_some(), "{ty:?}");
1295            assert_eq!(head_of(ty).is_some(), ret_of(ty, 0).is_some(), "{ty:?}");
1296        }
1297    }
1298
1299    /// A hundred and twenty eight bit float arrives in a vector register like the two narrower
1300    /// formats, and it takes one of them rather than two: the psABI classifies it SSE and SSEUP,
1301    /// and what that pair names is the one register both eightbytes are in.
1302    ///
1303    /// The second float here is what says so. If the quad had taken two vector registers the
1304    /// `double` after it would be in `xmm2`.
1305    #[test]
1306    fn a_quad_float_arrives_in_one_vector_register_and_not_in_two() {
1307        let quad = Type::float(rucc_ir::Float::F128);
1308        let f64 = Type::float(rucc_ir::Float::F64);
1309        assert_eq!(
1310            bind(&[quad, f64], &SYSV),
1311            "mfunc @f {\nblock0:\n    %0:xmm($xmm0) = x64.arg_val_f128\n    \
1312             %1:xmm($xmm1) = x64.arg_val_f64\n}\n"
1313        );
1314    }
1315
1316    /// And it is not the width that was refused before there was an instruction to move it with,
1317    /// which is the one thing about this type that used to turn a whole function away.
1318    #[test]
1319    fn a_quad_float_is_no_longer_a_width_nothing_can_carry() {
1320        assert_eq!(refuses(Type::float(rucc_ir::Float::F128), &X86_64), None);
1321        assert_eq!(refuses(Type::float(rucc_ir::Float::F80), &X86_64), Some(Missing::OnX87));
1322        assert_eq!(refuses(Type::int(128), &X86_64), Some(Missing::Width));
1323    }
1324
1325    /// A float arrives in the other file, and the two files are counted apart on SysV: the
1326    /// integer here is the first integer argument and the float is the first float one, so they
1327    /// are in `rdi` and `xmm0` rather than in the first and second of anything.
1328    #[test]
1329    fn a_float_arrives_in_a_vector_register_and_is_counted_apart_from_the_integers() {
1330        let f32 = Type::float(rucc_ir::Float::F32);
1331        let f64 = Type::float(rucc_ir::Float::F64);
1332        assert_eq!(
1333            bind(&[Type::int(32), f64, f32], &SYSV),
1334            "mfunc @f {\nblock0:\n    %0:gpr($rdi) = x64.arg_val_32\n    \
1335             %1:xmm($xmm0) = x64.arg_val_f64\n    %2:xmm($xmm1) = x64.arg_val_f32\n}\n"
1336        );
1337    }
1338
1339    /// Windows counts the two files together, so the same three arguments land in different
1340    /// registers: the float is the second argument and takes the second vector register rather
1341    /// than the first, which is the difference that makes a mismatched call read the wrong value.
1342    #[test]
1343    fn the_other_convention_counts_the_two_files_as_one_run_of_positions() {
1344        let f64 = Type::float(rucc_ir::Float::F64);
1345        assert_eq!(
1346            bind(&[Type::int(32), f64, Type::int(64)], &WIN64),
1347            "mfunc @f {\nblock0:\n    %0:gpr($rcx) = x64.arg_val_32\n    \
1348             %1:xmm($xmm1) = x64.arg_val_f64\n    %2:gpr($r8) = x64.arg_val_64\n}\n"
1349        );
1350    }
1351
1352    /// A `long double` is in neither file and travels in the argument area, which is what SysV's
1353    /// X87 classification comes to. So it arrives the way a structure the classification put in
1354    /// memory arrives, as the address of its bytes in a general purpose register, and it does that
1355    /// while the vector file is untouched: this one is in the argument area because of what it is
1356    /// rather than because the registers ran out.
1357    #[test]
1358    fn a_long_double_arrives_as_the_address_of_its_bytes_in_the_argument_area() {
1359        let params = [Type::int(32), Type::float(rucc_ir::Float::F80)];
1360        assert_eq!(
1361            bind(&params, &SYSV),
1362            "mfunc @f {\nblock0:\n    %0:gpr($rdi) = x64.arg_val_32\n    \
1363             %1:gpr = x64.lea_64 [$rsp]\n}\n"
1364        );
1365    }
1366
1367    /// It still cannot come back beside another value, and what it is turned away for says which
1368    /// file is in the way rather than calling eighty bits a width no register holds. A pair comes
1369    /// back in a pair of registers and there is no pair with the x87 stack in it.
1370    #[test]
1371    fn a_long_double_in_a_pair_is_reported_as_the_x87_stack_it_travels_on() {
1372        let returns = [Type::float(rucc_ir::Float::F80), Type::int(64)];
1373        assert_eq!(
1374            make(&[], &returns, false, &SYSV).2,
1375            Err(Refused { argument: None, missing: Missing::OnX87 })
1376        );
1377    }
1378
1379    /// One call to `g`, with a register for each argument arriving in the block that makes it.
1380    ///
1381    /// A variadic call here names none of its arguments, which is the shape that asks the most of a
1382    /// convention. [`made_naming`] is for the tests that care where the line between the named ones
1383    /// and the rest actually falls.
1384    fn make(
1385        args: &[Type],
1386        returns: &[Type],
1387        variadic: bool,
1388        conv: &CallRegs,
1389    ) -> (Interner, mir::Func, Result<Made, Refused>) {
1390        let named = if variadic { 0 } else { args.len() };
1391        made_naming(args, returns, named, variadic, conv)
1392    }
1393
1394    /// The same, for a callee whose signature names that many of the arguments.
1395    fn made_naming(
1396        args: &[Type],
1397        returns: &[Type],
1398        named: usize,
1399        variadic: bool,
1400        conv: &CallRegs,
1401    ) -> (Interner, mir::Func, Result<Made, Refused>) {
1402        let mut names = Interner::new();
1403        let mut out = mir::Func::new(names.intern("f"));
1404        let block = out.create_block();
1405        let passed: Vec<Passing> = args
1406            .iter()
1407            .map(|&ty| Passing {
1408                ty,
1409                reg: out.append_param(block, class_of(ty, conv)),
1410                abi: Abi::Plain,
1411            })
1412            .collect();
1413        let callee = Callee::Named(names.intern("g"));
1414        let what = Calling { callee, args: &passed, returns, variadic, named, at: Span::DUMMY };
1415        let made = call(&mut out, block, &what, conv, &X86_64, &mut names);
1416        (names, out, made)
1417    }
1418
1419    /// What the call in that function reads and writes, by register name, in the order the
1420    /// operands are in.
1421    fn operands(func: &mir::Func) -> (Vec<String>, Vec<String>) {
1422        let block = func.entry().expect("a function with a block in it");
1423        let call = func.terminator(block).expect("the call is the last thing in the block");
1424        let name = |operand: &mir::Operand| match (operand.reg.phys(), operand.constraint) {
1425            (Some(reg), _) | (None, Constraint::Fixed(reg)) => {
1426                REGS.name(operand.class, reg).expect("a register the file describes").to_string()
1427            }
1428            _ => format!("{:?}", operand.reg),
1429        };
1430        let mut written = Vec::new();
1431        let mut read = Vec::new();
1432        for operand in &func[func[call].operands] {
1433            let into = if operand.role == mir::Role::Use { &mut read } else { &mut written };
1434            into.push(name(operand));
1435        }
1436        (written, read)
1437    }
1438
1439    #[test]
1440    fn a_call_passes_its_arguments_where_the_convention_puts_them() {
1441        let i32 = Type::int(32);
1442        let (_, func, made) = make(&[i32, i32, i32], &[], false, &SYSV);
1443        assert_eq!(made.expect("three integers all fit in registers").results, []);
1444        assert_eq!(operands(&func).1, ["rdi", "rsi", "rdx"]);
1445    }
1446
1447    #[test]
1448    fn the_other_convention_passes_the_same_arguments_somewhere_else() {
1449        let i64 = Type::int(64);
1450        let (_, func, made) = make(&[i64, i64], &[], false, &WIN64);
1451        // Thirty two bytes of stack for a call that passes nothing on the stack, which is what
1452        // Windows asks a caller to leave the callee whether the callee uses it or not.
1453        assert_eq!(made.expect("two integers fit in registers").outgoing, 32);
1454        assert_eq!(operands(&func).1, ["rcx", "rdx"]);
1455    }
1456
1457    #[test]
1458    fn what_a_call_gives_back_comes_out_of_the_register_the_convention_returns_in() {
1459        let (names, func, made) = make(&[], &[Type::int(32)], false, &SYSV);
1460        let made = made.expect("an integer comes back");
1461        let [result] = made.results[..] else { panic!("one register") };
1462        // The first thing written is the result, and it is the only thing written that is a value
1463        // rather than a register the callee destroyed.
1464        assert_eq!(operands(&func).0.first().map(String::as_str), Some("rax"));
1465        assert_eq!(func.class_of(result), Some(SYSV.int_class));
1466        assert!(mir::print_func(&func, &names, &REGS).contains("x64.call"));
1467    }
1468
1469    #[test]
1470    fn every_register_the_callee_may_destroy_is_written_by_the_call() {
1471        let (_, func, _) = make(&[Type::int(64)], &[Type::int(64)], false, &SYSV);
1472        let (written, read) = operands(&func);
1473        // The callee saved registers are not here, because a value in one of those survives a
1474        // call and that is the whole difference between the two halves of the convention.
1475        for saved in ["rbx", "rbp", "r12", "r13", "r14", "r15"] {
1476            assert!(!written.contains(&saved.to_string()), "{saved} survives a call");
1477        }
1478        // Every other integer register is, once. The two named ones are named by the result and
1479        // by the argument instead, and naming one twice would be blocking it twice.
1480        for destroyed in ["rcx", "rdx", "rsi", "r8", "r9", "r10", "r11"] {
1481            let count = written.iter().filter(|name| *name == destroyed).count();
1482            assert_eq!(count, 1, "{destroyed} is destroyed by a call and is written {count} times");
1483        }
1484        assert_eq!(written.iter().filter(|name| *name == "rax").count(), 1);
1485        assert_eq!(read, ["rdi"]);
1486        // The vector registers are all destroyed on SysV, and they are in the other class.
1487        assert!(written.contains(&"xmm0".to_string()));
1488    }
1489
1490    #[test]
1491    fn a_variadic_call_says_how_many_vector_registers_it_passed_arguments_in() {
1492        let (names, func, made) = make(&[Type::int(64)], &[], true, &SYSV);
1493        made.expect("an integer argument to a variadic callee");
1494        let (_, read) = operands(&func);
1495        // Zero of them here, and `al` is where a SysV callee looks for it. Leaving whatever was in
1496        // the register there would make a callee that saves its vector registers save ones it was
1497        // never given.
1498        assert_eq!(read, ["rdi", "rax"]);
1499        assert_eq!(
1500            mir::print_func(&func, &names, &REGS).lines().nth(2),
1501            Some("    %1:gpr = x64.mov_ri_32 0")
1502        );
1503
1504        // Two of them here, which is the number that decides how much of the register save area a
1505        // callee like `printf` fills in. A count of zero with a float in `xmm0` would be a callee
1506        // reading its first `%f` out of a register nothing wrote.
1507        let f64 = Type::float(rucc_ir::Float::F64);
1508        let (names, func, made) = make(&[Type::int(64), f64, f64], &[], true, &SYSV);
1509        made.expect("one integer and two floats all fit in registers");
1510        assert_eq!(operands(&func).1, ["rdi", "xmm0", "xmm1", "rax"]);
1511        assert!(mir::print_func(&func, &names, &REGS).contains("x64.mov_ri_32 2"));
1512    }
1513
1514    /// Windows passes a float the callee has no prototype for in both files at once, because the
1515    /// callee has no way to know which file to look in and its walk over the arguments reads the
1516    /// general purpose one. This is the whole of what `printf("%f", x)` needs from the caller.
1517    #[test]
1518    fn a_float_a_variadic_callee_has_no_prototype_for_travels_in_both_files_on_windows() {
1519        let f64 = Type::float(rucc_ir::Float::F64);
1520        let (names, func, made) = made_naming(&[Type::int(32), f64], &[], 1, true, &WIN64);
1521        made.expect("an integer and a float both fit in registers");
1522
1523        // The float is the second argument, so its position is one and both of its registers are
1524        // the second of their file. `rdx` holds the bits and nothing converts them, which is what
1525        // the `movq` is: the callee reads bits out of it and not a value of any type.
1526        assert_eq!(operands(&func).1, ["rcx", "xmm1", "rdx"]);
1527        let text = mir::print_func(&func, &names, &REGS);
1528        assert!(text.contains("x64.movq_from_xmm %1"), "{text}");
1529    }
1530
1531    /// An argument the signature does name needs no second copy, since the callee's parameter says
1532    /// where it is, and neither does one on a convention that keeps the two files apart.
1533    #[test]
1534    fn an_argument_the_signature_names_travels_in_one_file() {
1535        let f64 = Type::float(rucc_ir::Float::F64);
1536        let (names, func, made) = made_naming(&[Type::int(32), f64], &[], 2, true, &WIN64);
1537        made.expect("both are named");
1538        assert_eq!(operands(&func).1, ["rcx", "xmm1"]);
1539        assert!(!mir::print_func(&func, &names, &REGS).contains("movq_from_xmm"));
1540
1541        let (names, func, made) = made_naming(&[Type::int(32), f64], &[], 1, true, &SYSV);
1542        made.expect("an integer and a float");
1543        assert!(!mir::print_func(&func, &names, &REGS).contains("movq_from_xmm"));
1544    }
1545
1546    /// A float past the position the registers run out at is in the argument area and nowhere else,
1547    /// which is where the second copy stops being a thing there is room for. The callee reads it
1548    /// out of memory whichever file it would have been in.
1549    #[test]
1550    fn a_float_the_registers_ran_out_before_gets_no_second_copy() {
1551        let f64 = Type::float(rucc_ir::Float::F64);
1552        let (names, func, made) = made_naming(&[f64; 6], &[], 0, true, &WIN64);
1553        made.expect("four in registers and two in memory");
1554        let text = mir::print_func(&func, &names, &REGS);
1555        assert_eq!(text.matches("movq_from_xmm").count(), 4, "{text}");
1556        assert!(text.contains("x64.movsd_mr %4, [$rsp + 32]"), "{text}");
1557    }
1558
1559    #[test]
1560    fn a_call_through_an_address_reads_it_in_front_of_the_arguments() {
1561        let i32 = Type::int(32);
1562        let mut names = Interner::new();
1563        let mut out = mir::Func::new(names.intern("f"));
1564        let block = out.create_block();
1565        let address = out.append_param(block, SYSV.int_class);
1566        let reg = out.append_param(block, SYSV.int_class);
1567        let passed = vec![Passing { ty: i32, reg, abi: Abi::Plain }];
1568        let what = Calling {
1569            callee: Callee::Through(address),
1570            args: &passed,
1571            returns: &[i32],
1572            variadic: false,
1573            named: passed.len(),
1574            at: Span::DUMMY,
1575        };
1576        call(&mut out, block, &what, &SYSV, &X86_64, &mut names)
1577            .expect("one integer fits in a register");
1578
1579        // The address is the first thing read and the arguments follow it, which is the order the
1580        // assembler counts on, and it is in no particular register because every register a call
1581        // could insist on is one the call has already spoken for.
1582        let text = mir::print_func(&out, &names, &REGS);
1583        assert!(text.contains("= x64.call_reg %0, %1($rdi)\n"), "{text}");
1584        assert!(!text.contains("@g"), "a call through an address names nobody: {text}");
1585    }
1586
1587    #[test]
1588    fn a_call_with_no_register_left_writes_the_argument_into_the_outgoing_area() {
1589        let i64 = Type::int(64);
1590        let (names, func, made) = make(&[i64; 7], &[], false, &SYSV);
1591        let made = made.expect("the seventh goes to memory");
1592
1593        // At the stack pointer, because the outgoing area is at the bottom of the frame, and in
1594        // front of the call rather than as an operand of it.
1595        let text = mir::print_func(&func, &names, &REGS);
1596        assert!(text.contains("x64.mov_mr_64 %6, [$rsp]\n"), "{text}");
1597        let store = text.find("x64.mov_mr_64").expect("the store");
1598        assert!(store < text.find("x64.call").expect("the call"), "{text}");
1599        // One word of it, which is what the frame has to reserve for this call.
1600        assert_eq!(made.outgoing, 8);
1601    }
1602
1603    /// One call to `g`, passing that many words, then an object of that size and alignment by
1604    /// value, then one more integer, which is `int g(long.., struct Big, int)` after the
1605    /// classification.
1606    fn pass_bytes(
1607        before: usize,
1608        size: u64,
1609        align: u32,
1610        conv: &CallRegs,
1611    ) -> (Interner, mir::Func, Result<Made, Refused>) {
1612        let mut names = Interner::new();
1613        let mut out = mir::Func::new(names.intern("f"));
1614        let block = out.create_block();
1615        let mut args: Vec<Passing> = (0..before)
1616            .map(|_| Passing {
1617                ty: Type::int(64),
1618                reg: out.append_param(block, conv.int_class),
1619                abi: Abi::Plain,
1620            })
1621            .collect();
1622        args.push(Passing {
1623            ty: Type::PTR,
1624            reg: out.append_param(block, conv.int_class),
1625            abi: Abi::ByVal { size, align, drains: Drains::Nothing },
1626        });
1627        args.push(Passing {
1628            ty: Type::int(32),
1629            reg: out.append_param(block, conv.int_class),
1630            abi: Abi::Plain,
1631        });
1632        let callee = Callee::Named(names.intern("g"));
1633        let what = Calling {
1634            callee,
1635            args: &args,
1636            returns: &[],
1637            variadic: false,
1638            named: args.len(),
1639            at: Span::DUMMY,
1640        };
1641        let made = call(&mut out, block, &what, conv, &X86_64, &mut names);
1642        (names, out, made)
1643    }
1644
1645    #[test]
1646    fn a_structure_passed_by_value_in_memory_is_copied_into_the_outgoing_area() {
1647        let (names, func, made) = pass_bytes(1, 24, 8, &SYSV);
1648        let made = made.expect("an object of three words is copied a word at a time");
1649
1650        // The bytes travel and the address does not, so the copy is a load and a store for each
1651        // word of it, in front of the call, and the callee's copy is at the bottom of the outgoing
1652        // area. The caller owes it this copy: the callee is free to write to what it was handed,
1653        // so what it was handed cannot be the object itself.
1654        let text = mir::print_func(&func, &names, &REGS);
1655        assert!(text.contains("x64.mov_mr_64 %3, [$rsp]\n"), "{text}");
1656        assert!(text.contains("x64.mov_mr_64 %4, [$rsp + 8]\n"), "{text}");
1657        assert!(text.contains("x64.mov_mr_64 %5, [$rsp + 16]\n"), "{text}");
1658        assert_eq!(text.matches("x64.mov_rm_64").count(), 3, "{text}");
1659        assert!(text.find("x64.mov_mr_64") < text.find("x64.call"), "{text}");
1660        assert_eq!(made.outgoing, 24);
1661    }
1662
1663    #[test]
1664    fn the_integers_beside_it_still_travel_in_registers() {
1665        let (_, func, _) = pass_bytes(1, 24, 8, &SYSV);
1666
1667        // An object in the argument area takes no argument register, so the integer behind it is
1668        // in the second one and not the third. Counting it as a register is the mistake that would
1669        // shift every argument after it along by one.
1670        let (clobbered, read) = operands(&func);
1671        assert_eq!(read, ["rdi", "rsi"]);
1672        assert!(clobbered.contains(&"rdx".to_owned()), "the third is free: {clobbered:?}");
1673    }
1674
1675    #[test]
1676    fn an_object_wanting_more_alignment_than_a_word_gets_it() {
1677        let (names, func, made) = pass_bytes(7, 24, 16, &SYSV);
1678        let made = made.expect("an object of three words");
1679
1680        // Six of the integers took the registers and the seventh is at the bottom of the area, so
1681        // the object cannot start where it left off: sixteen byte alignment moves it up to the
1682        // next multiple of sixteen and leaves a word of nothing behind it. The integer after it is
1683        // above all three of its words.
1684        let text = mir::print_func(&func, &names, &REGS);
1685        assert!(text.contains("x64.mov_mr_64 %9, [$rsp + 16]\n"), "{text}");
1686        assert!(text.contains("x64.mov_mr_32 %8, [$rsp + 40]\n"), "{text}");
1687        assert_eq!(made.outgoing, 48);
1688    }
1689
1690    #[test]
1691    fn an_object_too_large_to_copy_a_word_at_a_time_is_copied_by_the_runtime() {
1692        let (names, func, made) = pass_bytes(1, 4096, 8, &SYSV);
1693        let made = made.expect("an object the runtime copies");
1694
1695        // Five hundred and twelve words is past what unrolling is worth, so the copy is the call
1696        // the same size of `memcpy` in the IR becomes: the address in the outgoing area, the
1697        // address of the object, and the count, and then the call the object was an argument of.
1698        let text = mir::print_func(&func, &names, &REGS);
1699        assert!(text.contains("x64.lea_64 [$rsp]"), "{text}");
1700        assert!(text.contains("x64.mov_ri_32 4096"), "{text}");
1701        assert_eq!(text.matches("x64.call").count(), 2, "the copy and the call: {text}");
1702        assert!(text.find("@memcpy") < text.find("@g"), "the copy comes first: {text}");
1703
1704        // And the argument area is the object, because a call passing three words in registers
1705        // needs none of its own.
1706        assert_eq!(made.outgoing, 4096);
1707    }
1708
1709    #[test]
1710    fn an_object_too_large_to_count_the_bytes_of_is_reported_rather_than_passed() {
1711        let (_, _, made) = pass_bytes(1, 1 << 31, 8, &SYSV);
1712
1713        // Two gigabytes is more than the immediate the byte count travels in holds, and a count
1714        // that does not fit is the whole of what is left to refuse. No program passes a structure
1715        // that size by value, and one that tried would rather hear about it than be handed a copy
1716        // of the low part of it.
1717        assert_eq!(made, Err(Refused { argument: Some(1), missing: Missing::TooBig }));
1718        assert_eq!(
1719            Missing::TooBig.why(),
1720            "is more bytes than a count of them can be written down as"
1721        );
1722    }
1723
1724    /// The other convention runs out three arguments earlier and starts its argument area above the
1725    /// shadow space it also has to reserve, and both of those are what `Places` already said.
1726    #[test]
1727    fn where_the_outgoing_area_starts_is_the_convention_s_answer() {
1728        let i64 = Type::int(64);
1729        let (names, func, made) = make(&[i64; 7], &[], false, &WIN64);
1730        assert_eq!(made.expect("the last three go to memory").outgoing, 56);
1731
1732        // Thirty two bytes of shadow space first, which the caller writes nothing into and the
1733        // callee owns, and the fifth argument above it.
1734        let text = mir::print_func(&func, &names, &REGS);
1735        assert!(text.contains("x64.mov_mr_64 %4, [$rsp + 32]\n"), "{text}");
1736        assert!(text.contains("x64.mov_mr_64 %5, [$rsp + 40]\n"), "{text}");
1737        assert!(text.contains("x64.mov_mr_64 %6, [$rsp + 48]\n"), "{text}");
1738    }
1739
1740    /// What a stack argument is written with is its own width and its own register file, matching
1741    /// what the callee reads it back with.
1742    #[test]
1743    fn a_narrow_or_floating_argument_keeps_its_own_store() {
1744        let i64 = Type::int(64);
1745        let narrow = [i64, i64, i64, i64, i64, i64, Type::int(8)];
1746        let (names, func, made) = make(&narrow, &[], false, &SYSV);
1747        made.expect("the seventh goes to memory");
1748        let text = mir::print_func(&func, &names, &REGS);
1749        assert!(text.contains("x64.mov_mr_8 %6, [$rsp]\n"), "{text}");
1750
1751        let f32 = Type::float(rucc_ir::Float::F32);
1752        let (names, func, made) = make(&[f32; 9], &[], false, &SYSV);
1753        made.expect("the ninth goes to memory");
1754        let text = mir::print_func(&func, &names, &REGS);
1755        assert!(text.contains("x64.movss_mr %8, [$rsp]\n"), "{text}");
1756    }
1757
1758    /// The count a SysV variadic callee reads is a count of registers, so an argument that went to
1759    /// memory instead is not in it.
1760    #[test]
1761    fn an_argument_in_memory_is_not_counted_as_a_vector_register() {
1762        let f64 = Type::float(rucc_ir::Float::F64);
1763        let (names, func, made) = make(&[f64; 9], &[], true, &SYSV);
1764        made.expect("the ninth goes to memory");
1765        let text = mir::print_func(&func, &names, &REGS);
1766        assert!(text.contains("x64.mov_ri_32 8\n"), "eight registers, not nine: {text}");
1767    }
1768
1769    /// The two lists of widths answer for the same set of types, so that a value the callee can
1770    /// read out of the argument area is one the caller can write into it.
1771    #[test]
1772    fn what_can_be_read_can_be_written() {
1773        let types = [
1774            Type::int(1),
1775            Type::int(8),
1776            Type::int(16),
1777            Type::int(32),
1778            Type::int(64),
1779            Type::int(128),
1780            Type::PTR,
1781            Type::float(rucc_ir::Float::F32),
1782            Type::float(rucc_ir::Float::F64),
1783            Type::float(rucc_ir::Float::F80),
1784        ];
1785        for ty in types {
1786            assert_eq!(load_of(ty).is_some(), store_of(ty).is_some(), "{ty:?}");
1787        }
1788    }
1789
1790    /// A float travels in the other file at both ends of a call, and the register it comes back in
1791    /// is the first of that file rather than the first of the other one.
1792    #[test]
1793    fn a_call_passes_and_returns_a_float_in_a_vector_register() {
1794        let f64 = Type::float(rucc_ir::Float::F64);
1795        let (_, func, made) = make(&[Type::int(32), f64], &[f64], false, &SYSV);
1796        let result = made.expect("an integer and a float both fit in registers");
1797        let (written, read) = operands(&func);
1798        assert_eq!(read, ["rdi", "xmm0"]);
1799        assert_eq!(written.first().map(String::as_str), Some("xmm0"));
1800        assert_eq!(func.class_of(result.results[0]), Some(SYSV.sse_class));
1801        // Written once, because the register the result comes back in is already blocked by being
1802        // named and a clobber that repeated it would be blocking it twice. `rax` is a clobber here
1803        // rather than the result, which is the same register number in the other file and is the
1804        // whole reason the two lists are counted apart.
1805        assert_eq!(written.iter().filter(|name| *name == "xmm0").count(), 1);
1806        assert!(written.contains(&"rax".to_string()));
1807    }
1808
1809    #[test]
1810    fn a_call_at_a_width_no_register_holds_is_reported_on_either_side() {
1811        let i128 = Type::int(128);
1812        assert_eq!(
1813            make(&[i128], &[], false, &SYSV).2,
1814            Err(Refused { argument: Some(0), missing: Missing::Width })
1815        );
1816        assert_eq!(
1817            make(&[], &[i128], false, &SYSV).2,
1818            Err(Refused { argument: None, missing: Missing::Width })
1819        );
1820    }
1821
1822    #[test]
1823    fn an_argument_wider_than_a_register_has_no_name() {
1824        assert_eq!(head_of(Type::int(128)), None);
1825        assert_eq!(head_of(Type::int(8)), Some("x64.arg_val_8"));
1826        assert_eq!(head_of(Type::int(64)), Some("x64.arg_val_64"));
1827    }
1828
1829    /// An address arrives in a general purpose register like any other integer of its width, and
1830    /// used to be turned away here as a width no register holds, which is what issue 274 is.
1831    /// `int g(char *s)` is the smallest program that was.
1832    #[test]
1833    fn an_address_arrives_in_a_register_like_the_integer_it_is() {
1834        assert_eq!(head_of(Type::PTR), Some("x64.arg_val_64"));
1835        assert_eq!(
1836            bind(&[Type::PTR], &SYSV),
1837            "mfunc @f {\nblock0:\n    %0:gpr($rdi) = x64.arg_val_64\n}\n"
1838        );
1839        // And it travels the same way at a call, on both sides of one.
1840        assert!(make(&[Type::PTR], &[Type::PTR], false, &SYSV).2.is_ok());
1841    }
1842}