Skip to main content

rucc_opt/
switch_conv.rs

1//! A `switch` whose arms are a function of the label, which is arithmetic and not branches.
2//!
3//! Design: `spec/optimizer/24-switch-lowering.md` section 24.1, which is the transformation the GCC
4//! file `tree-switch-conversion.cc` is named after, and section 24.4, which puts it in the middle
5//! end rather than in the lowering and says why: what it produces is ordinary arithmetic that every
6//! pass after it optimizes, and what it needs to see is arms whose constancy earlier passes made
7//! visible.
8//!
9//! # The shape
10//!
11//! ```c
12//! switch (x) { case 0: return 1; case 1: return 2; case 2: return 3; case 3: return 4; }
13//! return 0;
14//! ```
15//!
16//! Four labels, four arms, and the arm for label `k` gives `k + 1`. The labels run consecutively
17//! and the answers run consecutively with them, so the whole statement is one range check and one
18//! addition. gcc reduces thirty three labels of this to a comparison and a `lea`, which is
19//! tamnd/rucc#728, and rucc emitted a comparison and a jump per label.
20//!
21//! Where the answers are an affine function of the label, `a * x + b`, there is nothing to look
22//! up. That covers the shape above with `a` of one and `b` of one, the shape where every arm gives
23//! the same answer with `a` of zero, and the scaled ones in between.
24//!
25//! Where they are not, the answers are a table. The arm for label `k` gives the constant in cell
26//! `k - low` of a read only array, and every arm becomes one load from it. That is section 24.4's
27//! other half, and it is what gcc calls a `CSWTCH` array. The array is asked for through
28//! `crate::readonly`, because a pass is handed one function and the array is the module's.
29//!
30//! # What it rewrites and what it leaves
31//!
32//! The `switch` stays a `switch`. Every case edge is pointed at one new block, which works the
33//! answer out and hands it on, and the default edge is not touched at all. What that buys is that
34//! the range check is not written here: a `switch` whose cases are consecutive and all go to one
35//! place is exactly `crates/rucc-codegen/src/switch.rs`'s `Cluster::Run`, which is one subtraction
36//! and one unsigned comparison however long the run is, and which already gets the modular
37//! arithmetic and the run that covers a whole type right. Writing a second range check here would
38//! be a second place for section 24.6's overflow to be got wrong.
39//!
40//! The default is untouched for the reason section 24.6 gives, which is that the default is never
41//! dropped. A value that matches no case went to the default before this ran and goes to the same
42//! place afterwards, because the edge it goes down is the same edge.
43//!
44//! # What has to be true
45//!
46//! For arithmetic, the labels are consecutive. A hole in the labels is a stretch the lowering
47//! would then have to cut the run at, and one comparison becomes several for a function that was
48//! only fitted to the labels either side of it.
49//!
50//! For a table, the labels may have holes. Where the default hands on what the arms hand on with a
51//! constant in the answer's place, which is `default: return 0;` and `default: y = 0; break;`, a
52//! hole's cell is that constant and the hole is given a case of its own going to the load, as gcc's
53//! `gather_default_values` does. The labels are then one run, which the lowering checks with one
54//! comparison, where a run with holes in it is a comparison and a bit test, and the bit test is a
55//! branch a stream of values mispredicts. Where the default does anything else, the `switch` still
56//! sends a value in a hole to the default, the hole's cell is never read, and it is written as zero
57//! only because an array has to have something there. What bounds the holes is size: the table spans at most eight cells for every label it replaces, which is gcc's
58//! `switch-conversion-max-branch-ratio`, so a `switch` over three labels a thousand apart stays a
59//! `switch`.
60//!
61//! Every arm is a block nothing else reaches, holding nothing but the constants it hands on, and
62//! ending the same way as every other arm. The same way means a jump to the same block, or a
63//! return, and in either case with the same values in every position but one. That one is the
64//! answer. Section 24.5 gives up on arms that assign more than one thing and so does this.
65//!
66//! A case that goes where the default goes is not an arm. That is a `case 15:` written just above
67//! `default:`, which the front end gives a block of its own that only jumps to the default's, or a
68//! case edge straight to the default block. It leaves the `switch` before the arms are looked at, so
69//! its label goes to the default down the default edge, and for a table it is a hole like any
70//! other. Until tamnd/rucc#1932 one case like that made the arms differ and kept the whole `switch`.
71//!
72//! Labels written one above another, `case 10: case 11: case 12: return 1;`, reach their arm
73//! through blocks that only jump. The front end gives every label a block, and the blocks for `10`
74//! and `11` hold nothing but a jump to the next one, so the arm under `12` is reached from two of
75//! them as well as from the `switch`. That is not an arm something else reaches. A case edge is
76//! followed through such blocks to where it arrives, and the blocks it went through go with the
77//! arms as long as nothing but the `switch` and each other reaches them. `simplify-cfg` would take
78//! them out, but it runs after this does. Until tamnd/rucc#728 a run of labels like that kept the
79//! whole `switch`, which is gcc's `CSWTCH` table for a character classifier.
80//!
81//! The answers are `a * label + b` at every label, checked at every label rather than fitted to two
82//! of them and believed. The check is done in the answer's own width with wrapping, because that is
83//! what the arithmetic this writes will do, and the arithmetic is written with no flags on it so
84//! that wrapping is what it is allowed to do.
85//!
86//! For arithmetic, the label and the answer are the same width. A `switch` on an `int` whose arms
87//! give a `long` is the same transformation with a widening in front of the multiply, and which
88//! widening it is depends on how the label is read, which is a question this would have to answer
89//! and currently declines to ask. A table does not have that question, because the label only
90//! picks a cell and the cell is already as wide as the answer, so a table may be any whole number
91//! of bytes wide up to eight whatever the label is. A label wider than a word gets no table, since
92//! the index into one is a word.
93//!
94//! # Why three labels and not two
95//!
96//! Two labels and a default is a shape `phiopt` already has something to say about, and what it
97//! says is a `select` between two constants that cost nothing to materialize. The arithmetic this
98//! writes is a multiply and an add against a range check, which is not obviously better than that
99//! and is worse when `a` is not one. From three labels up the chain being replaced is at least six
100//! instructions and what replaces it is at most five, so it is a win at three and grows from there.
101//!
102//! A table is held to the same three. What replaces the chain is a subtraction, the range check,
103//! a widening and a load, which is five again, and the load is from a line the program keeps
104//! reading if the `switch` is hot.
105//!
106//! # Where the table goes
107//!
108//! The array is internal, constant and aligned to its cell, which puts it in `.rodata`. It is
109//! named `CSWTCH.` and a number, as gcc names it, which nothing written in C can spell.
110//!
111//! When the goal is size a cell is as narrow as the answers allow, and the load is widened back to
112//! the answer's width with a sign or without one, whichever holds every answer. gcc 16 does the
113//! same at `-Os` and not at `-O2`, where the widening is an instruction on the path and the bytes
114//! it saves are data rather than code. Three `int` answers under ten are twelve bytes at `-O2` and
115//! three at `-Os`, in gcc and here.
116//!
117//! # A table of where the answers are
118//!
119//! An arm may give an address rather than a number, which is `case 0: return "zero";`. gcc 16
120//! makes those a `CSWTCH` array of pointers only under `-fno-pic`. In a position independent
121//! executable it keeps the compares, because every cell would be an address the loader has to
122//! write at startup, and the array would have to be in `.data.rel.ro` to be written at all. rucc
123//! only builds position independent code, so it does what clang does instead: a cell is four
124//! bytes holding how far the answer is from the table, which the linker works out once and the
125//! loader never touches. The answer is the table's address plus the cell, so the array stays in
126//! `.rodata` and what replaces the chain is one load and one addition.
127//!
128//! The distance has to be a number once the program is linked, so every answer has to be read
129//! only data this file defines and no other object can replace, which is what
130//! `crate::image::Images` holds, or a constant number of bytes into it, which is `&table[2]` and
131//! is the same distance with the bytes added. And it needs a four byte relocation measured from where it is
132//! written, which x86-64 ELF has and `crate::ReadOnly` says so. Anywhere else the `switch` stays
133//! a `switch`.
134//!
135//! The sum is made as an integer and turned back into a pointer, rather than added to the table's
136//! address, because the answer is not in the table and `crate::alias::origin` would say it was.
137
138use std::cmp::Ordering;
139use std::collections::HashSet;
140
141use rucc_base::Symbol;
142use rucc_ir::{
143    Block, BlockCall, Builder, Def, Extra, Flags, Func, Imm, Inst, InstData, MemInfo, MemOrder,
144    Opcode, Restrict, Type, Value,
145};
146
147use rucc_cost::Goal;
148use rucc_cost::heuristics::SWITCH_CONVERSION_MAX_GROWTH;
149
150use crate::cfg::Cfg;
151use crate::{Analyses, Fuel, Pass, Preserved, ReadOnly, Stats};
152
153/// What is reported when a `switch` becomes arithmetic.
154const CONVERTED: &str = "switch replaced by a range check and the arithmetic its arms were doing";
155
156/// What is reported when a `switch` becomes a load from a table of what its arms gave.
157const TABLED: &str = "switch replaced by a range check and a load from a table of its answers";
158
159/// What is reported when a `switch` becomes a load from a table of how far its answers are.
160const PLACED: &str =
161    "switch replaced by a range check and a load from a table of how far away its answers are";
162
163/// What is reported when the pass ran out of fuel with a `switch` it was about to convert.
164const NO_FUEL: &str = "switch left alone, the pass ran out of fuel";
165
166/// What is reported for a `switch` with too few labels to pay for the arithmetic.
167const TOO_FEW: &str = "switch left alone, it has too few labels for arithmetic to be cheaper";
168
169/// What is reported for a `switch` whose labels have holes in them.
170const NOT_CONSECUTIVE: &str = "switch left alone, its labels are not consecutive";
171
172/// What is reported for a `switch` with an arm that is not a block of its own.
173const ARM_IS_SHARED: &str = "switch left alone, an arm is reached from somewhere other than it";
174
175/// What is reported for a `switch` with an arm that does something.
176const ARM_DOES_WORK: &str = "switch left alone, an arm does more than work out a constant";
177
178/// What is reported for a `switch` whose arms do not end alike.
179const ARMS_DIFFER: &str = "switch left alone, its arms do not all hand on the same thing";
180
181/// What is reported for a `switch` whose answers are not a line.
182const NOT_AFFINE: &str = "switch left alone, its answers are not a fixed multiple of the label \
183                          plus a constant";
184
185/// What is reported for a `switch` whose answers are a different width from its labels.
186const WIDTHS_DIFFER: &str = "switch left alone, its answers are not as wide as its labels";
187
188/// What is reported for a `switch` whose labels are too far apart for a table of them.
189const TOO_SPARSE: &str = "switch left alone, a table of its answers would be mostly holes";
190
191/// What is reported for a `switch` whose label is wider than an index into a table.
192const LABEL_TOO_WIDE: &str = "switch left alone, its label is wider than a word";
193
194/// What is reported for a `switch` whose answers are not something a table cell holds.
195const CELL_IS_ODD: &str =
196    "switch left alone, its answers are not a whole number of bytes of integer";
197
198/// What is reported for a `switch` whose answers are addresses a table cannot say where are.
199const PLACE_IS_ODD: &str =
200    "switch left alone, its answers are not all addresses of read only data this file defines";
201
202/// The fewest labels worth converting, per the module documentation.
203const LABELS: usize = 3;
204
205/// How many cells a table may have for every label it stands for, per the module documentation.
206const GROWTH: i128 = SWITCH_CONVERSION_MAX_GROWTH as i128;
207
208/// The pass.
209#[derive(Debug)]
210pub struct SwitchConv;
211
212impl Pass for SwitchConv {
213    fn name(&self) -> &'static str {
214        "switch-conv"
215    }
216
217    fn describe(&self) -> &'static str {
218        "a switch whose arms give constants becomes a range check and arithmetic or a table load"
219    }
220
221    fn preserves(&self) -> Preserved {
222        // A block appears, the arms go, and every case edge moves.
223        Preserved::NONE
224    }
225
226    fn run(&self, func: &mut Func, an: &mut Analyses, fuel: &mut Fuel) -> Stats {
227        convert(func, an, fuel, None)
228    }
229
230    fn run_emitting(
231        &self,
232        func: &mut Func,
233        an: &mut Analyses,
234        fuel: &mut Fuel,
235        data: &mut ReadOnly<'_>,
236    ) -> Stats {
237        convert(func, an, fuel, Some(data))
238    }
239}
240
241/// The pass, with somewhere to put a table or without one.
242///
243/// Without one is what a caller that is not the pipeline gets, and it is arithmetic or nothing.
244fn convert(
245    func: &mut Func,
246    an: &mut Analyses,
247    fuel: &mut Fuel,
248    mut data: Option<&mut ReadOnly<'_>>,
249) -> Stats {
250    let mut stats = Stats::new();
251    if func.entry().is_none() {
252        return stats;
253    }
254    let cfg = an.cfg(func);
255    let found: Vec<Inst> = func
256        .blocks()
257        .filter_map(|block| func.terminator(block))
258        .filter(|&inst| func[inst].opcode == Opcode::Switch)
259        .collect();
260
261    let index_bits = data.as_ref().map(|data| data.pointer_bits());
262    let near = near(func, an, data.as_deref());
263    let small = an.machine().goal() == Goal::Size;
264    let mut plans = Vec::new();
265    for inst in found {
266        match plan(func, cfg, inst, index_bits, small, &near) {
267            Ok(plan) => plans.push(plan),
268            Err(why) => stats.missed(why),
269        }
270    }
271
272    let mut changed = false;
273    for plan in plans {
274        if !fuel.take() {
275            stats.missed(NO_FUEL);
276            continue;
277        }
278        let table = match (&plan.how, data.as_deref_mut()) {
279            (How::Table { cell, cells, .. }, Some(data)) => {
280                Some(data.table(cell.ty, cells.clone()))
281            }
282            (How::Distances { to, .. }, Some(data)) => Some(data.distances(to)),
283            _ => None,
284        };
285        stats.optimized(match (&plan.how, table) {
286            (_, None) => CONVERTED,
287            (How::Distances { .. }, Some(_)) => PLACED,
288            (_, Some(_)) => TABLED,
289        });
290        apply(func, &plan, table);
291        changed = true;
292    }
293    if changed {
294        an.clear();
295    }
296    stats
297}
298
299/// The names an answer may be the address of, when a table may hold how far away they are.
300///
301/// Empty when it may not, which is on a target with no four byte distance or for a caller with
302/// nowhere to put a table. Only the names this function takes the address of are asked about,
303/// since no other name can be an answer.
304fn near(func: &Func, an: &Analyses, data: Option<&ReadOnly<'_>>) -> HashSet<Symbol> {
305    if !data.is_some_and(ReadOnly::measures) {
306        return HashSet::new();
307    }
308    let images = an.images();
309    func.blocks()
310        .flat_map(|block| func.insts(block))
311        .filter_map(|inst| match func[inst] {
312            InstData { opcode: Opcode::GlobalAddr, extra: Extra::Symbol(name), .. } => Some(name),
313            _ => None,
314        })
315        .filter(|&name| images.holds(name))
316        .collect()
317}
318
319/// How the arms of one `switch` hand their answer on.
320#[derive(Clone, Copy, Debug, PartialEq, Eq)]
321enum Hands {
322    /// To this block, as one of its parameters.
323    On(Block),
324    /// Out of the function, as one of its results.
325    Back,
326}
327
328/// One `switch` and what it is about to become.
329#[derive(Debug)]
330struct Plan {
331    /// The `switch` itself.
332    inst: Inst,
333    /// What it switches on, which is what the answer is a function of.
334    value: Value,
335    /// The width of the label.
336    ty: Type,
337    /// Where the answer goes.
338    hands: Hands,
339    /// What every arm handed on, with the answer's position holding whatever the first arm had
340    /// there. That position is rewritten and the rest are passed on as they were.
341    args: Vec<Value>,
342    /// Which of `args` is the answer.
343    answer: usize,
344    /// How the answer is worked out from the label.
345    how: How,
346    /// The blocks the arms were, which nothing reaches once the case edges have moved.
347    arms: Vec<Block>,
348    /// Values between two labels that get a case of their own going to the load, because the
349    /// default gives what their cell holds.
350    holes: Vec<i128>,
351    /// Where in the `switch`'s list of cases the ones that went where the default goes are. They
352    /// leave the `switch`.
353    defaulted: Vec<usize>,
354}
355
356/// How the answer is worked out from the label.
357#[derive(Debug)]
358enum How {
359    /// As `scale * label + offset`, at the label's width.
360    Line {
361        /// The multiple of the label.
362        scale: i128,
363        /// What is added to it.
364        offset: i128,
365    },
366    /// As cell `label - low` of a table.
367    Table {
368        /// The lowest label, which is cell zero.
369        low: i128,
370        /// The width of the answer.
371        ty: Type,
372        /// What a cell is, which is the answer unless the goal is size.
373        cell: Cell,
374        /// Every cell, with zero in the holes.
375        cells: Vec<i128>,
376        /// The width of an index into the table, which is a word on the target.
377        index_bits: u32,
378    },
379    /// As the table's address plus cell `label - low`, which is how far the answer is from it.
380    Distances {
381        /// The lowest label, which is cell zero.
382        low: i128,
383        /// What each cell is the distance to, as a name and how many bytes into it, with `None` in
384        /// a hole nothing reads.
385        to: Vec<Option<(Symbol, i128)>>,
386        /// The width of an index into the table, which is a word on the target.
387        index_bits: u32,
388    },
389}
390
391/// How a cell of a table is held, and how it is made an answer again.
392#[derive(Clone, Copy, Debug, PartialEq, Eq)]
393struct Cell {
394    /// The width of a cell.
395    ty: Type,
396    /// Whether a cell narrower than the answer is widened with its sign.
397    signed: bool,
398}
399
400/// What one `switch` becomes, or why it stays as it is.
401///
402/// `index_bits` is the width of an address when a table may be made and `None` when it may not,
403/// `small` is whether the goal is size, which is what narrows a cell, and `near` is the names a
404/// table may hold the distance to.
405fn plan(
406    func: &Func,
407    cfg: &Cfg,
408    inst: Inst,
409    index_bits: Option<u32>,
410    small: bool,
411    near: &HashSet<Symbol>,
412) -> Result<Plan, &'static str> {
413    let Extra::Switch(info) = func[inst].extra else { return Err(ARMS_DIFFER) };
414    let info = func[info];
415    let Some(&value) = func[func[inst].args].first() else { return Err(ARMS_DIFFER) };
416    let ty = func[value].ty;
417    if !ty.is_int() {
418        return Err(WIDTHS_DIFFER);
419    }
420    let calls: Vec<BlockCall> = func[info.targets].to_vec();
421    let labels: Vec<i128> = func[info.cases].iter().map(|imm| imm.signed(ty)).collect();
422    let Some((&default, arms)) = calls.split_first() else { return Err(ARMS_DIFFER) };
423    if arms.len() != labels.len() || arms.len() < LABELS {
424        return Err(TOO_FEW);
425    }
426    // A case edge is followed through the blocks that only jump on to where it arrives, which is
427    // its arm or the default. A case that goes where the default goes is the default's. The blocks
428    // it went through that nothing but the `switch` and each other reach go with the arms, and a
429    // case whose arm is behind one that something else reaches has a shared arm.
430    let switch = func.block_of(inst).ok_or(ARMS_DIFFER)?;
431    let mut walks = Vec::with_capacity(arms.len());
432    for &call in arms {
433        walks.push(walk(func, call, default).ok_or(ARM_IS_SHARED)?);
434    }
435    let passed =
436        settle(cfg, switch, walks.iter().flat_map(|(_, through)| through.clone()).collect());
437    let mut defaulted = Vec::new();
438    let mut gone = Vec::new();
439    let mut kept = Vec::with_capacity(arms.len());
440    let mut kept_labels = Vec::with_capacity(labels.len());
441    for (at, ((end, through), &label)) in walks.iter().zip(&labels).enumerate() {
442        gone.extend(through.iter().copied().filter(|block| passed.contains(block)));
443        if there(func, *end, default) {
444            defaulted.push(at);
445        } else if through.iter().all(|block| passed.contains(block)) {
446            kept.push(*end);
447            kept_labels.push(label);
448        } else {
449            return Err(ARM_IS_SHARED);
450        }
451    }
452    let (arms, labels) = (&kept[..], kept_labels);
453    if arms.len() < LABELS {
454        return Err(TOO_FEW);
455    }
456    // A block that is both an arm and the default is not an arm this may take away, and it looks
457    // like one from here: the predecessor count below says one, because one block reaching another
458    // down two edges is one predecessor, and the arm being removed would take the default with it.
459    if arms.iter().any(|call| call.block == default.block) {
460        return Err(ARM_IS_SHARED);
461    }
462
463    // Consecutive and ascending. The front end sorts nothing, so this is asked of the list as it
464    // arrived rather than of a sorted copy: what is wanted is that the labels are a run, and a run
465    // read out of order is still a run only if it is sorted first, which is work this declines to
466    // do before it knows the answers are a line. Only a line asks, since a table indexes by the
467    // label whatever order the labels came in.
468    let consecutive = labels.windows(2).all(|pair| pair[1].checked_sub(pair[0]) == Some(1));
469    if !consecutive && index_bits.is_none() {
470        return Err(NOT_CONSECUTIVE);
471    }
472
473    // Every arm is a block of its own that works out constants and hands them on, and the way it
474    // hands them on is the way every other arm does.
475    let mut hands = None;
476    let mut shared: Option<Vec<Value>> = None;
477    let mut answer = None;
478    let mut handed = Vec::new();
479    for call in arms {
480        if !call.args.is_empty() {
481            return Err(ARM_DOES_WORK);
482        }
483        let outside = |from: &Block| *from != switch && !passed.contains(from);
484        if cfg.predecessors(call.block).iter().any(outside) {
485            return Err(ARM_IS_SHARED);
486        }
487        // And a block an image holds the address of is shared whatever the graph says, because what
488        // arrives there is a `goto *p` that can be in another function.
489        if func.block_name(call.block).is_some() {
490            return Err(ARM_IS_SHARED);
491        }
492        let (way, args) = tail(func, call.block)?;
493        if *hands.get_or_insert(way) != way {
494            return Err(ARMS_DIFFER);
495        }
496        let previous = shared.get_or_insert_with(|| args.clone());
497        if previous.len() != args.len() {
498            return Err(ARMS_DIFFER);
499        }
500        // The one position they disagree about is the answer, and it is the same position every
501        // time. The first arm sets nothing, since it agrees with itself everywhere.
502        for (index, (&mine, &theirs)) in previous.iter().zip(&args).enumerate() {
503            if mine == theirs {
504                continue;
505            }
506            if *answer.get_or_insert(index) != index {
507                return Err(ARMS_DIFFER);
508            }
509        }
510        handed.push(args);
511    }
512    let (Some(hands), Some(args)) = (hands, shared) else { return Err(ARMS_DIFFER) };
513    let answer = answer.ok_or(NOT_AFFINE)?;
514    let kind = func[args[answer]].ty;
515    // Read once the position is known and not while it was being found. Until the second arm
516    // disagrees with the first nobody knows which position the answer is in, and reading the first
517    // arm's answer from a guess of the first position would take whatever was there, which is a
518    // constant every arm passed if the arms pass one, and a line fitted through that is wrong at
519    // the first label.
520    let (how, holes) = if kind.is_ptr() {
521        let index_bits = index_bits.ok_or(PLACE_IS_ODD)?;
522        let mut places = Vec::with_capacity(handed.len());
523        for args in &handed {
524            places.push(place(func, args[answer], near).ok_or(PLACE_IS_ODD)?);
525        }
526        let fill = fallback(func, default, hands, &args, answer)
527            .and_then(|given| place(func, given, near));
528        distances(&labels, &places, ty, index_bits, fill)?
529    } else {
530        let mut answers = Vec::with_capacity(handed.len());
531        for args in &handed {
532            let Some(number) = constant(func, args[answer]) else { return Err(NOT_AFFINE) };
533            answers.push(number);
534        }
535        let line = if consecutive && kind == ty { line(&labels, &answers, ty) } else { None };
536        match (line, index_bits) {
537            (Some((scale, offset)), _) => (How::Line { scale, offset }, Vec::new()),
538            (None, Some(index_bits)) => {
539                let fill = fallback(func, default, hands, &args, answer)
540                    .and_then(|given| constant(func, given));
541                let shape = Shape { ty, kind, index_bits, small };
542                table(&labels, &answers, shape, fill)?
543            }
544            (None, None) if kind != ty => return Err(WIDTHS_DIFFER),
545            (None, None) => return Err(NOT_AFFINE),
546        }
547    };
548    Ok(Plan {
549        inst,
550        value,
551        ty,
552        hands,
553        args,
554        answer,
555        how,
556        arms: arms.iter().map(|call| call.block).chain(gone).collect(),
557        holes,
558        defaulted,
559    })
560}
561
562/// Whether a case edge goes where the default goes, which is its block with its values.
563fn there(func: &Func, call: BlockCall, default: BlockCall) -> bool {
564    call.block == default.block && func[call.args] == func[default.args]
565}
566
567/// Where a case edge arrives once it has gone through every block that only jumps on, and the
568/// blocks it went through, in order.
569///
570/// A block is gone through when it holds nothing but a jump, is handed nothing, has no name an
571/// image could take the address of, and is not the default's. `None` is a case that goes round a
572/// ring of such blocks forever and never arrives anywhere.
573fn walk(func: &Func, call: BlockCall, default: BlockCall) -> Option<(BlockCall, Vec<Block>)> {
574    let mut at = call;
575    let mut through = Vec::new();
576    loop {
577        if at.block == default.block || !at.args.is_empty() || func.block_name(at.block).is_some() {
578            return Some((at, through));
579        }
580        let Some(last) = func.terminator(at.block) else { return Some((at, through)) };
581        if func.insts(at.block).count() != 1 || func[last].opcode != Opcode::Jump {
582            return Some((at, through));
583        }
584        let Some(next) = func.successors(last).next() else { return Some((at, through)) };
585        if through.contains(&at.block) {
586            return None;
587        }
588        through.push(at.block);
589        at = next;
590    }
591}
592
593/// Which of the blocks gone through are reached from nowhere but the `switch` and each other, which
594/// are the ones nothing reaches once the case edges stop going through them.
595///
596/// A block something else reaches stays, and then so does every block below it, since the block
597/// that stays still jumps there. So this is a fixed point rather than one look at each block.
598fn settle(cfg: &Cfg, switch: Block, mut through: HashSet<Block>) -> HashSet<Block> {
599    loop {
600        let outside = |from: &Block| *from != switch && !through.contains(from);
601        let stays: Vec<Block> = through
602            .iter()
603            .copied()
604            .filter(|&block| cfg.predecessors(block).iter().any(outside))
605            .collect();
606        if stays.is_empty() {
607            return through;
608        }
609        for block in stays {
610            through.remove(&block);
611        }
612    }
613}
614
615/// What the default gives in the answer's place, when that is all it does differently from an arm.
616///
617/// Two shapes of default qualify. One is a block of its own that works out constants and hands
618/// them on the way the arms do, which is `default: return 0;`. The other is an edge straight to
619/// where the arms hand their answer, carrying the answer itself, which is what is left of
620/// `default: y = 0; break;` once the empty block is gone. Either way every position but the answer
621/// has to be what the arms pass, since a hole given a case is about to pass that instead. The
622/// default block is only read here and never taken away, so it may be shared.
623///
624/// What comes back is the value in the answer's place, and whether it is a number or an address a
625/// table can hold is for the caller to ask.
626fn fallback(
627    func: &Func,
628    default: BlockCall,
629    hands: Hands,
630    args: &[Value],
631    answer: usize,
632) -> Option<Value> {
633    let theirs = if default.args.is_empty() {
634        let (way, theirs) = tail(func, default.block).ok()?;
635        if way != hands {
636            return None;
637        }
638        theirs
639    } else if hands == Hands::On(default.block) {
640        func[default.args].to_vec()
641    } else {
642        return None;
643    };
644    if theirs.len() != args.len() {
645        return None;
646    }
647    let agrees =
648        args.iter().zip(&theirs).enumerate().all(|(at, (mine, it))| at == answer || mine == it);
649    if !agrees {
650        return None;
651    }
652    Some(theirs[answer])
653}
654
655/// What a table is made for: the label's width, the answer's, an index's, and whether the goal is
656/// size.
657#[derive(Clone, Copy, Debug)]
658struct Shape {
659    /// The width of the label.
660    ty: Type,
661    /// The width of the answer.
662    kind: Type,
663    /// The width of an index into the table.
664    index_bits: u32,
665    /// Whether a cell may be narrower than the answer.
666    small: bool,
667}
668
669/// The table the answers make when one is worth making, and the holes that get a case of their own.
670///
671/// A hole gets one only when `fill` is what the default gives, and then its cell is that. The
672/// labels are read with their own sign and so are ordered that way, which is only a question
673/// of which one is cell zero. What the index is at run time is the label less the lowest one at the
674/// label's width, and for a label that is a case that difference is the distance between the two
675/// however the bits are read, because the table is short and the distance fits.
676fn table(
677    labels: &[i128],
678    answers: &[i128],
679    shape: Shape,
680    fill: Option<i128>,
681) -> Result<(How, Vec<i128>), &'static str> {
682    let Shape { ty, kind, index_bits, small } = shape;
683    if ty.bits() > 64 {
684        return Err(LABEL_TOO_WIDE);
685    }
686    if !kind.is_int() || !matches!(kind.bits(), 8 | 16 | 32 | 64) {
687        return Err(CELL_IS_ODD);
688    }
689    let (low, cells) = spread(labels, answers)?;
690    let holes = if fill.is_some() { holes(low, &cells) } else { Vec::new() };
691    let cells: Vec<i128> = cells.into_iter().map(|cell| cell.or(fill).unwrap_or(0)).collect();
692    let cell = if small { narrowest(&cells, kind) } else { Cell { ty: kind, signed: false } };
693    Ok((How::Table { low, ty: kind, cell, cells, index_bits }, holes))
694}
695
696/// The table of distances the answers make when one is worth making, and the holes that get a case
697/// of their own.
698///
699/// The same as [`table`] but for what a cell is, which is four bytes whatever the label is. It is
700/// how far the answer is from the table, and nothing in one program is four gigabytes from
701/// anything else in it. A hole with nothing to fill it is `None`, a cell nothing reads.
702fn distances(
703    labels: &[i128],
704    places: &[(Symbol, i128)],
705    ty: Type,
706    index_bits: u32,
707    fill: Option<(Symbol, i128)>,
708) -> Result<(How, Vec<i128>), &'static str> {
709    if ty.bits() > 64 {
710        return Err(LABEL_TOO_WIDE);
711    }
712    let (low, to) = spread(labels, places)?;
713    let holes = if fill.is_some() { holes(low, &to) } else { Vec::new() };
714    let to = to.into_iter().map(|cell| cell.or(fill)).collect();
715    Ok((How::Distances { low, to, index_bits }, holes))
716}
717
718/// The answers laid out by label from the lowest one, with `None` in the holes, and the lowest
719/// label.
720///
721/// Refused when the labels are too far apart for the table to be worth its size, which is the
722/// growth limit in the module documentation.
723fn spread<T: Copy>(labels: &[i128], answers: &[T]) -> Result<(i128, Vec<Option<T>>), &'static str> {
724    let (Some(&low), Some(&high)) = (labels.iter().min(), labels.iter().max()) else {
725        return Err(TOO_FEW);
726    };
727    let span = high - low + 1;
728    if span > GROWTH * labels.len() as i128 {
729        return Err(TOO_SPARSE);
730    }
731    let mut cells = vec![None; usize::try_from(span).map_err(|_| TOO_SPARSE)?];
732    for (&label, &answer) in labels.iter().zip(answers) {
733        let at = usize::try_from(label - low).map_err(|_| TOO_SPARSE)?;
734        cells[at] = Some(answer);
735    }
736    Ok((low, cells))
737}
738
739/// The labels between the lowest and the highest that are not cases, from a spread of the answers.
740fn holes<T>(low: i128, cells: &[Option<T>]) -> Vec<i128> {
741    (low..).zip(cells).filter(|(_, cell)| cell.is_none()).map(|(label, _)| label).collect()
742}
743
744/// The narrowest cell every answer fits in, read back to the answer's width.
745///
746/// With a sign first, because an answer below zero only fits that way, and then without one, which
747/// is what fits a `200` in a byte. An answer that fits neither way at a width is an answer that
748/// needs the next one, and the answer's own width always fits.
749fn narrowest(answers: &[i128], kind: Type) -> Cell {
750    let whole = 1i128 << kind.bits();
751    for bits in [8u32, 16, 32] {
752        if bits >= kind.bits() {
753            break;
754        }
755        let half = 1i128 << (bits - 1);
756        if answers.iter().all(|&answer| (-half..half).contains(&answer)) {
757            return Cell { ty: Type::int(bits), signed: true };
758        }
759        if answers.iter().all(|&answer| answer.rem_euclid(whole) < half * 2) {
760            return Cell { ty: Type::int(bits), signed: false };
761        }
762    }
763    Cell { ty: kind, signed: false }
764}
765
766/// What a block hands on, when handing something on is the whole of what it does.
767///
768/// Every instruction in it but the last has to be a constant or an address, because the last one
769/// is about to be written somewhere else and anything the block worked out for it would be left
770/// behind. A constant is the exception because a constant is rewritten rather than moved, and an
771/// address is too, since what a table holds is how far away it is. Which addresses a table can
772/// hold is for [`place`] to say, and an arm that hands on one it cannot is refused there.
773fn tail(func: &Func, block: Block) -> Result<(Hands, Vec<Value>), &'static str> {
774    let Some(last) = func.terminator(block) else { return Err(ARM_DOES_WORK) };
775    for inst in func.insts(block) {
776        let opcode = func[inst].opcode;
777        if inst != last && !matches!(opcode, Opcode::IConst | Opcode::GlobalAddr | Opcode::PtrAdd) {
778            return Err(ARM_DOES_WORK);
779        }
780    }
781    let args: Vec<Value> = match func[last].opcode {
782        Opcode::Jump => {
783            let Some(call) = func.successors(last).next() else { return Err(ARM_DOES_WORK) };
784            let args = func[call.args].to_vec();
785            return Ok((Hands::On(call.block), args));
786        }
787        Opcode::Return => func[func[last].args].to_vec(),
788        _ => return Err(ARM_DOES_WORK),
789    };
790    Ok((Hands::Back, args))
791}
792
793/// The answer as `scale * label + offset`.
794fn arithmetic(builder: &mut Builder<'_>, plan: &Plan, scale: i128, offset: i128) -> Value {
795    let scaled = match scale {
796        0 => builder.iconst(plan.ty, offset),
797        1 => plan.value,
798        scale => {
799            let by = builder.iconst(plan.ty, scale);
800            builder.binary(Opcode::Mul, plan.value, by, Flags::NONE)
801        }
802    };
803    if offset == 0 || scale == 0 {
804        scaled
805    } else {
806        let by = builder.iconst(plan.ty, offset);
807        builder.binary(Opcode::Add, scaled, by, Flags::NONE)
808    }
809}
810
811/// The address of the table called `name`, and cell `label - low` of it.
812///
813/// The subtraction is at the label's width and wraps, and then the difference is made a word. Only
814/// a label that is a case gets here, so the difference is below the table's length and the
815/// widening is the same with or without a sign, and it is written without one.
816fn look_up(
817    builder: &mut Builder<'_>,
818    plan: &Plan,
819    name: Symbol,
820    low: i128,
821    ty: Type,
822    index_bits: u32,
823) -> (Value, Value) {
824    let from = if low == 0 {
825        plan.value
826    } else {
827        let by = builder.iconst(plan.ty, low);
828        builder.binary(Opcode::Sub, plan.value, by, Flags::NONE)
829    };
830    let word = Type::int(index_bits);
831    let index = match plan.ty.bits().cmp(&index_bits) {
832        Ordering::Less => builder.unary(Opcode::ZExt, from, word),
833        Ordering::Greater => builder.unary(Opcode::Trunc, from, word),
834        Ordering::Equal => from,
835    };
836    let bytes = ty.bits() / 8;
837    let distance = if bytes == 1 {
838        index
839    } else {
840        let by = builder.iconst(word, i128::from(bytes));
841        builder.binary(Opcode::Mul, index, by, Flags::NONE)
842    };
843    let base = builder.value(
844        InstData { extra: Extra::Symbol(name), ..InstData::new(Opcode::GlobalAddr) },
845        Type::PTR,
846    );
847    let cell = builder.binary(Opcode::PtrAdd, base, distance, Flags::NONE);
848    let info = MemInfo {
849        size: u64::from(bytes),
850        align: bytes,
851        order: MemOrder::NotAtomic,
852        tbaa: None,
853        owns: 0,
854        restrict: Restrict::NONE,
855    };
856    (base, builder.load(ty, cell, info, Flags::NONE))
857}
858
859/// The answer as the address of the table called `name` plus cell `label - low` of it.
860///
861/// The cell is four bytes with a sign, since an answer can be on either side of the table. The sum
862/// is worked out on integers and made a pointer at the end, so that where the answer came from is
863/// somewhere nobody can follow back, which is true, and not the table, which is not.
864fn far(builder: &mut Builder<'_>, plan: &Plan, name: Symbol, low: i128, index_bits: u32) -> Value {
865    let (base, read) = look_up(builder, plan, name, low, Type::int(32), index_bits);
866    let word = Type::int(index_bits);
867    let wider = index_bits > 32; // not a threshold: the cell is 32 bits
868    let away = if wider { builder.unary(Opcode::SExt, read, word) } else { read };
869    let start = builder.unary(Opcode::PtrToInt, base, word);
870    let at = builder.binary(Opcode::Add, start, away, Flags::NONE);
871    builder.unary(Opcode::IntToPtr, at, Type::PTR)
872}
873
874/// The name an answer is the address of and how many bytes into it, when it is one a table may
875/// hold the distance to.
876///
877/// The bytes are held to what four bytes can say, since they are added to the distance in the
878/// cell and a cell is four bytes.
879fn place(func: &Func, value: Value, near: &HashSet<Symbol>) -> Option<(Symbol, i128)> {
880    let Def::Result { inst, .. } = func[value].def else { return None };
881    let data = func[inst];
882    match (data.opcode, data.extra) {
883        (Opcode::GlobalAddr, Extra::Symbol(name)) if near.contains(&name) => Some((name, 0)),
884        (Opcode::PtrAdd, _) => {
885            let args = &func[data.args];
886            let (name, bytes) = place(func, args[0], near)?;
887            let bytes = bytes.checked_add(constant(func, args[1])?)?;
888            i32::try_from(bytes).is_ok().then_some((name, bytes))
889        }
890        _ => None,
891    }
892}
893
894/// The value of an integer constant, read with its own sign.
895fn constant(func: &Func, value: Value) -> Option<i128> {
896    crate::discharge::constant(func, value)
897}
898
899/// The multiple and the offset that give every answer from its label, when one pair does.
900///
901/// Fitted to the first two labels, which is exact because they are one apart, and then checked at
902/// every label including those two. Checked rather than trusted because the arithmetic that is
903/// about to be written wraps at the type's width, and a fit that is right about the numbers and
904/// wrong about the wrapping is a miscompile that only shows up at the ends of the range.
905fn line(labels: &[i128], answers: &[i128], ty: Type) -> Option<(i128, i128)> {
906    let [first, second, ..] = *labels else { return None };
907    let [low, high, ..] = *answers else { return None };
908    debug_assert_eq!(second - first, 1, "the labels were checked to be consecutive");
909    let scale = high.checked_sub(low)?;
910    let offset = low.checked_sub(scale.checked_mul(first)?)?;
911    for (&label, &answer) in labels.iter().zip(answers) {
912        let want = scale.checked_mul(label)?.checked_add(offset)?;
913        if wrap(want, ty) != answer {
914            return None;
915        }
916    }
917    Some((scale, offset))
918}
919
920/// A number as the machine will hold it at that width, read back with its own sign.
921///
922/// An immediate is stored in exactly the width its type has, so building one and reading it back is
923/// the truncation, and it is the same one every other part of the compiler uses.
924fn wrap(value: i128, ty: Type) -> i128 {
925    Imm::int(value, ty).signed(ty)
926}
927
928/// Writes the block the arms become and points every case edge at it.
929///
930/// `table` is the name of the table a plan for one was given, and `None` for a line.
931fn apply(func: &mut Func, plan: &Plan, table: Option<Symbol>) {
932    let span = func.span(plan.inst);
933    let hit = func.create_block();
934    let mut builder = Builder::new(func, hit).at(span);
935    let answer = match (&plan.how, table) {
936        (&How::Line { scale, offset }, _) => arithmetic(&mut builder, plan, scale, offset),
937        (&How::Table { low, ty, cell, index_bits, .. }, Some(name)) => {
938            let (_, read) = look_up(&mut builder, plan, name, low, cell.ty, index_bits);
939            match (cell.ty == ty, cell.signed) {
940                (true, _) => read,
941                (false, true) => builder.unary(Opcode::SExt, read, ty),
942                (false, false) => builder.unary(Opcode::ZExt, read, ty),
943            }
944        }
945        (&How::Distances { low, index_bits, .. }, Some(name)) => {
946            far(&mut builder, plan, name, low, index_bits)
947        }
948        (How::Table { .. } | How::Distances { .. }, None) => {
949            unreachable!("a table was planned with nowhere to put it")
950        }
951    };
952    let mut args = plan.args.clone();
953    args[plan.answer] = answer;
954    match plan.hands {
955        Hands::On(block) => builder.jump(block, &args),
956        Hands::Back => builder.ret(&args),
957    };
958
959    // Every case edge, and only the case edges: the default is the first target and stays where it
960    // was pointing. A case that went where the default goes leaves, and the default edge takes it.
961    let Extra::Switch(info) = func[plan.inst].extra else { return };
962    let empty = func.push_values(&[]);
963    let before: Vec<Imm> = func[func[info].cases].to_vec();
964    let mut calls: Vec<BlockCall> = func[func[info].targets][..1].to_vec();
965    let mut cases = Vec::with_capacity(before.len() + plan.holes.len());
966    for (at, &case) in before.iter().enumerate() {
967        if plan.defaulted.contains(&at) {
968            continue;
969        }
970        // No hint: the cases that had one had one each, and a single edge standing for all of them
971        // cannot carry a number that was true of one arm.
972        calls.push(BlockCall::new(hit, empty));
973        cases.push(case);
974    }
975    for &hole in &plan.holes {
976        calls.push(BlockCall::new(hit, empty));
977        cases.push(Imm::int(hole, plan.ty));
978    }
979    let targets = func.push_block_calls(&calls);
980    let cases = func.push_imms(&cases);
981    let info = func.add_switch(rucc_ir::SwitchInfo { targets, cases });
982    func[plan.inst].extra = Extra::Switch(info);
983
984    // The arms are unreachable now. Two labels sharing one arm is a shape that survives the checks
985    // above only when the answer does not depend on the label, so the same block can be here twice.
986    let mut gone = HashSet::new();
987    for &arm in &plan.arms {
988        if gone.insert(arm) {
989            func.remove_block(arm);
990        }
991    }
992}
993
994#[cfg(test)]
995mod tests {
996    use std::collections::{HashMap, HashSet};
997
998    use std::sync::Arc;
999
1000    use rucc_base::{Interner, Symbol};
1001    use rucc_cost::Goal;
1002    use rucc_ir::{
1003        Block, Builder, Datum, Extra, Flags, Func, Global, InstData, Linkage, Module, Opcode, Pic,
1004        Signature, Type, Value,
1005    };
1006    use rucc_target::{TargetInfo, Triple};
1007
1008    use super::{PLACE_IS_ODD, SwitchConv, TOO_FEW};
1009    use crate::image::Images;
1010    use crate::stats::Kind;
1011    use crate::{Fuel, Pass, ReadOnly, Stats, Table};
1012
1013    /// The width everything here switches on and answers in, unless a test says otherwise.
1014    fn i32() -> Type {
1015        Type::int(32)
1016    }
1017
1018    /// Runs the pass with as much fuel as it wants.
1019    fn convert(func: &mut Func) -> Stats {
1020        SwitchConv.run(func, &mut crate::machine::fixtures::analyses(), &mut Fuel::unlimited())
1021    }
1022
1023    /// Runs the pass the way the pipeline does, with a place for tables, and hands back the
1024    /// tables it asked for.
1025    fn tabled(func: &mut Func) -> (Stats, Vec<Table>) {
1026        tabled_for(func, Goal::Speed)
1027    }
1028
1029    /// The same for a goal, which is what decides how wide a cell is.
1030    fn tabled_for(func: &mut Func, goal: Goal) -> (Stats, Vec<Table>) {
1031        let mut names = Interner::new();
1032        let taken = HashSet::new();
1033        let mut data = ReadOnly::new(&mut names, &taken, 64, 0);
1034        let mut an = crate::Analyses::new(crate::Machine::with(None, goal));
1035        let stats = SwitchConv.run_emitting(func, &mut an, &mut Fuel::unlimited(), &mut data);
1036        (stats, data.into_tables())
1037    }
1038
1039    /// A function that switches on its parameter and returns a constant per label.
1040    ///
1041    /// The default returns a constant of its own that is not on any line these tests fit, so a
1042    /// test that says the pass fired is saying it fired on the cases and not on the whole thing.
1043    fn returning(ty: Type, labels: &[i128], answers: &[i128]) -> Func {
1044        let mut names = Interner::new();
1045        let mut func = Func::new(names.intern("f"), Signature::new());
1046        let head = func.create_block();
1047        let value = func.append_param(head, ty);
1048        let default = func.create_block();
1049        let arms: Vec<Block> = answers.iter().map(|_| func.create_block()).collect();
1050        for (&arm, &answer) in arms.iter().zip(answers) {
1051            let mut build = Builder::new(&mut func, arm);
1052            let it = build.iconst(ty, answer);
1053            build.ret(&[it]);
1054        }
1055        let mut build = Builder::new(&mut func, default);
1056        let it = build.iconst(ty, 999);
1057        build.ret(&[it]);
1058        let cases: Vec<(i128, Block)> = labels.iter().copied().zip(arms.iter().copied()).collect();
1059        Builder::new(&mut func, head).switch(value, default, &cases);
1060        func
1061    }
1062
1063    /// The same as [`returning`] at 32 bits, with more labels that go where the default goes.
1064    ///
1065    /// Each goes through a block of its own that only jumps to the default when `through` is set,
1066    /// which is what the front end makes of a `case` written just above `default:`, and straight to
1067    /// the default when it is not.
1068    fn falling(labels: &[i128], answers: &[i128], extra: &[i128], through: bool) -> Func {
1069        let ty = i32();
1070        let mut names = Interner::new();
1071        let mut func = Func::new(names.intern("f"), Signature::new());
1072        let head = func.create_block();
1073        let value = func.append_param(head, ty);
1074        let default = func.create_block();
1075        let arms: Vec<Block> = answers.iter().map(|_| func.create_block()).collect();
1076        for (&arm, &answer) in arms.iter().zip(answers) {
1077            let mut build = Builder::new(&mut func, arm);
1078            let it = build.iconst(ty, answer);
1079            build.ret(&[it]);
1080        }
1081        let mut build = Builder::new(&mut func, default);
1082        let it = build.iconst(ty, 999);
1083        build.ret(&[it]);
1084        let mut cases: Vec<(i128, Block)> =
1085            labels.iter().copied().zip(arms.iter().copied()).collect();
1086        for &label in extra {
1087            let to = if through {
1088                let block = func.create_block();
1089                Builder::new(&mut func, block).jump(default, &[]);
1090                block
1091            } else {
1092                default
1093            };
1094            cases.push((label, to));
1095        }
1096        Builder::new(&mut func, head).switch(value, default, &cases);
1097        func
1098    }
1099
1100    /// A function like [`returning`] at 32 bits where each answer is given by a run of labels
1101    /// written one above another, `case 10: case 11: return 1;`.
1102    ///
1103    /// The front end gives every label but the last in a run a block of its own that only jumps to
1104    /// the next label's, and the last label's block is the arm.
1105    fn runs(runs: &[(&[i128], i128)]) -> Func {
1106        let ty = i32();
1107        let mut names = Interner::new();
1108        let mut func = Func::new(names.intern("f"), Signature::new());
1109        let head = func.create_block();
1110        let value = func.append_param(head, ty);
1111        let default = func.create_block();
1112        let mut build = Builder::new(&mut func, default);
1113        let it = build.iconst(ty, 999);
1114        build.ret(&[it]);
1115        let mut cases = Vec::new();
1116        for &(labels, answer) in runs {
1117            let arm = func.create_block();
1118            let mut build = Builder::new(&mut func, arm);
1119            let it = build.iconst(ty, answer);
1120            build.ret(&[it]);
1121            let (&last, rest) = labels.split_last().expect("a run has a label");
1122            let mut next = arm;
1123            let mut run = vec![(last, arm)];
1124            for &label in rest.iter().rev() {
1125                let block = func.create_block();
1126                Builder::new(&mut func, block).jump(next, &[]);
1127                run.push((label, block));
1128                next = block;
1129            }
1130            cases.extend(run.into_iter().rev());
1131        }
1132        Builder::new(&mut func, head).switch(value, default, &cases);
1133        func
1134    }
1135
1136    /// The labels the `switch` still has, in the order it has them.
1137    fn labels(func: &Func) -> Vec<i128> {
1138        let head = func.entry().expect("a function with blocks in it");
1139        let term = func.terminator(head).expect("a head block has one");
1140        let Extra::Switch(info) = func[term].extra else { panic!("the head ends in no switch") };
1141        func[func[info].cases].iter().map(|imm| imm.signed(i32())).collect()
1142    }
1143
1144    /// The blocks every case edge goes to, which is one block when the pass has fired.
1145    fn cases(func: &Func) -> Vec<usize> {
1146        let head = func.entry().expect("a function with blocks in it");
1147        let term = func.terminator(head).expect("a head block has one");
1148        func.successors(term).skip(1).map(|call| call.block.index()).collect()
1149    }
1150
1151    /// The block every case edge goes to, when there is exactly one of them.
1152    fn arm(func: &Func) -> Block {
1153        let blocks = cases(func);
1154        let first = blocks[0];
1155        assert!(blocks.iter().all(|&block| block == first), "the case edges did not all move");
1156        Block::from_usize(first)
1157    }
1158
1159    /// The opcodes a block holds, in order.
1160    fn opcodes(func: &Func, block: Block) -> Vec<Opcode> {
1161        func.insts(block).map(|inst| func[inst].opcode).collect()
1162    }
1163
1164    /// What the block the case edges go to answers, given that label.
1165    ///
1166    /// An interpreter of exactly the three instructions this pass writes, because what the pass
1167    /// has to get right is the number and not the shape. Anything else in the block is a test
1168    /// that has drifted away from what it is testing, so it stops rather than guesses.
1169    fn answer(func: &Func, block: Block, label: i128) -> i128 {
1170        looked_up(func, block, label, &[])
1171    }
1172
1173    /// What the block the case edges go to answers, given that label and the tables it may read.
1174    ///
1175    /// The address of a table is its cell zero counted in bytes, which is all a load from one
1176    /// needs, and a load stops the test if it is not on a cell or not inside the table.
1177    fn looked_up(func: &Func, block: Block, label: i128, tables: &[Table]) -> i128 {
1178        let head = func.entry().expect("a function with blocks in it");
1179        let mut values: HashMap<Value, i128> = HashMap::new();
1180        values.insert(func[head].params[0], label);
1181        for inst in func.insts(block) {
1182            let data = func[inst];
1183            let Some(result) = data.first_result else {
1184                let args = func[data.args].to_vec();
1185                let handed = match data.opcode {
1186                    Opcode::Return => args[0],
1187                    Opcode::Jump => {
1188                        func[func.successors(inst).next().expect("a jump goes").args][0]
1189                    }
1190                    other => panic!("a block this pass wrote ends in {other:?}"),
1191                };
1192                return values[&handed];
1193            };
1194            let args: Vec<i128> = func[data.args].iter().map(|arg| values[arg]).collect();
1195            let it = match data.opcode {
1196                Opcode::IConst => {
1197                    let (imm, ty) = crate::fold::constant(func, result).expect("a constant is one");
1198                    imm.signed(ty)
1199                }
1200                Opcode::Mul => args[0].wrapping_mul(args[1]),
1201                Opcode::Add => args[0].wrapping_add(args[1]),
1202                Opcode::Sub => args[0].wrapping_sub(args[1]),
1203                // Unsigned, which is what the widening is, and then the width it widens to.
1204                Opcode::ZExt => {
1205                    let from = func[func[data.args][0]].ty;
1206                    super::wrap(args[0], from).rem_euclid(1 << from.bits())
1207                }
1208                Opcode::SExt | Opcode::PtrToInt | Opcode::IntToPtr => args[0],
1209                Opcode::GlobalAddr => 0,
1210                Opcode::PtrAdd => args[0] + args[1],
1211                // A cell that is how far a name is from the table is where the name is, since the
1212                // table is at zero.
1213                Opcode::Load => {
1214                    assert_eq!(tables.len(), 1, "a load with no single table to read");
1215                    let table = &tables[0];
1216                    let bytes = i128::from(table.ty.bits() / 8);
1217                    assert_eq!(args[0] % bytes, 0, "a load between two cells");
1218                    let at = usize::try_from(args[0] / bytes).expect("a load before the table");
1219                    let cell = *table.cells.get(at).expect("a load after the table");
1220                    cell + table.to.get(at).copied().flatten().map_or(0, spot)
1221                }
1222                other => panic!("this pass does not write {other:?}"),
1223            };
1224            // An address is a number of bytes into a table and has no width to wrap at.
1225            let ty = func[result].ty;
1226            values.insert(result, if ty.is_int() { super::wrap(it, ty) } else { it });
1227        }
1228        panic!("a block with no terminator");
1229    }
1230
1231    /// Where the tests say a name is, which is somewhere a distance from a table at zero shows.
1232    fn spot(name: Symbol) -> i128 {
1233        1000 * (i128::from(name.raw()) + 1)
1234    }
1235
1236    /// Whether the pass says it changed the function.
1237    fn fired(stats: &Stats) -> bool {
1238        stats.total(Kind::Optimized) > 0
1239    }
1240
1241    #[test]
1242    fn labels_that_run_with_their_answers_become_one_addition() {
1243        let mut func = returning(i32(), &[0, 1, 2, 3], &[1, 2, 3, 4]);
1244        assert!(fired(&convert(&mut func)));
1245        let arm = arm(&func);
1246        assert_eq!(opcodes(&func, arm), [Opcode::IConst, Opcode::Add, Opcode::Return]);
1247        for label in 0..4 {
1248            assert_eq!(answer(&func, arm, label), label + 1);
1249        }
1250    }
1251
1252    #[test]
1253    fn answers_that_are_a_multiple_of_the_label_become_a_multiplication() {
1254        let mut func = returning(i32(), &[3, 4, 5, 6], &[30, 40, 50, 60]);
1255        assert!(fired(&convert(&mut func)));
1256        let arm = arm(&func);
1257        assert_eq!(opcodes(&func, arm), [Opcode::IConst, Opcode::Mul, Opcode::Return]);
1258        for label in 3..7 {
1259            assert_eq!(answer(&func, arm, label), label * 10);
1260        }
1261    }
1262
1263    #[test]
1264    fn answers_that_are_all_the_same_become_the_constant_they_all_were() {
1265        let mut func = returning(i32(), &[7, 8, 9, 10], &[9, 9, 9, 9]);
1266        assert!(fired(&convert(&mut func)));
1267        let arm = arm(&func);
1268        assert_eq!(opcodes(&func, arm), [Opcode::IConst, Opcode::Return]);
1269        assert_eq!(answer(&func, arm, 8), 9);
1270    }
1271
1272    #[test]
1273    fn labels_that_run_below_zero_are_a_run_like_any_other() {
1274        let mut func = returning(i32(), &[-2, -1, 0, 1], &[-4, -2, 0, 2]);
1275        assert!(fired(&convert(&mut func)));
1276        let arm = arm(&func);
1277        for label in -2..2 {
1278            assert_eq!(answer(&func, arm, label), label * 2);
1279        }
1280    }
1281
1282    /// The line has to hold at the type's width and not at the arithmetic's.
1283    ///
1284    /// A hundred times two is two hundred, which is not a number an `i8` holds, and the answer the
1285    /// program gave at that label is what two hundred comes to there. The pass writes a
1286    /// multiplication with no flags on it, which wraps the same way, so this is a fit and not a
1287    /// refusal, and the number is the point of the test.
1288    #[test]
1289    fn a_line_that_only_holds_by_wrapping_still_holds() {
1290        let ty = Type::int(8);
1291        let mut func = returning(ty, &[0, 1, 2], &[0, 100, -56]);
1292        assert!(fired(&convert(&mut func)));
1293        let arm = arm(&func);
1294        assert_eq!(answer(&func, arm, 2), -56);
1295    }
1296
1297    #[test]
1298    fn labels_with_a_hole_in_them_are_left_alone_where_no_table_can_be_made() {
1299        let mut func = returning(i32(), &[0, 1, 3], &[1, 2, 4]);
1300        assert!(!fired(&convert(&mut func)));
1301        assert_eq!(cases(&func).len(), 3);
1302    }
1303
1304    #[test]
1305    fn answers_that_are_not_a_line_are_left_alone_where_no_table_can_be_made() {
1306        let mut func = returning(i32(), &[0, 1, 2], &[5, 9, 2]);
1307        assert!(!fired(&convert(&mut func)));
1308    }
1309
1310    #[test]
1311    fn two_labels_are_not_enough_to_pay_for_the_arithmetic() {
1312        let mut func = returning(i32(), &[0, 1], &[1, 2]);
1313        assert!(!fired(&convert(&mut func)));
1314    }
1315
1316    #[test]
1317    fn an_answer_wider_than_its_label_is_left_alone_where_no_table_can_be_made() {
1318        let mut names = Interner::new();
1319        let mut func = Func::new(names.intern("f"), Signature::new());
1320        let head = func.create_block();
1321        let value = func.append_param(head, i32());
1322        let default = func.create_block();
1323        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1324        for (index, &arm) in arms.iter().enumerate() {
1325            let mut build = Builder::new(&mut func, arm);
1326            let it = build.iconst(Type::int(64), index as i128 + 1);
1327            build.ret(&[it]);
1328        }
1329        let mut build = Builder::new(&mut func, default);
1330        let it = build.iconst(Type::int(64), 0);
1331        build.ret(&[it]);
1332        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1333        Builder::new(&mut func, head).switch(value, default, &cases);
1334        assert!(!fired(&convert(&mut func)));
1335    }
1336
1337    #[test]
1338    fn an_arm_something_else_reaches_is_left_alone() {
1339        let mut func = returning(i32(), &[0, 1, 2], &[1, 2, 3]);
1340        // The default jumps into the first arm instead of returning, so the arm is a block two
1341        // edges arrive at and is not one this may take away.
1342        let default = Block::from_usize(1);
1343        let arm = Block::from_usize(2);
1344        let term = func.terminator(default).expect("the default returns");
1345        func.remove_inst(term);
1346        Builder::new(&mut func, default).jump(arm, &[]);
1347        assert!(!fired(&convert(&mut func)));
1348    }
1349
1350    #[test]
1351    fn an_arm_that_is_also_the_default_is_left_alone() {
1352        let mut names = Interner::new();
1353        let mut func = Func::new(names.intern("f"), Signature::new());
1354        let head = func.create_block();
1355        let value = func.append_param(head, i32());
1356        let shared = func.create_block();
1357        let mut build = Builder::new(&mut func, shared);
1358        let it = build.iconst(i32(), 1);
1359        build.ret(&[it]);
1360        let others: Vec<Block> = (0..2).map(|_| func.create_block()).collect();
1361        for (index, &arm) in others.iter().enumerate() {
1362            let mut build = Builder::new(&mut func, arm);
1363            let it = build.iconst(i32(), index as i128 + 2);
1364            build.ret(&[it]);
1365        }
1366        let cases = [(0, shared), (1, others[0]), (2, others[1])];
1367        Builder::new(&mut func, head).switch(value, shared, &cases);
1368        assert!(!fired(&convert(&mut func)));
1369    }
1370
1371    #[test]
1372    fn arms_that_join_keep_what_they_pass_beside_the_answer() {
1373        let mut names = Interner::new();
1374        let mut func = Func::new(names.intern("f"), Signature::new());
1375        let head = func.create_block();
1376        let value = func.append_param(head, i32());
1377        let alongside = func.append_param(head, i32());
1378        let join = func.create_block();
1379        let handed = func.append_param(join, i32());
1380        let carried = func.append_param(join, i32());
1381        Builder::new(&mut func, join).ret(&[handed, carried]);
1382        let default = func.create_block();
1383        let mut build = Builder::new(&mut func, default);
1384        let it = build.iconst(i32(), 999);
1385        build.jump(join, &[it, alongside]);
1386        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1387        for (index, &arm) in arms.iter().enumerate() {
1388            let mut build = Builder::new(&mut func, arm);
1389            let it = build.iconst(i32(), index as i128 + 1);
1390            build.jump(join, &[it, alongside]);
1391        }
1392        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1393        Builder::new(&mut func, head).switch(value, default, &cases);
1394        assert!(fired(&convert(&mut func)));
1395
1396        let arm = arm(&func);
1397        assert_eq!(answer(&func, arm, 2), 3);
1398        // The second argument is what it always was, which is the parameter every arm passed.
1399        let term = func.terminator(arm).expect("the block ends in a jump");
1400        let call = func.successors(term).next().expect("a jump goes somewhere");
1401        assert_eq!(func[call.args][1], alongside);
1402    }
1403
1404    #[test]
1405    fn arms_that_hand_on_two_different_things_are_left_alone() {
1406        let mut names = Interner::new();
1407        let mut func = Func::new(names.intern("f"), Signature::new());
1408        let head = func.create_block();
1409        let value = func.append_param(head, i32());
1410        let join = func.create_block();
1411        let first = func.append_param(join, i32());
1412        let second = func.append_param(join, i32());
1413        Builder::new(&mut func, join).ret(&[first, second]);
1414        let default = func.create_block();
1415        let mut build = Builder::new(&mut func, default);
1416        let it = build.iconst(i32(), 999);
1417        build.jump(join, &[it, it]);
1418        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1419        for (index, &arm) in arms.iter().enumerate() {
1420            let mut build = Builder::new(&mut func, arm);
1421            let one = build.iconst(i32(), index as i128 + 1);
1422            let two = build.iconst(i32(), index as i128 + 10);
1423            build.jump(join, &[one, two]);
1424        }
1425        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1426        Builder::new(&mut func, head).switch(value, default, &cases);
1427        assert!(!fired(&convert(&mut func)));
1428    }
1429
1430    #[test]
1431    fn an_arm_that_does_something_is_left_alone() {
1432        let mut names = Interner::new();
1433        let mut func = Func::new(names.intern("f"), Signature::new());
1434        let head = func.create_block();
1435        let value = func.append_param(head, i32());
1436        let default = func.create_block();
1437        let mut build = Builder::new(&mut func, default);
1438        let it = build.iconst(i32(), 999);
1439        build.ret(&[it]);
1440        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1441        for (index, &arm) in arms.iter().enumerate() {
1442            let mut build = Builder::new(&mut func, arm);
1443            let it = build.iconst(i32(), index as i128 + 1);
1444            // An addition the arm did, which is work the answer would have been left without.
1445            let sum = build.binary(Opcode::Add, it, value, Flags::NONE);
1446            build.ret(&[sum]);
1447        }
1448        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1449        Builder::new(&mut func, head).switch(value, default, &cases);
1450        assert!(!fired(&convert(&mut func)));
1451    }
1452
1453    #[test]
1454    fn the_default_goes_where_it_went() {
1455        let mut func = returning(i32(), &[0, 1, 2, 3], &[1, 2, 3, 4]);
1456        let head = func.entry().expect("a function with blocks in it");
1457        let before = func.terminator(head).expect("a head block has one");
1458        let was = func.successors(before).next().expect("a switch has a default").block;
1459        assert!(fired(&convert(&mut func)));
1460        let after = func.terminator(head).expect("a head block has one");
1461        let now = func.successors(after).next().expect("a switch has a default").block;
1462        assert_eq!(was, now, "the default moved");
1463    }
1464
1465    /// The opcodes of a block that answers from a table, from the label at zero.
1466    const LOOKUP: [Opcode; 6] = [
1467        Opcode::ZExt,
1468        Opcode::IConst,
1469        Opcode::Mul,
1470        Opcode::GlobalAddr,
1471        Opcode::PtrAdd,
1472        Opcode::Load,
1473    ];
1474
1475    #[test]
1476    fn answers_that_are_not_a_line_are_one_load_from_a_table() {
1477        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, 9, 2, 7]);
1478        let (stats, tables) = tabled(&mut func);
1479        assert!(fired(&stats));
1480        assert_eq!(tables.len(), 1);
1481        assert_eq!(tables[0].ty, i32());
1482        assert_eq!(tables[0].cells, [5, 9, 2, 7]);
1483        let arm = arm(&func);
1484        let mut want = LOOKUP.to_vec();
1485        want.push(Opcode::Return);
1486        assert_eq!(opcodes(&func, arm), want);
1487        for (label, answer) in [(0, 5), (1, 9), (2, 2), (3, 7)] {
1488            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1489        }
1490    }
1491
1492    /// A hole gets the default's answer and a case of its own, when the default only gives one.
1493    ///
1494    /// The default here returns 999 and nothing else, so the value in the hole reads 999 out of
1495    /// the table and gets what it got before, and the labels are one run with no hole in it.
1496    #[test]
1497    fn a_hole_is_filled_with_what_a_default_that_only_answers_gives() {
1498        let mut func = returning(i32(), &[1, 2, 4, 5], &[10, 20, 40, 55]);
1499        let head = func.entry().expect("a function with blocks in it");
1500        let before = func.terminator(head).expect("a head block has one");
1501        let default = func.successors(before).next().expect("a switch has a default").block;
1502        let (stats, tables) = tabled(&mut func);
1503        assert!(fired(&stats));
1504        assert_eq!(tables[0].cells, [10, 20, 999, 40, 55]);
1505        assert_eq!(cases(&func).len(), 5, "the hole was not given a case");
1506        let after = func.terminator(head).expect("a head block has one");
1507        assert_eq!(func.successors(after).next().map(|call| call.block), Some(default));
1508        let arm = arm(&func);
1509        for (label, answer) in [(1, 10), (2, 20), (3, 999), (4, 40), (5, 55)] {
1510            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1511        }
1512    }
1513
1514    /// The shape of tamnd/rucc#1932: the last case is written just above `default:` and falls into
1515    /// it, so its block only jumps to the default's. It leaves the `switch`, the rest are a line,
1516    /// and the block it went through goes with the arms.
1517    #[test]
1518    fn a_case_that_falls_into_the_default_is_the_defaults() {
1519        let mut func = falling(&[0, 1, 2, 3], &[1, 2, 3, 4], &[4], true);
1520        assert!(fired(&convert(&mut func)));
1521        assert_eq!(labels(&func), [0, 1, 2, 3]);
1522        assert_eq!(func.blocks().count(), 3, "the head, the default and the arithmetic");
1523        let arm = arm(&func);
1524        for label in 0..4 {
1525            assert_eq!(answer(&func, arm, label), label + 1);
1526        }
1527    }
1528
1529    /// A case edge straight to the default block was an arm that was also the default, which kept
1530    /// the `switch` before.
1531    #[test]
1532    fn a_case_edge_straight_to_the_default_is_the_defaults() {
1533        let mut func = falling(&[0, 1, 2, 3], &[1, 2, 3, 4], &[4], false);
1534        assert!(fired(&convert(&mut func)));
1535        assert_eq!(labels(&func), [0, 1, 2, 3]);
1536        let arm = arm(&func);
1537        for label in 0..4 {
1538            assert_eq!(answer(&func, arm, label), label + 1);
1539        }
1540    }
1541
1542    /// The shape of tamnd/rucc#728's `switch-runs`: runs of labels written one above another, each
1543    /// run giving one answer. Every label but the last in a run reaches the arm through blocks that
1544    /// only jump, and those go with the arms.
1545    #[test]
1546    fn labels_written_one_above_another_reach_their_arm_through_the_blocks_between() {
1547        let mut func = runs(&[(&[10, 11, 12], 1), (&[13, 14, 15], 2), (&[16, 17], 7)]);
1548        let (stats, tables) = tabled(&mut func);
1549        assert!(fired(&stats));
1550        assert_eq!(tables[0].cells, [1, 1, 1, 2, 2, 2, 7, 7]);
1551        assert_eq!(func.blocks().count(), 3, "the head, the default and the load");
1552        let arm = arm(&func);
1553        for (label, answer) in [(10, 1), (11, 1), (12, 1), (13, 2), (15, 2), (16, 7), (17, 7)] {
1554            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1555        }
1556    }
1557
1558    /// A block between that something else reaches is not one this may take away, and neither is
1559    /// the arm below it.
1560    #[test]
1561    fn a_block_between_that_something_else_reaches_keeps_the_switch() {
1562        let mut func = runs(&[(&[0, 1, 2], 1), (&[3, 4, 5], 2), (&[6, 7], 3)]);
1563        let default = Block::from_usize(1);
1564        let between = Block::from_usize(cases(&func)[0]);
1565        let term = func.terminator(default).expect("the default returns");
1566        func.remove_inst(term);
1567        Builder::new(&mut func, default).jump(between, &[]);
1568        let (stats, _) = tabled(&mut func);
1569        assert!(!fired(&stats));
1570    }
1571
1572    /// A case two blocks above `default:` goes through both on its way there and is the default's,
1573    /// and both blocks go.
1574    #[test]
1575    fn a_case_that_falls_through_two_blocks_into_the_default_is_the_defaults() {
1576        let mut func = falling(&[0, 1, 2, 3], &[1, 2, 3, 4], &[4], true);
1577        let default = Block::from_usize(1);
1578        let through = Block::from_usize(cases(&func)[4]);
1579        let below = func.create_block();
1580        Builder::new(&mut func, below).jump(default, &[]);
1581        let term = func.terminator(through).expect("it jumps");
1582        func.remove_inst(term);
1583        Builder::new(&mut func, through).jump(below, &[]);
1584        assert!(fired(&convert(&mut func)));
1585        assert_eq!(labels(&func), [0, 1, 2, 3]);
1586        assert_eq!(func.blocks().count(), 3, "the head, the default and the arithmetic");
1587    }
1588
1589    /// Between two labels a case that falls into the default is a hole, and the default only
1590    /// answers, so the hole reads what the default gives out of the table.
1591    #[test]
1592    fn a_case_that_falls_into_the_default_between_two_labels_is_a_hole() {
1593        let mut func = falling(&[0, 1, 3, 4], &[5, 9, 2, 7], &[2], true);
1594        let (stats, tables) = tabled(&mut func);
1595        assert!(fired(&stats));
1596        assert_eq!(tables[0].cells, [5, 9, 999, 2, 7]);
1597        assert_eq!(labels(&func), [0, 1, 3, 4, 2]);
1598        let arm = arm(&func);
1599        for (label, answer) in [(0, 5), (1, 9), (2, 999), (3, 2), (4, 7)] {
1600            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1601        }
1602    }
1603
1604    /// What is left once the default's cases are out is what has to pay for the arithmetic.
1605    #[test]
1606    fn two_labels_left_once_the_defaults_are_out_are_not_enough() {
1607        let mut func = falling(&[0, 1], &[1, 2], &[2, 3], true);
1608        let stats = convert(&mut func);
1609        assert!(!fired(&stats));
1610        assert_eq!(stats.count(Kind::Missed, TOO_FEW), 1, "{stats:?}");
1611        assert_eq!(labels(&func), [0, 1, 2, 3]);
1612    }
1613
1614    /// A hole is a cell nothing reads when the default does something other than answer.
1615    ///
1616    /// This default returns the label, which is no constant, so the value in the hole has to
1617    /// keep going to it down the edge it always went down.
1618    #[test]
1619    fn a_hole_still_goes_to_a_default_that_does_more_than_answer() {
1620        let mut func = returning(i32(), &[1, 2, 4, 5], &[10, 20, 40, 55]);
1621        let head = func.entry().expect("a function with blocks in it");
1622        let before = func.terminator(head).expect("a head block has one");
1623        let default = func.successors(before).next().expect("a switch has a default").block;
1624        let label = func[func[before].args][0];
1625        let ret = func.terminator(default).expect("the default returns");
1626        func.remove_inst(ret);
1627        Builder::new(&mut func, default).ret(&[label]);
1628        let (stats, tables) = tabled(&mut func);
1629        assert!(fired(&stats));
1630        assert_eq!(tables[0].cells, [10, 20, 0, 40, 55]);
1631        assert_eq!(cases(&func).len(), 4, "a hole was given a case");
1632        let after = func.terminator(head).expect("a head block has one");
1633        assert_eq!(func.successors(after).next().map(|call| call.block), Some(default));
1634        let arm = arm(&func);
1635        for (label, answer) in [(1, 10), (2, 20), (4, 40), (5, 55)] {
1636            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1637        }
1638    }
1639
1640    /// Cell zero is the lowest label, which here is below zero and reached by wrapping.
1641    ///
1642    /// Every label of a `signed char` is tried, which is the whole of what can reach the block
1643    /// and the whole of what can go wrong with the subtraction and the widening after it.
1644    #[test]
1645    fn labels_below_zero_index_from_the_lowest_of_them() {
1646        let ty = Type::int(8);
1647        let labels = [-128, -3, -1, 0, 2, 127];
1648        let answers = [7, -5, 11, 3, -100, 42];
1649        let mut func = returning(ty, &labels, &answers);
1650        // Far apart at the ends, so a table is only allowed because the ratio is eight to one and
1651        // there are six labels: two hundred and fifty six cells is more than forty eight.
1652        let (stats, _) = tabled(&mut func);
1653        assert!(!fired(&stats), "a table of mostly holes was made");
1654
1655        let labels = [-3, -2, -1, 0, 2];
1656        let answers = [7, -5, 11, 3, -100];
1657        let mut func = returning(ty, &labels, &answers);
1658        let (stats, tables) = tabled(&mut func);
1659        assert!(fired(&stats));
1660        // The default returns 999, which is -25 as a `signed char`, and the hole at 1 is given it.
1661        assert_eq!(tables[0].cells, [7, -5, 11, 3, -25, -100]);
1662        let arm = arm(&func);
1663        assert_eq!(opcodes(&func, arm)[..2], [Opcode::IConst, Opcode::Sub]);
1664        for (&label, &answer) in labels.iter().zip(&answers).chain([(&1, &-25)]) {
1665            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1666        }
1667    }
1668
1669    /// An `int` label and a `long` answer, which a line declines and a table does not mind.
1670    #[test]
1671    fn an_answer_wider_than_its_label_is_a_table_of_the_wider_type() {
1672        let mut names = Interner::new();
1673        let answers = [1i128 << 40, 3, -1, 1 << 33];
1674        let mut func = Func::new(names.intern("f"), Signature::new());
1675        let head = func.create_block();
1676        let value = func.append_param(head, i32());
1677        let default = func.create_block();
1678        let arms: Vec<Block> = answers.iter().map(|_| func.create_block()).collect();
1679        for (&arm, &answer) in arms.iter().zip(&answers) {
1680            let mut build = Builder::new(&mut func, arm);
1681            let it = build.iconst(Type::int(64), answer);
1682            build.ret(&[it]);
1683        }
1684        let mut build = Builder::new(&mut func, default);
1685        let it = build.iconst(Type::int(64), 0);
1686        build.ret(&[it]);
1687        let cases: Vec<(i128, Block)> = (10..14).zip(arms.iter().copied()).collect();
1688        Builder::new(&mut func, head).switch(value, default, &cases);
1689        let (stats, tables) = tabled(&mut func);
1690        assert!(fired(&stats));
1691        assert_eq!(tables[0].ty, Type::int(64));
1692        let arm = arm(&func);
1693        for (label, &answer) in (10..14).zip(&answers) {
1694            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1695        }
1696    }
1697
1698    #[test]
1699    fn labels_too_far_apart_for_a_table_are_left_alone() {
1700        let mut func = returning(i32(), &[0, 100, 200], &[1, 5, 3]);
1701        let (stats, tables) = tabled(&mut func);
1702        assert!(!fired(&stats));
1703        assert!(tables.is_empty());
1704    }
1705
1706    #[test]
1707    fn a_line_is_still_arithmetic_where_a_table_could_be_made() {
1708        let mut func = returning(i32(), &[0, 1, 2, 3], &[1, 2, 3, 4]);
1709        let (stats, tables) = tabled(&mut func);
1710        assert!(fired(&stats));
1711        assert!(tables.is_empty(), "a table was made for a line");
1712    }
1713
1714    #[test]
1715    fn a_label_wider_than_a_word_gets_no_table() {
1716        let mut func = returning(Type::int(128), &[0, 1, 2, 3], &[5, 9, 2, 7]);
1717        let (stats, tables) = tabled(&mut func);
1718        assert!(!fired(&stats));
1719        assert!(tables.is_empty());
1720    }
1721
1722    /// The answer is in the second place and the first is a constant every arm passes.
1723    ///
1724    /// The first arm's answer used to be read from the first place, because which place the
1725    /// answer is in is not known until a second arm disagrees. Here that reads one at the first
1726    /// label, and one, two and three are a line, so the pass returned one where the program said
1727    /// ten. What it must do is see ten, two and three, which is not a line.
1728    #[test]
1729    fn the_answer_is_read_from_the_place_the_arms_disagree_about() {
1730        let mut names = Interner::new();
1731        let mut func = Func::new(names.intern("f"), Signature::new());
1732        let head = func.create_block();
1733        let value = func.append_param(head, i32());
1734        let join = func.create_block();
1735        let first = func.append_param(join, i32());
1736        let second = func.append_param(join, i32());
1737        Builder::new(&mut func, join).ret(&[second, first]);
1738        let default = func.create_block();
1739        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1740        let mut build = Builder::new(&mut func, head);
1741        let one = build.iconst(i32(), 1);
1742        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1743        build.switch(value, default, &cases);
1744        let mut build = Builder::new(&mut func, default);
1745        let it = build.iconst(i32(), 999);
1746        build.jump(join, &[one, it]);
1747        for (&arm, answer) in arms.iter().zip([10, 2, 3]) {
1748            let mut build = Builder::new(&mut func, arm);
1749            let it = build.iconst(i32(), answer);
1750            build.jump(join, &[one, it]);
1751        }
1752        assert!(!fired(&convert(&mut func)), "ten, two and three were taken for a line");
1753        let (stats, tables) = tabled(&mut func);
1754        assert!(fired(&stats));
1755        assert_eq!(tables[0].cells, [10, 2, 3]);
1756    }
1757
1758    /// At the size goal a cell is a byte when every answer is one, and the load is widened back.
1759    ///
1760    /// Below zero is widened with a sign and two hundred without one, so both are tried, and the
1761    /// speed goal keeps the answer's own width, which is what gcc 16 does at the two levels.
1762    #[test]
1763    fn a_table_for_size_has_cells_as_narrow_as_its_answers() {
1764        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, -9, 2, 7]);
1765        let (stats, tables) = tabled_for(&mut func, Goal::Size);
1766        assert!(fired(&stats));
1767        assert_eq!(tables[0].ty, Type::int(8));
1768        let at = arm(&func);
1769        assert!(opcodes(&func, at).contains(&Opcode::SExt));
1770        for (label, answer) in [(0, 5), (1, -9), (2, 2), (3, 7)] {
1771            assert_eq!(looked_up(&func, at, label, &tables), answer);
1772        }
1773
1774        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, 200, 2, 255]);
1775        let (_, tables) = tabled_for(&mut func, Goal::Size);
1776        assert_eq!(tables[0].ty, Type::int(8));
1777        let at = arm(&func);
1778        assert!(opcodes(&func, at).contains(&Opcode::ZExt));
1779        for (label, answer) in [(0, 5), (1, 200), (2, 2), (3, 255)] {
1780            assert_eq!(looked_up(&func, at, label, &tables), answer);
1781        }
1782
1783        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, -300, 2, 40000]);
1784        let (_, tables) = tabled_for(&mut func, Goal::Size);
1785        assert_eq!(tables[0].ty, i32(), "a cell narrower than an answer that needs all of it");
1786
1787        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, -9, 2, 7]);
1788        let (_, tables) = tabled_for(&mut func, Goal::Speed);
1789        assert_eq!(tables[0].ty, i32());
1790    }
1791
1792    /// A function whose arms return addresses, the module that defines what they are the
1793    /// addresses of, and the names of those, the default's last.
1794    struct Pointing {
1795        names: Interner,
1796        module: Module,
1797        func: Func,
1798        places: Vec<Symbol>,
1799    }
1800
1801    /// A function that switches on its parameter and returns the address of a name per label,
1802    /// with the module that defines the names.
1803    ///
1804    /// The names are `s0` and on, one per label, and the default returns the address of one more.
1805    /// Each is read only and `static` unless `written` names it, and a name that is written to
1806    /// is not somewhere a table may hold the distance to. The arm for the `k`th label returns an
1807    /// address `k * into` bytes into its name, which is `&s[k]` for an array of `into` bytes.
1808    fn pointing(labels: &[i128], written: Option<usize>, into: i128) -> Pointing {
1809        let mut names = Interner::new();
1810        let target = TargetInfo::new("x86_64-unknown-linux-gnu".parse::<Triple>().unwrap());
1811        let mut module = Module::new(names.intern("t.c"), &target);
1812        let places: Vec<Symbol> =
1813            (0..=labels.len()).map(|k| names.intern(&format!("s{k}"))).collect();
1814        for (k, &name) in places.iter().enumerate() {
1815            let mut global = Global::new(name, 4, 1);
1816            global.linkage = Linkage::Internal;
1817            global.constant = written != Some(k);
1818            global.init = Some(module.push_data(&[Datum::Zero(4)]));
1819            module.add_global(global);
1820        }
1821        let mut func = Func::new(names.intern("f"), Signature::new());
1822        let head = func.create_block();
1823        let value = func.append_param(head, i32());
1824        let blocks: Vec<Block> = places.iter().map(|_| func.create_block()).collect();
1825        for ((k, &block), &name) in blocks.iter().enumerate().zip(&places) {
1826            let mut build = Builder::new(&mut func, block);
1827            let data = InstData { extra: Extra::Symbol(name), ..InstData::new(Opcode::GlobalAddr) };
1828            let mut it = build.value(data, Type::PTR);
1829            if into != 0 && k < labels.len() {
1830                let bytes = build.iconst(Type::int(64), into * k as i128);
1831                it = build.binary(Opcode::PtrAdd, it, bytes, Flags::NONE);
1832            }
1833            build.ret(&[it]);
1834        }
1835        let (&default, arms) = blocks.split_last().expect("a default");
1836        let cases: Vec<(i128, Block)> = labels.iter().copied().zip(arms.iter().copied()).collect();
1837        Builder::new(&mut func, head).switch(value, default, &cases);
1838        Pointing { names, module, func, places }
1839    }
1840
1841    /// Runs the pass the way the pipeline does on x86-64 ELF when `measures` is true, and on a
1842    /// target with no four byte distance when it is not.
1843    fn placed(pointing: &mut Pointing, measures: bool) -> (Stats, Vec<Table>) {
1844        let taken = HashSet::new();
1845        let mut data = ReadOnly::new(&mut pointing.names, &taken, 64, 0).measuring(measures);
1846        let images = Arc::new(Images::of(&pointing.module, Pic::Executable));
1847        let mut an = crate::Analyses::new(crate::Machine::with(None, Goal::Speed)).reading(images);
1848        let stats =
1849            SwitchConv.run_emitting(&mut pointing.func, &mut an, &mut Fuel::unlimited(), &mut data);
1850        (stats, data.into_tables())
1851    }
1852
1853    #[test]
1854    fn answers_that_are_addresses_are_a_table_of_how_far_they_are_from_it() {
1855        let mut pointing = pointing(&[0, 1, 2, 3], None, 0);
1856        let (stats, tables) = placed(&mut pointing, true);
1857        assert!(fired(&stats));
1858        assert_eq!(tables.len(), 1);
1859        assert_eq!(tables[0].ty, i32());
1860        assert_eq!(tables[0].cells, [0, 0, 0, 0]);
1861        let want: Vec<Option<Symbol>> = pointing.places[..4].iter().copied().map(Some).collect();
1862        assert_eq!(tables[0].to, want);
1863        assert_eq!(tables[0].cells, [0, 0, 0, 0]);
1864        let func = &pointing.func;
1865        let arm = arm(func);
1866        let mut want = LOOKUP.to_vec();
1867        want.extend([
1868            Opcode::SExt,
1869            Opcode::PtrToInt,
1870            Opcode::Add,
1871            Opcode::IntToPtr,
1872            Opcode::Return,
1873        ]);
1874        assert_eq!(opcodes(func, arm), want);
1875        for label in 0..4 {
1876            let place = pointing.places[label as usize];
1877            assert_eq!(looked_up(func, arm, label, &tables), spot(place));
1878        }
1879    }
1880
1881    /// `&s[k]` for each label `k`, where the cell is the distance to the name with the bytes into
1882    /// it added.
1883    #[test]
1884    fn an_address_part_way_into_a_name_is_the_distance_to_it_and_the_bytes_in() {
1885        let mut pointing = pointing(&[0, 1, 2, 3], None, 4);
1886        let (stats, tables) = placed(&mut pointing, true);
1887        assert!(fired(&stats));
1888        assert_eq!(tables[0].cells, [0, 4, 8, 12]);
1889        let arm = arm(&pointing.func);
1890        for label in 0..4 {
1891            let place = pointing.places[label as usize];
1892            let got = looked_up(&pointing.func, arm, label, &tables);
1893            assert_eq!(got, spot(place) + 4 * label, "{label}");
1894        }
1895    }
1896
1897    /// The default returns the address of `s4`, which is somewhere a table can say it is, so the
1898    /// hole reads that and gets what it got before.
1899    #[test]
1900    fn a_hole_in_a_table_of_addresses_is_where_the_default_points() {
1901        let mut pointing = pointing(&[1, 2, 4, 5], None, 0);
1902        let (stats, tables) = placed(&mut pointing, true);
1903        assert!(fired(&stats));
1904        let places = &pointing.places;
1905        let want = [places[0], places[1], places[4], places[2], places[3]].map(Some);
1906        assert_eq!(tables[0].to, want);
1907        assert_eq!(cases(&pointing.func).len(), 5, "the hole was not given a case");
1908        let arm = arm(&pointing.func);
1909        for (label, place) in [(1, 0), (2, 1), (3, 4), (4, 2), (5, 3)] {
1910            let got = looked_up(&pointing.func, arm, label, &tables);
1911            assert_eq!(got, spot(places[place]), "{label}");
1912        }
1913    }
1914
1915    /// A name the program writes to is still a name this file defines, but it is not in the images,
1916    /// and the images are what says a name is somewhere nothing else can move.
1917    #[test]
1918    fn an_answer_that_is_not_read_only_data_keeps_its_switch() {
1919        let mut pointing = pointing(&[0, 1, 2, 3], Some(2), 0);
1920        let (stats, tables) = placed(&mut pointing, true);
1921        assert!(!fired(&stats));
1922        assert!(tables.is_empty());
1923        assert_eq!(stats.count(Kind::Missed, PLACE_IS_ODD), 1, "{stats:?}");
1924    }
1925
1926    #[test]
1927    fn a_target_with_no_four_byte_distance_keeps_its_switch() {
1928        let mut pointing = pointing(&[0, 1, 2, 3], None, 0);
1929        let (stats, tables) = placed(&mut pointing, false);
1930        assert!(!fired(&stats));
1931        assert!(tables.is_empty());
1932    }
1933}