Skip to main content

rucc_opt/
switch_conv.rs

1//! A `switch` whose arms are a function of the label, which is arithmetic and not branches.
2//!
3//! Design: `spec/optimizer/24-switch-lowering.md` section 24.1, which is the transformation the GCC
4//! file `tree-switch-conversion.cc` is named after, and section 24.4, which puts it in the middle
5//! end rather than in the lowering and says why: what it produces is ordinary arithmetic that every
6//! pass after it optimizes, and what it needs to see is arms whose constancy earlier passes made
7//! visible.
8//!
9//! # The shape
10//!
11//! ```c
12//! switch (x) { case 0: return 1; case 1: return 2; case 2: return 3; case 3: return 4; }
13//! return 0;
14//! ```
15//!
16//! Four labels, four arms, and the arm for label `k` gives `k + 1`. The labels run consecutively
17//! and the answers run consecutively with them, so the whole statement is one range check and one
18//! addition. gcc reduces thirty three labels of this to a comparison and a `lea`, which is
19//! tamnd/rucc#728, and rucc emitted a comparison and a jump per label.
20//!
21//! Where the answers are an affine function of the label, `a * x + b`, there is nothing to look
22//! up. That covers the shape above with `a` of one and `b` of one, the shape where every arm gives
23//! the same answer with `a` of zero, and the scaled ones in between.
24//!
25//! Where they are not, the answers are a table. The arm for label `k` gives the constant in cell
26//! `k - low` of a read only array, and every arm becomes one load from it. That is section 24.4's
27//! other half, and it is what gcc calls a `CSWTCH` array. The array is asked for through
28//! `crate::readonly`, because a pass is handed one function and the array is the module's.
29//!
30//! # What it rewrites and what it leaves
31//!
32//! The `switch` stays a `switch`. Every case edge is pointed at one new block, which works the
33//! answer out and hands it on, and the default edge is not touched at all. What that buys is that
34//! the range check is not written here: a `switch` whose cases are consecutive and all go to one
35//! place is exactly `crates/rucc-codegen/src/switch.rs`'s `Cluster::Run`, which is one subtraction
36//! and one unsigned comparison however long the run is, and which already gets the modular
37//! arithmetic and the run that covers a whole type right. Writing a second range check here would
38//! be a second place for section 24.6's overflow to be got wrong.
39//!
40//! The default is untouched for the reason section 24.6 gives, which is that the default is never
41//! dropped. A value that matches no case went to the default before this ran and goes to the same
42//! place afterwards, because the edge it goes down is the same edge.
43//!
44//! # What has to be true
45//!
46//! For arithmetic, the labels are consecutive. A hole in the labels is a stretch the lowering
47//! would then have to cut the run at, and one comparison becomes several for a function that was
48//! only fitted to the labels either side of it.
49//!
50//! For a table, the labels may have holes. Where the default hands on what the arms hand on with a
51//! constant in the answer's place, which is `default: return 0;` and `default: y = 0; break;`, a
52//! hole's cell is that constant and the hole is given a case of its own going to the load, as gcc's
53//! `gather_default_values` does. The labels are then one run, which the lowering checks with one
54//! comparison, where a run with holes in it is a comparison and a bit test, and the bit test is a
55//! branch a stream of values mispredicts. Where the default does anything else, the `switch` still
56//! sends a value in a hole to the default, the hole's cell is never read, and it is written as zero
57//! only because an array has to have something there. What bounds the holes is size: the table spans at most eight cells for every label it replaces, which is gcc's
58//! `switch-conversion-max-branch-ratio`, so a `switch` over three labels a thousand apart stays a
59//! `switch`.
60//!
61//! Every arm is a block nothing else reaches, holding nothing but the constants it hands on, and
62//! ending the same way as every other arm. The same way means a jump to the same block, or a
63//! return, and in either case with the same values in every position but one. That one is the
64//! answer. Section 24.5 gives up on arms that assign more than one thing and so does this.
65//!
66//! The answers are `a * label + b` at every label, checked at every label rather than fitted to two
67//! of them and believed. The check is done in the answer's own width with wrapping, because that is
68//! what the arithmetic this writes will do, and the arithmetic is written with no flags on it so
69//! that wrapping is what it is allowed to do.
70//!
71//! For arithmetic, the label and the answer are the same width. A `switch` on an `int` whose arms
72//! give a `long` is the same transformation with a widening in front of the multiply, and which
73//! widening it is depends on how the label is read, which is a question this would have to answer
74//! and currently declines to ask. A table does not have that question, because the label only
75//! picks a cell and the cell is already as wide as the answer, so a table may be any whole number
76//! of bytes wide up to eight whatever the label is. A label wider than a word gets no table, since
77//! the index into one is a word.
78//!
79//! # Why three labels and not two
80//!
81//! Two labels and a default is a shape `phiopt` already has something to say about, and what it
82//! says is a `select` between two constants that cost nothing to materialize. The arithmetic this
83//! writes is a multiply and an add against a range check, which is not obviously better than that
84//! and is worse when `a` is not one. From three labels up the chain being replaced is at least six
85//! instructions and what replaces it is at most five, so it is a win at three and grows from there.
86//!
87//! A table is held to the same three. What replaces the chain is a subtraction, the range check,
88//! a widening and a load, which is five again, and the load is from a line the program keeps
89//! reading if the `switch` is hot.
90//!
91//! # Where the table goes
92//!
93//! The array is internal, constant and aligned to its cell, which puts it in `.rodata`. It is
94//! named `CSWTCH.` and a number, as gcc names it, which nothing written in C can spell.
95//!
96//! When the goal is size a cell is as narrow as the answers allow, and the load is widened back to
97//! the answer's width with a sign or without one, whichever holds every answer. gcc 16 does the
98//! same at `-Os` and not at `-O2`, where the widening is an instruction on the path and the bytes
99//! it saves are data rather than code. Three `int` answers under ten are twelve bytes at `-O2` and
100//! three at `-Os`, in gcc and here.
101
102use std::cmp::Ordering;
103use std::collections::HashSet;
104
105use rucc_base::Symbol;
106use rucc_ir::{
107    Block, BlockCall, Builder, Extra, Flags, Func, Imm, Inst, InstData, MemInfo, MemOrder, Opcode,
108    Restrict, Type, Value,
109};
110
111use rucc_cost::Goal;
112use rucc_cost::heuristics::SWITCH_CONVERSION_MAX_GROWTH;
113
114use crate::cfg::Cfg;
115use crate::{Analyses, Fuel, Pass, Preserved, ReadOnly, Stats};
116
117/// What is reported when a `switch` becomes arithmetic.
118const CONVERTED: &str = "switch replaced by a range check and the arithmetic its arms were doing";
119
120/// What is reported when a `switch` becomes a load from a table of what its arms gave.
121const TABLED: &str = "switch replaced by a range check and a load from a table of its answers";
122
123/// What is reported when the pass ran out of fuel with a `switch` it was about to convert.
124const NO_FUEL: &str = "switch left alone, the pass ran out of fuel";
125
126/// What is reported for a `switch` with too few labels to pay for the arithmetic.
127const TOO_FEW: &str = "switch left alone, it has too few labels for arithmetic to be cheaper";
128
129/// What is reported for a `switch` whose labels have holes in them.
130const NOT_CONSECUTIVE: &str = "switch left alone, its labels are not consecutive";
131
132/// What is reported for a `switch` with an arm that is not a block of its own.
133const ARM_IS_SHARED: &str = "switch left alone, an arm is reached from somewhere other than it";
134
135/// What is reported for a `switch` with an arm that does something.
136const ARM_DOES_WORK: &str = "switch left alone, an arm does more than work out a constant";
137
138/// What is reported for a `switch` whose arms do not end alike.
139const ARMS_DIFFER: &str = "switch left alone, its arms do not all hand on the same thing";
140
141/// What is reported for a `switch` whose answers are not a line.
142const NOT_AFFINE: &str = "switch left alone, its answers are not a fixed multiple of the label \
143                          plus a constant";
144
145/// What is reported for a `switch` whose answers are a different width from its labels.
146const WIDTHS_DIFFER: &str = "switch left alone, its answers are not as wide as its labels";
147
148/// What is reported for a `switch` whose labels are too far apart for a table of them.
149const TOO_SPARSE: &str = "switch left alone, a table of its answers would be mostly holes";
150
151/// What is reported for a `switch` whose label is wider than an index into a table.
152const LABEL_TOO_WIDE: &str = "switch left alone, its label is wider than a word";
153
154/// What is reported for a `switch` whose answers are not something a table cell holds.
155const CELL_IS_ODD: &str =
156    "switch left alone, its answers are not a whole number of bytes of integer";
157
158/// The fewest labels worth converting, per the module documentation.
159const LABELS: usize = 3;
160
161/// How many cells a table may have for every label it stands for, per the module documentation.
162const GROWTH: i128 = SWITCH_CONVERSION_MAX_GROWTH as i128;
163
164/// The pass.
165#[derive(Debug)]
166pub struct SwitchConv;
167
168impl Pass for SwitchConv {
169    fn name(&self) -> &'static str {
170        "switch-conv"
171    }
172
173    fn describe(&self) -> &'static str {
174        "a switch whose arms give constants becomes a range check and arithmetic or a table load"
175    }
176
177    fn preserves(&self) -> Preserved {
178        // A block appears, the arms go, and every case edge moves.
179        Preserved::NONE
180    }
181
182    fn run(&self, func: &mut Func, an: &mut Analyses, fuel: &mut Fuel) -> Stats {
183        convert(func, an, fuel, None)
184    }
185
186    fn run_emitting(
187        &self,
188        func: &mut Func,
189        an: &mut Analyses,
190        fuel: &mut Fuel,
191        data: &mut ReadOnly<'_>,
192    ) -> Stats {
193        convert(func, an, fuel, Some(data))
194    }
195}
196
197/// The pass, with somewhere to put a table or without one.
198///
199/// Without one is what a caller that is not the pipeline gets, and it is arithmetic or nothing.
200fn convert(
201    func: &mut Func,
202    an: &mut Analyses,
203    fuel: &mut Fuel,
204    mut data: Option<&mut ReadOnly<'_>>,
205) -> Stats {
206    let mut stats = Stats::new();
207    if func.entry().is_none() {
208        return stats;
209    }
210    let cfg = an.cfg(func);
211    let found: Vec<Inst> = func
212        .blocks()
213        .filter_map(|block| func.terminator(block))
214        .filter(|&inst| func[inst].opcode == Opcode::Switch)
215        .collect();
216
217    let index_bits = data.as_ref().map(|data| data.pointer_bits());
218    let small = an.machine().goal() == Goal::Size;
219    let mut plans = Vec::new();
220    for inst in found {
221        match plan(func, cfg, inst, index_bits, small) {
222            Ok(plan) => plans.push(plan),
223            Err(why) => stats.missed(why),
224        }
225    }
226
227    let mut changed = false;
228    for plan in plans {
229        if !fuel.take() {
230            stats.missed(NO_FUEL);
231            continue;
232        }
233        let table = match (&plan.how, data.as_deref_mut()) {
234            (How::Table { cell, cells, .. }, Some(data)) => {
235                Some(data.table(cell.ty, cells.clone()))
236            }
237            _ => None,
238        };
239        stats.optimized(if table.is_some() { TABLED } else { CONVERTED });
240        apply(func, &plan, table);
241        changed = true;
242    }
243    if changed {
244        an.clear();
245    }
246    stats
247}
248
249/// How the arms of one `switch` hand their answer on.
250#[derive(Clone, Copy, Debug, PartialEq, Eq)]
251enum Hands {
252    /// To this block, as one of its parameters.
253    On(Block),
254    /// Out of the function, as one of its results.
255    Back,
256}
257
258/// One `switch` and what it is about to become.
259#[derive(Debug)]
260struct Plan {
261    /// The `switch` itself.
262    inst: Inst,
263    /// What it switches on, which is what the answer is a function of.
264    value: Value,
265    /// The width of the label.
266    ty: Type,
267    /// Where the answer goes.
268    hands: Hands,
269    /// What every arm handed on, with the answer's position holding whatever the first arm had
270    /// there. That position is rewritten and the rest are passed on as they were.
271    args: Vec<Value>,
272    /// Which of `args` is the answer.
273    answer: usize,
274    /// How the answer is worked out from the label.
275    how: How,
276    /// The blocks the arms were, which nothing reaches once the case edges have moved.
277    arms: Vec<Block>,
278    /// Values between two labels that get a case of their own going to the load, because the
279    /// default gives what their cell holds.
280    holes: Vec<i128>,
281}
282
283/// How the answer is worked out from the label.
284#[derive(Debug)]
285enum How {
286    /// As `scale * label + offset`, at the label's width.
287    Line {
288        /// The multiple of the label.
289        scale: i128,
290        /// What is added to it.
291        offset: i128,
292    },
293    /// As cell `label - low` of a table.
294    Table {
295        /// The lowest label, which is cell zero.
296        low: i128,
297        /// The width of the answer.
298        ty: Type,
299        /// What a cell is, which is the answer unless the goal is size.
300        cell: Cell,
301        /// Every cell, with zero in the holes.
302        cells: Vec<i128>,
303        /// The width of an index into the table, which is a word on the target.
304        index_bits: u32,
305    },
306}
307
308/// How a cell of a table is held, and how it is made an answer again.
309#[derive(Clone, Copy, Debug, PartialEq, Eq)]
310struct Cell {
311    /// The width of a cell.
312    ty: Type,
313    /// Whether a cell narrower than the answer is widened with its sign.
314    signed: bool,
315}
316
317/// What one `switch` becomes, or why it stays as it is.
318///
319/// `index_bits` is the width of an address when a table may be made and `None` when it may not,
320/// and `small` is whether the goal is size, which is what narrows a cell.
321fn plan(
322    func: &Func,
323    cfg: &Cfg,
324    inst: Inst,
325    index_bits: Option<u32>,
326    small: bool,
327) -> Result<Plan, &'static str> {
328    let Extra::Switch(info) = func[inst].extra else { return Err(ARMS_DIFFER) };
329    let info = func[info];
330    let Some(&value) = func[func[inst].args].first() else { return Err(ARMS_DIFFER) };
331    let ty = func[value].ty;
332    if !ty.is_int() {
333        return Err(WIDTHS_DIFFER);
334    }
335    let calls: Vec<BlockCall> = func[info.targets].to_vec();
336    let labels: Vec<i128> = func[info.cases].iter().map(|imm| imm.signed(ty)).collect();
337    let Some((&default, arms)) = calls.split_first() else { return Err(ARMS_DIFFER) };
338    if arms.len() != labels.len() || arms.len() < LABELS {
339        return Err(TOO_FEW);
340    }
341    // A block that is both an arm and the default is not an arm this may take away, and it looks
342    // like one from here: the predecessor count below says one, because one block reaching another
343    // down two edges is one predecessor, and the arm being removed would take the default with it.
344    if arms.iter().any(|call| call.block == default.block) {
345        return Err(ARM_IS_SHARED);
346    }
347
348    // Consecutive and ascending. The front end sorts nothing, so this is asked of the list as it
349    // arrived rather than of a sorted copy: what is wanted is that the labels are a run, and a run
350    // read out of order is still a run only if it is sorted first, which is work this declines to
351    // do before it knows the answers are a line. Only a line asks, since a table indexes by the
352    // label whatever order the labels came in.
353    let consecutive = labels.windows(2).all(|pair| pair[1].checked_sub(pair[0]) == Some(1));
354    if !consecutive && index_bits.is_none() {
355        return Err(NOT_CONSECUTIVE);
356    }
357
358    // Every arm is a block of its own that works out constants and hands them on, and the way it
359    // hands them on is the way every other arm does.
360    let mut hands = None;
361    let mut shared: Option<Vec<Value>> = None;
362    let mut answer = None;
363    let mut handed = Vec::new();
364    for call in arms {
365        if !call.args.is_empty() {
366            return Err(ARM_DOES_WORK);
367        }
368        if cfg.predecessors(call.block).len() != 1 {
369            return Err(ARM_IS_SHARED);
370        }
371        // And a block an image holds the address of is shared whatever the graph says, because what
372        // arrives there is a `goto *p` that can be in another function.
373        if func.block_name(call.block).is_some() {
374            return Err(ARM_IS_SHARED);
375        }
376        let (way, args) = tail(func, call.block)?;
377        if *hands.get_or_insert(way) != way {
378            return Err(ARMS_DIFFER);
379        }
380        let previous = shared.get_or_insert_with(|| args.clone());
381        if previous.len() != args.len() {
382            return Err(ARMS_DIFFER);
383        }
384        // The one position they disagree about is the answer, and it is the same position every
385        // time. The first arm sets nothing, since it agrees with itself everywhere.
386        for (index, (&mine, &theirs)) in previous.iter().zip(&args).enumerate() {
387            if mine == theirs {
388                continue;
389            }
390            if *answer.get_or_insert(index) != index {
391                return Err(ARMS_DIFFER);
392            }
393        }
394        handed.push(args);
395    }
396    let (Some(hands), Some(args)) = (hands, shared) else { return Err(ARMS_DIFFER) };
397    let answer = answer.ok_or(NOT_AFFINE)?;
398    // Read once the position is known and not while it was being found. Until the second arm
399    // disagrees with the first nobody knows which position the answer is in, and reading the first
400    // arm's answer from a guess of the first position would take whatever was there, which is a
401    // constant every arm passed if the arms pass one, and a line fitted through that is wrong at
402    // the first label.
403    let mut answers = Vec::with_capacity(handed.len());
404    for args in &handed {
405        let Some(number) = constant(func, args[answer]) else { return Err(NOT_AFFINE) };
406        answers.push(number);
407    }
408    let kind = func[args[answer]].ty;
409
410    let line = if consecutive && kind == ty { line(&labels, &answers, ty) } else { None };
411    let (how, holes) = match (line, index_bits) {
412        (Some((scale, offset)), _) => (How::Line { scale, offset }, Vec::new()),
413        (None, Some(index_bits)) => {
414            let fill = fallback(func, default, hands, &args, answer);
415            let shape = Shape { ty, kind, index_bits, small };
416            table(&labels, &answers, shape, fill)?
417        }
418        (None, None) if kind != ty => return Err(WIDTHS_DIFFER),
419        (None, None) => return Err(NOT_AFFINE),
420    };
421    Ok(Plan {
422        inst,
423        value,
424        ty,
425        hands,
426        args,
427        answer,
428        how,
429        arms: arms.iter().map(|call| call.block).collect(),
430        holes,
431    })
432}
433
434/// What the default gives in the answer's place, when that is all it does differently from an arm.
435///
436/// Two shapes of default qualify. One is a block of its own that works out constants and hands
437/// them on the way the arms do, which is `default: return 0;`. The other is an edge straight to
438/// where the arms hand their answer, carrying the answer itself, which is what is left of
439/// `default: y = 0; break;` once the empty block is gone. Either way every position but the answer
440/// has to be what the arms pass, since a hole given a case is about to pass that instead. The
441/// default block is only read here and never taken away, so it may be shared.
442fn fallback(
443    func: &Func,
444    default: BlockCall,
445    hands: Hands,
446    args: &[Value],
447    answer: usize,
448) -> Option<i128> {
449    let theirs = if default.args.is_empty() {
450        let (way, theirs) = tail(func, default.block).ok()?;
451        if way != hands {
452            return None;
453        }
454        theirs
455    } else if hands == Hands::On(default.block) {
456        func[default.args].to_vec()
457    } else {
458        return None;
459    };
460    if theirs.len() != args.len() {
461        return None;
462    }
463    let agrees =
464        args.iter().zip(&theirs).enumerate().all(|(at, (mine, it))| at == answer || mine == it);
465    if !agrees {
466        return None;
467    }
468    constant(func, theirs[answer])
469}
470
471/// What a table is made for: the label's width, the answer's, an index's, and whether the goal is
472/// size.
473#[derive(Clone, Copy, Debug)]
474struct Shape {
475    /// The width of the label.
476    ty: Type,
477    /// The width of the answer.
478    kind: Type,
479    /// The width of an index into the table.
480    index_bits: u32,
481    /// Whether a cell may be narrower than the answer.
482    small: bool,
483}
484
485/// The table the answers make when one is worth making, and the holes that get a case of their own.
486///
487/// A hole gets one only when `fill` is what the default gives, and then its cell is that. The
488/// labels are read with their own sign and so are ordered that way, which is only a question
489/// of which one is cell zero. What the index is at run time is the label less the lowest one at the
490/// label's width, and for a label that is a case that difference is the distance between the two
491/// however the bits are read, because the table is short and the distance fits.
492fn table(
493    labels: &[i128],
494    answers: &[i128],
495    shape: Shape,
496    fill: Option<i128>,
497) -> Result<(How, Vec<i128>), &'static str> {
498    let Shape { ty, kind, index_bits, small } = shape;
499    if ty.bits() > 64 {
500        return Err(LABEL_TOO_WIDE);
501    }
502    if !kind.is_int() || !matches!(kind.bits(), 8 | 16 | 32 | 64) {
503        return Err(CELL_IS_ODD);
504    }
505    let (Some(&low), Some(&high)) = (labels.iter().min(), labels.iter().max()) else {
506        return Err(TOO_FEW);
507    };
508    let span = high - low + 1;
509    if span > GROWTH * labels.len() as i128 {
510        return Err(TOO_SPARSE);
511    }
512    let mut cells = vec![None; usize::try_from(span).map_err(|_| TOO_SPARSE)?];
513    for (&label, &answer) in labels.iter().zip(answers) {
514        let at = usize::try_from(label - low).map_err(|_| TOO_SPARSE)?;
515        cells[at] = Some(answer);
516    }
517    let holes: Vec<i128> = match fill {
518        Some(_) => (low..=high).filter(|&label| cells[(label - low) as usize].is_none()).collect(),
519        None => Vec::new(),
520    };
521    let cells: Vec<i128> = cells.into_iter().map(|cell| cell.or(fill).unwrap_or(0)).collect();
522    let cell = if small { narrowest(&cells, kind) } else { Cell { ty: kind, signed: false } };
523    Ok((How::Table { low, ty: kind, cell, cells, index_bits }, holes))
524}
525
526/// The narrowest cell every answer fits in, read back to the answer's width.
527///
528/// With a sign first, because an answer below zero only fits that way, and then without one, which
529/// is what fits a `200` in a byte. An answer that fits neither way at a width is an answer that
530/// needs the next one, and the answer's own width always fits.
531fn narrowest(answers: &[i128], kind: Type) -> Cell {
532    let whole = 1i128 << kind.bits();
533    for bits in [8u32, 16, 32] {
534        if bits >= kind.bits() {
535            break;
536        }
537        let half = 1i128 << (bits - 1);
538        if answers.iter().all(|&answer| (-half..half).contains(&answer)) {
539            return Cell { ty: Type::int(bits), signed: true };
540        }
541        if answers.iter().all(|&answer| answer.rem_euclid(whole) < half * 2) {
542            return Cell { ty: Type::int(bits), signed: false };
543        }
544    }
545    Cell { ty: kind, signed: false }
546}
547
548/// What a block hands on, when handing something on is the whole of what it does.
549///
550/// Every instruction in it but the last has to be a constant, because the last one is about to be
551/// written somewhere else and anything the block worked out for it would be left behind. A constant
552/// is the exception because a constant is rewritten rather than moved.
553fn tail(func: &Func, block: Block) -> Result<(Hands, Vec<Value>), &'static str> {
554    let Some(last) = func.terminator(block) else { return Err(ARM_DOES_WORK) };
555    for inst in func.insts(block) {
556        if inst != last && func[inst].opcode != Opcode::IConst {
557            return Err(ARM_DOES_WORK);
558        }
559    }
560    let args: Vec<Value> = match func[last].opcode {
561        Opcode::Jump => {
562            let Some(call) = func.successors(last).next() else { return Err(ARM_DOES_WORK) };
563            let args = func[call.args].to_vec();
564            return Ok((Hands::On(call.block), args));
565        }
566        Opcode::Return => func[func[last].args].to_vec(),
567        _ => return Err(ARM_DOES_WORK),
568    };
569    Ok((Hands::Back, args))
570}
571
572/// The answer as `scale * label + offset`.
573fn arithmetic(builder: &mut Builder<'_>, plan: &Plan, scale: i128, offset: i128) -> Value {
574    let scaled = match scale {
575        0 => builder.iconst(plan.ty, offset),
576        1 => plan.value,
577        scale => {
578            let by = builder.iconst(plan.ty, scale);
579            builder.binary(Opcode::Mul, plan.value, by, Flags::NONE)
580        }
581    };
582    if offset == 0 || scale == 0 {
583        scaled
584    } else {
585        let by = builder.iconst(plan.ty, offset);
586        builder.binary(Opcode::Add, scaled, by, Flags::NONE)
587    }
588}
589
590/// The answer as cell `label - low` of the table called `name`.
591///
592/// The subtraction is at the label's width and wraps, and then the difference is made a word. Only
593/// a label that is a case gets here, so the difference is below the table's length and the
594/// widening is the same with or without a sign, and it is written without one.
595fn look_up(
596    builder: &mut Builder<'_>,
597    plan: &Plan,
598    name: Symbol,
599    low: i128,
600    ty: Type,
601    index_bits: u32,
602) -> Value {
603    let from = if low == 0 {
604        plan.value
605    } else {
606        let by = builder.iconst(plan.ty, low);
607        builder.binary(Opcode::Sub, plan.value, by, Flags::NONE)
608    };
609    let word = Type::int(index_bits);
610    let index = match plan.ty.bits().cmp(&index_bits) {
611        Ordering::Less => builder.unary(Opcode::ZExt, from, word),
612        Ordering::Greater => builder.unary(Opcode::Trunc, from, word),
613        Ordering::Equal => from,
614    };
615    let bytes = ty.bits() / 8;
616    let distance = if bytes == 1 {
617        index
618    } else {
619        let by = builder.iconst(word, i128::from(bytes));
620        builder.binary(Opcode::Mul, index, by, Flags::NONE)
621    };
622    let base = builder.value(
623        InstData { extra: Extra::Symbol(name), ..InstData::new(Opcode::GlobalAddr) },
624        Type::PTR,
625    );
626    let cell = builder.binary(Opcode::PtrAdd, base, distance, Flags::NONE);
627    let info = MemInfo {
628        size: u64::from(bytes),
629        align: bytes,
630        order: MemOrder::NotAtomic,
631        tbaa: None,
632        owns: 0,
633        restrict: Restrict::NONE,
634    };
635    builder.load(ty, cell, info, Flags::NONE)
636}
637
638/// The value of an integer constant, read with its own sign.
639fn constant(func: &Func, value: Value) -> Option<i128> {
640    crate::discharge::constant(func, value)
641}
642
643/// The multiple and the offset that give every answer from its label, when one pair does.
644///
645/// Fitted to the first two labels, which is exact because they are one apart, and then checked at
646/// every label including those two. Checked rather than trusted because the arithmetic that is
647/// about to be written wraps at the type's width, and a fit that is right about the numbers and
648/// wrong about the wrapping is a miscompile that only shows up at the ends of the range.
649fn line(labels: &[i128], answers: &[i128], ty: Type) -> Option<(i128, i128)> {
650    let [first, second, ..] = *labels else { return None };
651    let [low, high, ..] = *answers else { return None };
652    debug_assert_eq!(second - first, 1, "the labels were checked to be consecutive");
653    let scale = high.checked_sub(low)?;
654    let offset = low.checked_sub(scale.checked_mul(first)?)?;
655    for (&label, &answer) in labels.iter().zip(answers) {
656        let want = scale.checked_mul(label)?.checked_add(offset)?;
657        if wrap(want, ty) != answer {
658            return None;
659        }
660    }
661    Some((scale, offset))
662}
663
664/// A number as the machine will hold it at that width, read back with its own sign.
665///
666/// An immediate is stored in exactly the width its type has, so building one and reading it back is
667/// the truncation, and it is the same one every other part of the compiler uses.
668fn wrap(value: i128, ty: Type) -> i128 {
669    Imm::int(value, ty).signed(ty)
670}
671
672/// Writes the block the arms become and points every case edge at it.
673///
674/// `table` is the name of the table a plan for one was given, and `None` for a line.
675fn apply(func: &mut Func, plan: &Plan, table: Option<Symbol>) {
676    let span = func.span(plan.inst);
677    let hit = func.create_block();
678    let mut builder = Builder::new(func, hit).at(span);
679    let answer = match (&plan.how, table) {
680        (&How::Line { scale, offset }, _) => arithmetic(&mut builder, plan, scale, offset),
681        (&How::Table { low, ty, cell, index_bits, .. }, Some(name)) => {
682            let read = look_up(&mut builder, plan, name, low, cell.ty, index_bits);
683            match (cell.ty == ty, cell.signed) {
684                (true, _) => read,
685                (false, true) => builder.unary(Opcode::SExt, read, ty),
686                (false, false) => builder.unary(Opcode::ZExt, read, ty),
687            }
688        }
689        (How::Table { .. }, None) => unreachable!("a table was planned with nowhere to put it"),
690    };
691    let mut args = plan.args.clone();
692    args[plan.answer] = answer;
693    match plan.hands {
694        Hands::On(block) => builder.jump(block, &args),
695        Hands::Back => builder.ret(&args),
696    };
697
698    // Every case edge, and only the case edges: the default is the first target and stays where it
699    // was pointing.
700    let Extra::Switch(info) = func[plan.inst].extra else { return };
701    let empty = func.push_values(&[]);
702    let mut calls: Vec<BlockCall> = func[func[info].targets].to_vec();
703    for call in &mut calls[1..] {
704        // No hint: the cases that had one had one each, and a single edge standing for all of them
705        // cannot carry a number that was true of one arm.
706        *call = BlockCall::new(hit, empty);
707    }
708    let mut cases: Vec<Imm> = func[func[info].cases].to_vec();
709    for &hole in &plan.holes {
710        calls.push(BlockCall::new(hit, empty));
711        cases.push(Imm::int(hole, plan.ty));
712    }
713    let targets = func.push_block_calls(&calls);
714    let cases = func.push_imms(&cases);
715    let info = func.add_switch(rucc_ir::SwitchInfo { targets, cases });
716    func[plan.inst].extra = Extra::Switch(info);
717
718    // The arms are unreachable now. Two labels sharing one arm is a shape that survives the checks
719    // above only when the answer does not depend on the label, so the same block can be here twice.
720    let mut gone = HashSet::new();
721    for &arm in &plan.arms {
722        if gone.insert(arm) {
723            func.remove_block(arm);
724        }
725    }
726}
727
728#[cfg(test)]
729mod tests {
730    use std::collections::{HashMap, HashSet};
731
732    use rucc_base::Interner;
733    use rucc_cost::Goal;
734    use rucc_ir::{Block, Builder, Func, Opcode, Signature, Type, Value};
735
736    use super::SwitchConv;
737    use crate::stats::Kind;
738    use crate::{Fuel, Pass, ReadOnly, Stats, Table};
739
740    /// The width everything here switches on and answers in, unless a test says otherwise.
741    fn i32() -> Type {
742        Type::int(32)
743    }
744
745    /// Runs the pass with as much fuel as it wants.
746    fn convert(func: &mut Func) -> Stats {
747        SwitchConv.run(func, &mut crate::machine::fixtures::analyses(), &mut Fuel::unlimited())
748    }
749
750    /// Runs the pass the way the pipeline does, with a place for tables, and hands back the
751    /// tables it asked for.
752    fn tabled(func: &mut Func) -> (Stats, Vec<Table>) {
753        tabled_for(func, Goal::Speed)
754    }
755
756    /// The same for a goal, which is what decides how wide a cell is.
757    fn tabled_for(func: &mut Func, goal: Goal) -> (Stats, Vec<Table>) {
758        let mut names = Interner::new();
759        let taken = HashSet::new();
760        let mut data = ReadOnly::new(&mut names, &taken, 64, 0);
761        let mut an = crate::Analyses::new(crate::Machine::with(None, goal));
762        let stats = SwitchConv.run_emitting(func, &mut an, &mut Fuel::unlimited(), &mut data);
763        (stats, data.into_tables())
764    }
765
766    /// A function that switches on its parameter and returns a constant per label.
767    ///
768    /// The default returns a constant of its own that is not on any line these tests fit, so a
769    /// test that says the pass fired is saying it fired on the cases and not on the whole thing.
770    fn returning(ty: Type, labels: &[i128], answers: &[i128]) -> Func {
771        let mut names = Interner::new();
772        let mut func = Func::new(names.intern("f"), Signature::new());
773        let head = func.create_block();
774        let value = func.append_param(head, ty);
775        let default = func.create_block();
776        let arms: Vec<Block> = answers.iter().map(|_| func.create_block()).collect();
777        for (&arm, &answer) in arms.iter().zip(answers) {
778            let mut build = Builder::new(&mut func, arm);
779            let it = build.iconst(ty, answer);
780            build.ret(&[it]);
781        }
782        let mut build = Builder::new(&mut func, default);
783        let it = build.iconst(ty, 999);
784        build.ret(&[it]);
785        let cases: Vec<(i128, Block)> = labels.iter().copied().zip(arms.iter().copied()).collect();
786        Builder::new(&mut func, head).switch(value, default, &cases);
787        func
788    }
789
790    /// The blocks every case edge goes to, which is one block when the pass has fired.
791    fn cases(func: &Func) -> Vec<usize> {
792        let head = func.entry().expect("a function with blocks in it");
793        let term = func.terminator(head).expect("a head block has one");
794        func.successors(term).skip(1).map(|call| call.block.index()).collect()
795    }
796
797    /// The block every case edge goes to, when there is exactly one of them.
798    fn arm(func: &Func) -> Block {
799        let blocks = cases(func);
800        let first = blocks[0];
801        assert!(blocks.iter().all(|&block| block == first), "the case edges did not all move");
802        Block::from_usize(first)
803    }
804
805    /// The opcodes a block holds, in order.
806    fn opcodes(func: &Func, block: Block) -> Vec<Opcode> {
807        func.insts(block).map(|inst| func[inst].opcode).collect()
808    }
809
810    /// What the block the case edges go to answers, given that label.
811    ///
812    /// An interpreter of exactly the three instructions this pass writes, because what the pass
813    /// has to get right is the number and not the shape. Anything else in the block is a test
814    /// that has drifted away from what it is testing, so it stops rather than guesses.
815    fn answer(func: &Func, block: Block, label: i128) -> i128 {
816        looked_up(func, block, label, &[])
817    }
818
819    /// What the block the case edges go to answers, given that label and the tables it may read.
820    ///
821    /// The address of a table is its cell zero counted in bytes, which is all a load from one
822    /// needs, and a load stops the test if it is not on a cell or not inside the table.
823    fn looked_up(func: &Func, block: Block, label: i128, tables: &[Table]) -> i128 {
824        let head = func.entry().expect("a function with blocks in it");
825        let mut values: HashMap<Value, i128> = HashMap::new();
826        values.insert(func[head].params[0], label);
827        for inst in func.insts(block) {
828            let data = func[inst];
829            let Some(result) = data.first_result else {
830                let args = func[data.args].to_vec();
831                let handed = match data.opcode {
832                    Opcode::Return => args[0],
833                    Opcode::Jump => {
834                        func[func.successors(inst).next().expect("a jump goes").args][0]
835                    }
836                    other => panic!("a block this pass wrote ends in {other:?}"),
837                };
838                return values[&handed];
839            };
840            let args: Vec<i128> = func[data.args].iter().map(|arg| values[arg]).collect();
841            let it = match data.opcode {
842                Opcode::IConst => {
843                    let (imm, ty) = crate::fold::constant(func, result).expect("a constant is one");
844                    imm.signed(ty)
845                }
846                Opcode::Mul => args[0].wrapping_mul(args[1]),
847                Opcode::Add => args[0].wrapping_add(args[1]),
848                Opcode::Sub => args[0].wrapping_sub(args[1]),
849                // Unsigned, which is what the widening is, and then the width it widens to.
850                Opcode::ZExt => {
851                    let from = func[func[data.args][0]].ty;
852                    super::wrap(args[0], from).rem_euclid(1 << from.bits())
853                }
854                Opcode::SExt => args[0],
855                Opcode::GlobalAddr => 0,
856                Opcode::PtrAdd => args[0] + args[1],
857                Opcode::Load => {
858                    assert_eq!(tables.len(), 1, "a load with no single table to read");
859                    let table = &tables[0];
860                    let bytes = i128::from(table.ty.bits() / 8);
861                    assert_eq!(args[0] % bytes, 0, "a load between two cells");
862                    let at = usize::try_from(args[0] / bytes).expect("a load before the table");
863                    *table.cells.get(at).expect("a load after the table")
864                }
865                other => panic!("this pass does not write {other:?}"),
866            };
867            // An address is a number of bytes into a table and has no width to wrap at.
868            let ty = func[result].ty;
869            values.insert(result, if ty.is_int() { super::wrap(it, ty) } else { it });
870        }
871        panic!("a block with no terminator");
872    }
873
874    /// Whether the pass says it changed the function.
875    fn fired(stats: &Stats) -> bool {
876        stats.total(Kind::Optimized) > 0
877    }
878
879    #[test]
880    fn labels_that_run_with_their_answers_become_one_addition() {
881        let mut func = returning(i32(), &[0, 1, 2, 3], &[1, 2, 3, 4]);
882        assert!(fired(&convert(&mut func)));
883        let arm = arm(&func);
884        assert_eq!(opcodes(&func, arm), [Opcode::IConst, Opcode::Add, Opcode::Return]);
885        for label in 0..4 {
886            assert_eq!(answer(&func, arm, label), label + 1);
887        }
888    }
889
890    #[test]
891    fn answers_that_are_a_multiple_of_the_label_become_a_multiplication() {
892        let mut func = returning(i32(), &[3, 4, 5, 6], &[30, 40, 50, 60]);
893        assert!(fired(&convert(&mut func)));
894        let arm = arm(&func);
895        assert_eq!(opcodes(&func, arm), [Opcode::IConst, Opcode::Mul, Opcode::Return]);
896        for label in 3..7 {
897            assert_eq!(answer(&func, arm, label), label * 10);
898        }
899    }
900
901    #[test]
902    fn answers_that_are_all_the_same_become_the_constant_they_all_were() {
903        let mut func = returning(i32(), &[7, 8, 9, 10], &[9, 9, 9, 9]);
904        assert!(fired(&convert(&mut func)));
905        let arm = arm(&func);
906        assert_eq!(opcodes(&func, arm), [Opcode::IConst, Opcode::Return]);
907        assert_eq!(answer(&func, arm, 8), 9);
908    }
909
910    #[test]
911    fn labels_that_run_below_zero_are_a_run_like_any_other() {
912        let mut func = returning(i32(), &[-2, -1, 0, 1], &[-4, -2, 0, 2]);
913        assert!(fired(&convert(&mut func)));
914        let arm = arm(&func);
915        for label in -2..2 {
916            assert_eq!(answer(&func, arm, label), label * 2);
917        }
918    }
919
920    /// The line has to hold at the type's width and not at the arithmetic's.
921    ///
922    /// A hundred times two is two hundred, which is not a number an `i8` holds, and the answer the
923    /// program gave at that label is what two hundred comes to there. The pass writes a
924    /// multiplication with no flags on it, which wraps the same way, so this is a fit and not a
925    /// refusal, and the number is the point of the test.
926    #[test]
927    fn a_line_that_only_holds_by_wrapping_still_holds() {
928        let ty = Type::int(8);
929        let mut func = returning(ty, &[0, 1, 2], &[0, 100, -56]);
930        assert!(fired(&convert(&mut func)));
931        let arm = arm(&func);
932        assert_eq!(answer(&func, arm, 2), -56);
933    }
934
935    #[test]
936    fn labels_with_a_hole_in_them_are_left_alone_where_no_table_can_be_made() {
937        let mut func = returning(i32(), &[0, 1, 3], &[1, 2, 4]);
938        assert!(!fired(&convert(&mut func)));
939        assert_eq!(cases(&func).len(), 3);
940    }
941
942    #[test]
943    fn answers_that_are_not_a_line_are_left_alone_where_no_table_can_be_made() {
944        let mut func = returning(i32(), &[0, 1, 2], &[5, 9, 2]);
945        assert!(!fired(&convert(&mut func)));
946    }
947
948    #[test]
949    fn two_labels_are_not_enough_to_pay_for_the_arithmetic() {
950        let mut func = returning(i32(), &[0, 1], &[1, 2]);
951        assert!(!fired(&convert(&mut func)));
952    }
953
954    #[test]
955    fn an_answer_wider_than_its_label_is_left_alone_where_no_table_can_be_made() {
956        let mut names = Interner::new();
957        let mut func = Func::new(names.intern("f"), Signature::new());
958        let head = func.create_block();
959        let value = func.append_param(head, i32());
960        let default = func.create_block();
961        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
962        for (index, &arm) in arms.iter().enumerate() {
963            let mut build = Builder::new(&mut func, arm);
964            let it = build.iconst(Type::int(64), index as i128 + 1);
965            build.ret(&[it]);
966        }
967        let mut build = Builder::new(&mut func, default);
968        let it = build.iconst(Type::int(64), 0);
969        build.ret(&[it]);
970        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
971        Builder::new(&mut func, head).switch(value, default, &cases);
972        assert!(!fired(&convert(&mut func)));
973    }
974
975    #[test]
976    fn an_arm_something_else_reaches_is_left_alone() {
977        let mut func = returning(i32(), &[0, 1, 2], &[1, 2, 3]);
978        // The default jumps into the first arm instead of returning, so the arm is a block two
979        // edges arrive at and is not one this may take away.
980        let default = Block::from_usize(1);
981        let arm = Block::from_usize(2);
982        let term = func.terminator(default).expect("the default returns");
983        func.remove_inst(term);
984        Builder::new(&mut func, default).jump(arm, &[]);
985        assert!(!fired(&convert(&mut func)));
986    }
987
988    #[test]
989    fn an_arm_that_is_also_the_default_is_left_alone() {
990        let mut names = Interner::new();
991        let mut func = Func::new(names.intern("f"), Signature::new());
992        let head = func.create_block();
993        let value = func.append_param(head, i32());
994        let shared = func.create_block();
995        let mut build = Builder::new(&mut func, shared);
996        let it = build.iconst(i32(), 1);
997        build.ret(&[it]);
998        let others: Vec<Block> = (0..2).map(|_| func.create_block()).collect();
999        for (index, &arm) in others.iter().enumerate() {
1000            let mut build = Builder::new(&mut func, arm);
1001            let it = build.iconst(i32(), index as i128 + 2);
1002            build.ret(&[it]);
1003        }
1004        let cases = [(0, shared), (1, others[0]), (2, others[1])];
1005        Builder::new(&mut func, head).switch(value, shared, &cases);
1006        assert!(!fired(&convert(&mut func)));
1007    }
1008
1009    #[test]
1010    fn arms_that_join_keep_what_they_pass_beside_the_answer() {
1011        let mut names = Interner::new();
1012        let mut func = Func::new(names.intern("f"), Signature::new());
1013        let head = func.create_block();
1014        let value = func.append_param(head, i32());
1015        let alongside = func.append_param(head, i32());
1016        let join = func.create_block();
1017        let handed = func.append_param(join, i32());
1018        let carried = func.append_param(join, i32());
1019        Builder::new(&mut func, join).ret(&[handed, carried]);
1020        let default = func.create_block();
1021        let mut build = Builder::new(&mut func, default);
1022        let it = build.iconst(i32(), 999);
1023        build.jump(join, &[it, alongside]);
1024        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1025        for (index, &arm) in arms.iter().enumerate() {
1026            let mut build = Builder::new(&mut func, arm);
1027            let it = build.iconst(i32(), index as i128 + 1);
1028            build.jump(join, &[it, alongside]);
1029        }
1030        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1031        Builder::new(&mut func, head).switch(value, default, &cases);
1032        assert!(fired(&convert(&mut func)));
1033
1034        let arm = arm(&func);
1035        assert_eq!(answer(&func, arm, 2), 3);
1036        // The second argument is what it always was, which is the parameter every arm passed.
1037        let term = func.terminator(arm).expect("the block ends in a jump");
1038        let call = func.successors(term).next().expect("a jump goes somewhere");
1039        assert_eq!(func[call.args][1], alongside);
1040    }
1041
1042    #[test]
1043    fn arms_that_hand_on_two_different_things_are_left_alone() {
1044        let mut names = Interner::new();
1045        let mut func = Func::new(names.intern("f"), Signature::new());
1046        let head = func.create_block();
1047        let value = func.append_param(head, i32());
1048        let join = func.create_block();
1049        let first = func.append_param(join, i32());
1050        let second = func.append_param(join, i32());
1051        Builder::new(&mut func, join).ret(&[first, second]);
1052        let default = func.create_block();
1053        let mut build = Builder::new(&mut func, default);
1054        let it = build.iconst(i32(), 999);
1055        build.jump(join, &[it, it]);
1056        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1057        for (index, &arm) in arms.iter().enumerate() {
1058            let mut build = Builder::new(&mut func, arm);
1059            let one = build.iconst(i32(), index as i128 + 1);
1060            let two = build.iconst(i32(), index as i128 + 10);
1061            build.jump(join, &[one, two]);
1062        }
1063        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1064        Builder::new(&mut func, head).switch(value, default, &cases);
1065        assert!(!fired(&convert(&mut func)));
1066    }
1067
1068    #[test]
1069    fn an_arm_that_does_something_is_left_alone() {
1070        let mut names = Interner::new();
1071        let mut func = Func::new(names.intern("f"), Signature::new());
1072        let head = func.create_block();
1073        let value = func.append_param(head, i32());
1074        let default = func.create_block();
1075        let mut build = Builder::new(&mut func, default);
1076        let it = build.iconst(i32(), 999);
1077        build.ret(&[it]);
1078        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1079        for (index, &arm) in arms.iter().enumerate() {
1080            let mut build = Builder::new(&mut func, arm);
1081            let it = build.iconst(i32(), index as i128 + 1);
1082            // An addition the arm did, which is work the answer would have been left without.
1083            let sum = build.binary(Opcode::Add, it, value, rucc_ir::Flags::NONE);
1084            build.ret(&[sum]);
1085        }
1086        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1087        Builder::new(&mut func, head).switch(value, default, &cases);
1088        assert!(!fired(&convert(&mut func)));
1089    }
1090
1091    #[test]
1092    fn the_default_goes_where_it_went() {
1093        let mut func = returning(i32(), &[0, 1, 2, 3], &[1, 2, 3, 4]);
1094        let head = func.entry().expect("a function with blocks in it");
1095        let before = func.terminator(head).expect("a head block has one");
1096        let was = func.successors(before).next().expect("a switch has a default").block;
1097        assert!(fired(&convert(&mut func)));
1098        let after = func.terminator(head).expect("a head block has one");
1099        let now = func.successors(after).next().expect("a switch has a default").block;
1100        assert_eq!(was, now, "the default moved");
1101    }
1102
1103    /// The opcodes of a block that answers from a table, from the label at zero.
1104    const LOOKUP: [Opcode; 6] = [
1105        Opcode::ZExt,
1106        Opcode::IConst,
1107        Opcode::Mul,
1108        Opcode::GlobalAddr,
1109        Opcode::PtrAdd,
1110        Opcode::Load,
1111    ];
1112
1113    #[test]
1114    fn answers_that_are_not_a_line_are_one_load_from_a_table() {
1115        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, 9, 2, 7]);
1116        let (stats, tables) = tabled(&mut func);
1117        assert!(fired(&stats));
1118        assert_eq!(tables.len(), 1);
1119        assert_eq!(tables[0].ty, i32());
1120        assert_eq!(tables[0].cells, [5, 9, 2, 7]);
1121        let arm = arm(&func);
1122        let mut want = LOOKUP.to_vec();
1123        want.push(Opcode::Return);
1124        assert_eq!(opcodes(&func, arm), want);
1125        for (label, answer) in [(0, 5), (1, 9), (2, 2), (3, 7)] {
1126            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1127        }
1128    }
1129
1130    /// A hole gets the default's answer and a case of its own, when the default only gives one.
1131    ///
1132    /// The default here returns 999 and nothing else, so the value in the hole reads 999 out of
1133    /// the table and gets what it got before, and the labels are one run with no hole in it.
1134    #[test]
1135    fn a_hole_is_filled_with_what_a_default_that_only_answers_gives() {
1136        let mut func = returning(i32(), &[1, 2, 4, 5], &[10, 20, 40, 55]);
1137        let head = func.entry().expect("a function with blocks in it");
1138        let before = func.terminator(head).expect("a head block has one");
1139        let default = func.successors(before).next().expect("a switch has a default").block;
1140        let (stats, tables) = tabled(&mut func);
1141        assert!(fired(&stats));
1142        assert_eq!(tables[0].cells, [10, 20, 999, 40, 55]);
1143        assert_eq!(cases(&func).len(), 5, "the hole was not given a case");
1144        let after = func.terminator(head).expect("a head block has one");
1145        assert_eq!(func.successors(after).next().map(|call| call.block), Some(default));
1146        let arm = arm(&func);
1147        for (label, answer) in [(1, 10), (2, 20), (3, 999), (4, 40), (5, 55)] {
1148            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1149        }
1150    }
1151
1152    /// A hole is a cell nothing reads when the default does something other than answer.
1153    ///
1154    /// This default returns the label, which is no constant, so the value in the hole has to
1155    /// keep going to it down the edge it always went down.
1156    #[test]
1157    fn a_hole_still_goes_to_a_default_that_does_more_than_answer() {
1158        let mut func = returning(i32(), &[1, 2, 4, 5], &[10, 20, 40, 55]);
1159        let head = func.entry().expect("a function with blocks in it");
1160        let before = func.terminator(head).expect("a head block has one");
1161        let default = func.successors(before).next().expect("a switch has a default").block;
1162        let label = func[func[before].args][0];
1163        let ret = func.terminator(default).expect("the default returns");
1164        func.remove_inst(ret);
1165        Builder::new(&mut func, default).ret(&[label]);
1166        let (stats, tables) = tabled(&mut func);
1167        assert!(fired(&stats));
1168        assert_eq!(tables[0].cells, [10, 20, 0, 40, 55]);
1169        assert_eq!(cases(&func).len(), 4, "a hole was given a case");
1170        let after = func.terminator(head).expect("a head block has one");
1171        assert_eq!(func.successors(after).next().map(|call| call.block), Some(default));
1172        let arm = arm(&func);
1173        for (label, answer) in [(1, 10), (2, 20), (4, 40), (5, 55)] {
1174            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1175        }
1176    }
1177
1178    /// Cell zero is the lowest label, which here is below zero and reached by wrapping.
1179    ///
1180    /// Every label of a `signed char` is tried, which is the whole of what can reach the block
1181    /// and the whole of what can go wrong with the subtraction and the widening after it.
1182    #[test]
1183    fn labels_below_zero_index_from_the_lowest_of_them() {
1184        let ty = Type::int(8);
1185        let labels = [-128, -3, -1, 0, 2, 127];
1186        let answers = [7, -5, 11, 3, -100, 42];
1187        let mut func = returning(ty, &labels, &answers);
1188        // Far apart at the ends, so a table is only allowed because the ratio is eight to one and
1189        // there are six labels: two hundred and fifty six cells is more than forty eight.
1190        let (stats, _) = tabled(&mut func);
1191        assert!(!fired(&stats), "a table of mostly holes was made");
1192
1193        let labels = [-3, -2, -1, 0, 2];
1194        let answers = [7, -5, 11, 3, -100];
1195        let mut func = returning(ty, &labels, &answers);
1196        let (stats, tables) = tabled(&mut func);
1197        assert!(fired(&stats));
1198        // The default returns 999, which is -25 as a `signed char`, and the hole at 1 is given it.
1199        assert_eq!(tables[0].cells, [7, -5, 11, 3, -25, -100]);
1200        let arm = arm(&func);
1201        assert_eq!(opcodes(&func, arm)[..2], [Opcode::IConst, Opcode::Sub]);
1202        for (&label, &answer) in labels.iter().zip(&answers).chain([(&1, &-25)]) {
1203            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1204        }
1205    }
1206
1207    /// An `int` label and a `long` answer, which a line declines and a table does not mind.
1208    #[test]
1209    fn an_answer_wider_than_its_label_is_a_table_of_the_wider_type() {
1210        let mut names = Interner::new();
1211        let answers = [1i128 << 40, 3, -1, 1 << 33];
1212        let mut func = Func::new(names.intern("f"), Signature::new());
1213        let head = func.create_block();
1214        let value = func.append_param(head, i32());
1215        let default = func.create_block();
1216        let arms: Vec<Block> = answers.iter().map(|_| func.create_block()).collect();
1217        for (&arm, &answer) in arms.iter().zip(&answers) {
1218            let mut build = Builder::new(&mut func, arm);
1219            let it = build.iconst(Type::int(64), answer);
1220            build.ret(&[it]);
1221        }
1222        let mut build = Builder::new(&mut func, default);
1223        let it = build.iconst(Type::int(64), 0);
1224        build.ret(&[it]);
1225        let cases: Vec<(i128, Block)> = (10..14).zip(arms.iter().copied()).collect();
1226        Builder::new(&mut func, head).switch(value, default, &cases);
1227        let (stats, tables) = tabled(&mut func);
1228        assert!(fired(&stats));
1229        assert_eq!(tables[0].ty, Type::int(64));
1230        let arm = arm(&func);
1231        for (label, &answer) in (10..14).zip(&answers) {
1232            assert_eq!(looked_up(&func, arm, label, &tables), answer);
1233        }
1234    }
1235
1236    #[test]
1237    fn labels_too_far_apart_for_a_table_are_left_alone() {
1238        let mut func = returning(i32(), &[0, 100, 200], &[1, 5, 3]);
1239        let (stats, tables) = tabled(&mut func);
1240        assert!(!fired(&stats));
1241        assert!(tables.is_empty());
1242    }
1243
1244    #[test]
1245    fn a_line_is_still_arithmetic_where_a_table_could_be_made() {
1246        let mut func = returning(i32(), &[0, 1, 2, 3], &[1, 2, 3, 4]);
1247        let (stats, tables) = tabled(&mut func);
1248        assert!(fired(&stats));
1249        assert!(tables.is_empty(), "a table was made for a line");
1250    }
1251
1252    #[test]
1253    fn a_label_wider_than_a_word_gets_no_table() {
1254        let mut func = returning(Type::int(128), &[0, 1, 2, 3], &[5, 9, 2, 7]);
1255        let (stats, tables) = tabled(&mut func);
1256        assert!(!fired(&stats));
1257        assert!(tables.is_empty());
1258    }
1259
1260    /// The answer is in the second place and the first is a constant every arm passes.
1261    ///
1262    /// The first arm's answer used to be read from the first place, because which place the
1263    /// answer is in is not known until a second arm disagrees. Here that reads one at the first
1264    /// label, and one, two and three are a line, so the pass returned one where the program said
1265    /// ten. What it must do is see ten, two and three, which is not a line.
1266    #[test]
1267    fn the_answer_is_read_from_the_place_the_arms_disagree_about() {
1268        let mut names = Interner::new();
1269        let mut func = Func::new(names.intern("f"), Signature::new());
1270        let head = func.create_block();
1271        let value = func.append_param(head, i32());
1272        let join = func.create_block();
1273        let first = func.append_param(join, i32());
1274        let second = func.append_param(join, i32());
1275        Builder::new(&mut func, join).ret(&[second, first]);
1276        let default = func.create_block();
1277        let arms: Vec<Block> = (0..3).map(|_| func.create_block()).collect();
1278        let mut build = Builder::new(&mut func, head);
1279        let one = build.iconst(i32(), 1);
1280        let cases: Vec<(i128, Block)> = (0..3).zip(arms.iter().copied()).collect();
1281        build.switch(value, default, &cases);
1282        let mut build = Builder::new(&mut func, default);
1283        let it = build.iconst(i32(), 999);
1284        build.jump(join, &[one, it]);
1285        for (&arm, answer) in arms.iter().zip([10, 2, 3]) {
1286            let mut build = Builder::new(&mut func, arm);
1287            let it = build.iconst(i32(), answer);
1288            build.jump(join, &[one, it]);
1289        }
1290        assert!(!fired(&convert(&mut func)), "ten, two and three were taken for a line");
1291        let (stats, tables) = tabled(&mut func);
1292        assert!(fired(&stats));
1293        assert_eq!(tables[0].cells, [10, 2, 3]);
1294    }
1295
1296    /// At the size goal a cell is a byte when every answer is one, and the load is widened back.
1297    ///
1298    /// Below zero is widened with a sign and two hundred without one, so both are tried, and the
1299    /// speed goal keeps the answer's own width, which is what gcc 16 does at the two levels.
1300    #[test]
1301    fn a_table_for_size_has_cells_as_narrow_as_its_answers() {
1302        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, -9, 2, 7]);
1303        let (stats, tables) = tabled_for(&mut func, Goal::Size);
1304        assert!(fired(&stats));
1305        assert_eq!(tables[0].ty, Type::int(8));
1306        let at = arm(&func);
1307        assert!(opcodes(&func, at).contains(&Opcode::SExt));
1308        for (label, answer) in [(0, 5), (1, -9), (2, 2), (3, 7)] {
1309            assert_eq!(looked_up(&func, at, label, &tables), answer);
1310        }
1311
1312        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, 200, 2, 255]);
1313        let (_, tables) = tabled_for(&mut func, Goal::Size);
1314        assert_eq!(tables[0].ty, Type::int(8));
1315        let at = arm(&func);
1316        assert!(opcodes(&func, at).contains(&Opcode::ZExt));
1317        for (label, answer) in [(0, 5), (1, 200), (2, 2), (3, 255)] {
1318            assert_eq!(looked_up(&func, at, label, &tables), answer);
1319        }
1320
1321        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, -300, 2, 40000]);
1322        let (_, tables) = tabled_for(&mut func, Goal::Size);
1323        assert_eq!(tables[0].ty, i32(), "a cell narrower than an answer that needs all of it");
1324
1325        let mut func = returning(i32(), &[0, 1, 2, 3], &[5, -9, 2, 7]);
1326        let (_, tables) = tabled_for(&mut func, Goal::Speed);
1327        assert_eq!(tables[0].ty, i32());
1328    }
1329}