Module retry 

The read modify writes the machine has no single instruction for, as a loop around the compare and exchange.

Design: spec/10-backend.md section 10.5, which names this as the third exemption from the rule that every lowering is a rule in the table.

What is here and why it is not a rule

Opcode::AtomicRmw carries thirteen operations. x86-64 has an instruction for three of them: xchg puts a value there, lock xadd adds one, and a subtraction is the same instruction over the negated operand. The other ten have no instruction at any width, and what stands in for one is the loop every architecture manual writes out by hand: read what is there, work out what should be there instead, put it back if nothing else got in first, and go round again when something did.

That loop is blocks, and blocks are why this is a pass rather than a rule. A rule rewrites one instruction into instructions; it has no way to say that control leaves a block here and arrives somewhere else. So the shape is built in the IR, before anything below has been told what the blocks are, and by the time the selector sees it there is nothing left but a compare and exchange it already has a rule for.

Where it runs

Straight after switch::switches and before expand::orderings.

After the switches because both of these create blocks and expand may not: every pass in expand.rs rewrites an instruction in the place it stands, and the two passes that change the shape of the control flow are kept together at the front where the function is still the one the optimizer handed over.

Before the orderings because the head of the loop reads the address with an atomic_load, and it is expand::orderings that turns that into the plain load this machine does anyway. Running the other way round would leave an ordered access nothing below understands.

What the loop is

For old = atomic_rmw op, addr, operand in a block, with tail for whatever followed it:

head:                              ; what was in front of the instruction
  first = atomic_load ty, addr     ; relaxed, because the compare and exchange carries the order
  jump spin(first)
spin(seen):
  want = <op> seen, operand
  got, ok = cmpxchg addr, seen, want
  br_if ok, done(seen), spin(got)
done(before):                      ; `tail`, with every use of `old` reading `before`

The value the loop answers is the one that was there before, which is what AtomicRmw answers, so the value handed to done is seen and not want. A name that asked for the value afterwards got the arithmetic that works one out from the other back in rucc-lower, over the result of the instruction this pass is rewriting, and that arithmetic is in tail and needs nothing from here.

The failing edge carries got rather than going back to the load. That is the whole reason the compare and exchange answers what it found: a second read would be a second chance to be wrong, and the value the exchange saw is the freshest one there is.

The edge from spin to itself is critical, since spin has two ways out and is arrived at two ways. Nothing here splits it, because split::critical runs below and splitting it twice is worse than splitting it once.

What it leaves alone

An exchange, an addition and a subtraction, because those three have instructions and a loop would be slower and larger for no reason. Anything whose value is not an integer the machine compares and exchanges at, which is the two floating operations: a compare and exchange of a float wants the value carried through an integer of the same width, an eighty bit float has no such width, and until that is worked out the refusal in lower.rs is the honest answer.

Functions

loops

Rewrites every read modify write this machine has no instruction for into a loop around the compare and exchange, and leaves the rest of them alone.