Module inline 

The inliner, for the calls gcc inlines at every level, those to an always_inline function, and from -O1 up for the calls to a small function declared inline.

Design: spec/optimizer/33-inlining.md, and tamnd/rucc#392.

extern inline __attribute__((always_inline, gnu_inline)) int
printf (const char *fmt, ...)
{
  return __printf_chk (1, fmt, __builtin_va_arg_pack ());
}

always_inline is not a hint. gcc inlines every direct call to one of these whatever the level, -O0 included, and a header that defines one is relying on that in two ways. The first is that the definition above is an inline definition, so no unit is obliged to have a copy of it out of line. The second is __builtin_va_arg_pack, which stands for the anonymous arguments of the call the body was inlined into and means nothing anywhere else. glibc’s fortified headers are written this way, and so is va-arg-pack-1.c in the torture suite.

So this runs first, before anything else in the pipeline and at every level, since objsize right behind it wants to see the caller’s objects through the wrapper’s parameters. Each function is settled before it is inlined anywhere, so a body goes in with the always_inline calls inside it already gone, and a function that reaches itself again through such calls is refused rather than unrolled.

A call is spliced in place. The block it is in is split after it, and the part after becomes a block that takes the call’s results as parameters. The callee’s blocks are copied in with every side table they point into, its entry is jumped to with the arguments, a return becomes a jump to the second half, and an alloca of a fixed size goes to the caller’s entry block, where the verifier wants it.

A va_arg_pack in the callee is the last argument of a call, since that is the one place sema lets it be written, and it is replaced by the anonymous arguments of the call being inlined. What makes that more than a list splice is the calling convention: the lowering has already decided which of those arguments go in registers and which go in memory, and it decided for the outer call. SysV x86-64 puts all of a structure in registers or none of it, so a structure that travelled as two registers in the outer call and would find only one left in the inner one goes to memory there instead, stored to a slot in the caller just ahead of the call. The one case refused is the other way round, a small structure in memory that the inner call would have room for. A call that is refused stays a call. A va_arg_pack_len is replaced by the count.

What is left is the out of line copy of a function that still holds either of them, which is a function nothing can emit. It becomes a declaration, which is gcc’s answer too: gcc emits nothing for an inline definition, so a call the inliner left goes to whatever the rest of the program defines under the name, which for a glibc wrapper is the library function.

From -O1 up the same splice takes a call to a function declared inline whose body, once its own calls are settled, is no larger than max-inline-insns-single, the limit gcc gives such a callee. It is the declared half of gcc’s early inliner and not the rest of it: a function nobody declared inline is left alone however small it is, and nothing here weighs the call against the growth the way section 33.4 wants the later inliner to. What it is for is the code after it. A __builtin_constant_p in the body of such a function asks about a parameter, and only once the body is where the call was can the answer be the constant the caller passed, which is what gcc answers and what bcp-1.c checks. -fno-inline turns this half off and leaves always_inline alone, which is what the flag does in gcc.

A body that takes the address of one of its own labels is copied with the label, so each copy has an address of its own, which is what gcc does and what 990208-1.c checks. A body that jumps to such an address, or whose labels a static table holds, is refused, since the copy would still be reaching into the original.

Enums

InlineFailure

Why a call to an always_inline function was not inlined.

Constants

NAME

What the step calls itself in a remark, and the name -fno-inline turns the declared half off by.

Functions

run

Inlines every call to an always_inline function that can be, and with a limit every call to a function declared inline whose body is no larger than that, and says what it did where.