Module buffer 

A buffer that grows without moving.

The problem this solves

A KV cache grows a token at a time. The obvious implementation reallocates and copies, and that costs three things beyond the copy itself:

• The address changes. Anything holding the old pointer is wrong. A captured device graph recorded the old address, so the capture is dead and has to be retaken.
• Peak memory doubles at the seam. Old and new are live at once, so a grow can fail at a tier that is merely full rather than over-subscribed.
• The copy is O(everything so far), paid on every growth step.

Reserving address space is free — it costs no memory, only address bits, and 64-bit processes have plenty. So reserve for the largest the buffer could ever be, and commit physical pages behind it as it actually grows. The base address is fixed at reservation and never moves again.

What it costs instead

Growth rounds up to the platform’s mapping granularity — 64 KiB on Windows, a page on unix — so a buffer that grows by a hundred bytes commits a whole granule. That is a real overhead on small buffers and irrelevant on the large ones this exists for.

Leasing

Only committed bytes are leased. Reserving is not an allocation and charging for it would make a governor refuse a buffer that will never use the address space it reserved — which is precisely the arrangement that makes reserving generously safe.

Structs

VirtualBuffer

A growable region whose base address never changes.

Enums

VirtualBufferError

What went wrong growing or shrinking a VirtualBuffer.