Module uring_write 

io_uring arm of archive_write. Linux only — the module is empty elsewhere rather than substituting a pwrite path under an io_uring name. Impl 3 — the io_uring arm: one submission, four ops, kernel-enforced ordering.

SafeWriter buys its durability with four blocking syscalls and four round-trips through the scheduler:

  pwrite(blob)  fsync(blob)  write(journal)  fsync(journal)
     ↑ ring 0      ↑ ring 0       ↑ ring 0        ↑ ring 0     4 syscalls

Here the same four operations are one io_uring_enter(2). Each of the first three SQEs carries IOSQE_IO_LINK, so the kernel will not start an op until its predecessor has completed — the ordering contract is enforced by the kernel rather than by the caller blocking between calls:

  [ Write(blob) ]→[ Fsync(blob) ]→[ WriteFixed(journal) ]→[ Fsync(journal) ]
    IO_LINK         IO_LINK          IO_LINK                (chain end)
  ────────────────────── one io_uring_enter ──────────────────────

That is the same ordering as SafeWriter, not a second one: blob bytes durable before the row that references them. A crash anywhere in the chain leaves orphan payload nobody points at. If a link fails the kernel cancels the rest of the chain with ECANCELED, which is exactly the semantics wanted: no journal row is ever written after a failed blob fsync.

Registered buffers, and the copy that is not being avoided

register_buffers pins a set of stable buffers once, so per-op the kernel skips get_user_pages/put_page on them. That only works for buffers whose address the ring can be told about up front, which the caller’s transient &[u8] is not. So this writer splits the two:

• journal → a registered one-page staging buffer and WriteFixed. This is the buffer that is stable across appends, and it is where the win is real. Crucially it means there is no arrow BufWriter here at all: the serialized IPC message is written into the registered buffer and the kernel is handed that. You cannot have both arrow’s buffered StreamWriter and a registered buffer; the brief says take the io_uring side, and this does.
• blob → plain Write against the caller’s own pointer. Not registered (it cannot be), but also not copied in userspace: the pushed pack bytes go from the caller’s slice straight into the ring.

Saying this plainly matters: the registered buffer removes per-op page pinning on the journal, not a memcpy on the pack.

What a registration COSTS, and why the staging buffer is one page

IORING_REGISTER_BUFFERS pins its pages against RLIMIT_MEMLOCK, and the kernel charges them to user->locked_vm — a counter kept on the user_struct, so it is per-UID and shared by every process that user is running, not per-process and not per-ring (io_uring/rsrc.c: io_account_mem -> __io_account_mem).

Every store gets its own writer, therefore its own ring, therefore its own registration. So the pinned pages are stores × ceil(JOURNAL_STAGING / PAGE_SIZE) and the ceiling is a hard, shared, uid-wide one. The default on oden (and on stock Debian/Ubuntu) is 8 MiB.

MEASURED on oden 2026-08-14, RLIMIT_MEMLOCK 8 MiB, one fresh process per reading, three readings each, identical code with only the buffer size varied:

  registered per ring   rings that fit        what refused the next one
  ───────────────────   ──────────────        ─────────────────────────
  64 KiB (16 pages)     87, 88, 87            IORING_REGISTER_BUFFERS
   4 KiB  (1 page)      397, 422, 419         io_uring_setup

with

  uring: register_buffers: Cannot allocate memory (os error 12)

and every store opened after that refused. ~87 io_uring stores per process, and gunnar’s multiuser_scaling alone provisions 102 users with a repository each; a gunnar serve holds one store — therefore one ring, therefore one registration — per repository it has open. FastWriter and SafeWriter register nothing and have no such ceiling, which is exactly the shape the sweep showed: four workloads and eight forge arms red on both io_uring columns and green on fast and safe, with the index arm varied underneath and making no difference.

Cutting the registration to one page moves that to ~400 rings, a 4.6× ceiling — and past it the thing that refuses is no longer the registration at all but io_uring_setup. A bare ring of [RING_ENTRIES] costs about 21 KiB of the same budget (8 MiB / ~400), so the old writer spent ~21 pages per store and the new one spends ~6. It is not the 16× the buffer sizes suggest, because the ring was always paying five pages of it.

End to end rather than at the writer: 150 whole GitStores on this arm, open at once in one process, each with a real 5 653 302-byte / 2 687-object pack pushed, indexed and read back — 3.4–4.0 s per store, oden 2026-08-14, load average 15–30. Every one of them past the old ceiling.

…and every number in the two paragraphs above was measured in the WRONG

PROCESS: the page you register is charged as the huge page it sits in

Found 2026-08-14, after the ceiling above had already been “fixed” once.

io_buffer_account_pin does not charge the pages you named. It charges the compound_head of each of them. If the 4 KiB you register happens to live inside a transparent huge page, the kernel charges the whole 2 MiB — 512 pages, not one (io_uring/rsrc.c: io_buffer_account_pin, the PageCompound branch: imu->acct_pages += page_size(hpage) >> PAGE_SHIFT).

MEASURED on oden 2026-08-14, RLIMIT_MEMLOCK 8 MiB, transparent_hugepage = [madvise], one registration of exactly one page, charge read back off the kernel by binary-searching what could still be registered afterwards:

  where the one page came from                       pages charged
  ────────────────────────────────────────────────   ─────────────
  its own anonymous mmap, MADV_NOHUGEPAGE                        1
  glibc malloc(4096) in a small C program                        2
  inside a MADV_HUGEPAGE arena                                 512

Box<[u8]> — what this file registered until now — is whatever the process allocator gives you, and gunnar serve runs on mimalloc, which madvise(MADV_HUGEPAGE)s its arenas. Measured on the running server: AnonHugePages: 16384 kB in one VMA, and every staging buffer allocated out of it. So the real ceiling in the process that matters was not ~400 stores. It was three:

  RLIMIT_MEMLOCK    io_uring stores one `gunnar serve` could open
  ──────────────    ─────────────────────────────────────────────
  8 MiB                             3   (+ the control store = 4 × 2 MiB)
  4 MiB                             1

— measured by pushing to distinct repositories one at a time until the server refused, oden 2026-08-14. Four huge pages fit in 8 MiB and that is the whole arithmetic. The fourth push onwards died with register_buffers (4096 B, 1 page(s)): Cannot allocate memory, which reads like the ceiling this file already documents and is a different one: it is not how MANY pages are registered, it is WHOSE page each one is.

That is why [Staging] does not ask the allocator for the buffer. It takes its own one-page anonymous mapping and madvise(MADV_NOHUGEPAGE)s it, so the registration is charged one page whatever the executable’s allocator does — and the “~400 rings” arithmetic above becomes true instead of merely plausible. enough_uring_writers_for_a_population_coexist_in_one_process could never have caught this: a cargo test binary is on the system allocator, whose 4 KiB allocations are not huge-page backed, so the guard measured a process that did not have the bug. The guard that does catch it is [tests::a_registration_costs_one_page_and_not_the_huge_page_it_might_sit_in], which measures the CHARGE rather than the count.

It is tighter than “N stores at once”, because the kernel reclaims lazily

io_uring teardown runs off a workqueue after the ring’s last descriptor closes, so the pages of a dropped writer stay charged for a while. Opening and dropping one at a time and holding nothing, the 64 KiB writer was refused at the 63rd. A server that opens a store per repository and lets it go — gunnar’s store_cache map holds Weaks, so that is its shape while seeding — therefore hits this after about sixty repositories, which is where the sweep’s vs_forge_* arms died seeding.

The accounting is on the user_struct, so it also crosses process boundaries: a fresh process was refused its second ring while a previous test process’s rings were still being torn down. That is also why there is no guard on the serial shape — one written against it failed at 54 of 160 purely on the residue of the guard that ran before it, and a test whose verdict depends on what else the box did in the last few seconds is not a guard. The serial ceiling is bounded by the same per-store cost the guards below do measure, and it clears with pacing: 600 rings created and dropped 5 ms apart did not fail once.

If the kernel cannot do it

UringWriter::create probes for IORING_OP_WRITE, IORING_OP_WRITE_FIXED and IORING_OP_FSYNC and fails with a named error if any is missing, rather than silently degrading to pwrite and reporting an io_uring number that is not one. Same for io_uring_setup being blocked outright (kernel.io_uring_disabled=2, seccomp, a container without the syscall).

Structs

UringWriter

io_uring writer: write → fsync → journal → fsync as one linked submission.