Crate yo_alloc 

A global allocator that turns an accidental heap allocation on a shard path into a crash.

Y7 says there is no global allocator call on a command path. That is easy to write down and impossible to keep by review once more than one person is in the codebase, because the allocating constructs in Rust are the comfortable ones: format!, to_vec, collect, Box::new, a Vec that grows, a String built to make an error message. Each costs tens of nanoseconds against a 150 ns budget, and none of them looks wrong in a diff.

So the rule is enforced instead of reviewed. A shard thread marks itself enter_no_alloc before the command loop, and from that point any allocation aborts the process with a message naming the size. Setup, arena growth and anything else that legitimately needs the heap wraps itself in allow, which is a visible, greppable, deliberate act.

Cost when it is off

The check is one thread local load and a branch, on a path that already calls into the system allocator. It is not measurable next to malloc. Non shard threads never set the flag and pay the same single branch.

The claims are cheaper still. allow and the three named forms of it come down to a relaxed load of a static, false in any process that has never armed a thread, which is every shipped binary. That is what lets a claim sit inside a loop where the growth actually happens rather than being hoisted to the top of a function it does not describe.

Three modes, and why an abort is not the only one

An abort tells you about one violation per run, which is the wrong tool for finding out how many there are. Nothing in this project had been checked against Y7 since the rule was written down, so the first question is not “stop on the first one” but “what is the list”.

Mode::Report answers that. It suspends the check, captures a backtrace, prints each distinct site once and counts the repeats, and lets the allocation through. It allocates while it does this, on purpose and with the check turned off around it, because a debugging mode that cannot use the heap cannot tell you where you are.

Mode::Abort is the rule as written, for a build that is expected to be clean. Mode::Off is the default, so installing the allocator does not change what a shipped binary does until somebody asks for it.

Off costs nothing rather than costing a branch, because nothing arms the thread: guard is where the mode is read, and when it is off the thread flag is never set and the allocator’s check is the same false it would be on any other thread.

Getting the list

yodb installs this and pump wraps its dispatch in guard, so YO_ALLOC=report yodb serve answers the question, and cargo xtask alloc is that with a server, a workload and a parser around it. It builds a debug yodb, drives it with about nine thousand commands covering every type and prints one line per distinct site. Debug on purpose: release inlines the interesting frames into each other and the report comes back naming serve_command for everything.

Run it before touching anything here, and believe it over anything written down. Twice during this work the list contradicted what was already recorded about what was left, and both times the list was right. A report mode exists so that this is measured rather than argued about, and that only works if it is the list somebody actually looks at.

It is also the gate. An empty list exits 0 and anything else exits 1, and ci.yml runs it on every push, so a new allocation on a command path fails the pull request that added it rather than being found a release later.

What it found, and the four piles it sorted into

The first arming reported 31 distinct sites, and they were never one problem. The list is empty now, and it got there four different ways.

Most of them were the first touch of a key. Creating a set, hash, list or zset allocates the body, and the slab that holds bodies of that type doubles when it fills. That is real allocation on a command path and it is also the only sensible place for it, so those sites wanted a claim written down and a first_touch around them rather than a fix. They have one now.

Then the ones worth having, which were per command and in steady state, which is exactly what Y7 is about. A to_vec of the value in APPEND, SETRANGE, EXPIRE, GETEX, INCRBYFLOAT and the string arm of COPY. A Vec built per call to hold the operands of a set operation. A Vec of indices in LREM. An owned key out of RANDOMKEY. The record copy in RENAME. The engine’s own list of free decoder slots. The hash table a SUNION walked everything into, which was the largest of them and is now a table the database keeps. And the old value out of SET ... GET, GETSET and GETDEL, which looked like a signature question and was not: the owning method stayed for the embedded caller and the wire took a _with form that hands the value over where it lies, because the wire writes it into the reply and never looks at it again. Every one of those is gone.

Third, memory that is proportional to what is stored rather than to what is served. An intset run gets longer as members go into it and there is no arrangement of that code which stores ten thousand integers in the room it had for eight. That is for_the_data, and the test of the claim is that a workload which stops adding data stops allocating.

Fourth, scratch buffers the database keeps and refills: the ZRANDMEMBER permutation and the set algebra tables. Those allocate when a call is bigger than every call before it and never otherwise. That is high_water, and the test of the claim is a unit test making the same call twice and counting zero the second time. Every site wrapped in it has one.

So the order is: report first, sort the list into the piles, fix the ones that are per command and put the right claim on the ones that are not. The three claims are all allow underneath and differ only in what they say, which is the entire point of having three of them: git grep first_touch is a list of the places a key is created, and it stays one.

Using it

#[global_allocator]
static ALLOC: YoAlloc = YoAlloc::new();

The engine installs this in yodb. A library consumer of yodb does not get it, because choosing a global allocator is the application’s call and never a library’s.

Structs

Guard

The mark from guard, undone when it goes out of scope.

YoAlloc

The allocator. Delegates to the system allocator and checks the flag first.

Enums

Mode

What happens when a marked thread allocates.

Functions

allow

Run f with allocation permitted, then restore the previous state.

enter_no_alloc

Mark this thread as a shard thread: from here on, allocating aborts.

exit_no_alloc

Undo one enter_no_alloc.

first_touch

Run f, which is a key coming into existence for the first time.

for_the_data

Run f, which is a collection getting bigger because more was put in it.

guard

Mark this thread for the length of the returned value, if the mode says so.

high_water

Run f, which is a buffer the database keeps reaching a size it has never been asked for before.

is_forbidden

Whether allocation is currently forbidden on this thread.

mode

What a marked thread does when it allocates.

seen

Everything Mode::Report collected, as (sites, allocations).

set_mode

Set what a marked thread does when it allocates.

set_mode_from_env

Set the mode from YO_ALLOC, which is off, report or abort.