Module kernel_registry 

Sealed kernel-lookup registry: makes a missing kernel loud instead of silently slow.

Why this exists

The worst bug class this engine has shipped is not a wrong kernel but an absent one, silently replaced by a correct-but-slow fallback. Two real examples:

• IQ4_XS batched prefill ran on the CPU because crate::weight_matrix::WeightMatrix’s metal_kind_supported predicate and apply_gpu_batch’s per-kind dispatch table disagreed about one kind. The only symptom was a benchmark row 13.7x behind.
• Gemma-4-E2B is slower on Metal than on CPU. Output is correct, so nothing fails; the model simply never reaches a batched path.

Both are lookups that missed. Neither produced a diagnostic.

The shape

Every dispatch decision that asks “is there a kernel for this (backend, op, quant kind)?” records the answer here.

• Build phase. While the model is constructed, each weight matrix is probed eagerly (crate::weight_matrix::WeightMatrix::probe_kernels): the same predicates the hot path will consult are evaluated once per weight and recorded. This is what turns “why is this row slow” into a startup line.
• seal. Called once the model is loaded. It summarises the build picture, warns about quantized weights that will run off the selected accelerator, and switches on post-seal checking.
• Run phase. After sealing, a dispatch-site lookup that misses a combination the build probe never saw is by definition an unpredicted slow path. It warns once, loudly, naming the call site (#[track_caller]) and the quant kind, and is a hard error under FERROX_STRICT_KERNELS=1 so CI and benchmarks can run closed.

Cost

This sits next to dispatch, so it follows the OnceLock discipline used by metal_dense_enabled / min_task_macs: the environment is read exactly once per process, never per dispatch.

On the hot path the added cost is zero instructions on a hit — hits are only ever recorded by the build probe, never by a dispatch site. A dispatch site records only when it is about to take a fallback, i.e. only when it is already paying orders of magnitude more than the bookkeeping costs.

And that bookkeeping never takes an exclusive lock in steady state: a repeat lookup is a shared RwLock read plus one relaxed fetch_add. The write lock is taken once per distinct call site, to create the row and decide whether to warn. A dispatch path must not be able to serialize rayon workers behind a mutex.

FERROX_KERNEL_REGISTRY=0 reduces every entry point to one relaxed atomic load and a return.

What it must not do

Observe only. Nothing in this module may change a dispatch decision; the predicates it calls are the same ones the dispatch takes, read a second time, and their results are recorded rather than acted on.

Modules

op

Canonical op names. Free-form &'static str is accepted everywhere, but sticking to these keeps the report groupable.

Structs

Entry

One deduplicated row of the registry.

Key

The identity of one lookup. Deduplicated on this; a repeated lookup bumps Entry::count instead of adding a row.

Lookup

A lookup being reported, minus the call site (which #[track_caller] supplies).

Registry

A registry instance. There is one global instance; tests build their own so they never race the process-wide one.

SealReport

The picture at seal time.

StrictKernelError

Raised by seal_or_error under FERROX_STRICT_KERNELS=1.

Enums

Backend

Which execution backend a lookup was resolved against.

Outcome

What the lookup resolved to.

Phase

Whether the lookup happened while the model was being built or after seal.

Severity

How bad a miss is. Recorded at the call site, which is the only place that knows — inferring it at report time from the kind or the backend is how a signal turns into noise nobody reads.

Functions

enabled

Whether the registry records at all.

global

The process-wide registry.

hit

Record a dispatch-site lookup that found a kernel. Dispatch sites do not normally call this — hits cost nothing precisely because they are not recorded on the hot path — but it exists for completeness.

miss

Record a dispatch-site lookup that missed and is taking a slower fallback than the selected backend.

miss_by_design

Record a dispatch-site lookup that missed, where the fallback is the deliberate choice rather than a gap. Never warns; recorded so the report is a complete picture instead of a filtered one.

record_build

Record an eager, load-time lookup against the global registry.

seal

Seal the global registry: print what the build probe found, warn loudly about quantized weights that will run off the selected accelerator, and switch on post-seal checking.

seal_or_error

seal, but returns Err under FERROX_STRICT_KERNELS=1 when a quantized weight has no kernel on the selected accelerator.

strict

Whether a missing kernel is a hard error. Set this in CI and in benchmark harnesses so a slow path cannot be published as a number.

verbose

Whether the full build-phase table is printed at seal.