Module judge_cache 

Ranked dispatch tables — Phase 6b’s O(1) runtime lookup.

Moved here from fuel-tensor/src/judge/cache.rs (board #109 PR-D follow-up). docs/restructure-migration-design.md §5.1 Row 3 (RULED 2026-09-02) called this move clean, but that was measured before board #109 landed judge/mod.rs itself in fuel-tensor (it needs Tensor to realize profiling ops). The ORIGINAL cache.rs had exactly one Tensor-dependent call (populate_dispatch_table ran judge::Judge::run directly) blocking a bare move — moving it as-is would have recreated the fuel-dispatch -> fuel-tensor cycle #305 avoided. The fix inverts that one write path: this module no longer RUNS the judge, it only STORES an already-computed ProfileReport via store. fuel-tensor/src/judge/cache.rs keeps a thin populate_dispatch_table that runs Judge::run (needs Tensor) and hands the result to store — a downward call, no cycle.

Judge (mentioned above) is fuel_tensor::judge::Judge — plain text, not a doc link, since this crate cannot resolve a path in a crate above it in the dependency graph. This module has no dependency on it.

Criteria

• Fastest — pick the backend with the lowest median latency. Excludes the reference backend by default (it’s the oracle, not a production path — pathologically slow by design).
• MostAccurate — pick the backend with the lowest max_rel_error against the reference backend. Also excludes reference (reference vs reference is by definition 0, but dispatching to it defeats the purpose of having an optimized backend at all). Ties broken by latency.
• Balanced — minimise latency_ns * (1 + accuracy_penalty * rel_error). Penalty coefficient defaults to 100 so a 1% rel-error bump is equivalent to a 2× latency penalty — steep enough that numerically-unsound fast paths don’t win by default.

Process-wide cache

The dispatch table is hardware-determined: it doesn’t depend on the app, only on the CPU/GPU configuration. So a single process-wide instance is the right granularity, with three states:

1. In-memory — populated by an explicit store call this process. Authoritative.
2. On-disk — persisted from a prior process; the same hardware was profiled previously. Lazy-loaded on first cached call.
3. Absent — no profile yet, or hardware changed since the persisted profile was taken. Routed ops fall through to the Router’s default backend until store runs successfully.

Structs

DispatchOptions

Options for building a DispatchTable.

DispatchTable

O(1) runtime dispatch table, constructed once from a ProfileReport and then queried at realize time.

Pick

Where the dispatch table decided an op should run, and which kernel sibling won at that decision point.

ProfileEntry

Single (op_kind, dtype, size_class) × (backend, device_index, kernel_source) datum produced by one measurement run.

ProfileReport

A persistable table of every profile measurement the Judge produced in one run.

SizeClass

Dispatch size bucket. One u32 field carries two representations, disambiguated by the OpKind that always sits beside it in the dispatch key (a matmul key is only ever compared to matmul keys):

Enums

Criterion

A selection criterion — what “best” means for a lookup.

OpKind

Op kinds the Judge profiles. Adding a variant + a Judge match arm extends the profile matrix; existing reports parse forward thanks to #[non_exhaustive].

Constants

DEFAULT_ACCURACY_PENALTY

Default weight applied to max_rel_error in the Balanced criterion’s cost function.

Functions

cached

The currently-active dispatch table, if any.

cached_oracle

The currently-active ProfileJudgeOracle, if any — same lifecycle and data source as cached, but exposing EVERY raw measurement (losing alternatives included) instead of winners only. The pipelined bridge attaches it to crate::plan::PlanOptions::with_judge so compile_plan refines Layer-1 static costs with measured latencies.

invalidate

Drop the in-memory cache and delete the persisted profile on disk. The next populate call (fuel-tensor’s populate_dispatch_table, which runs the Judge and calls store) will re-run the probe + judge from scratch.

store

Populate the dispatch table from an already-computed ProfileReport and persist it to disk. The caller runs the actual Judge measurement (that needs Tensor, hence lives in fuel-tensor) and hands the finished report here — this function only stores and serves it.