Module bench_profile 

Where a machine’s measured bandwidth profile lives, and when it may be trusted.

crate::qstar can turn a BandwidthProfile into a split policy, but only if something hands it one. This module is that something: it decides which file on disk is this machine’s profile, reads it, and refuses it when it describes a different machine. Without it every deployment falls back to QStarPolicy::fixed_cap’s one-fetch-per-step default, which is safe and slow – the whole point of benchmarking a host is that the result is then found again on the next run.

One file per GPU

The profile is stored per GPU UUID, at $XDG_CACHE_HOME/ferrox/benchbw/<uuid>.json, not once per box. Bandwidth is a property of a slot, not of a machine: two identical cards in the same chassis routinely sit behind different links (x16 off the CPU vs. x4 off the chipset), and the q* fraction that balances one of them starves the other. Machines with a single card and older benchmarks use the legacy benchbw.json next to it.

Lookup order, and the one place it stops

An explicit path wins, then PROFILE_PATH_ENV, then this card’s benchbw/<uuid>.json, then the legacy benchbw.json.

A candidate that is simply absent is skipped – that is the whole reason the legacy file is in the list. A candidate that exists but does not parse is not skipped: the lookup returns None on the spot and the caller keeps its unbenchmarked default. Falling through would mean a truncated or half-written profile for the card in slot 1 silently promotes slot 0’s numbers to describe slot 1, and a wrong fetch fraction is worse than no fetch fraction: it does not degrade to “a bit slower”, it puts every decode step’s misses on the wrong side of a link that cannot carry them. Corruption is a reason to stop, not a reason to guess.

Naming

The environment variable is FERROX_BENCHBW_PATH and the cache directory is ferrox/ (matching ferrox-core’s registry_dir); FreeToken spells the same two FREETOKEN_BENCHBW_PATH and freetoken/. The on-disk layout is otherwise identical – benchbw/<uuid>.json plus the legacy benchbw.json, same JSON document – so a profile written by either tool is readable by the other once it is in the right directory.

Ported 1:1 from FreeToken’s moe/bench_profile.py (Apache-2.0); see docs/THIRD_PARTY_NOTICES.md.

Structs

Measured

A measured (format, card) pair, before it becomes a profile entry.

Enums

NotMeasurable

Why a measurement cannot become a profile.

Constants

LEGACY_PROFILE_FILE

The single-file profile that predates the per-GPU layout.

PROFILE_PATH_ENV

Overrides the whole lookup with one path. Empty means unset.

PROFILE_SUBDIR

The per-GPU profile directory, under the cache directory.

Functions

bench_format

The bench format key an engine quant name is measured under.

cache_dir

$XDG_CACHE_HOME/ferrox, else $HOME/.cache/ferrox, else a temporary directory.

default_profile_path

default_profile_path_in under the real cache_dir.

default_profile_path_in

<cache_dir>/benchbw/<uuid>.json, or the legacy <cache_dir>/benchbw.json when the card has no UUID.

entry_from

Turns a measurement into the entry a profile stores, deriving the verdict the reader consults.

env_profile_path

The path PROFILE_PATH_ENV names, if it names one.

latest_profile_path

latest_profile_path_in under the real cache_dir.

latest_profile_path_in

The newest benchbw/*.json under cache_dir, else the legacy benchbw.json, else None.

load_backend_recommendation

The bench-recommended offload-family backend for quant_format on this card, or None.

load_hybrid_fetch_fraction

The benched hybrid fetch fraction for quant_format, or None.

load_policy

The split policy to serve quant_format with on this card.

read_profile

The profile document at path, or None when it is absent, unreadable, or not a profile.

usable_profile

usable_profile_in under the real cache_dir, with PROFILE_PATH_ENV standing in for an absent path.

usable_profile_in

The profile this machine may actually be served with, or None.

write_profile

Writes profile where read_profile will find it.