Module throughput

Expand description

Throughput related utilities.

Structs§

AchievedThroughput: One resource’s achieved rate during a measured run, against its modeled peak.
CmmaDims: The M, N, and K dimensions of a matrix multiplication.
ComputeCmmaConfig: Configuration for a matrix multiplication (CMMA) operation.
KernelConfig: Configuration and payload for a benchmarkable compute kernel.
MemoryCurve: Memory throughput measured across a range of working sets.
MemoryPoint: One measured point of a MemoryCurve.
MemorySpec: What a memory mode asks of a probe.
Ranked: What a ranking pass found: how fast the shape answered, and the iteration count it settled on for the next shape to start from.
ResourceBound: A resource’s roofline record: how much of it a run must move, against the peak rate that resource can sustain.
ThroughputBenchmarker: A marker for measuring throughput of compute kernels.
ThroughputCache: Caches the ThroughputValue for a given ThroughputKey.
ThroughputKey: Represents a key/configuration used to identify the throughput of a computation.
ThroughputValue: Represents the throughput of a computation, including the number of operations and the duration.

DEFAULT_WORKING_SET_BYTES: Bytes one buffer of a ThroughputMode::Memory probe moves per pass at its default working set. The probe’s buffer is a multiple of this, and both are clamped to the device’s maximum allocation when the probe runs.
MIN_WORKING_SET: The smallest working set a curve is measured at, in bytes moved per pass.
PROBE_VERSION: What the probes measure, as opposed to which release ran them. Bump it when a probe changes what it reports.

binding_achieved: The resource that actually governed the run’s duration: binding_resource’s reduction applied to achieved scores instead of raw bounds. Every entry here shares one duration, so ranking by fraction_of_peak gives the same order ranking the underlying bounds by time_at_peak would.
binding_resource: The roofline convention this module uses throughout: a run cannot finish faster than its slowest resource allows even under perfect overlap between them, so that resource is the one binding the achievable duration, whatever the others manage. binding_achieved applies the same reduction to a run’s actually measured rates.
compute_throughput_key: Constructs a compute ThroughputKey based on CMMA tile availability and types.
score_resources: Scores an actual run against a set of ResourceBounds, one resource at a time.
select_cmma_tile: Resolves the largest supported CMMA or MMA tile size (m, n, k).
sweep_size: The sweep size a working set of bytes is probed at: the power of two at or below it, never below MIN_WORKING_SET.
working_set_sweep: The working sets a curve is measured at: powers of two from MIN_WORKING_SET up to cap, which is where the device runs out of allocation.