Module throughput

Expand description

Throughput related utilities.

Structs§

AchievedThroughput: One resource’s achieved rate during a measured run, against its modeled peak.
CmmaDims: The M, N, and K dimensions of a matrix multiplication.
ComputeCmmaConfig: Configuration for a matrix multiplication (CMMA) operation.
KernelConfig: Configuration and payload for a benchmarkable compute kernel.
MemoryCurve: Memory throughput measured across a range of working sets.
MemoryPoint: One measured point of a MemoryCurve.
ResourceBound: A resource’s roofline record: how much of it a run must move, against the peak rate that resource can sustain.
ThroughputBenchmarker: A marker for measuring throughput of compute kernels.
ThroughputCache: Caches the ThroughputValue for a given ThroughputKey.
ThroughputKey: Represents a key/configuration used to identify the throughput of a computation.
ThroughputValue: Represents the throughput of a computation, including the number of operations and the duration.

DEFAULT_BUFFER_BYTES: Bytes per buffer of the single-size memory probes, ThroughputMode::Memory, ThroughputMode::MemoryRead, and ThroughputMode::MemoryWrite. Clamped to the device’s maximum allocation when the probe runs.
MIN_WORKING_SET: The smallest working set a curve is measured at, in bytes moved per pass.

binding_achieved: The resource that actually governed the run’s duration: binding_resource’s reduction applied to achieved scores instead of raw bounds. Every entry here shares one duration, so ranking by fraction_of_peak gives the same order ranking the underlying bounds by time_at_peak would.
binding_resource: The roofline convention this module uses throughout: a run cannot finish faster than its slowest resource allows even under perfect overlap between them, so that resource is the one binding the achievable duration, whatever the others manage. binding_achieved applies the same reduction to a run’s actually measured rates.
compute_throughput_key: Constructs a compute ThroughputKey based on CMMA tile availability and types.
score_resources: Scores an actual run against a set of ResourceBounds, one resource at a time.
select_cmma_tile: Resolves the largest supported CMMA or MMA tile size (m, n, k).
working_set_sweep: The working sets a curve is measured at: powers of two from MIN_WORKING_SET up to cap, which is where the device runs out of allocation.