Struct Hardware 

pub struct Hardware {
    pub properties: Arc<DeviceProperties>,
    pub accumulator_as_a: bool,
}

The device, as the routines see it.

A value rather than a client, so a routine’s geometry is unit-tested against a device that is not in the room. Cloning one costs a count increment: the properties are the client’s own.

Fields

properties: Arc<DeviceProperties>

What the runtime reports about the device. Every capability and limit a routine adapts to is read here, as cubecl states it.

accumulator_as_a: bool

Whether an mma accumulator holds its cells where the A operand of a contraction as deep as it is wide holds them, so a product contracted again reads it as that operand in the same registers (the tile DSL’s accumulator_as_a).

True on CUDA, and on Metal, whose simdgroup matrix gives every lane two cells of one row in each operand. False on RDNA, whose A runs along its rows against an accumulator down its columns, and whose RDNA3 A is held twice across the wave.

Implementations

impl Hardware

pub fn new(properties: Arc<DeviceProperties>, target: &TargetProperties) -> Self

The device properties describe, compiling for target.

pub fn from_client(client: &Client) -> Self

The device client runs on, compiling for its target.

pub fn plane_dim(&self) -> Result<u32, Refusal>

The exact plane width, for a kernel whose correctness depends on one.

The only way a routine reaches a plane width, and fallible, because both shortcuts are wrong. plane_size_max > 1 does not mean planes work: the wgpu runtime reports 8..=128 for an adapter with no subgroups at all, and a plane_sum there is undefined. And plane_size_max is not the width: wgpu reports a range on AMD RDNA (32 to 64) and Intel (8 to 32), and a cube sized to the max sums over a fraction of each row.

Errors

Refusal::PlanesUnavailable without plane operations, Refusal::ImprecisePlaneDim where the runtime reports a range.

pub fn plane_size(&self) -> Result<u32, Refusal>

Units that cooperate as one plane: the device’s width where it commits to one, and 1 where a plane is a single unit.

The width a derivation asks for. plane_dim is the width a kernel asks for when it issues plane instructions; a geometry that only divides a microkernel between cooperating units gets 1 on a runtime with none, the width the tile engine builds its cube from there. A single-unit plane is read off what makes it one, a committed width of one and no plane operations, as cubecl’s CPU runtime reports.

Errors

plane_dim’s, for a fabricated width or a range.

pub fn dispatch_plane_size(&self) -> u32

The width the engine builds a cube from, whatever the device committed to: sound only where no axis of the space rides Unit scope.

cubek builds cube_dim.x from plane_size_max and asserts only that the axes riding Unit scope multiply to it. A cut that hands disjoint work to whole planes has no Unit axis, so this is the width the launch is going to use rather than a guess about the device. Units that share an item and sum across each other need plane_size instead.

pub fn full_cube_planes(&self) -> usize

The most planes of the width the device dispatches that one cube holds: a cube as full as the device allows. On a CPU runtime, where a plane is one unit, a unit on every core.

pub fn cube_dim_1d(&self, units: u32) -> CubeDim

A flat cube of units, clamped to what the device spawns: for the element-wise kernels that want a reasonable block, where the count is a throughput preference.

pub fn shared_memory_budget(&self) -> usize

Shared memory a cube may claim: the reported maximum, less what the driver reserves for cube-scoped matrices. Every budget goes through here: reading the maximum directly over-subscribes on a runtime that reserves, as Vulkan does, and fails the launch.

pub fn workers(&self) -> Option<u32>

Independent workers to spread across: a CPU runtime’s cores, or a GPU’s compute units. None where nothing reports one, and a caller keeps its behaviour there rather than guess.

pub fn units_to_fill(&self) -> Option<u32>

Units a launch wants in flight to keep this device busy, or None when the device does not say how wide it is.

Units, not cubes: a worker hides memory latency behind other units’ requests, so one cube per worker starves it however large the cube, and counting cubes would make the answer depend on a cube size decided after it. A CPU runtime’s units are threads, one per core.

The per-worker figure is a floor no runtime reports. Two measurements agree on it: a shared-memory gemv’s best target was 8192 units, and the decode attention that measured fastest on an M2 Pro ran 256 single-plane cubes, also 8192 units, about 431 per worker.

pub fn cubes_to_fill(&self, units_per_cube: u32) -> Option<u32>

Cubes of units_per_cube units that fill this device, or None when it does not say how wide it is. A preference: the grid may exceed it.

pub fn load_values(&self, elem: ElemType) -> usize

How many values of elem one load instruction holds: what a step counts its loads in.

pub fn vector_ceiling(&self, elem: ElemType) -> usize

The widest vector this device reads elem in: its IO width over the element’s, capped by the most components a vector takes. The device’s half of a ceiling: a launch reads at the microkernel’s, narrowed until the operands admit it.

pub fn transaction_bytes(&self) -> usize

Bytes the memory system moves in one transaction, whatever share of them a read wants: GPU_TRANSACTION_BYTES on a GPU, CPU_CACHE_LINE_BYTES on a CPU runtime. No runtime reports one, so it is stated per kind of runtime.

What it bounds is waste: a walk whose units read consecutive addresses is coalesced however many they are, and this decides whether the run they cover per step fills a transaction.

pub fn staggered_pitch(&self, run: usize, elem: ElemType) -> usize

The distance, in elements, between the starts of consecutive runs of run elements of elem laid one after another: a whole and odd number of transactions.

Odd because a memory system that picks a channel from low power-of-two address bits sends runs a power of two bytes apart to one channel; an odd step reaches every slot of such an interleave. It costs at most one transaction of padding a run.

pub fn plane_count_per_cube(&self, working_units: usize, plane_dim: u32) -> u32

Planes per cube for a plane-parallel launch of working_units units: on a CPU runtime capped at the core count, on a GPU a power of two up to eight, and never a cube of no unit on a device whose unit budget is narrower than its plane.

pub fn strip_budget(&self, plane_size: usize) -> usize

The tallest cube, in planes of plane_size, the device spawns once the plane width has taken cube_dim.x: the cap on cube_dim.y.

A budget, not a preference. A bound that lowers what the caller asked for would also erase the differences between the values autotune measures, so what a taller cube is worth is measured and only what the device can spawn is stated here.

pub fn shared_f32_atomic_add(&self) -> bool

Whether a cube’s shared memory takes an atomic add on an f32: what a cube-scope microkernel sums its planes’ partials through.

pub fn bulk_tensor_copy(&self) -> bool

Whether a stage may be filled by the TMA engine: the bulk tensor copy, and the barriers its copies land on.

pub fn unit_async_copy(&self) -> bool

Whether a stage may be filled by the units’ async copies: the copy engine, and the barriers its copies land on.

pub fn barrier_ops(&self) -> bool

Whether the runtime carries a barrier type, which is how a stage’s producer and consumer meet when they are different planes.

Trait Implementations

impl Clone for Hardware

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Hardware

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<&Client> for Hardware

fn from(client: &Client) -> Self

Converts to this type from the input type.