Crate cubecl_server 

The toolkit for implementing a CubeCL runtime.

A runtime depends on this crate, not on cubecl-runtime directly: every module of the runtime API is re-exported here under the same name, so an implementation sees one tree — the contract it implements and the pieces it implements it with.

Error handling

A failure belongs to the memory that work left unwritten and travels the dataflow from there: nothing here holds error state beside the buffers. memory_management::Taint, memory_management::ErrorGraph, stream::Failures and stream::ExecuteScope are the four pieces, and ERROR_HANDLING.md at the repository root is the argument for why they are shaped that way — read it before changing any of them.

Re-exports

pub use serde;

Modules

allocator

Allocators moddule.

benchmark

Module for benchmark timings

client

Compute client module.

command

One unit of work against the device, and what a backend supplies for it.

compiler

Compiler trait, and the compilation caches in front of one. Compilation for a runtime: the one path every backend loads a kernel by, the two halves a backend contributes to it, the persistent stores and the in-memory cache in front of them, and what the environment records of a compilation. The Compiler contract itself lives in cubecl-runtime and is re-exported here.

config

CubeCL config module.

device

The device type of each runtime, and which hardware a device is. The device type of each runtime, named without naming the runtime crate, and which hardware a device is.

device_events

Device events, and the two things built on them.

driver

Turning a driver’s status code into an error the runtime understands.

execution

What the process does with the work it is asked to run, and what that work triggered.

id

Various identifier types used in CubeCL.

kernel

Kernel related traits, and the compiled kernel a launch produces. The compile step of a launch: what a CubeKernel becomes once a Compiler has seen it.

launched

The kernels a workload launches, collected while it replays: what an environment shipped for that workload has to keep, and all it has to.

logging

Logging utilities to be used by a compute server.

memory_management

Memory management module. Memory pools and the bookkeeping behind a Handle.

metadata_cache

Cache of per-launch kernel metadata info buffers. Cache of per-launch metadata info buffers (kernel shapes, strides and scalars) keyed by the exact info bytes they were built from (see InfoCacheKey).

runtime

Runtime trait and related types

server

Compute server module. The server contract, re-exported from cubecl-runtime.

storage

Compute Storage module. Storage implementations. The ComputeStorage contract is cubecl-runtime’s and re-exported here.

stream

Stream related utilities.

throughput

Throughput related utilities.

timestamp_profiler

Simple system profiling using timestamps.

tma

TMA-related runtime types

tune

Autotune module

validation

Validation utils for shared properties

Macros

local_tuner

Create a local tuner with the provided name.

storage_id_type

Create a new storage ID type.