Crate cubecl_server 

The toolkit for implementing a CubeCL runtime.

A runtime depends on this crate, not on cubecl-runtime directly: every module of the runtime API is re-exported here under the same name, so an implementation sees one tree — the contract it implements and the pieces it implements it with.

Error handling

A failure belongs to the memory that work left unwritten and travels the dataflow from there: nothing here holds error state beside the buffers. memory_management::Taint, memory_management::ErrorGraph, stream::Failures and stream::ExecuteScope are the four pieces, and ERROR_HANDLING.md at the repository root is the argument for why they are shaped that way — read it before changing any of them.

Modules

allocator

Allocators moddule.

benchmark

Module for benchmark timings

client

Compute client module.

command

One unit of work against the device, and what a backend supplies for it.

compiler

Compiler trait, and the compilation caches in front of one. Compilation caching for a runtime: the persistent store and the in-memory cache in front of it. The [Compiler] contract itself lives in cubecl-runtime and is re-exported here.

config

CubeCL config module.

device

The device type of each runtime. The device type of each runtime, named without naming the runtime crate.

device_events

Device events, and the two things built on them.

driver

Turning a driver’s status code into an error the runtime understands.

dry_run

Running a workload for the compilation and tuning it provokes, without running the workload itself. Running a workload for the compilation and tuning it provokes, without running the workload itself.

id

Various identifier types used in CubeCL.

kernel

Kernel related traits, and the compiled kernel a launch produces. The compile step of a launch: what a [CubeKernel] becomes once a [Compiler] has seen it.

launched

The kernels a workload launches, collected while it replays: what an environment shipped for that workload has to keep, and all it has to.

logging

Logging utilities to be used by a compute server.

memory_management

Memory management module. Memory pools and the bookkeeping behind a Handle.

metadata_cache

Cache of per-launch kernel metadata info buffers. Cache of per-launch metadata info buffers (kernel shapes, strides and scalars) keyed by the exact info bytes they were built from (see [InfoCacheKey]).

runtime

Runtime trait and related types

server

Compute server module. The server contract, re-exported from cubecl-runtime.

storage

Compute Storage module. Storage implementations. The [ComputeStorage] contract is cubecl-runtime’s and re-exported here.

stream

Stream related utilities.

throughput

Throughput related utilities.

timestamp_profiler

Simple system profiling using timestamps.

tma

TMA-related runtime types

tune

Autotune module

validation

Validation utils for shared properties

Macros

local_tuner

Create a local tuner with the provided name.

storage_id_type

Create a new storage ID type.