cubecl_runtime/execution/
mod.rs

1//! What the process does with the work it is asked to run, and what that
2//! work triggered.
3//!
4//! Two levels decide what a launch does. The [`ProcessMode`] is the whole
5//! process's, on every device: it is where the state it governs lives — each
6//! server's compile queue, the tune cache keyed by tune key, measurements
7//! any other work on a device would skew, pages shared across streams. A
8//! stream only knows whether its launches run, its [`StreamMode`]: the process mode
9//! sets every stream's default, and a measurement — autotune's candidates, a
10//! throughput probe — switches the stream it measures on to
11//! [`StreamMode::Execute`] while it measures. The stream never knows why.
12//!
13//! Under [`ProcessMode::CompileOnly`] every launch and every tune's
14//! candidates only queue their kernels, so a pass gathers everything a
15//! workload and its tuning reach; nothing compiles until a later launch loads
16//! a kernel — the first of a [`CompileAndAutotune`](ProcessMode::CompileAndAutotune)
17//! pass, or the first tune before it measures — which compiles the whole
18//! queue in one batch. Under `CompileAndAutotune` every launch compiles its
19//! kernel and is discarded, and the tunes measure. A workload then pays for
20//! compilation and tuning without paying for the work that provoked them,
21//! which is what makes producing a shippable environment affordable.
22//!
23//! A [`ProcessModeOverride`] counts what it triggers into a
24//! [`StatisticsCollector`], which a [`StatisticsReader`] reads from any
25//! thread: the kernels obtained and the tunes measured, counted to it by the
26//! code doing the work, on every device.
27//!
28//! **Buffers are left as they were** under either process mode that discards
29//! launches, so anything read back is meaningless. It only suits a pass
30//! driven by the *shapes* it produces, which is what keys the caches, and
31//! never one that branches on a computed value.
32//!
33//! The verdict is resolved where the launch is issued, on the stream it goes
34//! out on, and handed to the server as a [`LaunchMode`].
35
36mod base;
37mod issued;
38mod process_mode;
39mod statistics;
40mod stream_mode;
41
42pub use base::LaunchMode;
43pub use process_mode::{ProcessMode, ProcessModeOverride};
44pub use statistics::{
45    AutotuneStatistics, CompilationStatistics, ExecutionStatistics, KernelHandling, KernelOutcome,
46    KernelRegistration, Registration, SettledKernel, StatisticsCollector, StatisticsReader,
47};
48pub(crate) use statistics::{StatisticsRecorder, TuneOutcome, TuneRegistration};
49// Only a persisted pick names it.
50pub(crate) use issued::IssuedRecorder;
51#[cfg_attr(not(persistence), allow(unused_imports))]
52pub(crate) use statistics::TunePick;
53pub(crate) use stream_mode::{STREAM_MODES, ServiceStream};
54pub use stream_mode::{StreamMode, StreamModeOverride};
cubecl_runtime/execution/mod.rs

cubecl_runtime/execution/
mod.rs
