Struct Device 

pub struct Device { /* private fields */ }

A Metal GPU with its command queue, kernel library and pipeline cache. Cheap to clone.

Implementations

impl Device

pub fn new_buffer(&self, len: usize) -> Result<Buffer>

A buffer of len bytes from the caching allocator. Its contents are unspecified (a reused buffer holds stale bytes); fails with GpuError::OutOfMemory beyond the memory limit.

pub fn new_gpu_buffer(&self, len: usize) -> Result<Buffer>

A buffer of len bytes for GPU work to write: as new_buffer, but it may be memory that committed GPU work still uses (as MLX reuses memory). GPU work encoded with it is ordered after those uses; the CPU cannot access it until they complete (Buffer::in_use).

pub fn map_file(&self, path: &Path) -> Result<Buffer>

The file at path, memory-mapped read-only and wrapped without a copy (newBufferWithBytesNoCopy): its pages are the page cache’s, read from disk when first touched by the CPU or the GPU. The buffer cannot be written (CPU writes and GPU writes are refused). The mapping is released when the buffer and every GPU use of it are gone.

A shared read-only mapping is used because a private (copy-on-write) one costs about 4× more to fault in on first use.

The file must not be modified (written or truncated, by this or another process) while it is mapped: the bytes Buffer::contents returns would change underneath a &[u8] (undefined behaviour), and access past a truncated end raises SIGBUS. Map only files nothing writes while they are in use, such as model weights.

pub fn buffer_from(&self, bytes: &[u8]) -> Result<Buffer>

A buffer holding a copy of bytes.

impl Device

pub fn command_buffer(&self) -> Result<CommandBuffer>

A new command buffer on this device’s queue. It is unordered: its work does not wait for other command buffers, so binding reused memory whose previous owner’s GPU work is still running (Device::new_gpu_buffer) is refused with GpuError::BufferInUse; use an ordered_command_buffer for that.

impl Device

pub fn timeline(&self) -> Result<Timeline>

A new, empty timeline.

pub fn ordered_command_buffer(&self) -> Result<CommandBuffer>

A new command buffer whose work reads the outputs of earlier ordered command buffers (committed on this device from any thread) safely while they are in flight: before a dispatch reads or writes a buffer with a pending GPU write, it waits for the writing command buffer’s fence. Independent command buffers overlap on the GPU, as in MLX.

Reads wait for the pending writer; writes wait for every pending user (writing a buffer that earlier work still reads). Other command buffers do not wait.

pub fn command_buffer_on( &self, timeline: &mut Timeline, ) -> Result<CommandBuffer>

A new command buffer whose work runs after the work of every command buffer created earlier on timeline (commit them in creation order). Like command_buffer, it refuses reused memory whose previous owner’s GPU work is still running.

impl Device

pub fn system_default() -> Option<Device>

The system’s default Metal device, or None if there is none (for example in some virtual machines).

pub fn shared() -> Option<&'static Device>

The process-wide default device, created on first use: one allocator, kernel cache and queue shared by everything that uses the GPU.

pub fn memory(&self) -> MemoryStats

Memory use and limits of this device’s allocator.

pub fn set_memory_limit(&self, bytes: usize) -> usize

Sets the memory limit and returns the previous one (as MLX’s set_memory_limit).

The limit bounds active + mapped (MemoryStats::active_with_mapped): an allocation or a file mapping that would pass it fails with GpuError::OutOfMemory before anything is wired, so a server’s memory budget can refuse work instead of swapping. This deliberately differs from MLX 0.32.4, whose limit is “a guideline for the maximum amount of memory to use during graph evaluation” (allocations fail there only when no RAM or swap is left). A limit below what is already in use is accepted; the next allocation or mapping then fails. The default is min(1.5 × recommended working set, 0.95 × RAM), as MLX.

pub fn set_cache_limit(&self, bytes: usize) -> usize

Sets the cache limit, trimming the cache to it, and returns the previous one.

pub fn set_wired_limit(&self, bytes: usize) -> Result<usize>

Sets the wired limit and returns the previous one (MLX’s set_wired_limit; 0 by default). Buffers, mapped files included, are kept resident up to this many bytes while GPU work runs, instead of being made resident per command buffer, which can page memory in again under pressure. Lowering it unwires buffers until the wired set fits. A limit above the recommended working set is an error. On a GPU without residency sets (before Metal 3) the limit is recorded but nothing is wired.

pub fn can_wire(&self) -> bool

Whether buffers can be wired on this system (residency sets: a Metal 3 GPU).

pub fn clear_cache(&self)

Releases every cached buffer.

pub fn in_flight(&self) -> usize

Command buffers committed on this device that have not completed yet.

pub fn wait_for_capacity(&self, max: usize)

Blocks until fewer than max command buffers are in flight (back-pressure for callers that submit without waiting).

pub fn reset_peak(&self)

Resets the peaks to the current use: peak to active, peak_with_mapped to active + mapped.

pub fn info(&self) -> &DeviceInfo

What this GPU is and can do.

pub fn pipeline(&self, name: &str) -> Result<Pipeline>

The compute pipeline for the kernel name from nuvec’s kernel library, compiled on first use and cached for the life of the device. The cache is keyed by kernel name and function constants, so it is bounded by the library’s kernels times the constant combinations the operations use (a few thousand at most); pipelines from source are keyed by their name (one entry per distinct custom kernel instantiation).

pub fn pipeline_with( &self, name: &str, constants: &[(usize, bool)], ) -> Result<Pipeline>

The pipeline for kernel name specialised with boolean function constants (index, value), compiled on first use and cached per name and constants.

pub fn pipeline_with_constants( &self, name: &str, constants: &[(usize, Constant)], ) -> Result<Pipeline>

The pipeline for kernel name specialised with boolean or integer function constants (index, value), compiled on first use and cached per name and constants.

pub fn pipeline_from_source( &self, name: &str, source: impl FnOnce() -> String, ) -> Result<Pipeline>

The pipeline for kernel name from Metal source that source generates, compiled at runtime on first use and cached by name (so a name must always mean the same source; source runs only on a cache miss). Source kernels have their own namespace: a source kernel named like one of nuvec’s library kernels is a different pipeline. The source may include nuvec’s headers (#include "common.h"); it is compiled as nuvec’s own kernels are, with Metal 3.2 and IEEE-conformant math.

The source is trusted code: the pipeline is run with the unsafe CommandBuffer::dispatch family, whose contract (in-bounds accesses only) the kernel must keep.

Trait Implementations

impl Clone for Device

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Device

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.