Struct SelectionRequest

Source

pub struct SelectionRequest {
    pub require_gpu: bool,
    pub require_fp16: bool,
    pub require_bf16: bool,
    pub require_fp8: bool,
    pub require_tensor_cores: bool,
    pub require_unified_memory: bool,
    pub require_peer_access: bool,
    pub pin: Option<BackendKind>,
}

Expand description

Required-feature predicate used to filter backends during selection.

A backend is eligible only if it is available and its capabilities satisfy every requested flag. All fields default to “don’t care”.

Fields§

§require_gpu: bool

Require a GPU backend (exclude the CPU fallback from the primary pick).

§require_fp16: bool

Require native FP16 compute.

§require_bf16: bool

Require native BF16 compute.

§require_fp8: bool

Require FP8 compute.

§require_tensor_cores: bool

Require Tensor-Core / matrix units.

§require_unified_memory: bool

Require unified / managed memory.

§require_peer_access: bool

Require peer-to-peer device access.

§pin: Option<BackendKind>

If Some, only this exact backend kind is acceptable.

Implementations§

Source §

impl SelectionRequest

Source

pub const fn any() -> Self

A request with no constraints (any available backend qualifies).

Source

pub const fn require_gpu() -> Self

A request that demands a GPU (used by callers that must not silently fall back to the CPU reference path).

Source

pub const fn pinned(kind: BackendKind) -> Self

Pin selection to one specific backend kind.

Source

pub const fn for_workload( self, workload_bytes: usize, gpu_threshold_bytes: usize, ) -> Self

Narrow this request to the CPU reference backend when the workload is too small to pay for a GPU dispatch.

A GPU dispatch costs a host→device copy, a command submission and a synchronisation; below some byte count that overhead dominates the kernel and the host path finishes first. This helper expresses that policy at selection time — i.e. before any memory is allocated — so that the whole workload (allocations included) lands on one backend. Per-operation routing is deliberately not offered: a device pointer belongs to the backend that allocated it, so a per-op switch would require duplicating every buffer.

The request is returned unchanged when the caller has already expressed an explicit preference (require_gpu or a pin), or when workload_bytes >= gpu_threshold_bytes.

§Example

use oxicuda_backend::{BackendKind, SelectionRequest};

// 1 KiB with a 64 KiB threshold → pinned to the host backend.
let small = SelectionRequest::any().for_workload(1024, 64 * 1024);
assert_eq!(small.pin, Some(BackendKind::Cpu));

// 1 MiB → unchanged, so the GPU can win the selection.
let large = SelectionRequest::any().for_workload(1024 * 1024, 64 * 1024);
assert_eq!(large, SelectionRequest::any());