Module gpu 

Low-level GPU inference support via ONNX Runtime ExecutionProviders.

This module handles the ort integration layer – it configures ONNX Runtime SessionBuilder instances with the appropriate ExecutionProvider (CUDA, CoreML, DirectML, TensorRT) and manages device string parsing for the engine.

Feature-gated: cuda, coreml, directml, tensorrt features enable respective providers. Auto-detection tries available providers and falls back to CPU.

For the high-level, user-facing device enumeration and selection API, see crate::device.

Structs

DeviceInfo

Information about an available compute device.

Enums

DeviceType

Supported GPU device types.

Functions

configure_session_builder

Configure an ONNX Runtime session builder with the appropriate ExecutionProvider.

list_devices

List all available compute devices.

parse_device_string

Parse a device string (“cpu”, “cuda”, “cuda:0”, “cuda:1”, “coreml”, “directml”, “directml:2”, “tensorrt”, “tensorrt:0”, “auto”) into a DeviceType.