Struct OpenVINO 

pub struct OpenVINO { /* private fields */ }

OpenVINO execution provider for Intel CPUs/GPUs/NPUs.

Implementations

impl OpenVINO

pub fn build(self) -> ExecutionProviderDispatch

impl OpenVINO

pub fn with_device_type(self, device_type: impl AsRef<str>) -> OpenVINO

Overrides the accelerator hardware type and precision.

device_type should be in the format CPU, NPU, GPU, GPU.0, GPU.1, etc. Heterogenous combinations are also supported in the format HETERO:NPU,GPU.

let ep = ep::OpenVINO::default().with_device_type("GPU.0").build();

pub fn with_num_threads(self, num_threads: usize) -> OpenVINO

Overrides the accelerator default value of number of threads with this value at runtime. If this option is not explicitly set, default value of 8 is used during build time.

pub fn with_cache_dir(self, dir: impl AsRef<str>) -> OpenVINO

Explicitly specify the path to save and load the blobs, enabling model caching.

pub fn with_opencl_throttling(self, enable: bool) -> OpenVINO

This option enables OpenCL queue throttling for GPU devices (reduces CPU utilization when using GPU).

pub fn with_qdq_optimizer(self, enable: bool) -> OpenVINO

pub fn with_dynamic_shapes(self, enable: bool) -> OpenVINO

This option if enabled works for dynamic shaped models whose shape will be set dynamically based on the infer input image/data shape at run time in CPU. This gives best result for running multiple inferences with varied shaped images/data.

pub fn with_num_streams(self, num_streams: u8) -> OpenVINO

pub fn with_precision(self, precision: impl AsRef<str>) -> OpenVINO

Trait Implementations

impl ArbitrarilyConfigurableExecutionProvider for OpenVINO

fn with_arbitrary_config( self, key: impl ToString, value: impl ToString, ) -> OpenVINO

impl Clone for OpenVINO

fn clone(&self) -> OpenVINO

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OpenVINO

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for OpenVINO

fn default() -> OpenVINO

Returns the “default value” for a type.

impl ExecutionProvider for OpenVINO

fn name(&self) -> &'static str

Returns the identifier of this execution provider used internally by ONNX Runtime.

fn register(&self, session_builder: &mut SessionBuilder) -> Result<(), Error>

Attempts to register this execution provider on the given session.

fn is_available(&self) -> Result<bool, Error>

Returns Ok(true) if ONNX Runtime was compiled with support for this execution provider, and Ok(false) otherwise.

impl From<OpenVINO> for ExecutionProviderDispatch

fn from(value: OpenVINO) -> ExecutionProviderDispatch

Converts to this type from the input type.