Crate onnxruntime_ng 

ONNX Runtime

This crate is a (safe) wrapper around Microsoft’s ONNX Runtime through its C API.

From its GitHub page:

ONNX Runtime is a cross-platform, high performance ML inferencing and training accelerator.

The (highly) unsafe C API is wrapped using bindgen as onnxruntime-sys-ng.

The unsafe bindings are wrapped in this crate to expose a safe API.

For now, efforts are concentrated on the inference API. Training is not supported.

Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to onnxruntime.

First, an environment must be created using and EnvBuilder:

let environment = Environment::builder()
    .with_name("test")
    .with_log_level(LoggingLevel::Verbose)
    .build()?;

Then a Session is created from the environment, some options and an ONNX archive:

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_from_file("squeezenet.onnx")?;

Instead of loading a model from file using with_model_from_file(), a model can be fetched directly from the ONNX Model Zoo using with_model_downloaded() method (requires the model-fetching feature).

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_downloaded(ImageClassification::SqueezeNet)?;

See AvailableOnnxModel for the different models available to download.

Inference will be run on data passed as an ndarray::Array.

let array = ndarray::Array::linspace(0.0_f32, 1.0, 100);
// Multiple inputs and outputs are possible
let input_tensor = vec![array];
let outputs: Vec<OrtOwnedTensor<f32,_>> = session.run(input_tensor)?;

The outputs are of type OrtOwnedTensors inside a vector, with the same length as the inputs.

See the sample.rs example for more details.

Re-exports

pub use error::OrtApiError;

pub use error::OrtError;

pub use error::Result;

pub use ndarray;

Modules

download

Module controlling models downloadable from ONNX Model Zoo

environment

Module containing environment types

error

Module containing error definitions.

session

Module containing session types

tensor

Module containing tensor types.

Enums

AllocatorType

Allocator type

GraphOptimizationLevel

Optimization level performed by ONNX Runtime of the loaded graph

LoggingLevel

Logging level of the ONNX Runtime C API

MemType

Memory type

TensorElementDataType

Enum mapping ONNX Runtime’s supported tensor types

Traits

TypeToTensorElementDataType

Trait used to map Rust types (for example f32) to ONNX types (for example Float)

Utf8Data

Adapter for common Rust string types to Onnx strings.