Crate onnxruntime_ng

Source

Expand description

ONNX Runtime

This crate is a (safe) wrapper around Microsoft’s ONNX Runtime through its C API.

From its GitHub page:

ONNX Runtime is a cross-platform, high performance ML inferencing and training accelerator.

The (highly) unsafe C API is wrapped using bindgen as onnxruntime-sys-ng.

The unsafe bindings are wrapped in this crate to expose a safe API.

For now, efforts are concentrated on the inference API. Training is not supported.

§Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to onnxruntime.

First, an environment must be created using and EnvBuilder:

let environment = Environment::builder()
    .with_name("test")
    .with_log_level(LoggingLevel::Verbose)
    .build()?;

Then a Session is created from the environment, some options and an ONNX archive:

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_from_file("squeezenet.onnx")?;

Instead of loading a model from file using with_model_from_file(), a model can be fetched directly from the ONNX Model Zoo using with_model_downloaded() method (requires the model-fetching feature).

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_downloaded(ImageClassification::SqueezeNet)?;

See AvailableOnnxModel for the different models available to download.

Inference will be run on data passed as an ndarray::Array.

let array = ndarray::Array::linspace(0.0_f32, 1.0, 100);
// Multiple inputs and outputs are possible
let input_tensor = vec![array];
let outputs: Vec<OrtOwnedTensor<f32,_>> = session.run(input_tensor)?;

The outputs are of type OrtOwnedTensors inside a vector, with the same length as the inputs.

See the sample.rs example for more details.

Re-exports§

pub use error::OrtApiError;
pub use error::OrtError;
pub use error::Result;
pub use ndarray;

Modules§

download: Module controlling models downloadable from ONNX Model Zoo
environment: Module containing environment types
error: Module containing error definitions.
session: Module containing session types
tensor: Module containing tensor types.

Enums§

AllocatorType: Allocator type
GraphOptimizationLevel: Optimization level performed by ONNX Runtime of the loaded graph
LoggingLevel: Logging level of the ONNX Runtime C API
MemType: Memory type
TensorElementDataType: Enum mapping ONNX Runtime’s supported tensor types

Traits§