Crate onnxruntime_ng

source ·
Expand description

ONNX Runtime

This crate is a (safe) wrapper around Microsoft’s ONNX Runtime through its C API.

From its GitHub page:

ONNX Runtime is a cross-platform, high performance ML inferencing and training accelerator.

The (highly) unsafe C API is wrapped using bindgen as onnxruntime-sys-ng.

The unsafe bindings are wrapped in this crate to expose a safe API.

For now, efforts are concentrated on the inference API. Training is not supported.

Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to onnxruntime.

First, an environment must be created using and EnvBuilder:

let environment = Environment::builder()
    .with_name("test")
    .with_log_level(LoggingLevel::Verbose)
    .build()?;

Then a Session is created from the environment, some options and an ONNX archive:

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_from_file("squeezenet.onnx")?;

Instead of loading a model from file using with_model_from_file(), a model can be fetched directly from the ONNX Model Zoo using with_model_downloaded() method (requires the model-fetching feature).

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_downloaded(ImageClassification::SqueezeNet)?;

See AvailableOnnxModel for the different models available to download.

Inference will be run on data passed as an ndarray::Array.

let array = ndarray::Array::linspace(0.0_f32, 1.0, 100);
// Multiple inputs and outputs are possible
let input_tensor = vec![array];
let outputs: Vec<OrtOwnedTensor<f32,_>> = session.run(input_tensor)?;

The outputs are of type OrtOwnedTensors inside a vector, with the same length as the inputs.

See the sample.rs example for more details.

Re-exports

Modules

  • download
    Module controlling models downloadable from ONNX Model Zoo
  • environment
    Module containing environment types
  • error
    Module containing error definitions.
  • session
    Module containing session types
  • tensor
    Module containing tensor types.

Enums

Traits