Crate onnxruntime[−][src]

Expand description

ONNX Runtime

This crate is a (safe) wrapper around Microsoft’s ONNX Runtime through its C API.

From its GitHub page:

ONNX Runtime is a cross-platform, high performance ML inferencing and training accelerator.

The (highly) unsafe C API is wrapped using bindgen as onnxruntime-sys.

The unsafe bindings are wrapped in this crate to expose a safe API.

For now, efforts are concentrated on the inference API. Training is not supported.

Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to onnxruntime.

First, an environment must be created using and EnvBuilder:

let environment = Environment::builder()
    .with_name("test")
    .with_log_level(LoggingLevel::Verbose)
    .build()?;

Then a Session is created from the environment, some options and an ONNX archive:

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_from_file("squeezenet.onnx")?;

Instead of loading a model from file using with_model_from_file(), a model can be fetched directly from the ONNX Model Zoo using with_model_downloaded() method (requires the model-fetching feature).

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_downloaded(ImageClassification::SqueezeNet)?;

See AvailableOnnxModel for the different models available to download.

Inference will be run on data passed as an ndarray::Array.

let array = ndarray::Array::linspace(0.0_f32, 1.0, 100);
// Multiple inputs and outputs are possible
let input_tensor = vec![array];
let outputs: Vec<OrtOwnedTensor<f32,_>> = session.run(input_tensor)?;