Crate onnxruntime[][src]

Expand description

ONNX Runtime

This crate is a (safe) wrapper around Microsoft’s ONNX Runtime through its C API.

From its GitHub page:

ONNX Runtime is a cross-platform, high performance ML inferencing and training accelerator.

The (highly) unsafe C API is wrapped using bindgen as onnxruntime-sys.

The unsafe bindings are wrapped in this crate to expose a safe API.

For now, efforts are concentrated on the inference API. Training is not supported.

Example

The C++ example that uses the C API (C_Api_Sample.cpp) was ported to onnxruntime.

First, an environment must be created using and EnvBuilder:

let environment = Environment::builder()
    .with_name("test")
    .with_log_level(LoggingLevel::Verbose)
    .build()?;

Then a Session is created from the environment, some options and an ONNX archive:

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_from_file("squeezenet.onnx")?;

Instead of loading a model from file using with_model_from_file(), a model can be fetched directly from the ONNX Model Zoo using with_model_downloaded() method (requires the model-fetching feature).

let mut session = environment
    .new_session_builder()?
    .with_optimization_level(GraphOptimizationLevel::Basic)?
    .with_number_threads(1)?
    .with_model_downloaded(ImageClassification::SqueezeNet)?;

See AvailableOnnxModel for the different models available to download.

Inference will be run on data passed as an ndarray::Array.

let array = ndarray::Array::linspace(0.0_f32, 1.0, 100);
// Multiple inputs and outputs are possible
let input_tensor = vec![array];
let outputs: Vec<OrtOwnedTensor<f32,_>> = session.run(input_tensor)?;

The outputs are of type OrtOwnedTensors inside a vector, with the same length as the inputs.

See the sample.rs example for more details.

Re-exports

pub use error::OrtApiError;
pub use error::OrtError;
pub use error::Result;
pub use ndarray;

Modules

Module controlling models downloadable from ONNX Model Zoom

Module containing environment types

Module containing error definitions.

Module containing session types

Module containing tensor types.

Enums

Allocator type

Optimization level performed by ONNX Runtime of the loaded graph

Logging level of the ONNX Runtime C API

Memory type

Enum mapping ONNX Runtime’s supported tensor types

Traits

Trait used to map Rust types (for example f32) to ONNX types (for example Float)

Adapter for common Rust string types to Onnx strings.