Crate rten

rten is a runtime for machine learning models.

RTen uses models that are exported from other frameworks such as PyTorch into ONNX format and then converted into the inference-optimized .rten format by the tools in this repository.

Loading and running models

The basic workflow for loading and running a model is:

1. Load the model using Model::load.
2. Load the input data (images, audio, text etc.)
3. Pre-process the input data to convert it into tensors in the format the model expects. For this you can use RTen’s own tensor types (see rten-tensor) and pre-processing libraries, or popular Rust crates such as ndarray.
4. Execute the model using Model::run (or one of the other run_ methods)
5. Post-process the results to convert them into meaningful outputs.

See the example projects in rten-examples to see how all these pieces fit together.

Supported operators

RTen currently implements a subset of ONNX operators. See the schema.fbs FlatBuffers schema for currently supported operators and attributes.

Some operators require additional dependencies and are only available if certain crate features are enabled:

• The random feature enables operators that generate random numbers (eg. RandomUniform).

Re-exports

• pub use ops::FloatOperators;
• pub use ops::Input;
• pub use ops::Operators;
• pub use ops::Output;
• pub use timer::Timer;

Modules

• ctc
  Connectionist Temporal Classification (CTC) sequence decoding tools.
• ops
  The ops module exposes the various operators available for machine-learning models.

Macros

• unroll_loop
  Generate an unrolled loop.

Structs

• Model
  The central type used to execute RTen machine learning models.
• ModelMetadata
  Metadata for an RTen model.
• NodeInfo
  Provides access to metadata about a graph node.
• OpRegistry
  Registry used to instantiate operators when loading a model file.
• RunOptions
  Options that control logging and other behaviors when executing a Model.
• TensorPool
  A pool which enables reuse of data buffers from tensors and other containers.

Enums

• Dimension
  Represents the size of a dimension of a runtime-provided value, such as an operator input, output or intermediate value.
• ModelLoadError
  Errors reported by Model::load.
• ReadOpError
  Error type for errors that occur when de-serializing an operator.
• TimingSort
  Specifies sort order for graph run timings.

Traits

• DefaultOperatorFactory
  Trait that that creates the default/built-in implementation of an operator, for use with OpRegistry::register_op.

Type Aliases

• NodeId
  ID of a node in a Model graph.