Struct OrtInfer 

pub struct OrtInfer { /* private fields */ }

Core ONNX Runtime inference engine with support for pooling and configurable sessions.

Implementations

impl OrtInfer

pub fn new( model_path: impl AsRef<Path>, input_name: Option<&str>, ) -> Result<OrtInfer, OCRError>

Creates a new OrtInfer instance with default ONNX Runtime settings and a single session.

pub fn from_config( common: &ModelInferenceConfig, model_path: impl AsRef<Path>, input_name: Option<&str>, ) -> Result<OrtInfer, OCRError>

Creates a new OrtInfer instance from ModelInferenceConfig, applying ORT session configuration.

impl OrtInfer

pub fn model_path(&self) -> &Path

Returns the model path associated with this inference engine.

pub fn model_name(&self) -> &str

Returns the model name associated with this inference engine.

pub fn infer( &self, inputs: &[(&str, TensorInput<'_>)], ) -> Result<Vec<(String, TensorOutput)>, OCRError>

Unified inference method: multi-input -> multi-output.

This is the only public inference method. It accepts arbitrary named inputs and returns all model outputs without making assumptions about their semantic meaning, shape, or type.

Responsibilities

This method is responsible for:

• Managing the session pool (round-robin selection)
• Converting inputs to ONNX Runtime format
• Executing inference
• Extracting raw outputs

This method is NOT responsible for:

• Validating output shapes
• Converting outputs to specific dimensions
• Interpreting the semantic meaning of outputs

Arguments

• inputs - A slice of (name, tensor) pairs specifying the model inputs

Returns

A vector of (name, TensorOutput) pairs representing all model outputs. The caller (model layer) is responsible for interpreting and validating these outputs according to the model’s requirements.

Example

// Single input
let inputs = vec![("image", TensorInput::Array4(&batch))];
let outputs = inference.infer(&inputs)?;
let result = outputs[0].1.try_into_array2_f32()?;

// Multiple inputs
let inputs = vec![
    ("image", TensorInput::Array4(&batch)),
    ("scale_factor", TensorInput::Array2(&scale)),
    ("im_shape", TensorInput::Array2(&shape)),
];
let outputs = inference.infer(&inputs)?;

impl OrtInfer

pub fn input_name(&self) -> &str

Returns the input tensor name.

pub fn get_session( &self, idx: usize, ) -> Result<MutexGuard<'_, Session>, OCRError>

Gets a session from the pool.

pub fn input_names_from_model(&self) -> Vec<String>

Returns the declared input names from the model.

pub fn primary_input_shape(&self) -> Option<Vec<i64>>

Attempts to retrieve the primary input tensor shape from the first session.

Returns a vector of dimensions if available. Dynamic dimensions (e.g., -1) are returned as-is.

Trait Implementations

impl Debug for OrtInfer

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.