Struct AsyncSession 

pub struct AsyncSession { /* private fields */ }

Async wrapper for inference sessions.

Provides non-blocking inference operations using Tokio runtime. Ideal for web services, concurrent request handling, and high-throughput scenarios.

Example

use ronn_api::AsyncSession;
use std::collections::HashMap;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let session = AsyncSession::from_file("model.onnx").await?;

    let inputs = HashMap::new(); // Add your inputs here
    let outputs = session.run(inputs).await?;
    Ok(())
}

Implementations

impl AsyncSession

pub async fn from_file(path: impl AsRef<Path> + Send + 'static) -> Result<Self>

Create a new async session from an ONNX model file.

Arguments

• path - Path to the ONNX model file

Example

let session = AsyncSession::from_file("model.onnx").await?;

pub async fn with_options( path: impl AsRef<Path> + Send + 'static, options: SessionOptions, ) -> Result<Self>

Create a new async session with custom options.

Arguments

• path - Path to the ONNX model file
• options - Session configuration options

Example

let options = SessionOptions::new()
    .with_optimization_level(OptimizationLevel::O3);

let session = AsyncSession::with_options("model.onnx", options).await?;

pub async fn run( &self, inputs: HashMap<String, Tensor>, ) -> Result<HashMap<String, Tensor>>

Run async inference on the provided inputs.

This method is non-blocking and returns a future that resolves to the outputs.

Arguments

• inputs - Map of input names to tensors

Returns

A future that resolves to a map of output names to tensors.

Example

let outputs = session.run(inputs).await?;

pub async fn run_concurrent( &self, inputs: HashMap<String, Tensor>, ) -> Result<HashMap<String, Tensor>>

Run inference with read lock (allows concurrent reads if session supports it).

Note

This is the same as run() for now, but could be optimized for concurrent inference in the future.

pub fn clone_handle(&self) -> Self

Clone the async session for sharing across tasks.

This is cheap (only clones the Arc), allowing multiple tasks to share the same session.

Trait Implementations

impl Clone for AsyncSession

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.