ort-parallel 0.1.3

A library for running ONNX models in parallel using ORT(session pool)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# Ort Parallel
This crate is just a session pool for ONNX Runtime.
It will load more sessions with the same configuration & packed weights until the pool is full.

## Features
- sync
- async

## Exmaple
Sync
```rs
let builder = SessionBuilder::new()
    .unwrap()
    .with_optimization_level(ort::session::builder::GraphOptimizationLevel::Level3)
    .unwrap();
let pool = SessionPool::commit_from_file(builder, Path::new("model.onnx"), 10).unwrap();
pool.run(inputs!{...}).unwrap();
```

Async
```rs
let builder = SessionBuilder::new()
    .unwrap()
    .with_optimization_level(ort::session::builder::GraphOptimizationLevel::Level3)
    .unwrap();
let pool = AsyncSessionPool::commit_from_file(builder, Path::new("model.onnx"), 10).unwrap();
pool.load_all().await.unwrap();
pool.run_async(inputs! {...}, &RunOptions::new().unwrap())
    .await
    .unwrap();
```