atomr-accel-tensorrt 0.4.2

TensorRT engine builder + runtime as supervised atomr actors. Wraps libnvinfer / libnvonnxparser at runtime (proprietary library, not vendored). ONNX import, INT8/FP8 PTQ calibration, IPluginV3 Rust trampolines, dynamic shapes, refit.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

//! # atomr-accel-tensorrt
//!
//! TensorRT engine builder + runtime as supervised atomr actors.
//! Wraps NVIDIA's libnvinfer (and optionally libnvonnxparser) at
//! runtime — the library itself is **not** vendored because it is
//! proprietary; users opt in via the `tensorrt-link` feature and
//! either install TensorRT system-wide or set `LIBNVINFER_PATH`.
//!
//! ## Features
//!
//! - `tensorrt-link` — actually link libnvinfer at build time.
//!   Off-by-default so the crate compiles on hosts without
//!   TensorRT (used by CI + unit tests).
//! - `tensorrt-onnx` — pull in `nvonnxparser` for ONNX import.
//! - `tensorrt-plugin` — `IPluginV3` Rust trampolines.
//! - `tensorrt-int8` — INT8 calibration helpers (entropy / minmax).
//! - `tensorrt-fp8` — FP8 PTQ helpers (Hopper-class GPUs).
//!
//! ## Public surface
//!
//! - [`actor::TrtActor`] / [`actor::TrtMsg`] — sibling actor to
//!   `atomr_accel_cuda::DeviceActor`. Shares `Arc<CudaStream>` with
//!   the device actor so inference rides the same execution
//!   timeline.
//! - [`builder::IBuilderConfig`] — pure-Rust mirror of the TensorRT
//!   builder config, with knobs for precision, DLA, structured
//!   sparsity, tactic sources, timing cache, and engine refit.
//! - [`engine::TrtEngine`] — owned, immutable engine handle that's
//!   `Send + Sync` via newtype.
//! - [`runtime::TrtRuntime`] / [`runtime::ExecutionContext`] — load
//!   serialised plans + drive `enqueueV3` on a shared CUDA stream.
//! - [`onnx::OnnxParser`] — gated on `tensorrt-onnx`.
//! - [`calibration`] — gated on `tensorrt-int8` / `tensorrt-fp8`.
//! - [`plugin`] — gated on `tensorrt-plugin`.

#![allow(
    clippy::type_complexity,
    clippy::too_many_arguments,
    clippy::arc_with_non_send_sync
)]

pub mod actor;
pub mod builder;
pub mod engine;
pub mod error;
pub mod runtime;
pub mod sys;

#[cfg(feature = "tensorrt-onnx")]
pub mod onnx;

#[cfg(feature = "tensorrt-int8")]
pub mod calibration;

#[cfg(feature = "tensorrt-plugin")]
pub mod plugin;

pub use actor::{
    BuildFromOnnxReply, BuildReply, CreateContextReply, DeserializeReply, EnqueueReply,
    ExecuteReply, NetworkSource, RefitReply, RefitWeights, TrtActor, TrtMsg,
};
pub use builder::{
    BuilderFlags, DeviceType, IBuilderConfig, Precision, RefitPolicy, TacticSources,
};
pub use engine::{EnginePlan, TrtEngine, TrtRefitter};
pub use error::TrtError;
pub use runtime::{EnqueueRequest, ExecutionBindings, ExecutionContext, TensorShape, TrtRuntime};