Crate onnx_runtime_ep_api

Expand description

§`onnx-runtime-ep-api`

The Execution Provider (EP) interface for the ORT 2.0 runtime (see docs/ORT2.md §4). Every backend — CPU, CUDA, MLX, or a legacy ORT plugin loaded via dlopen — implements the same ExecutionProvider trait; only the loading mechanism differs.

This is a Phase 1 skeleton: trait and type signatures are real and reference the onnx_runtime_ir contract, but method bodies live in the concrete EP crates (e.g. onnx-runtime-ep-cpu). Zero-copy tensor views and the ORT ABI bridge require unsafe FFI and are stubbed here.

§Modules

provider — ExecutionProvider, EpConfig, device buffers/fences.
kernel — Kernel trait, KernelMatch, Cost.
registry — OpRegistry, OpKey, KernelFactory, EpRegistry.
epcontext — EpContext runtime form + source-keyed EpContextRegistry (§55).
tensor — TensorView / TensorMut zero-copy device views.
weight — capability-negotiated lazy WeightHandle delivery.
abi — ORT graph ABI bridge for legacy plugin EPs (Phase 2).

Re-exports§

pub use epcontext::EpContext;
pub use epcontext::EpContextRegistry;
pub use epcontext::build_ep_context_registry;
pub use kernel::ARG_KERNEL_VARIANT;
pub use kernel::ARG_KERNEL_VARIANT_REASON;
pub use kernel::CaptureSupport;
pub use kernel::Cost;
pub use kernel::Kernel;
pub use kernel::KernelInput;
pub use kernel::KernelMatch;
pub use kernel::KernelVariantSelection;
pub use kernel::ViewOutput;
pub use kernel::kernel_variant_tracing_enabled;
pub use kernel::record_kernel_variant_selection;
pub use kernel::record_kernel_variant_stage_selection;
pub use provider::CaptureRegionShapeStatus;
pub use provider::DeviceBuffer;
pub use provider::EpConfig;
pub use provider::EpId;
pub use provider::ExecutionProvider;
pub use provider::Fence;
pub use provider::OrtPluginExport;
pub use provider::StructuralCaptureDecline;
pub use registry::EpRegistry;
pub use registry::KernelFactory;
pub use registry::OpKey;
pub use registry::OpRegistry;
pub use tensor::DevicePtr;
pub use tensor::DevicePtrMut;
pub use tensor::ExternalMmapRegion;
pub use tensor::TensorBacking;
pub use tensor::TensorMut;
pub use tensor::TensorView;
pub use weight::ExecutionProviderCapabilities;
pub use weight::LazyDeviceWeightBinder;
pub use weight::LazyWeight;
pub use weight::LazyWeightBoundary;
pub use weight::NXRT_WEIGHT_PAGING_CAPABILITY;
pub use weight::NegotiatedWeight;
pub use weight::Phase3aHostOnlyBinder;
pub use weight::ResidentWeight;
pub use weight::ResidentWeightMaterializer;
pub use weight::WeightHandle;
pub use weight::WeightHandleError;

Modules§

abi: ORT graph ABI bridge for legacy plugin EPs (§3.4, §4.5).
epcontext: Runtime EpContext form and the source-keyed EpContextRegistry (docs/ORT2.md §55.1 / §55.6).
kernel: The Kernel trait and kernel-match / cost types (§4.2).
provider: The ExecutionProvider trait and its supporting types (§4.1).
registry: Op → kernel-factory registry and the EP registry (§4.3, §4.6).
tensor: Zero-copy device tensor views (§5.4) and their DLPack alignment (§5.3).
weight: Capability-negotiated lazy weight handles for executor-to-EP delivery.

Macros§

decline_capture: Decline the current capture-support query with an actionable reason.
deny: Decline the current supports_op call with an actionable reason.
record_kernel_variant: Record a selected kernel implementation and lazily formatted reason on the active executor op-span.
record_kernel_variant_stage: Record a named sub-decision kernel variant on the active executor op-span.
require: Require a claim condition, declining with an actionable reason when false.
require_capture: Require a capture precondition, declining with an actionable reason when false.

Structs§

DeviceId: A specific device instance: a DeviceType plus an ordinal index.

Enums§

DeviceType: A class of compute device / execution backend.
EpError: Errors produced by execution providers and kernels (subset of the runtime top-level Error, docs/ORT2.md §22).

Traits§

OptimizerPass: A single graph→graph rewrite (see docs/ORT2.md §18.1).

Type Aliases§

Result: A convenience Result alias for EP operations.