Struct InferenceSession 

pub struct InferenceSession { /* private fields */ }

A loaded model ready to run inference (§20.2).

Implementations

impl InferenceSession

pub fn load(path: impl AsRef<Path>) -> Result<Self, SessionError>

Primary entry point: load a model with auto device detection.

pub fn load_bytes(bytes: &[u8]) -> Result<Self, SessionError>

Load a model from an in-memory buffer.

pub fn from_graph(graph: Graph) -> Result<Self, SessionError>

Build a session directly from an in-memory IR Graph.

Initializer bytes are read from the graph’s inline [WeightRef]s, so no on-disk model or weight store is required. Useful for programmatically constructed graphs and tests.

pub fn builder() -> SessionBuilder

Start a configuration builder.

pub fn run( &mut self, inputs: &[(&str, &Tensor)], ) -> Result<Vec<Tensor>, SessionError>

Run inference with named inputs, returning the graph outputs in order.

pub fn inputs(&self) -> &[IoMeta]

Input metadata.

pub fn outputs(&self) -> &[IoMeta]

Output metadata.

pub fn cache_stats(&self) -> CacheStats

Kernel-cache statistics (§11.1); useful to observe warmup/run reuse.

pub fn control_flow_stats(&self) -> ControlFlowStats

Control-flow subgraph build/run statistics. A Loop or Scan body with a stable input-shape signature should build once and run many times.

pub fn warmup(&mut self, shapes: &[WarmupShape]) -> Result<(), SessionError>

Pre-compile kernels for common shapes to avoid first-inference latency (§11.3). Phase-1 minimal: the compiled plan’s shapes already key the cache, so this repopulates it for the plan; shapes are validated to name real inputs.

pub fn ep_context_config(&self) -> &EpContextDumpConfig

The EPContext dump configuration parsed from the ep.context_* session options (§21.4). Disabled by default.

pub fn graph(&self) -> &Graph

The session’s (post-optimize) compiled graph.

This is the graph the executor runs and the same one Self::export_ep_context serialises — a caller identifying the NodeIds of a compiled partition (the CompiledPartition::covered_nodes) must read them from here so they reference the exact nodes the exporter will splice out. This is the compiler-integration seam: a real compiling EP inspects this graph to choose the subgraphs it claims.

pub fn export_ep_context( &self, orig_path: &Path, partitions: &[CompiledPartition<'_>], ) -> Result<PathBuf, SessionError>

Export a com.microsoft::EPContext context-cache model for this session (§55.4 dump path), driven by the ep.context_* session options (Self::ep_context_config).

orig_path is the source model path the default output location (<orig>_ctx.onnx) is derived from when ep.context_file_path is unset. partitions are the EP-compiled partitions to serialise — each names the ExecutionProvider that compiled it, so the driver pulls the blob + SDK version via save_context and the source key via context_source_keys (§55.6 — nothing is hardcoded).

When ep.context_enable is false (the default) this is a no-op: no EP save_context is called and no files are written; it returns the path it would have written to.

Compiler-integration seam

The Phase-1 CPU EP has no compile step, so no real EP yet yields CompiledPartitions — partitions is therefore supplied by the caller (proven end-to-end with a mock compiling EP in the crate tests). TODO(compiler): when a real compiling EP lands, collect its partitions from the compile/placement stage and call this internally at build time so a session created with ep.context_enable=1 dumps automatically.