Struct OrtEngine 

pub struct OrtEngine { /* private fields */ }

An ort-loaded IMF artifact: encoder + decoder sessions plus the ONNX Runtime environment they share.

Implementations

impl OrtEngine

pub fn from_graphs(graphs: &ArtifactGraphs) -> Result<OrtEngine>

Build sessions from verified graphs (decoder-kv.onnx is selected for kv artifacts per the reference loader; this slice implements the plain greedy loop — kv sessions decode through the plain graph).

Trait Implementations

impl InferenceEngine for OrtEngine

fn encode(&mut self, input_ids: &[i64]) -> Result<Vec<f32>>

Encoder: single-row input_ids -> hidden states, shape (1, seq, d) flattened, row-major.

fn hidden_dims(&self, seq_len: usize) -> Vec<i64>

Hidden-state tensor dims (1, seq, d) — needed to build the decoder’s encoder_hidden_states input.

fn decode_step( &mut self, decoder_ids: &[i64], hidden: &[f32], ) -> Result<Vec<f32>>

Decoder step: full prefix decoder_ids -> logits of the last position (vocab-sized).