Skip to main content

InferenceEngine

interscript_rs::imf::infer

Trait InferenceEngine 

Source 
pub trait InferenceEngine {
    // Required methods
    fn encode(&mut self, input_ids: &[i64]) -> Result<Vec<f32>>;
    fn hidden_dims(&self, seq_len: usize) -> Vec<i64>;
    fn decode_step(
        &mut self,
        decoder_ids: &[i64],
        hidden: &[f32],
    ) -> Result<Vec<f32>>;
}
Expand description

The engine abstraction: one encoder pass + one decoder step at a time. ort sessions implement it; tests drive the loop with a deterministic mock.

Required Methods§

Source

fn encode(&mut self, input_ids: &[i64]) -> Result<Vec<f32>>

Encoder: single-row input_ids -> hidden states, shape (1, seq, d) flattened, row-major.

Source

fn hidden_dims(&self, seq_len: usize) -> Vec<i64>

Hidden-state tensor dims (1, seq, d) — needed to build the decoder’s encoder_hidden_states input.

Source

fn decode_step( &mut self, decoder_ids: &[i64], hidden: &[f32], ) -> Result<Vec<f32>>

Decoder step: full prefix decoder_ids -> logits of the last position (vocab-sized).

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§