Struct SingleBatchOutput 

pub struct SingleBatchOutput {
    pub outputs: Vec<(String, Value)>,
    pub attention_mask_array: Array2<i64>,
}

SingleBatchOutput contains the output of a single batch of inference.

In the future, each batch will need to deal with its own post-processing, such as pooling etc. This struct should contain all the necessary information for the post-processing to be performed.

Fields

outputs: Vec<(String, Value)>

attention_mask_array: Array2<i64>

Implementations

impl SingleBatchOutput

pub fn select_output( &self, precedence: &impl OutputPrecedence, ) -> Result<ArrayView<'_, f32, Dim<IxDynImpl>>>

Select the output from the session outputs based on the given precedence.

This returns a view into the tensor, which can be used to perform further operations.

pub fn select_and_pool_output( &self, precedence: &impl OutputPrecedence, pooling_opt: Option<Pooling>, ) -> Result<Array2<f32>>

Select the output from the session outputs based on the given precedence and pool it.

This function will pool the output based on the given pooling option, if any.