Struct SingleBatchOutput

pub struct SingleBatchOutput<'r, 's> {
    pub session_outputs: SessionOutputs<'r, 's>,
    pub attention_mask_array: Array2<i64>,
}

SingleBatchOutput contains the output of a single batch of inference.

In the future, each batch will need to deal with its own post-processing, such as pooling etc. This struct should contain all the necessary information for the post-processing to be performed.

Fields

session_outputs: SessionOutputs<'r, 's>

attention_mask_array: Array2<i64>

Implementations

impl SingleBatchOutput<'_, '_>

pub fn select_output( &self, precedence: &impl OutputPrecedence, ) -> Result<ArrayView<'_, f32, Dim<IxDynImpl>>>

Select the output from the session outputs based on the given precedence.

This returns a view into the tensor, which can be used to perform further operations.

pub fn select_and_pool_output( &self, precedence: &impl OutputPrecedence, pooling_opt: Option<Pooling>, ) -> Result<Array2<f32>>

Select the output from the session outputs based on the given precedence and pool it.

This function will pool the output based on the given pooling option, if any.