Struct StatefulInferer 

pub struct StatefulInferer<WrapStack: InfererWrapper, Inf: Inferer> { /* private fields */ }

An inferer that maintains state in wrappers around an inferer.

This is an alternative to direct wrapping of an inferer, which allows the inner inferer to be swapped out while maintaining state in the wrappers.

Implementations

impl<WrapStack: InfererWrapper, Inf: Inferer> StatefulInferer<WrapStack, Inf>

pub fn new(wrapper_stack: WrapStack, inferer: Inf) -> Self

Construct a new StatefulInferer by wrapping the given inferer with the given wrapper stack.

pub fn with_new_inferer<NewInf: Inferer>( self, new_inferer: NewInf, ) -> Result<StatefulInferer<WrapStack, NewInf>, (Self, Error)>

Replace the inner inferer with a new inferer while maintaining any state in wrappers.

Requires that the shapes of the policies are compatible, but they may be different inferer types. If this check fails, will return self unchanged.

pub fn replace_inferer(&mut self, new_inferer: Inf) -> Result<()>

Replace the inner inferer with a new inferer while maintaining any state in wrappers.

Requires that the shapes of the policies are compatible If this check fails, will not change self. Compared to [with_new_inferer], also requires that the new inferer has the same type as the old one.

pub fn check_compatible_shapes<Old: Inferer, New: Inferer>( old: &Old, new: &New, ) -> Result<(), Error>

Validate that [Old] and [New] are compatible with each other.

pub fn input_shapes(&self) -> &[(String, Vec<usize>)]

Returns the input shapes after all wrappers have been applied.

pub fn output_shapes(&self) -> &[(String, Vec<usize>)]

Returns the output shapes after all wrappers have been applied.

Trait Implementations

impl<WrapStack: InfererWrapper, Inf: Inferer> Inferer for StatefulInferer<WrapStack, Inf>

See Inferer for documentation.

fn select_batch_size(&self, max_count: usize) -> usize

Query the inferer for how many elements it can deal with in a single batch.

fn infer_raw(&self, batch: &mut ScratchPadView<'_>) -> Result<(), Error>

Execute the model on the provided pre-batched data.

fn raw_input_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model inputs.

fn raw_output_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model outputs.

fn begin_agent(&self, id: u64)

fn end_agent(&self, id: u64)

fn input_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model inputs. This API only contains the external API, so code-based transforms outside the model are hidden.

fn output_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model outputs. This API only contains the external API, so code-based transforms outside the model are hidden.