Struct BasicInferer 

pub struct BasicInferer { /* private fields */ }

The most basic inferer provided will deal with a single element at a time, at the cost of reduced (but predictable) performance per element.

Pros

• Requires no tuning
• Very predictable performance across different workloads

Cons

• Scales linearly unless it’s the only code executing

Implementations

impl BasicInferer

pub fn from_model(model: InferenceModel) -> TractResult<Self>

Create an inferer for the provided inference model.

Errors

Will only forward errors from the tract_core::model::Graph optimization and graph building steps.

pub fn from_typed(model: TypedModel) -> TractResult<Self>

Trait Implementations

impl Inferer for BasicInferer

fn select_batch_size(&self, _: usize) -> usize

Query the inferer for how many elements it can deal with in a single batch.

fn infer_raw(&self, pad: &mut ScratchPadView<'_>) -> Result<(), Error>

Execute the model on the provided pre-batched data.

fn raw_input_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model inputs.

fn raw_output_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model outputs.

fn begin_agent(&self, _id: u64)

fn end_agent(&self, _id: u64)

fn input_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model inputs. This API only contains the external API, so code-based transforms outside the model are hidden.

fn output_shapes(&self) -> &[(String, Vec<usize>)]

Retrieve the name and shapes of the model outputs. This API only contains the external API, so code-based transforms outside the model are hidden.

impl IntoStateful for BasicInferer

fn into_stateful<WrapStack: InfererWrapper>( self, wrapper_stack: WrapStack, ) -> StatefulInferer<WrapStack, Self>

Construct a StatefulInferer by wrapping this concrete inferer with the given wrapper stack.