Module tract_core::model

B

Models and their lifecycle

In order to reason on the model and performs optimisations, a model needs to be typed. This means all tensor exchanged between the nodes have a well defined element type (f32, i32, etc) and a shape ([1, 12, 53, 13]).

A model typically starts as an InferenceModel, with minimum or partial tensor type information. At this stage, the application developper can add types and shapes hints (like the model inputs and output element types and shapes), then tract will perform type inference propagating this information. Hopefully tract will be able to infer a type and shape for all tensors in the model graph.

At this stage, the model can be converted into a TypedModel.

InferanceModel and TypeModel are two variants of ModelImpl, Parameterized by a Fact implementation: TypedModel uses TypedFact, enforcing complete determination of element type and shape, and allowing a constant value for the tensor. InferenceModel uses InferenceFact, which can handle partial information.

A third type of Model exists and can be useful: NormalizedModel, using NormalizedFact. In this case, constant values are no longer allowed: all tensors exchanged in the network are actual variables. Parts of the graph producing constant values (like constant weights or hyper-parameter computation implemented in the graph) have been eliminated from the graph. This normal form is akin to an IR in compiler technologies. This is the favourite form on which tract optimisation should be implemented.

We call declutter the process getting the network closer to its normal form: as we just said, constant must be absorbed by the operator that will use them. For instance an Add node adding a constant input to a variable tensor input would be replaced by an unary Add operator taking only the variable input and for which the constant to add is a fixed construction attribute. In the same decluttering process, we try and replace proprietary operators (eg, from TensorFlow) by tract core operators: it is not always possible to simply map TensorFlow operators to tract-core while loading the network: their interfaces can be different (what is an input, what is an attribute) and constant propagation may be necessary before the right core operator could be chosen.

Re-exports

pub use self::order::eval_order;

pub use crate::analyser::types::InferenceFact;

pub use crate::ops::InferenceOp;

pub use crate::ops::Op;

pub use crate::ops::TypedOp;

Modules

order

Evaluation order for nodes.

Structs

BaseNode	A Node in an Model.
InletId	Identifier for a node input in the graph.
ModelImpl	Main model class
ModelPatch	A change to apply to a model.
NormalizedFact	Tensor information for Normalized models.
OutletFact	Information for each outlet of a node
OutletId	Identifier for a node output in the graph.
ShapeInfo	Fully determined dimension of a tensor.
StreamInfo	Streaming information for a streamed tensor.
TypedFact	Fully determined tensor information for TypedModel.

Traits

Fact	Type information about a tensor: shape, and element type, in various state of determination.
Model	Common methods for all variants of model.
ModelDsl	Extensions on ModelImpl to explore and build graph models more easily.
ModelDslConst	Extension to add constants to model that tolerates them.
ModelSpecialOps
ModelWireNode

Type Definitions

InferenceModel	A model with partially types and shapes, as produced by parsing ONNX or Tensorflow graphs.
InferenceModelPatch	A ModelPatch for InferenceModel.
InferenceNode	Node for InferenceModel graph
InferenceSimplePlan	An execution plan for InferenceModel.
InferenceSimpleState	An execution state for InferenceModel.
Node
NormalizedModel	A model with determined types and shapes, where constant have been eleminated from the graph.
NormalizedModelPatch	A ModelPatch for NormalizedModel.
NormalizedNode	A Node for NormalizedModel.
NormalizedSimplePlan	An execution plan for NormalizedModel.
NormalizedSimpleState	An execution state for TypedModel.
TVec	A Smallvec instantiation with 4 embeddable values.
TypedModel	A model with completely determined types and shapes.
TypedModelPatch	A ModelPatch for TypedModel.
TypedNode	Node for TypedModel graph
TypedSimplePlan	An execution plan for TypedModel.
TypedSimpleState	An execution state for TypedModel.