[−][src]Module tract_core::model
B
Models and their lifecycle
In order to reason on the model and performs optimisations, a model needs
to be typed
. This means all tensor exchanged between the nodes have a
well defined element type (f32, i32, etc) and a shape ([1, 12, 53, 13]).
A model typically starts as an InferenceModel
, with minimum or partial
tensor type information. At this stage, the application developper can add
types and shapes hints (like the model inputs and output element types
and shapes), then tract
will perform type inference propagating this
information. Hopefully tract
will be able to infer a type and shape for
all tensors in the model graph.
At this stage, the model can be converted into a TypedModel
.
InferanceModel and TypeModel are two variants of ModelImpl
, Parameterized
by a TensorInfo implementation: TypedModel uses TypedTensorInfo, enforcing
complete determination of element type and shape, and allowing a constant
value for the tensor. InferenceModel uses TensorFact, which can handle
partial information.
A third type of Model exists and can be useful: NormalizedModel, using NormalizedTensorInfo. In this case, constant values are no longer allowed: all tensors exchanged in the network are actual variables. Parts of the graph producing constant values (like constant weights or hyper-parameter computation implemented in the graph) have been eliminated from the graph. This normal form is akin to an IR in compiler technologies. This is the favourite form on which tract optimisation should be implemented.
We call declutter
the process getting the network closer to its normal
form: as we just said, constant must be absorbed by the operator that will
use them. For instance an Add node adding a constant input to a variable
tensor input would be replaced by an unary Add operator taking only the
variable input and for which the constant to add is a fixed construction
attribute. In the same decluttering process, we try and replace proprietary
operators (eg, from TensorFlow) by tract core operators: it is not always
possible to simply map TensorFlow operators to tract-core while loading the
network: their interfaces can be different (what is an input, what is an
attribute) and constant propagation may be necessary before the right
core operator could be chosen.
Re-exports
pub use self::order::eval_order; |
pub use crate::analyser::types::TensorFact; |
pub use crate::ops::InferenceOp; |
pub use crate::ops::Op; |
Modules
order | Evaluation order for nodes. |
Structs
BaseNode | A Node in an Model. |
InletId | Identifier for a node input in the graph. |
ModelImpl | Main model class |
ModelPatch | A change to apply to a model. |
NormalizedTensorInfo | Tensor information for Normalized models. |
OutletFact | Information for each outlet of a node |
OutletId | Identifier for a node output in the graph. |
ShapeInfo | Fully determined dimension of a tensor. |
StreamInfo | Streaming information for a streamed tensor. |
TypedTensorInfo | Fully determined tensor information for TypedModel. |
Traits
Model | Common methods for all variants of model. |
ModelDsl | Extensions on ModelImpl to explore and build graph models more easily. |
ModelDslConst | Extension to add constants to model that tolerates them. |
ModelDslInfer | Model extension for InferenceModel |
TensorInfo | Type information about a tensor: shape, and element type, in various state of determination. |
Type Definitions
InferenceModel | A model with partially types and shapes, as produced by parsing ONNX or Tensorflow graphs. |
InferenceModelPatch | A ModelPatch for InferenceModel. |
InferenceNode | Node for InferenceModel graph |
InferenceSimplePlan | An execution plan for InferenceModel. |
InferenceSimpleState | An execution state for InferenceModel. |
Node | |
NormalizedModel | A model with determined types and shapes, where constant have been eleminated from the graph. |
NormalizedModelPatch | A ModelPatch for NormalizedModel. |
NormalizedNode | A Node for NormalizedModel. |
NormalizedSimplePlan | An execution plan for NormalizedModel. |
NormalizedSimpleState | An execution state for TypedModel. |
TVec | A Smallvec instantiation with 4 embeddable values. |
TypedModel | A model with completely determined types and shapes. |
TypedModelPatch | A ModelPatch for TypedModel. |
TypedNode | Node for TypedModel graph |
TypedSimplePlan | An execution plan for TypedModel. |
TypedSimpleState | An execution state for TypedModel. |