Struct DeviceMemSchema 

pub struct DeviceMemSchema {
    pub model_num_nodes: usize,
    pub by_partition: Vec<Partition>,
    pub by_steps: Vec<Vec<Option<NodeMemReq>>>,
}

This struct represent a memory schema for node output memory that are handled by a GPU.

Fields

model_num_nodes: usize

Total numbef in the model.

by_partition: Vec<Partition>

by_steps: Vec<Vec<Option<NodeMemReq>>>

Implementations

impl DeviceMemSchema

pub fn size_by_partition(&self) -> Vec<TDim>

Returns memory size of each inner partitions.

pub fn eval_size_by_partition( &self, symbols: &SymbolValues, ) -> TractResult<Vec<i64>>

Evaluate memory size by partition for given symbol values.

pub fn memory_size(&self) -> TDim

Returns total memory size required for the schema.

pub fn eval_memory_size(&self, symbols: &SymbolValues) -> TractResult<i64>

Evaluate memory size required for the schema for given symbol values.

pub fn compute_offset_by_node( &self, symbols: &SymbolValues, ) -> TractResult<Vec<Option<usize>>>

Compute offsets for each node for given symbols. Node ids are indexes in the returned vector.

pub fn eval_peak_memory_size(&self, symbols: &SymbolValues) -> TractResult<i64>

Evaluate peak memory size for given symbols. The return value is lower or equal to the memory size of the schema. The difference between peak memory size and memory size represents the memory fragmentation introduced by the schema.

pub fn eval_usage(&self, symbols: &SymbolValues) -> TractResult<f32>

Evaluate the usage for given symbols as the ratio between schema memory size and peak memory size. A value of 1.0 means that the schema doesn’t introduce memory fragmentation.

impl DeviceMemSchema

pub fn resolve( &self, symbols: &SymbolValues, ) -> TractResult<DeviceResolvedMemSchema>

Resolve Memory schema with given symbols.

pub fn build( model: &TypedModel, order: &[usize], hint: &SymbolValues, ) -> TractResult<DeviceMemSchema>

Build a memory schema for given model and execution order. The hint is used to optimize the memory schema because it is based on symbolic dimensions. That doesn’t mean it will be optimal for all possible values for symbolic dimensions.

Trait Implementations

impl Clone for DeviceMemSchema

fn clone(&self) -> DeviceMemSchema

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DeviceMemSchema

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DeviceMemSchema

fn fmt(&self, fmt: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for DeviceMemSchema

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for DeviceMemSchema

fn eq(&self, other: &DeviceMemSchema) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for DeviceMemSchema

impl StructuralPartialEq for DeviceMemSchema