Enum DeviceTensor 

pub enum DeviceTensor {
    Owned(Box<dyn OwnedDeviceTensor>),
    View(DeviceView),
}

A GPU tensor: either a buffer of its own, or a window into someone else’s.

Variants

Owned(Box<dyn OwnedDeviceTensor>)

View(DeviceView)

Implementations

impl DeviceTensor

pub const SUPPORTED_DT: [DatumType; 11]

pub fn tname(dt: DatumType) -> TractResult<&'static str>

pub fn uninitialized_dt( dt: DatumType, shape: &[usize], ) -> TractResult<DeviceTensor>

Create an uninitialized DeviceTensor

pub fn uninitialized<T: Datum>(shape: &[usize]) -> TractResult<DeviceTensor>

pub fn uninitialized_exotic( exotic_fact: Box<dyn ExoticFact>, ) -> TractResult<DeviceTensor>

pub fn from_shape<T: Copy + Datum>( shape: &[usize], data: &[T], ) -> TractResult<DeviceTensor>

pub fn is_supported_dt(dt: DatumType) -> bool

pub fn datum_type(&self) -> DatumType

Get the datum type of the tensor.

pub fn rank(&self) -> usize

Get the number of dimensions (or axes) of the tensor.

pub fn shape(&self) -> &[usize]

Get the shape of the tensor.

pub fn len(&self) -> usize

Get the number of values in the tensor.

pub fn strides(&self) -> &[isize]

Get the strides of the tensor.

pub fn device_buffer(&self) -> &dyn DeviceBuffer

Get underlying inner buffer.

pub fn buffer_offset<I: Copy + 'static>(&self) -> I

where usize: AsPrimitive<I>,

Get underlying inner buffer offset

pub fn device_buffer_ptr(&self) -> *const c_void

pub fn description(&self) -> String

Returns short description of the inner tensor.

pub fn reshaped(&self, shape: TVec<usize>) -> TractResult<Self>

Reshaped tensor with given shape.

pub fn restrided(&self, strides: TVec<isize>) -> TractResult<Self>

pub fn into_tensor(self) -> Tensor

Convert device tensor to a Tensor backed by device storage.

The resulting tensor carries the real datum type and shape from the device tensor (e.g. F32 / [2,3]), rather than an exotic scalar wrapper.

pub fn prefix_window( &self, axis: usize, len: usize, ) -> TractResult<DeviceTensor>

The first len along axis, as a view keeping this tensor’s strides.

What a buffer holding spare capacity hands out: the live prefix costs nothing to describe, and the tail it leaves behind is simply not part of the shape. A DeviceView is turn-scoped, so this is for a reader downstream of the producer, never for the application.

The result is strided wherever axis is not the outermost: a consumer has to read shape and strides rather than assume a packed tensor.

pub fn to_host(&self) -> TractResult<Arc<Tensor>>

Synchronize the GPU Tensor by completing all current commands on GPU and returns the inner tensor.

Trait Implementations

impl Clone for DeviceTensor

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DeviceTensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DeviceTensor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for DeviceTensor

impl From<DeviceView> for DeviceTensor

fn from(view: DeviceView) -> Self

Converts to this type from the input type.

impl Hash for DeviceTensor

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl IntoDevice<DeviceTensor> for Tensor

fn into_device(self) -> TractResult<DeviceTensor>

impl IntoDevice<DeviceTensor> for Arc<Tensor>

fn into_device(self) -> TractResult<DeviceTensor>

impl PartialEq for DeviceTensor

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for DeviceTensor

impl TensorStorage for DeviceTensor

fn byte_len(&self) -> usize

fn is_empty(&self) -> bool

fn deep_clone(&self) -> Box<dyn TensorStorage>

fn as_plain_ram(&self) -> Option<&PlainStorage>

fn as_plain_ram_mut(&mut self) -> Option<&mut PlainStorage>

fn into_plain_ram(self: Box<Self>) -> Option<PlainStorage>

fn dyn_hash(&self, _state: &mut dyn Hasher)

fn is_exotic(&self) -> bool

True when the tensor’s datum type and shape do not describe it on their own, so a fact over it carries an ExoticFact.

fn in_ram(&self) -> bool

True when the bytes are in host memory, readable without a transfer.

fn slice( &self, dt: DatumType, shape: &[usize], axis: usize, start: usize, end: usize, ) -> TractResult<Option<Tensor>>

Slice along axis, if this storage can serve it in its own memory – for free where the slice is already a range it holds, by a copy it can make where it is otherwise.

fn exotic_fact( &self, _shape: &[usize], ) -> TractResult<Option<Box<dyn ExoticFact>>>

Build the ExoticFact that describes this storage for use in TypedFact.

fn materialize_plain_ram(&self) -> Result<&PlainStorage, Error>

Plain storage for the tensor’s bytes, producing it if this storage can.