The provided handle is invalid. This error is typically used by NNEngine
when interfacing with another API such as OpenCL or OpenVX which require
native handles for their internal API.
The tensor’s shape is invalid for the given operation. It differs from
the shape mismatch in that the shape is invalid on its own and not
relative to another related tensor. An example would be a shape with
more than one -1 dimension.
The internal kernel or subroutine required to complete an operation using
the engine plugin was missing. An example would be OpenCL or OpenVX
operation where the kernel implementation cannot be located.
Signals an API has not been implemented. Can be caught by the core
DeepViewRT library when interfacing with engine plugins to gracefully
fallback to the native implementation.
When attempting to run an operation and the input/output tensors have
invalid or unsupported shape combinations. Some operations require the
shapes to be the same while others, such as arithmetic broadcasting
operations, will support various shape combinations but if the provided
pairs are invalid then the shape mismatch is returned.
The tensor has no data or the data is not currently accessible. An
example of the latter would be attempting to call @ref nn_tensor_maprw
while the tensor was already mapped read-only or write-only.
When attempting to run an operation where the input/output tensors are
of different types and the operation does not support automatic type
conversions.
Affine quantization with separate parameters applied to each channel.
Also known as per-axis where the axis is always the channel “C” axis in
a NCHW, NHWC, and so-on shaped tensor.
Initializes an NNContext and allocates required memories. If any of the
pointers are NULL malloc will be called automatically to create the memory
using the provided sizes. For memory_size and cache_size if these are 0
then they will not be initialized.
Initializes an NNContext into the provided memory which MUST be at least
NN_CONTEXT_SIZEOF bytes. If any of the pointers are NULL malloc will be
called automatically to create the memory using the provided sizes. For
memory_size and cache_size if these are 0 then they will not be initialized.
Returns the actual size of the context structure. This size will be smaller
than @ref NN_CONTEXT_SIZEOF which contains additional padding for future
extension. Since @ref nn_context_sizeof() is called dynamically at runtime
it can return the true and unpadded size.
Runs layer with index from model within the given context object.
If index is invalid NN_ERROR_INVALID_LAYER is returned, this can be
used to determine when at the end of the model.
Loads the plugin to provided engine object. The plugin should point to an
engine plugin library either as an absolute or relative path or be found in
the standard OS search path for shared libraries.
The actual size of the NNEngine structure. This will differ from the size
defined by @ref NN_ENGINE_SIZEOF as the later is padded for future API
extensions while this function returns the actual size currently required.
Initializes the library with optional parameters. This function MUST be
called before any others (though nn_version and nn_strerror are safe) and
MUST not be called again unless care is taken to protect this call.
Returns the label of the given index within the given model object. If the
model contains no labels or the index is out of range then NULL will be
returned.
Returns an NNModelParameter from the model at the layer index defined by
layer using the parameter key. If the layer does not contain this parameter
NULL is returned.
Returns string data for parameter at layer index for string
array element . This is a convenience wrapper around acquiring the
parameter followed by acquiring the data.
Returns number of string elements in the data_str array for the specified
layer and parameter key. This is a convenience wrapper around acquiring the
parameter followed by acquiring the data.
Returns the array of quantization scales, and optionally the number of scales
in the array. The length will either be 0, 1, or equal to the number of
channels in an NHWC/NCHW tensor.
Returns the array of quantization zero-points, and optionally the number of
zero-points in the array. The length will either be 0, 1, or equal to the
number of channels in an NHWC/NCHW tensor.
Returns parameter string data at desired index. This data handler is
different from the others which return the array as strings are themselves
arrays and need special handling. Refer to @ref
nn_model_parameter_data_str_len() to query the size of the data_str array,
which refers to the number of strings in this parameter.
Returns the shape of the parameter data or NULL if no shape was defined. If
n_dims is non-NULL the number of dimensions will be stored there. The shape
attribute is not required for parameters but can be used either on its own
or as part of defining layout of data attributes.
Attempts to validate model, this is automatically called by nn_model_load and
nn_model_mmap. The function returns 0 on success, otherwise it will return
an error code which can be turned into a string by calling
@ref nn_model_validate_error() with the return value from
@ref nn_model_validate().
Assigns the tensor parameters and optionally data pointer. The default
implementation uses the data buffer as the internal storage for tensor data
and it MUST outlive the tensor. For engine plugins they may choose how to
use the data but for the OpenCL example if data is provided it will be copied
into the OpenCL buffer then otherwise never used again. If NULL is provided
for data the OpenCL engine would create the memory and leave it unassigned.
Element-wise comparison of two tensors within a given tolerance, returning
total number of errors relative to the left tensor. If the two tensors are
incompatible the volume of the left tensor is returned (all elements
invalid).
Loads a tensor with data from a user buffer
User has to maintain the buffer and ensure compatibility with NHWC tensor
Function will return error if there is a size mismatch
i.e (bufsize != nn_tensor_size(tensor)) or tensor is invalid
Fills the tensor with the provided constant. The constant is captured
as double precision (64-bit floating point) which has 53-bits of precision
on whole numbers. This means the constant CANNOT represent all 64-bit
integers but it CAN represent all 32-bit and lower integers. If full
64-bit integer support is required @ref nn_tensor_map can be used though
it is less efficient with some engines because of the addition memory
transfer required.
Initializes the tensor using provided memory. The memory MUST be at least
the size returned by @ref nn_tensor_sizeof(). This size does not include
the actual tensor data which is allocated separately, either by requesting
the implementation to allocate the buffer or attaching to externally
allocated memory.
Returns the I/O time information stored in the tensor. The time is returned
in nanoseconds of the duration of the last map/unmap pair. When tensors are
mapped to the CPU (no accelerator engine is loaded) then times are expected
to be zero time as no mapping is actually required and the internal pointer
is simply returned. When an accelerator engine is used, such as OpenVX,
then the io_time measures the time the map/unmap or copy operations took to
complete.
Loads an image from the provided buffer and decodes it accordingly, the
function uses the images headers to find an appropriate decoder. The
function will handle any required casting to the target tensor’s format.
Loads an image from the provided buffer and decodes it accordingly, the
function uses the images headers to find an appropriate decoder. The
function will handle any required casting to the target tensor’s format and
will apply image standardization (compatible with tensorflow’s
tf.image.per_image_standardization) if the proc parameter is set to
NN_IMAGE_PROC_WHITENING.
Maps the tensor’s memory and returns the client accessible pointer. This is
the read-only version which causes the engine to download buffers to the CPU
memory space if required but will not flush back to the device on unmap.
Maps the tensor’s memory and returns the client accessible pointer. This is
the read-write version which causes the engine to download buffers to the CPU
memory space if required and will also flush back to the device on unmap.
Maps the tensor’s memory and returns the client accessible pointer. This is
the write-only version which will not cause a download of the buffers to the
CPU memory space on map but will upload to the device on unmap.
Returns the offset of a given tensor using variable length dimensions. This
works the same as @ref nn_tensor_offset() but uses variable arguments. The
user must provide @p n_dims number of parameters after the
@p n_dims parameter.
nn_tensor_pad implements a padded Tensor to Tensor copy. This can be used to
achieve the various convolution padding strategies (SAME, FULL). For example
SAME conv2d would use the following padded_copy before running the conv2d
layer.
nn_tensor_padding calculates the paddings for the given tensor, padtype,
window, stride, and dilation given n_dims being queried from the tensor’s
nn_tensor_dims().
Writes the tensor inforamtion to the FILE stream provided. The format is
“[D0 D1 D2 D3]” where D0..D3 are the dimensions provided. If the data
parameter is true the format will be followed by “: …” where … is the
string representation of the tensor’s data.
Sets the panel size of the tensor. This is primarily an internal API used
to store the vectorization length when shuffling tensors into an optimized
tile format.
Returns the time information stored in the tensor. The time is returned
in nanoseconds of the duration of the last operation the wrote into this
tensor. causes a nn_tensor_sync on the target tensor.
Enumeration of all errors provided by DeepViewRT. Most functions will
return an NNError with NN_SUCCESS being zero. A common usage pattern for
client code is to check for err using if (err) ... as any error condition
will return non-zero.