Failed to verify graph generateed from model.
Internal error without a specific error code, catch-all error.
The requested axis for an operation was invalid or unsupported.
The requested engine is invalid.
The provided handle is invalid. This error is typically used by NNEngine
when interfacing with another API such as OpenCL or OpenVX which require
native handles for their internal API.
When working with a model a reference was made to a layer which did not
exist.
The requested ordering was invalid.
A required parameter was missing or NULL or simply invalid.
The quantization parameters are invalid.
The tensor’s shape is invalid for the given operation. It differs from
the shape mismatch in that the shape is invalid on its own and not
relative to another related tensor. An example would be a shape with
more than one -1 dimension.
The internal kernel or subroutine required to complete an operation using
the engine plugin was missing. An example would be OpenCL or OpenVX
operation where the kernel implementation cannot be located.
A required resource was missing or the reference invalid.
Failed to generate graph representation of model.
The model is invalid or corrupted.
An operation referenced a model but the model was not provided.
Signals an API has not been implemented. Can be caught by the core
DeepViewRT library when interfacing with engine plugins to gracefully
fallback to the native implementation.
Out of memory error, returned if a call to malloc returns NULL or similar
error from an underlying engine plugin.
Out of resources errors are similar to out of memory though sometimes
treated separately by underlying engine plugins.
When attempting to run an operation and the input/output tensors have
invalid or unsupported shape combinations. Some operations require the
shapes to be the same while others, such as arithmetic broadcasting
operations, will support various shape combinations but if the provided
pairs are invalid then the shape mismatch is returned.
The string was too large.
A system error occured when interfacing with an operating system
function. On some systems errno might be updated with the underlying
error code.
The tensor has no data or the data is not currently accessible. An
example of the latter would be attempting to call @ref nn_tensor_maprw
while the tensor was already mapped read-only or write-only.
The operation does not support the tensor’s type.
For operations which can operate on an array of inputs, the provided list
of inputs was too large.
When attempting to run an operation where the input/output tensors are
of different types and the operation does not support automatic type
conversions.
Successfull operation, no error.
Affine quantization with separate parameters applied to each channel.
Also known as per-axis where the axis is always the channel “C” axis in
a NCHW, NHWC, and so-on shaped tensor.
Affine quantization with parameters applied globally across the tensor.
Quantized using Dynamic Fixed Point.
No quantization for tensor.
Half precision (16-bit) floating point tensor data.
Single precision (32-bit) floating point tensor data.
Double precision (64-bit) floating point tensor data.
Signed 8-bit integer tensor data internally @ref int8_t
Signed 16-bit integer tensor data internally @ref int16_t
Signed 16-bit integer tensor data internally @ref int32_t
Signed 16-bit integer tensor data internally @ref int64_t
Raw byte-stream tensor, useful for encoded tensors such as PNG images.
The size of this tensor would be in bytes.
String tensor data, a single dimension would hold one null-terminated
string of variable length. A standard C char* array.
Unsigned 8-bit integer tensor data internally @ref uint8_t
Unsigned 16-bit integer tensor data internally @ref uint16_t
Unsigned 16-bit integer tensor data internally @ref uint32_t
Unsigned 16-bit integer tensor data internally @ref uint64_t
@public @memberof NNContext
@since 2.2
Returns the engine used by the given context object.
Initializes an NNContext and allocates required memories. If any of the
pointers are NULL malloc will be called automatically to create the memory
using the provided sizes. For memory_size and cache_size if these are 0
then they will not be initialized.
Initializes an NNContext into the provided memory which MUST be at least
NN_CONTEXT_SIZEOF bytes. If any of the pointers are NULL malloc will be
called automatically to create the memory using the provided sizes. For
memory_size and cache_size if these are 0 then they will not be initialized.
@public @memberof NNContext
@since 2.2
Returns the currently loaded model blob for the context.
Loads the model provided by the input into the context.
Frees the memory used by the model within the given context object.
Release the memory being used by the given context object.
Runs the model within the given context object.
Returns the actual size of the context structure. This size will be smaller
than @ref NN_CONTEXT_SIZEOF which contains additional padding for future
extension. Since @ref nn_context_sizeof() is called dynamically at runtime
it can return the true and unpadded size.
Runs layer with index from model within the given context object.
If index is invalid NN_ERROR_INVALID_LAYER is returned, this can be
used to determine when at the end of the model.
Returns the tensor with the given name within the model provided by the given
context object.
Returns the tensor at the given index with the model provided by the given
context object.
@public @memberof NNContext
@since 2.4
@public @memberof NNContext
@since 2.4
Initializes the NNEngine structure using the provided memory or allocating a
new buffer is none was provided.
Loads the plugin to provided engine object. The plugin should point to an
engine plugin library either as an absolute or relative path or be found in
the standard OS search path for shared libraries.
Returns the name of the engine object.
Returns handle of the NNEngine object.
Releases the memory that was being used by the engine.
The actual size of the NNEngine structure. This will differ from the size
defined by @ref NN_ENGINE_SIZEOF as the later is padded for future API
extensions while this function returns the actual size currently required.
Unloads the plugin from the given engine object.
Returns the version of the engine object.
Exposes the free() function
Initializes the library with optional parameters. This function MUST be
called before any others (though nn_version and nn_strerror are safe) and
MUST not be called again unless care is taken to protect this call.
Exposes the malloc() function
Returns the minimum cache size of a given model object.
Returns the optimum cache size of a given model object.
Returns the list of model input indices and optionally the number of inputs.
Returns the label of the given index within the given model object. If the
model contains no labels or the index is out of range then NULL will be
returned.
Returns the number of labels within a given model object.
Returns an optional icon resource for the provided label index.
Returns the natural data axis for the tensor or -1 if one is not set.
Returns the number of layers within a given model object.
Returns the datatype of a layer at the given index within the given model
object.
Returns the datatype of a layer at the given index within the given model
object.
Returns the number of inputs to a layer at the given index within the given
model object.
Returns the index of a given layer with the name provided in the given model
object.
Returns the name of a layer at a given index within the given model object.
Returns an NNModelParameter from the model at the layer index defined by
layer using the parameter key. If the layer does not contain this parameter
NULL is returned.
Returns float data for parameter at layer index . This is a
convenience wrapper around acquiring the parameter followed by acquiring the
data.
Returns int16 data for parameter at layer index . This is a
convenience wrapper around acquiring the parameter followed by acquiring the
data.
Returns raw data for parameter at layer index . This is a
convenience wrapper around acquiring the parameter followed by acquiring the
data.
Returns string data for parameter at layer index for string
array element . This is a convenience wrapper around acquiring the
parameter followed by acquiring the data.
Returns number of string elements in the data_str array for the specified
layer and parameter key. This is a convenience wrapper around acquiring the
parameter followed by acquiring the data.
Returns the shape of the model parameter for layer at index .
Returns the array of quantization scales, and optionally the number of scales
in the array. The length will either be 0, 1, or equal to the number of
channels in an NHWC/NCHW tensor.
Returns the shape of a layer at the given index within the given model
object.
Returns the type of a layer at the given index within the given model object.
Returns the type ID of the layer.
Returns the array of quantization zero-points, and optionally the number of
zero-points in the array. The length will either be 0, 1, or equal to the
number of channels in an NHWC/NCHW tensor.
Returns the memory size of the given model object.
Returns the name of the given model object. Names are optional and if the
model does not contain a name then NULL will be returned.
Returns the list of model output indices and optionally the number of
outputs.
Returns parameter float data, length of the array is optionally stored into
the length parameter if non-NULL.
Returns parameter int8_t data, length of the array is optionally stored into
the length parameter if non-NULL.
Returns parameter int16_t data, length of the array is optionally stored into
the length parameter if non-NULL.
Returns parameter int32_t data, length of the array is optionally stored into
the length parameter if non-NULL.
Returns parameter raw data pointer, length of the array is optionally stored
into the length parameter if non-NULL.
Returns parameter string data at desired index. This data handler is
different from the others which return the array as strings are themselves
arrays and need special handling. Refer to @ref
nn_model_parameter_data_str_len() to query the size of the data_str array,
which refers to the number of strings in this parameter.
Returns the number of strings in the parameter’s data_str attribute.
Returns the shape of the parameter data or NULL if no shape was defined. If
n_dims is non-NULL the number of dimensions will be stored there. The shape
attribute is not required for parameters but can be used either on its own
or as part of defining layout of data attributes.
Retrieves a reference to the resource with the given name.
Retrieves a reference to the resource at the provided index.
The number of resources defined in the model.
Returns the raw binary data for the resource, the size of the data will be
saved in @p data_size if non-NULL.
Returns the meta string for the resource.
Returns the mime type string for the resource.
The unique name of the resource as can be used to retrieve the resource using
@ref nn_model_resource().
Currently returns 0
Currently returns NULL (UPDATE WHEN FUNCTION IS UPDATED)
Attempts to validate model, this is automatically called by nn_model_load and
nn_model_mmap. The function returns 0 on success, otherwise it will return
an error code which can be turned into a string by calling
@ref nn_model_validate_error() with the return value from
@ref nn_model_validate().
Returns the string associated with a given error returned from
@ref nn_model_validate().
Returns the string associated with a given error.
Allocates the internal memory for the tensor.
Assigns the tensor parameters and optionally data pointer. The default
implementation uses the data buffer as the internal storage for tensor data
and it MUST outlive the tensor. For engine plugins they may choose how to
use the data but for the OpenCL example if data is provided it will be copied
into the OpenCL buffer then otherwise never used again. If NULL is provided
for data the OpenCL engine would create the memory and leave it unassigned.
Returns the auxiliary object’s free function, or NULL if none is attached.
Frees the auxiliary object associated with the given name parameter.
Returns the auxiliary object for the tensor, or NULL if none is attached.
Acquire the auxiliary object associated with the given name parameter.
Returns the natural data axis of the tensor.
Element-wise comparison of two tensors within a given tolerance, returning
total number of errors relative to the left tensor. If the two tensors are
incompatible the volume of the left tensor is returned (all elements
invalid).
nn_tensor_concat concatenates all of the given input tensors into
the given output tensor.
Copies the contents of source tensor into destination tensor.
Loads a tensor with data from a user buffer
User has to maintain the buffer and ensure compatibility with NHWC tensor
Function will return error if there is a size mismatch
i.e (bufsize != nn_tensor_size(tensor)) or tensor is invalid
De-quantizes the source tensor into the destination tensor.
De-quantizes the source tensor into the destination buffer.
Returns the number of dimensions of the given tensor object.
Returns the element size of a given tensor object.
Returns the engine owning this tensor, could be NULL.
Fills the tensor with the provided constant. The constant is captured
as double precision (64-bit floating point) which has 53-bits of precision
on whole numbers. This means the constant CANNOT represent all 64-bit
integers but it CAN represent all 32-bit and lower integers. If full
64-bit integer support is required @ref nn_tensor_map can be used though
it is less efficient with some engines because of the addition memory
transfer required.
Initializes the tensor using provided memory. The memory MUST be at least
the size returned by @ref nn_tensor_sizeof(). This size does not include
the actual tensor data which is allocated separately, either by requesting
the implementation to allocate the buffer or attaching to externally
allocated memory.
Returns the I/O time information stored in the tensor. The time is returned
in nanoseconds of the duration of the last map/unmap pair. When tensors are
mapped to the CPU (no accelerator engine is loaded) then times are expected
to be zero time as no mapping is actually required and the internal pointer
is simply returned. When an accelerator engine is used, such as OpenVX,
then the io_time measures the time the map/unmap or copy operations took to
complete.
Loads an image from file into the provided tensor.
Loads an image from file into the provided tensor.
Loads an image from the provided buffer and decodes it accordingly, the
function uses the images headers to find an appropriate decoder. The
function will handle any required casting to the target tensor’s format.
Loads an image from the provided buffer and decodes it accordingly, the
function uses the images headers to find an appropriate decoder. The
function will handle any required casting to the target tensor’s format and
will apply image standardization (compatible with tensorflow’s
tf.image.per_image_standardization) if the proc parameter is set to
NN_IMAGE_PROC_WHITENING.
Returns the tensor’s mapping count, 0 means the tensor is unmapped.
Maps the tensor’s memory and returns the client accessible pointer. This is
the read-only version which causes the engine to download buffers to the CPU
memory space if required but will not flush back to the device on unmap.
Maps the tensor’s memory and returns the client accessible pointer. This is
the read-write version which causes the engine to download buffers to the CPU
memory space if required and will also flush back to the device on unmap.
Maps the tensor’s memory and returns the client accessible pointer. This is
the write-only version which will not cause a download of the buffers to the
CPU memory space on map but will upload to the device on unmap.
Returns the native handle of the tensor object. This is an internal API for
access internal structures.
Returns the offset of a given tensor. This function can be used to calculate
the index across numerous dimensions.
Returns the offset of a given tensor using variable length dimensions. This
works the same as @ref nn_tensor_offset() but uses variable arguments. The
user must provide @p n_dims number of parameters after the
@p n_dims parameter.
nn_tensor_pad implements a padded Tensor to Tensor copy. This can be used to
achieve the various convolution padding strategies (SAME, FULL). For example
SAME conv2d would use the following padded_copy before running the conv2d
layer.
nn_tensor_padding calculates the paddings for the given tensor, padtype,
window, stride, and dilation given n_dims being queried from the tensor’s
nn_tensor_dims().
Retrieves the panel size of the tensor when it has been panel-shuffled for
improved tiling performance. The panel size is the vectorization length.
Writes the tensor inforamtion to the FILE stream provided. The format is
“[D0 D1 D2 D3]” where D0..D3 are the dimensions provided. If the data
parameter is true the format will be followed by “: …” where … is the
string representation of the tensor’s data.
Internal API used by the RTM loader to associate quantization parameters to
the tensor.
Returns the quantization type for the tensor.
Quantizes the source tensor into the destination tensor.
Quantizes the source buffer into the destination tensor.
Randomizes the data within the tensor.
Releases the memory used by the tensor object.
Requantizes the source tensor into the destination tensor.
Reshapes the given tensor to the provided new shape.
Returns the scales array for the tensor and optionally the number of scales.
Configures an auxiliary object for the tensor. This is a private API used
for attaching auxiliary buffers.
Extended version of the auxiliary object API which allows additional objects
to be attached to the tensor using name-based indexing.
Configures the channel axis of the tensor. This refers to the “C” in
orderings such as NHWC and NCHW.
Sets the tensor objects native handle to the one provided.
Sets the panel size of the tensor. This is primarily an internal API used
to store the vectorization length when shuffling tensors into an optimized
tile format.
Sets the quantization scales for the tensor. If n_scales>1 it should match
the channel dimension (axis) of the tensor.
Sets the type of a given tensor object.
Sets the quantization zero-points for the tensor. If n_zeros>1 it should
match the channel dimension (axis) of the tensor.
Returns the shape of the given tensor object.
Copys the source shape array to the destination array.
Tensor shape comparison.
Shuffles (transpose) the tensor moving the current dimensions into the
ordering defined in the order parameter.
Calculates the total byte size of the tensor (volume * element_size).
Returns the size of the tensor object for preparing memory allocations.
nn_tensor_slice copies a slice of the tensor into output. For a version which
supports strides see @ref nn_tensor_strided_slice.
Returns the strides of the given tensor object.
Synchronize the tensor and all preceeding events in the chain.
Returns the time information stored in the tensor. The time is returned
in nanoseconds of the duration of the last operation the wrote into this
tensor. causes a nn_tensor_sync on the target tensor.
Returns the type of a given tensor object.
Releases the tensor mapping, if the reference count reaches 0 it will be
fully unmapped and will force the flush to the device, if required.
Maps the tensor using the memory from the parent tensor.
Calculates the total tensor volume (product of dimensions).
Returns the zero-points for the tensor and optionally the number of
zero-points.
DeepViewRT library version as “MAJOR.MINOR.PATCH”.