Struct IKVCacheUpdateLayer 

pub struct IKVCacheUpdateLayer { /* private fields */ }

! \class IKVCacheUpdateLayer ! ! \brief Layer that represents a KVCacheUpdate operation. ! ! The KVCacheUpdate layer is used to cache the key or value tensors for the attention mechanism. ! K and V use separate KVCacheUpdate layers. ! ! An IKVCacheUpdateLayer has three inputs (cache, update, writeIndices) and one output. ! In kLINEAR mode, for each batch element i, the layer copies the update tensor into the cache starting at ! position writeIndices[i]. Assuming no out-of-bounds writes occur, the operation for each sequence position ! s in [0, sequenceLength) is: ! \code ! output[i, :, writeIndices[i] + s, :] = update[i, :, s, :] ! \endcode ! ! The output performs in-place updates on the cache tensor, so they must share the same device memory address. ! ! \warning Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI. !

Trait Implementations

impl ExternType for IKVCacheUpdateLayer

type Id = (n, v, i, n, f, e, r, _1, (), I, K, V, C, a, c, h, e, U, p, d, a, t, e, L, a, y, e, r)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Opaque

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl MakeCppStorage for IKVCacheUpdateLayer

unsafe fn allocate_uninitialized_cpp_storage() -> *mut IKVCacheUpdateLayer

Allocates heap space for this type in C++ and return a pointer to that space, but do not initialize that space (i.e. do not yet call a constructor).

unsafe fn free_uninitialized_cpp_storage(arg0: *mut IKVCacheUpdateLayer)

Frees a C++ allocation which has not yet had a constructor called.