| DeserializeParams is just a helper
| class to consolidate the parameters
| required for deserializing tensor
| data so they can be passed around more
| easily.
|
| It also contains some helper functions
| to perform some operations on the parameters
| that are shared by multiple deserialization
| functions.
|
this is a reader implemented by std::istream
| SerializeParams is just a helper class
| to consolidate the parameters required
| for serializing tensor data so they
| can be passed around more easily.
|
| It also contains some helper functions
| to perform some operations on the parameters
| that are shared by multiple serialization
| functions.
|
| @brief
|
| StringDeserializer is the deserializer
| for Strings.
|
| @brief
|
| StringSerializer is the serializer
| for
|
| String.
|
| StringSerializer takes in a blob that
| contains a String, and serializes it
| into a BlobProto protocol buffer.
|
| @brief
|
| TensorDeserializer is the deserializer
| for Tensors.
|
| The device that the deserialized Tensor
| will live under is determined by the
| device_detail field. If you want to
| specify the device of the deserialized
| tensor, change the TensorProto’s corresponding
| fields before calling
|
| Deserialize.
|
| ———–
| @brief
|
| TensorSerializer is the serializer
| for Tensors.
|
| TensorSerializer takes in a blob that
| contains a Tensor, and serializes it
| into a TensorProto protocol buffer.
|
compute CRC32 (byte algorithm) without lookup tables
compute CRC32 (byte algorithm) without lookup tables
compute CRC32 (bitwise algorithm)
| compute CRC32 using the fastest algorithm for
| large datasets on modern CPUs crc32_fast
| selects the fastest algorithm depending on
| flags (CRC32_USE_LOOKUP_…)
compute CRC32 (half-byte algoritm)
| Deserializes from a string containing
| either BlobProto or TensorProto. If the deserialization
| fails, the content in the blob should
| no longer be trusted.
|
| DeserializeTensorData() is specialized
| for each supported combination of
| SerializationFormat and output type.
|
| The default implementation throws
| an exception, but this function can
| be specialized to support different
| combinations.
|
Get dimensions from Tensor proto
| Get an empty Tensor from the TensorProto
| given the meta data in proto (data type
| and size of the Tensor) without actually
| filling in the data.
|
| We need this function because we want
| to construct a fully initialized Tensor
| in the beginning instead of keeping
| partially initialized Tensor around
| the process. Consider the case when
| we have a Tensor that is split into multiple
| protos during serialization, in deserialization,
| we have to fill the Tensor in multiple
| calls to Deserialize, therefore we
| need to create a new Tensor with the correct
| size and data type before the call to
|
| Deserialize, because otherwise we
| will have to check whether the function
| call is the first call to initialize
| the underlying Tensor, which makes
| the function stateful and complicated.
|
| The legacy code get away with this problem
| by passing in a partially initialized
| Tensor and use Resize and mutable_data
| to set the correct size, data type and
| allocate memory for the
|
| Tensor, so the state is encoded in these
| function calls. e.g. mutable_data
| will allocate memory on the first call
| and it will return a pointer to the allocated
| memory on later calls.
|
| Make space for new elements to be copied to
| the end of the repeated field.
|
| The new space is not guaranteed to be
| initialized.
Get data type from Tensor proto
| Return a mutable Range pointing to a
| portion of the tensor’s data field.
|
| Returns a Range pointing to the elements
| starting at the specified start index,
| and including the specified number
| of elements.
|
Returns a record to be appended to the local user extra
data entry in order to make data beginning aligned at
kFieldAlignment bytes boundary.
Get number of elements from Tensor proto
| Converts MessageLite to string while also
| checking that SerializeAsString succeeds. Pass
| description of class/function of the call if
| you’d like it appended to the error message.
| @brief
|
| Convenience function to serialize
| a blob to a string.
|
| This is a convenience function to serialize
| small Blobs that produce manageable
| serialized strings. To serialize big
| blobs such as large sparse tensors,
| use the fully-functional interface
| in blob_serializer_base.h.
|
| ———–
| @note
|
| this function doesn’t do chunking and
| might break with big tensors.
|
| Serializes the given blob, if possible.
|
| ———–
| @note
|
| this serialization uses the registration
| mechanism and one has to implement specific
| serialization approaches for specific
| classes.
|
| Acceptor should take care of writing
| data to the actual storage.
|
| Convert BlobProto to string with success
| checks.
|
swap endianess
| Get TensorOptions from Tensor proto
| Assumes TensorProto is not empty
|