tfrecord-rust
The crate provides the functionality to serialize and deserialize TFRecord data format from TensorFlow.
Features
- Provide both high level
EasyExample
type as well as low levelVec<u8>
bytes {,de}serialization. - Support async/await syntax. It's easy to work with futures-rs.
- Interoperability with serde.
Usage
Append this line to your Cargo.toml
.
tfrecord = "0.1.2"
Due to a bug in async-std 1.6.0, it cannot read file properly sometimes. Please add the patch at the end of Cargo.toml
unless the issue is fixed in future release.
[]
= { = "https://github.com/async-rs/async-std", = "master" }
The crate provides several cargo features that you can conditionally compile modules.
serde
: Enable interoperability with serde to serialize and deserialize example types.async_
: Enable async/await feature.dataset
: Enable the dataset API that can load records from multiple TFRecord files.full
: Enable all features above.
By default, the crate compiles the pre-built ProtocolBuffer code in the repository. If you would like to re-run the code generation, see Generate ProtocolBuffer code from TensorFlow section.
Documentation
See docs.rs for the API.
Example
File reading example
This is a snipplet copied from examples/tfrecord_info.rs.
use ;
Work with async/await syntax
The snipplet from examples/tfrecord_info_async.rs demonstrates the integration with async-std.
use TryStreamExt;
use ;
use ;
async
More examples
Also, we suggest visiting the test code for more detailed usage.
Generate ProtocolBuffer code from TensorFlow
The crate relies on ProtocolBuffer documents from TensorFlow. The crate ships pre-generated code from ProtocolBuffer documents by default. Most users don't need to bother with the code generation. The step is needed only in case of TensorFlow updates or your custom patch.
The build script accepts several ways to access the TensorFlow source code, controlled by the TFRECORD_BUILD_METHOD
environment variable. The generated code will be placed under prebuild_src
directory. See the examples below to understand the usage.
- Build from a source tarball
- Build from a source directory
- Build from a URL
- Build from installed TensorFlow on system. The build script will search
${install_prefix}/include/tensorflow
directory for protobuf documents.
License
MIT license. See LICENSE file for full license.