Crate tfrecord

The crate provides the functionality to serialize and deserialize TFRecord data format from TensorFlow.

Cargo Features

Optional features:

• full: Enable all features.
• async: Enable async/await feature.

Third-party crate supports:

• with-serde: Enable interoperability with serde to serialize and deserialize example types.
• with-tch: Enable tch types support.
• with-image: Enable image types support.
• with-ndarray: Enable ndarray types support.

Manualy ProtocolBuffer Code Generation

The crate compiles the pre-generated ProtocolBuffer code from TensorFlow. In the case of TensorFlow version update, you may generate the code manually. It accepts several ways to access the TensorFlow source code specified by TFRECORD_BUILD_METHOD environment variable. The generated code is placed under prebuild_src directory. See the examples below and change X.Y.Z to actual TensorFlow version.

Build from a source tarball

export TFRECORD_BUILD_METHOD="src_file:///home/myname/tensorflow-X.Y.Z.tar.gz"
cargo build --release --features generate_protobuf_src

Build from a source directory

export TFRECORD_BUILD_METHOD="src_dir:///home/myname/tensorflow-X.Y.Z"
cargo build --release --features generate_protobuf_src

Build from a URL

export TFRECORD_BUILD_METHOD="url://https://github.com/tensorflow/tensorflow/archive/vX.Y.Z.tar.gz"
cargo build --release --features generate_protobuf_src

Build from installed TensorFlow on system

The build script will search ${install_prefix}/include/tensorflow directory for protobuf documents.

export TFRECORD_BUILD_METHOD="install_prefix:///usr"
cargo build --release --features generate_protobuf_src

Re-exports

• pub use protobuf::Event;
• pub use protobuf::Example;
• pub use protobuf::Feature;
• pub use protobuf::HistogramProto;
• pub use protobuf::Summary;
• pub use error::*;
• pub use event::*;
• pub use event_writer::*;
• pub use protobuf_ext::*;
• pub use record::*;
• pub use record_reader::*;
• pub use record_writer::*;

Modules

• error
  Error types and error handling utilities.
• event
  Summary and event types.
• event_writer
  Summary record writer.
• indexer
  The indexer that enumerate record locations from one or multiple TFRecord files.
• io
  Low level synchronous and asynchronous I/O functions.
• protobuf
  ProtocolBuffer types compiled from TensorFlow.
• protobuf_ext
  Extension to ProtocolBuffer types.
• record
  Marker traits.
• record_reader
  TFRecord data reader.
• record_writer
  TFRecord data writer.