Expand description
§TOA Compression File Format
This project implements the TOA compression file format, an experimental compression format designed for streaming operation, parallel processing and corruption resilience. The format uses LZMA as the primary compression algorithm with BLAKE3 cryptographic hashing and Reed-Solomon error correction codes.
§Specification
The specification of the file format version 1.0 is finished and can be read in the SPECIFICATION.md in the source repository.
§Acknowledgement
- The lzma code is a hard copy of the lzma-rust crate (Apache 2 license).
- Original Author of the lzma-rust crate was dyz1990 (Apache 2 license).
- The lzma-rust2 crate was a rewrite of the XZ for Java by Lasse Collin (0BSD).
- Major parts of XZ for Java are based on code written by Igor Pavlov in the LZMA SDK (public domain).
§License
Licensed under the Apache License, Version 2.0.
Modules§
- filter
- Implements filter used by both the 7z and XZ file format.
- reed_
solomon - Stack only Reed–Solomon implementation.
Structs§
- CVStack
- Efficient chaining value stack for Blake3 hash computation as described in section 5.1.2 of the Blake3 paper.
- ECCDecoder
- Error Correction Code Decoder that applies Reed-Solomon decoding to compressed data.
- ECCEncoder
- Error Correction Code Writer that applies Reed-Solomon encoding to compressed data.
- TOABlock
Header - TOA block header containing size information, hash, and Reed-Solomon parity.
- TOABlock
Writer - A single block encoder for TOA format.
- TOAFile
Decoder std - A multi-threaded TOA file decoder that implements Read trait.
- TOAFile
Encoder std - A multi-threaded TOA file encoder that implements Read trait.
- TOAFile
Trailer - File trailer containing total uncompressed size, root hash, and Reed-Solomon parity.
- TOAHeader
- TOA file header containing format metadata and compression parameters.
- TOAMetadata
- Metadata information from an TOA file.
- TOAOptions
- Options for TOA compression.
- TOAStreaming
Decoder - A single-threaded streaming TOA decompressor.
- TOAStreaming
Encoder - A single-threaded streaming TOA compressor.
Enums§
- Error
Correction - Reed-Solomon error correction levels for data protection.
- Prefilter
- Prefilter types that can be applied before LZMA compression to improve compression ratios for specific data types like executable files.
- Simd
Override - SIMD override for controlling SIMD code paths.
Functions§
- copy_
wide std - Optimized copy function with 64KiB buffer for better performance.
Type Aliases§
- Result
std - Result type of the crate.