mzstd 0.2.1

Zstandard (RFC 8878) compression and decompression in safe Rust; single-threaded encoding is byte-identical to zstd 1.5.7
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232

//! Safe Rust Zstandard ([RFC 8878][rfc]) codecs.
//!
//! The encoder is a port of the reference implementation, zstd 1.5.7, and
//! writes the same bytes it does; the decoder reads every frame RFC 8878
//! defines apart from those that need a dictionary.
//!
//! # Byte identity
//!
//! For the same input and parameters, single-threaded compression produces
//! exactly the bytes of zstd 1.5.7: one-shot compression matches
//! `ZSTD_compress2`, and an encoder session matches `ZSTD_compressStream2`
//! driven with the same input chunks, output capacities and operations. The
//! contract covers compression levels -131072 to 22, the explicit
//! compression parameters (window, hash, chain and search logs, minimum
//! match, target length, strategy), long distance matching, the checksum and
//! content-size flags, declared input sizes and size hints.
//!
//! Parallel compression writes valid frames that decode to the input but are
//! not those of the reference's multithreaded mode. Dictionaries, the
//! magicless format, `targetCBlockSize` and external sequence producers are
//! not supported.
//!
//! # Choose your API
//!
//! Compression and decompression expose the same core I/O shapes. Pick the shape that
//! matches how your application already moves bytes. [`ParallelCompressor`] is a separate
//! execution strategy for compression: it splits one frame into independent segments and
//! hands them to your executor as tasks, either for a whole input at once or for input
//! that arrives in chunks. Each codec requires its Cargo feature; reader/writer adapters
//! and parallel compression are unavailable with `no_std`.
//!
//! | I/O shape | Compression | Decompression |
//! | --- | --- | --- |
//! | Return a new `Vec<u8>` | [`Compressor::compress`][compressor-compress] | [`Decompressor::decompress`][decompressor-decompress] |
//! | Append to an existing Vec | [`compress_into`][compressor-compress-into] | [`decompress_into`][decompressor-decompress-into] |
//! | Write into a caller-owned slice | [`compress_to_slice`][compressor-compress-to-slice], [`compress_to_uninit`][compressor-compress-to-uninit] | [`decompress_to_slice`][decompressor-decompress-to-slice] |
//! | Pull output through `std::io::Read` | [`Compressor::reader`][compressor-reader] | [`Decompressor::reader`][decompressor-reader] |
//! | Push input through `std::io::Write` | [`Compressor::writer`][compressor-writer] | [`Decompressor::writer`][decompressor-writer] |
//! | Drive input/output incrementally | [`start`][compressor-start] → [`EncoderSession`] | [`start`][decompressor-start] → [`DecoderSession`] |
//! | Same, with the session owning the codec | [`into_session`][compressor-into-session] → [`EncoderSessionOwned`] | [`into_session`][decompressor-into-session] → [`DecoderSessionOwned`] |
//! | Split one known input across your threads | [`prepare_slice`][parallel-prepare-slice] → [`ScopedParallelBatch`] | — |
//! | Stream chunked input across your threads | [`start`][parallel-start] → [`ParallelEncoderSession`] + [`ParallelTaskQueue`] | — |
//! | Same, with the session owning the compressor | [`into_session`][parallel-into-session] → [`ParallelEncoderSessionOwned`] | — |
//! | Push or pull through `std::io` across your threads | [`writer`][parallel-writer], [`reader`][parallel-reader] | — |
//!
//! [`EncoderConfig`] selects the level and frame flags, and
//! [`CompressionParams`] overrides individual parameters exactly as
//! `ZSTD_CCtx_setParameter` would. [`DecodeLimits`] bound the window, input
//! and output, so untrusted frames cannot make the decoder allocate without
//! limit.
//!
//! ```
//! # #[cfg(all(feature = "compression", feature = "decompression"))]
//! # {
//! use mzstd::{Compressor, Decompressor, EncoderConfig, Level};
//!
//! let mut encoder = Compressor::new(EncoderConfig::default().with_level(Level::try_from(19)?))?;
//! let payload = "the quick brown fox ".repeat(500);
//! let compressed = encoder.compress(payload.as_bytes())?;
//! assert!(compressed.len() < payload.len() / 50);
//! assert_eq!(Decompressor::default().decompress(&compressed)?, payload.as_bytes());
//! # }
//! # Ok::<(), Box<dyn std::error::Error>>(())
//! ```
//!
//! # Select only the codecs you need
//!
//! The default feature set is `std`, `compression`, and `decompression`. Disable default
//! features to select one codec, alongside `std` or `no_std` (which needs `alloc`):
//!
//! ```toml
//! mzstd = { version = "0.2", default-features = false, features = ["std", "decompression"] }
//! ```
//!
//! A disabled codec has no module and no re-exports. The `std::io` adapters
//! and parallel compression need `std`.
//!
//! [rfc]: https://www.rfc-editor.org/rfc/rfc8878
// Resolve available APIs locally; unavailable APIs link to feature selection.
#![cfg_attr(
    feature = "compression",
    doc = r#"
[compressor-compress]: crate::Compressor::compress
[compressor-compress-into]: crate::Compressor::compress_into
[compressor-compress-to-slice]: crate::Compressor::compress_to_slice
[compressor-compress-to-uninit]: crate::Compressor::compress_to_uninit
[compressor-start]: crate::Compressor::start
[`EncoderSession`]: crate::EncoderSession
[compressor-into-session]: crate::Compressor::into_session
[`EncoderSessionOwned`]: crate::EncoderSessionOwned
[`EncoderConfig`]: crate::EncoderConfig
[`CompressionParams`]: crate::CompressionParams
"#
)]
#![cfg_attr(
    not(feature = "compression"),
    doc = r#"
[compressor-compress]: #select-only-the-codecs-you-need
[compressor-compress-into]: #select-only-the-codecs-you-need
[compressor-compress-to-slice]: #select-only-the-codecs-you-need
[compressor-compress-to-uninit]: #select-only-the-codecs-you-need
[compressor-start]: #select-only-the-codecs-you-need
[`EncoderSession`]: #select-only-the-codecs-you-need
[compressor-into-session]: #select-only-the-codecs-you-need
[`EncoderSessionOwned`]: #select-only-the-codecs-you-need
[`EncoderConfig`]: #select-only-the-codecs-you-need
[`CompressionParams`]: #select-only-the-codecs-you-need
"#
)]
#![cfg_attr(
    feature = "decompression",
    doc = r#"
[decompressor-decompress]: crate::Decompressor::decompress
[decompressor-decompress-into]: crate::Decompressor::decompress_into
[decompressor-decompress-to-slice]: crate::Decompressor::decompress_to_slice
[decompressor-start]: crate::Decompressor::start
[`DecoderSession`]: crate::DecoderSession
[decompressor-into-session]: crate::Decompressor::into_session
[`DecoderSessionOwned`]: crate::DecoderSessionOwned
[`DecodeLimits`]: crate::DecodeLimits
"#
)]
#![cfg_attr(
    not(feature = "decompression"),
    doc = r#"
[decompressor-decompress]: #select-only-the-codecs-you-need
[decompressor-decompress-into]: #select-only-the-codecs-you-need
[decompressor-decompress-to-slice]: #select-only-the-codecs-you-need
[decompressor-start]: #select-only-the-codecs-you-need
[`DecoderSession`]: #select-only-the-codecs-you-need
[decompressor-into-session]: #select-only-the-codecs-you-need
[`DecoderSessionOwned`]: #select-only-the-codecs-you-need
[`DecodeLimits`]: #select-only-the-codecs-you-need
"#
)]
#![cfg_attr(
    all(feature = "compression", not(feature = "no_std")),
    doc = r#"
[compressor-reader]: crate::Compressor::reader
[compressor-writer]: crate::Compressor::writer
[`ParallelCompressor`]: crate::compressor::parallel::ParallelCompressor
[parallel-prepare-slice]: crate::compressor::parallel::ParallelCompressor::prepare_slice
[`ScopedParallelBatch`]: crate::compressor::parallel::ScopedParallelBatch
[parallel-start]: crate::compressor::parallel::ParallelCompressor::start
[`ParallelEncoderSession`]: crate::compressor::parallel::ParallelEncoderSession
[`ParallelTaskQueue`]: crate::compressor::parallel::ParallelTaskQueue
[parallel-into-session]: crate::compressor::parallel::ParallelCompressor::into_session
[`ParallelEncoderSessionOwned`]: crate::compressor::parallel::ParallelEncoderSessionOwned
[parallel-writer]: crate::compressor::parallel::ParallelCompressor::writer
[parallel-reader]: crate::compressor::parallel::ParallelCompressor::reader
"#
)]
#![cfg_attr(
    not(all(feature = "compression", not(feature = "no_std"))),
    doc = r#"
[compressor-reader]: #select-only-the-codecs-you-need
[compressor-writer]: #select-only-the-codecs-you-need
[`ParallelCompressor`]: #select-only-the-codecs-you-need
[parallel-prepare-slice]: #select-only-the-codecs-you-need
[`ScopedParallelBatch`]: #select-only-the-codecs-you-need
[parallel-start]: #select-only-the-codecs-you-need
[`ParallelEncoderSession`]: #select-only-the-codecs-you-need
[`ParallelTaskQueue`]: #select-only-the-codecs-you-need
[parallel-into-session]: #select-only-the-codecs-you-need
[`ParallelEncoderSessionOwned`]: #select-only-the-codecs-you-need
[parallel-writer]: #select-only-the-codecs-you-need
[parallel-reader]: #select-only-the-codecs-you-need
"#
)]
#![cfg_attr(
    all(feature = "decompression", not(feature = "no_std")),
    doc = r#"
[decompressor-reader]: crate::Decompressor::reader
[decompressor-writer]: crate::Decompressor::writer
"#
)]
#![cfg_attr(
    not(all(feature = "decompression", not(feature = "no_std"))),
    doc = r#"
[decompressor-reader]: #select-only-the-codecs-you-need
[decompressor-writer]: #select-only-the-codecs-you-need
"#
)]
#![cfg_attr(feature = "no_std", no_std)]
#![cfg_attr(not(test), forbid(unsafe_code))]
#![deny(missing_docs)]
#![deny(missing_debug_implementations)]
#![deny(rustdoc::broken_intra_doc_links)]

#[cfg(any(feature = "compression", feature = "decompression"))]
extern crate alloc;

#[cfg(any(feature = "compression", feature = "decompression"))]
mod backend;
#[cfg(any(feature = "compression", feature = "decompression"))]
mod retention;
#[cfg(any(feature = "compression", feature = "decompression"))]
mod shared;
#[cfg(any(feature = "compression", feature = "decompression"))]
pub use backend::Backend;
#[cfg(any(feature = "compression", feature = "decompression"))]
pub use retention::RetentionPolicy;

#[cfg(all(
    any(feature = "compression", feature = "decompression"),
    not(feature = "no_std")
))]
mod finish_error;
#[cfg(all(
    any(feature = "compression", feature = "decompression"),
    not(feature = "no_std")
))]
pub mod io;

#[cfg(feature = "compression")]
pub mod compressor;
#[cfg(feature = "compression")]
pub use compressor::{
    CompressionParams, Compressor, CompressorBuilder, ConfigError, EncodeError, EncoderConfig,
    EncoderSession, EncoderSessionOwned, EncoderStatus, InputSize, LdmParams, Level,
    LongDistanceMatching, Operation, Parameter, Progress, SizeOverflow, Strategy, StreamConfig,
};

#[cfg(feature = "decompression")]
pub mod decompressor;

#[cfg(feature = "decompression")]
pub use decompressor::{
    DecodeConfigError, DecodeError, DecodeFailure, DecodeLimits, DecodeOperation, DecodeProgress,
    DecoderConfig, DecoderSession, DecoderSessionOwned, DecoderStatus, Decompressor,
    DecompressorBuilder, FrameMode, InvalidDataKind,
};