Module zstd 

The zstd family (zstd, unzstd, zstdcat, zstdmt): stuffr behaving as zstd 1.5.7’s command line tool.

Ported from zstd 1.5.7 programs/ by Yann Collet and the zstd contributors (Meta Platforms, Inc.; BSD-3-Clause, the licence ships as LICENSE-zstd), one module per C file: cli.rs (zstdcli.c’s argument parsing, help texts and parameter printers), dibio.rs (dibio.c, --train*), util.rs (util.c), display.rs (fileio_common.h’s display macros) and this module (zstdcli.c’s main after the arguments). Every string is in messages.rs. libzstd itself is reached through stuffr::zstd_opts.

Names (spec section 1): the four map to this family, and the presets come from the invoked name by zstdcli.c’s exeNameMatch (the name, then the end or a .): unzstd decompresses, zstdcat decompresses to standard output with pass-through at display level 1, zstdmt compresses with -T0. zstdcli.c’s gzip, xz, lzma and lz4 presets are ported too, though dispatch never reaches them (those names belong to other families).

The machine (R3): the real process counts its cores as util.c does (util::Cores::detect); a sandboxed run ([run_in_counted]) sees 8 logical and 4 physical cores.

Known deviations (the source of truth; the README and the spec mirror this list):

1. The banner names stuffr wherever zstd prints WELCOME_MESSAGE (-V, -vV, -vvV, the first line of -H, and -v’s welcome line on standard error): *** Zstandard CLI (stuffr X.Y.Z; zstd 1.5.7 compatible) (64-bit) v1.5.7, by Yann Collet ***. -qV prints exactly 1.5.7, and -vV’s “supports:” line is zstd’s. -vvV prints zstd’s POSIX lines for the platform, but not its three library-version lines (the zlib, lz4 and lzma zstd links), which describe zstd’s own build.
2. The -v summary’s timing fields vary from run to run, as they do between two runs of zstd.
3. An input that is also the output is never lost.
4. Closed standard streams and a parent that ignores SIGPIPE: the Rust runtime reopens a closed standard descriptor on /dev/null and resets SIGPIPE before main, so neither is seen.
5. Out of scope, refused as a zstd built with ZSTD_NOBENCH and ZSTD_NOTRACE refuses them: -b, -e#, -i# (“Incorrect parameter: -b”, exit 1) and --trace (“Incorrect parameter: –trace”); -H omits their lines. -B#, -S and -p# keep the reference’s meaning. --[no-]asyncio is accepted and has no visible effect. --trace-file-stat is accepted and prints no trace.
   • -l on two malformed skippable frames (ruling T7-a): a size field of 0xFFFFFFF8 makes zstd loop forever, and stuffr answers exit 1 with could not find end of skippable frame; a skippable frame cut after 5 to 7 bytes makes zstd read uninitialised stack, and stuffr reads a deterministic header. Neither upstream defect is reproduced.
   • At display level 5 and above, EXM_THROW’s Error defined at <file>, line <n> line is not printed (ruling T4-b): it names the C implementation’s source lines.
6. --train-cover (ruling T1-a): libzstd 1.5.7’s cover sort breaks ties by comparing addresses (cover.c:300, :316), so a cover dictionary depends on heap layout, in zstd itself too. It is promised valid and usable by zstd, not byte for byte. --train, --train-fastcover and --train-legacy write zstd’s bytes. --adapt changes the level by timing, so its frames are compatible, not identical.

Spec deviation 7, the pure tier (zstd-pure without zstd-c), as far as compressing and decompressing go (fileio.rs):

• Every level compresses with ruzstd’s Fastest, so the sizes and ratios in the summaries and in -vvvvvv’s trace are the tier’s own; the reference decodes every frame. The compression parameters are range-checked as libzstd checks them, then ignored: -T, --auto-threads and zstdmt run single-threaded, and the LDM, --rsyncable, block-size and overlap options have no effect.
• Content size and window: our frames never record a content size, so --content-size has no effect and -l on our own output says the decompressed size is unknown. The window is 128 KiB whatever --long asks.
• Each input is compressed as one frame built in memory: memory use grows with the input’s size (a piped standard input is unbounded), and a run that exhausts memory ends outside zstd’s exit codes.
• Compressing with -D or --patch-from is refused, after everything zstd prints first: zstd: error 1 : dictionary compression is not available in this build (stuffr pure tier). Decompressing with either works (a --patch-from prefix through a hand-built raw-content dictionary, Task 1’s spike S1).
• --train* answers as a zstd built with ZSTD_NODICT: --train prints training mode not available, exit 1, and --train-cover, --train-fastcover and --train-legacy are no options there (Incorrect parameter). -H still lists the dictionary builder.
• --adapt is accepted and the level never moves (the encoder has no jobs to measure).
• A legacy frame (v0.5 to v0.7) is unsupported format, what a zstd built without legacy support says.
• A corrupt frame whose blocks overflow the output libzstd sized from its header can be named Data corruption detected where libzstd says Destination buffer is too small.
• --format=gzip|xz|lzma write flate2’s and xz_opts’ pure encoders’ bytes, which the reference decodes (compatible, not identical; the summaries’ sizes are the tier’s own).
• A gzip member’s deflate data goes through flate2 (miniz_oxide), which accepts some streams zlib refuses (a distance reaching before the output, a code with no end of block): where the reference says inflate error -3 and writes nothing, the pure tier may write bytes and then fail (premature gz end), or, when the trailer’s CRC and size match what miniz_oxide decoded, succeed with exit 0 and that output. With zlib-c it is zlib’s own inflate.
• An xz or lzma frame that fails to decode (the xz family’s P6 and P9): what is written before the error can be a strict prefix of what liblzma writes, and corruption in the last bytes of a .lzma reads as the input ending there (premature lzma end, the jobs before it written) where liblzma says lzma_code decoding error 9.

The foreign formats on both tiers (formats.rs; spec “Promises”): --format=gzip is zlib’s bytes with zlib-c, --format=xz|lzma liblzma’s with xz-c; --format=lz4 is compatible only on every tier: lz4_flex’s fast compressor at every level, inside liblz4’s frame descriptor (64 KiB linked blocks, content size, content checksum as fileio.c sets them).

Functions

run

Entry point for the four names; name is the canonical invoked name, argv0 the program as invoked (its last component is zstd’s programName) and args the arguments after it.