Module measure 

The timing harness for stuffr bench: one format at one level, run over the user’s input, compressed and decompressed, with only codec work inside a timed region, except the temp file’s writes when the output is too large to keep in memory (below).

A compression region holds encoder creation, write_all of the input and Sink::finish (which consumes the sink, so its teardown is inside too). A decompression region holds decoder creation and reading the decoder to its end into a buffer whose bytes are only counted. Building the destination, opening files, reading streamed input, the temp file’s creation and removal, and dropping the decoder are all outside.

In memory, each region is one pair of clock reads. Streamed from disk, the input’s reads happen between the encoder’s writes, so compression is the sum of several regions (encoder creation, each chunk’s write_all, finish); decompression is still one region, the decoder’s own reads of the compressed temp file being part of decoding.

The compressed output stays in memory whenever its worst case (input + 1/8 + 1 KiB, the same test for every codec) fits under the memory left for it: half of --memory-limit when the input is held in memory, all of it when the input is streamed. Only past that does it go to one temp file, and then its writes (to the page cache, through the 64 KiB buffer) fall inside the compression regions. Both destinations have the same shape: a 64 KiB BufWriter over a pre-reserved Vec, or over the temp file, so a codec that writes in small pieces pays the same either way. Decoding reads the same shape back: the Vec as a seekable, sized source, or the temp file through a 64 KiB BufReader, so a decoder that reads a byte at a time (LZW’s) costs a memory copy per byte, not a system call.

Each file’s length is taken once, when the input is loaded; every later read stops there, so a file still being written is benched as it was.

bench ignores STUFFR_THREADS; threads come only from –threads.

With streamed input and --threads above 1, a multi-threaded encoder’s workers keep running during the untimed input reads between regions, so its compression time can be under-reported against a single-threaded one. In memory there are no such gaps.

Structs

Plan

The conditions every row runs under.

RealClock

The real clock: nanoseconds since it was made.

Snapshot

A streamed file and its length when the input was loaded: every read of it stops at len, so bytes appended later are never benched.

Enums

Input

The input, in operand order: held in memory (one buffer per file) or streamed from the files.

Mode

One stream of the files concatenated, or one stream per file.

Traits

Clock

A monotonic clock in nanoseconds; injected so tests can count its reads.

Functions

governor_for

The encode governor for plan: pack --threads’s mapping, except that no --threads means single-threaded whatever STUFFR_THREADS says (resolved_budget would read it). With a CLI value the variable is already ignored: resolve_workers takes the CLI value first.

load_input

Loads paths into memory when their total size is at most half of memory_limit, leaving room for the compressed copy; otherwise the input is streamed from the files.

measure

Compresses and decompresses input with codec at level (None: the encoder’s own default) under plan. Every error, a rejected level included, is a failed row; nothing panics.

warm

One untimed read of every file, so the first timed run does not pay for a cold cache. Nothing to do for input already in memory.