Crate ratdmp 

ratdmp – a standalone crate (open source, dual-licensed MIT/Apache-2.0). Depends on serde (so ExtractedString can be serialized – the caller decides how to wrap it in JSON/its own protocol if needed, this crate only returns a plain Vec<ExtractedString>) and rayon, used only by the opt-in parallel scanning path (extract_strings_from_file_parallel) – the default sequential/ streaming path does not touch it.

Pulls printable ASCII / UTF-16LE strings out of a raw .dmp memory-dump file (or any other binary file). Pure post-processing over an already-existing file on disk (no WinAPI / process-permission involved) – scans the buffer sequentially, grouping contiguous runs of printable bytes (ASCII) and UTF-16LE runs (printable low byte, high byte == 0x00), the same principle as the Unix strings command. It does NOT parse the MDMP file format’s internal structure – scanning the whole payload is the only approach that doesn’t depend on the Windows/DbgHelp version that produced the dump.

Two things keep this fast and memory-safe on multi-gigabyte files:

• Chunked streaming (CHUNK_SIZE = 32MB per read) via Read, carrying exactly 1 trailing byte over to the next chunk so a UTF-16LE pair straddling an I/O boundary is always tested correctly – the whole file (which can be several GB) is NEVER loaded into RAM at once.
• ASCII and UTF-16LE are detected SIMULTANEOUSLY in a single pass (StringRunScanner::feed) instead of two independent loops – fewer passes over the buffer.

SPEED OPTIMIZATIONS (results unchanged, speed only): (1) a precomputed 256-entry PRINTABLE_TABLE lookup table built at compile time instead of a range-match per byte; (2) each byte is looked up in the printable table EXACTLY ONCE (the ascii branch and the utf16 branch, when that byte acts as the “lo” byte, share the same lookup); (3) fast-skip jumps straight over an entire run of consecutive non-printable bytes when no run is currently open (skipping the lookahead/aligned computation and the no-op flush() calls for each junk byte entirely) – non-printable binary regions typically make up the bulk of a real RAM dump, so this is the hottest path.

Noise filtering (higher filter accuracy than a naive strings-style scan, same ASCII/UTF-16LE run-grouping logic): a naive scan accepts ANY run meeting min_len as a valid “string”, including a run of a single repeated character (AAAAAAAA...) or two characters alternating (ABABABAB...) – an extremely common kind of noise in real RAM dumps (byte-fill/heap-fill patterns on alloc/free, alignment padding, repeated memset), which almost never carries useful information and needlessly bloats the result. is_low_information_repeat() filters exactly those two simple repeat shapes (period 1 and period 2, only triggered from a long-enough threshold so it doesn’t accidentally drop short real strings like “0000”/“====” which can still be real data) – it does NOT filter more broadly (no entropy/complex statistics) to avoid false negatives (missing real strings), staying true to the “only filter what’s certainly noise” spirit.

Structs

EntropyRegion

A fixed file region whose byte distribution has high Shannon entropy.

ExtractedString

NoiseConfig

Runtime-configurable noise-filter thresholds (see is_low_information_repeat). Previously these were the hardcoded constants NOISE_REPEAT1_MIN_LEN / NOISE_REPEAT2_MIN_LEN – now callers (including the CLI’s --noise-threshold <N>) can tune or fully disable the filter.

YaraMatch

A YARA rule that matched while a dump was being streamed.

Constants

ENTROPY_BLOCK_SIZE

Fixed window used by the entropy pass. Keeping this separate from the string-scan chunk makes entropy results comparable across files.

MAX_STRINGS

Default max_strings (see MIN_STRING_LEN above).

MIN_STRING_LEN

Default min_len when the caller doesn’t pass one – a reasonable baseline for the extract_strings* functions below when the caller wants a sensible default instead of picking their own.

Functions

extract_strings

Returns a list of ExtractedString, sorted by ascending offset, from a byte slice ALREADY in RAM. max_strings is an upper bound to avoid unbounded growth of memory / the JSON returned to the UI.

extract_strings_from_file

Extracts strings from a .dmp file, in chunks – does NOT read the whole file into RAM.

extract_strings_from_file_parallel

Same as extract_strings_from_file, but scans the file in parallel across multiple OS threads via rayon’s work-stealing thread pool instead of a single sequential pass. Best on multi-core machines with fast storage / a warm page cache, where the CPU-bound scan (not disk I/O) is the bottleneck. Peak memory is bounded by num_active_threads * (PARALLEL_CHUNK_SIZE + MAX_RUN_CHARS), not by the file size. Use rayon::ThreadPoolBuilder::num_threads (or the RAYON_NUM_THREADS env var) to control the degree of parallelism; by default rayon uses one thread per logical CPU.

extract_strings_from_file_streaming

File-based streaming variant of extract_strings_from_reader_streaming.

extract_strings_from_file_with_noise_config

Same as extract_strings_from_file, but with a configurable noise filter (see NoiseConfig) instead of the hardcoded defaults.

extract_strings_from_reader

Extracts strings directly from a Read, in chunks – NEVER loads more than CHUNK_SIZE bytes into RAM at once, suitable for multi-GB dump files. Carries exactly 1 trailing byte from each chunk into the next so a UTF-16LE pair straddling an I/O boundary is always tested correctly.

extract_strings_from_reader_streaming

Streams extracted strings to on_found while scanning a reader.

extract_strings_from_reader_with_noise_config

Same as extract_strings_from_reader, but with a configurable noise filter (see NoiseConfig) instead of the hardcoded defaults.

extract_strings_with_noise_config

Same as extract_strings, but with a configurable noise filter (see NoiseConfig) instead of the hardcoded defaults.

scan_entropy_from_file_streaming

Streams high-entropy fixed-size regions from a file.

scan_entropy_from_file_streaming_with_data

Like scan_entropy_from_file_streaming, but also provides each matching block’s bytes to the callback before the scan buffer is reused.

scan_yara_from_file_streaming

File-based convenience wrapper for scan_yara_from_reader_streaming.

scan_yara_from_reader_streaming

Compiles YARA source and scans a reader in bounded chunks.

shannon_entropy

Computes Shannon entropy in bits per byte for a byte slice.