Expand description
General-purpose PAR2 verification and repair engine.
A pure-Rust implementation of PAR2 (Parity Archive Volume Set v2.0): load a set, find out what is damaged, and repair it from the recovery data.
§Verifying a set
A PAR2 set is usually spread across several .par2 files. Packets from all
of them aggregate into one Par2FileSet, and verification runs against
that.
use par2_rs::{DiskFileAccess, Par2FileSet, Repairability, scan_packets_from_path, verify_all};
let packets = scan_packets_from_path(std::path::Path::new("release.par2"))?
.into_iter()
.map(|(packet, _offset)| packet)
.collect();
let set = Par2FileSet::from_packets(packets)?;
let access = DiskFileAccess::new("/downloads/release".into(), &set);
let result = verify_all(&set, &access);
println!("{} recovery blocks available", result.recovery_blocks_available);
match result.repairable {
Repairability::NotNeeded => println!("everything verified clean"),
Repairability::Repairable { blocks_needed, .. } => {
println!("repairable: {blocks_needed} blocks to rebuild")
}
Repairability::Insufficient { deficit, .. } => {
println!("not enough recovery data: {deficit} blocks short")
}
other => println!("{other:?}"),
}Verification is slice-level, using the CRC32 + MD5 pairs in IFSC packets,
so damage is localised to the slices that are actually wrong rather than
condemning the whole file. Sets carrying no IFSC data fall back to full-file
MD5, and quick_check_16k identifies a candidate file cheaply before
either.
§Verifying bytes that are not files
verify_all reads through the FileAccess trait, not the filesystem.
DiskFileAccess is the ordinary implementation; supply your own and a set
can be verified against bytes still arriving over a network, or assembled
from somewhere that has no paths at all. MemoryFileAccess is useful in
tests.
§Repair
Par2Repairer drives the whole sequence — scan, verify, solve, repair,
then verify again. Repair is placement-aware: files that were renamed or
moved are matched by content rather than by name, so a set still repairs
after its files have been reorganised.
§Repairing across a whole download
Par2RepairSession is the retained form: one session accumulates
evidence — per-slice verdicts, whole-file proofs — while the data is still
arriving, so assessment is incremental and repair runs from what is already
known instead of a fresh walk. Its sources may be files under a base
directory, or bytes served through a FileAccess handle
(Par2RepairSessionOptions::with_source_access) for sets that never
became files — and where the .par2 volumes themselves never became files
either, Par2RepairSessionOptions::from_set takes the parsed set
directly. Repair output is always real files either way.
§Damaged PAR2 files
A malformed or truncated packet does not fail the set. The scanner skips forward to the next valid packet, because the recovery data that survived is usually still enough — which is the entire point of parity.
§Feature flags
-
crypto-aws-lc(default): AWS-LC-backed MD5. Needs a C toolchain to buildaws-lc-sys, and is the configuration the published performance figures were measured with. -
crypto-rust: the portable RustCrypto (md-5) MD5 backend, for builds that must not carry a C/assembly dependency and forwasm, where AWS-LC is unavailable. Select it withdefault-features = false, features = ["crypto-rust"]. Expect slower hashing; nothing else changes. -
native-crypto: back-compat alias forcrypto-aws-lc.Exactly one backend is active: on a native target AWS-LC wins whenever
crypto-aws-lcis on, and enabling neither backend is a compile error. -
metal/wgpu: GPU-accelerated repair throughreedsolomon_rs, with repair fallback to CPU when no suitable device or driver is present. Themetalfeature also enables policy-driven creation on native Apple Silicon throughCreationBackend.CreationBackend::Autokeeps creation work below 16 GiB (slice size × source-slice count × recovery- slice count) on CPU; on supported native Apple Silicon at or above that threshold it preflights Metal and falls back to CPU when unavailable.
§Benchmarks
Heavy PAR2 repair against par2cmdline-turbo 1.4.0, from the deterministic
43-case rarpar-bench corpus. Each figure is the geometric mean of
reference wall time / rarpar wall time over par2-heavy-damage-28 and
par2-heavy-damage-250, so 2.0x means half the time.
| CPU | Arch | Instruction set | par2 (heavy) |
|---|---|---|---|
| AMD EPYC 9R14 (Zen 4) | x86-64 | GFNI + AVX-512 | 1.8x |
| Intel Xeon Platinum 8488C (Sapphire Rapids) | x86-64 | GFNI + AVX-512 | 1.7x |
| Intel Core i5-1240P (Alder Lake) | x86-64 | GFNI + AVX2 | 1.9x |
| AMD Ryzen 5 3600 (Zen 2) | x86-64 | AVX2 | 1.5x |
| Intel Atom C3538 (Denverton) | x86-64 | SSSE3 (no AVX) | 1.3x |
| Apple M5 Max | arm64 | NEON | 7.1x |
| Arm Cortex-A72 | arm64 | NEON | 1.2x |
| Arm Neoverse N1 | arm64 | NEON | 1.4x |
| Arm Neoverse V2 | arm64 | NEON | 1.5x |
The Apple row is the CPU lane, and is measured against upstream’s published macOS arm64 reference binary, which is much slower than the same version’s Linux and Windows builds; that lifts every macOS PAR2 figure.
Per-case charts for every machine, the full methodology, and the versions these numbers were measured with are in rarpar benchmarks.
The format is specified in the Parity Volume Set Specification 2.0.
Re-exports§
pub use checksum::FileHashState;pub use checksum::SliceChecksumState;pub use create::BlockSizing;pub use create::CreationBackend;pub use create::CreationSource;pub use create::ForwardKernel;pub use create::Par2CreateOutcome;pub use create::Par2CreatePlan;pub use create::Par2Creator;pub use create::Par2CreatorOptions;pub use create::Par2MemoryPlan;pub use create::RecoveryAmount;pub use create::RecoveryVolumePlan;pub use create::VolumeScheme;pub use disk::DiskFileAccess;pub use disk::MultiDirectoryFileAccess;pub use disk::PlacementFileAccess;pub use error::Par2Error;pub use error::Result;pub use evidence::CommittedFileEvidence;pub use evidence::ContiguousAssemblyProof;pub use evidence::FileStatFingerprint;pub use matrix::Matrix;pub use matrix::build_decode_matrix;pub use packet::CreatorPacket;pub use packet::DEFAULT_MAX_EXAMINED_PACKETS;pub use packet::DEFAULT_MAX_RETAINED_METADATA_BYTES;pub use packet::DEFAULT_MAX_RETAINED_PACKETS;pub use packet::FileDescriptionPacket;pub use packet::IfscPacket;pub use packet::MAX_RECOVERY_EXPONENT;pub use packet::MainPacket;pub use packet::Packet;pub use packet::PacketHeader;pub use packet::PacketScanBudget;pub use packet::PacketScanLimits;pub use packet::PacketSink;pub use packet::PacketType;pub use packet::RECOVERY_EXPONENT_DOMAIN;pub use packet::RecoverySliceData;pub use packet::RecoverySlicePacket;pub use packet::ScannedPacket;pub use packet::parse_packet;pub use packet::scan_packets;pub use packet::scan_packets_bounded;pub use packet::scan_packets_from_path;pub use packet::scan_packets_from_path_bounded;pub use packet::scan_packets_from_path_with_set_ids;pub use packet::scan_packets_from_path_with_set_ids_limited;pub use packet::scan_packets_with_limits;pub use par2_set::FileDescription;pub use par2_set::MergeResult;pub use par2_set::Par2Diagnostic;pub use par2_set::Par2FileSet;pub use par2_set::Par2ParseResult;pub use par2_set::RecoverySlice;pub use path::translate_par2_name_to_local_path;pub use path::translate_par2_name_to_relative;pub use placement::PlacementEntry;pub use placement::PlacementPlan;pub use placement::apply_placement_plan;pub use placement::scan_placement;pub use rename::MatchType;pub use rename::RenameSuggestion;pub use rename::SplitFileGroup;pub use rename::detect_split_files;pub use rename::identify_par2_files;pub use rename::scan_for_renames;pub use repair::NativeRepairSolver;pub use repair::RepairOptions;pub use repair::RepairPlan;pub use repair::RepairProblem;pub use repair::RepairSolver;pub use repair::SolverError;pub use repair::execute_repair;pub use repair::execute_repair_with_options;pub use repair::execute_repair_with_solver;pub use repair::plan_repair;pub use repair::plan_repair_with_memory_limit;pub use repair::prepare_recovery_buffers;pub use repair::reconstruct_and_write;pub use repair::xor_out_slice;pub use repair_session::DEFAULT_RETAINED_STATE_LIMIT;pub use repair_session::Par2RepairSession;pub use repair_session::Par2RepairSessionDiagnostics;pub use repair_session::Par2RepairSessionOptions;pub use repair_session::Par2SessionError;pub use repair_transform::TransformArm;pub use repair_transform::TransformArmStats;pub use repair_transform::set_transform_arm_override;pub use repair_transform::transform_arm_override;pub use repair_transform::transform_arm_stats;pub use repairer::BlockLocation;pub use repairer::BlockLocationKind;pub use repairer::CarryDiagnostics;pub use repairer::CarryRetryReason;pub use repairer::ExternalCarryError;pub use repairer::PacketDiagnostics;pub use repairer::PacketInventory;pub use repairer::Par2RepairOutcome;pub use repairer::Par2RepairStatus;pub use repairer::Par2Repairer;pub use repairer::Par2RepairerOptions;pub use repairer::ScanCarry;pub use repairer::ScanDiagnostics;pub use repairer::SourceBlock;pub use repairer::SourceFileEntry;pub use repairer::SourceLocation;pub use session::FeedDisposition;pub use session::FeedOutcome;pub use session::InStreamCrc32Proof;pub use session::InStreamCrc32ProofError;pub use session::SettleRead;pub use session::SliceEvidence;pub use session::SliceEvidenceStrength;pub use session::VerificationMemoryBudget;pub use session::VerificationSession;pub use session::VerificationSessionOptions;pub use types::CancellationToken;pub use types::ProgressCallback;pub use types::ProgressPhase;pub use types::ProgressStage;pub use types::ProgressUpdate;pub use types::FileId;pub use types::RecoveryExponent;pub use types::RecoverySetId;pub use types::SliceChecksum;pub use types::SliceIndex;pub use verify::FileAccess;pub use verify::FileStatus;pub use verify::FileVerification;pub use verify::MemoryFileAccess;pub use verify::Repairability;pub use verify::VerificationResult;pub use verify::VerifyOptions;pub use verify::quick_check_16k;pub use verify::verify_all;pub use verify::verify_all_with_options;pub use verify::verify_full_hash;pub use verify::verify_selected_file_ids;pub use verify::verify_selected_file_ids_with_options;pub use verify::verify_slices;pub use verify::verify_slices_from_crcs;
Modules§
- checksum
- create
- Validated PAR2 creation with deterministic packet allocation and transactional outputs. Output transactions detect ordinary replacement races, but assume no other process with equivalent filesystem permissions mutates their staging or backup paths.
- disk
- Filesystem-backed implementation of
FileAccess. - error
- evidence
- gf
- GF(2^16) field arithmetic for PAR2 Reed-Solomon coding.
- gf_pmul
- Element-wise GF(2^16) multiply:
dst[i] = a[i] * b[i]. - gf_simd
- SIMD-accelerated GF(2^16) region operations.
- matrix
- Matrix operations over GF(2^16) for PAR2 Reed-Solomon repair.
- matrix_
tiled - Rank-k tiled Gauss-Jordan inversion for PAR2 repair-matrix solves.
- md5_
simd - Multi-buffer MD5: compute several independent MD5 digests at once by putting one message per SIMD lane.
- packet
- par2_
set - path
- PAR2 filename translation.
- placement
- Content-placement scan: match on-disk files to PAR2 file descriptions by hash.
- rename
- Obfuscated filename recovery and split-file detection.
- repair
- PAR2 repair orchestration using Reed-Solomon decoding over GF(2^16).
- repair_
session - Retained PAR2 repair orchestration.
- repair_
transform - The transform arm of PAR2 repair: syndromes by DFT, solve by the m×m inverse.
- repairer
- High-level PAR2 verifier/repairer.
- session
- Streaming verification session for incremental PAR2 verification during download.
- types
- verify
Structs§
- Cache
Eviction Deferral - RAII guard deferring page-cache eviction until the outermost scope drops.
- Factor
Dst - A (factor, destination) pair for multi-region multiply-accumulate.
Functions§
- gf_add
- Addition in GF(2^16) is XOR.
- gf_inv
- Multiplicative inverse in GF(2^16).
- gf_mul
- Multiplication in GF(2^16) via log/antilog tables.
- gf_pow
- Exponentiation in GF(2^16):
base^exp. - input_
slice_ constants - Compute the PAR2 input slice constant assignment sequence.
- mul_
acc_ multi_ region - Multiply each u16 word in
srcby multiple factors and XOR-accumulate into corresponding destination buffers. - mul_
acc_ region - Multiply each u16 word in
srcbyfactorin GF(2^16) and XOR-accumulate intodst.