Skip to main content

Crate quichash_core

Crate quichash_core 

Source
Expand description

§quichash-core

quichash-core is the reusable engine behind the QuicHash command-line application. It hashes bytes, readers, files, and directory trees without parsing command-line arguments or owning terminal output.

The typed API built around Algorithm, DigestValue, HasherSet, Manifest, scan_folder, and verify_folder is recommended for new applications. The engine types in the individual modules remain available for compatibility with the QuicHash CLI.

§Dependency

Use the complete default configuration:

[dependencies]
quichash-core = "0.0.22"

When consuming the workspace directly:

[dependencies]
quichash-core = { path = "../hash-rs/crates/quichash-core" }

For a small BLAKE3-only build that hashes bytes and streams:

[dependencies]
quichash-core = { version = "0.0.22", default-features = false, features = ["blake3"] }

§Cargo features

The default feature set enables every feature below.

FeatureEnables
all-algorithmsAll algorithm features listed below
md5MD5
sha1SHA-1
sha2SHA-224, SHA-256, SHA-384, and SHA-512
sha3SHA3-224, SHA3-256, SHA3-384, and SHA3-512
blake2BLAKE2b-512 and BLAKE2s-256
blake3BLAKE3
xxhashXXH3-64 and XXH3-128
filesystemFolder scanning, verification, ignore files, deduplication, and wildcard expansion
parallelRayon-powered folder work and BLAKE3 parallel support
mmapMemory-mapped BLAKE3 file hashing
zstdReading and creating Zstandard-compressed manifest files
reportingJSON and timestamp support used by analysis and comparison reports

Calling an algorithm that was compiled out returns HashUtilityError::AlgorithmUnavailable. Algorithm::is_available and HashRegistry::list_algorithms can be used to discover the compiled algorithms at runtime.

§Algorithms

Algorithm provides stable identifiers, parsing, display names, output sizes, and feature availability for:

  • MD5 and SHA-1
  • SHA-224, SHA-256, SHA-384, and SHA-512
  • SHA3-224, SHA3-256, SHA3-384, and SHA3-512
  • BLAKE2b-512, BLAKE2s-256, and BLAKE3
  • XXH3-64 (xxh3) and XXH3-128 (xxh128)

MD5, SHA-1, and xxHash are useful for compatibility or non-adversarial file identification, but should not be selected for new security-sensitive uses. BLAKE3 and SHA-2/3 are the usual choices for integrity verification.

§Hash bytes

Use hash_bytes for an in-memory value. Multiple algorithms consume the input in one pass and results retain the requested order.

use quichash_core::{hash_bytes, Algorithm};

let digests = hash_bytes(b"hello", &[Algorithm::Blake3, Algorithm::Sha256])?;
assert_eq!(digests.len(), 2);
assert_eq!(digests[0].algorithm, Algorithm::Blake3);
assert_eq!(digests[0].to_hex().len(), 64);

DigestValue::from_hex validates both hexadecimal syntax and the expected length for its algorithm. DigestValue::as_bytes exposes the validated raw digest when binary storage or comparison is preferable.

§Incremental hashing

Use HasherSet when an application already receives chunks, such as from a network stream or archive decoder.

use quichash_core::{Algorithm, HasherSet};

let mut hashers = HasherSet::new(&[Algorithm::Blake3, Algorithm::Sha512])?;
hashers.update(b"first chunk");
hashers.update(b"second chunk");
let digests = hashers.finalize();
assert_eq!(digests.len(), 2);

HasherSet is synchronous and has no async-runtime dependency. An async application can update it with each buffer it reads, or move file work to its runtime’s blocking worker pool.

§Hash readers and files

hash_reader streams any std::io::Read implementation through all requested algorithms with a 1 MiB internal buffer:

use std::io::Cursor;
use quichash_core::{hash_reader, Algorithm};

let mut reader = Cursor::new(b"reader input");
let digests = hash_reader(&mut reader, &[Algorithm::Blake3])?;
assert_eq!(digests.len(), 1);

hash_file hashes an entire file. With blake3, parallel, and mmap enabled, a single BLAKE3 file hash uses the optimized memory-mapped path when possible. hash_file_mode additionally accepts HashMode.

use std::path::Path;
use quichash_core::{hash_file, Algorithm};

let digests = hash_file(Path::new("archive.tar"), &[Algorithm::Blake3])?;
println!("{}", digests[0].to_hex());

HashMode::Full reads the complete file and is required for cryptographic integrity. HashMode::Sampled hashes selected regions of large files for quick identification; it is intentionally a different digest domain and must not be treated as a complete integrity check.

§Progress and cancellation

Long-running typed APIs accept an OperationObserver. Its callbacks may be invoked from worker threads, so implementations must be Send + Sync and should return quickly. Cancellation is cooperative and returns HashUtilityError::Cancelled. Use NoopObserver when neither capability is needed.

use std::sync::atomic::{AtomicBool, Ordering};
use quichash_core::{OperationObserver, ProgressEvent};

#[derive(Default)]
struct Observer {
    cancelled: AtomicBool,
}

impl OperationObserver for Observer {
    fn on_progress(&self, event: &ProgressEvent) {
        let _completed = event.completed;
    }

    fn is_cancelled(&self) -> bool {
        self.cancelled.load(Ordering::Relaxed)
    }
}

hash_reader_observed reports streamed bytes. scan_folder reports file discovery and hashing, and verify_folder reports verification progress.

§Scan a folder

Folder APIs require the filesystem feature. scan_folder recursively hashes regular files, does not follow symbolic links, includes hidden files, and optionally applies .hashignore patterns. Paths stored in the manifest are relative to the supplied root.

use std::path::Path;
use quichash_core::{
    scan_folder, Algorithm, FailurePolicy, HashMode, NoopObserver, ScanOptions,
};

let options = ScanOptions::default()
    .with_algorithms(vec![Algorithm::Blake3, Algorithm::Sha256])
    .with_mode(HashMode::Full)
    .with_parallel(true)
    .with_hashignore(true)
    .with_failure_policy(FailurePolicy::FailFast)
    .with_exclude(None);
let report = scan_folder(Path::new("assets"), &options, &NoopObserver)?;
println!("hashed {} files", report.files_processed);
for root in report.folder_digests {
    println!("{} {}", root.algorithm, root.digest.to_hex());
}

Only regular files contribute to a folder digest. The digest commits to the canonical relative path, size, hash mode, algorithm names, and every stored file digest. Entry order does not affect it; renaming a file, changing modes, or changing any digest does. The domain-separated encoding is versioned as quichash-folder-v1.

ScanOptions::exclude omits one canonicalized file, which is useful when the manifest is written inside the directory being scanned.

§Manifest formats

Manifest is the typed in-memory representation. Each ManifestEntry contains a relative path, size, hash mode, and one or more validated digests. Call Manifest::canonicalize before custom serialization, or Manifest::folder_digests to derive tree hashes.

database::DatabaseHandler reads and writes:

  • QuicHash (.qh): one hash algorithm mode path row per digest. Repeated rows for the same path are merged into one multi-digest entry.
  • hashdeep (.hashdeep): size,hash1,hash2,...,filename, preserving every algorithm column declared by the header.
  • Zstandard: existing text databases can be read transparently from any .zst or .zstd path when zstd is enabled. New compression is limited to QuicHash content and creates .qh.zst.
use std::path::Path;
use quichash_core::database::{DatabaseFormat, DatabaseHandler};

let manifest = DatabaseHandler::read_manifest(Path::new("hashes.txt"))?;
let actual_path = DatabaseHandler::write_manifest_file(
    Path::new("converted.txt"),
    &manifest,
    DatabaseFormat::Quichash,
    false,
)?;
assert_eq!(actual_path, Path::new("converted.qh"));

DatabaseHandler::canonical_output_path normalizes requested output names to .qh, .qh.zst, or .hashdeep. DatabaseHandler::write_manifest_file writes to that canonical path and returns it. On compression failure the plain .qh file is retained. DatabaseHandler::write_manifest and the row writers remain available when an application intentionally owns its filenames and output streams.

Reading remains content-based for compatibility, so legacy uncompressed .txt and .db manifests and historical .txt.zst or .hashdeep.zst files continue to work.

Verification can additionally read conventional two-column checksum files with DatabaseHandler::read_checksum_manifest. The algorithm is inferred from extensions such as .md5, .sha256, or .blake3 (optionally followed by .zst or .zstd). Parsing accepts GNU text/binary markers and generic whitespace separators, and fails on malformed rows. This format is verification-only and is not accepted by manifest writers, analysis, or comparison.

use std::path::Path;
use quichash_core::{verify_folder, FailurePolicy, NoopObserver};
use quichash_core::database::DatabaseHandler;

let manifest = DatabaseHandler::read_checksum_manifest(Path::new("checks.sha256"))?;
let report = verify_folder(
    &manifest,
    Path::new("assets"),
    FailurePolicy::FailFast,
    &NoopObserver,
)?;
assert!(report.mismatches.is_empty());

DatabaseHandler::read_manifest is fail-fast. Use DatabaseHandler::read_manifest_with_policy with FailurePolicy::Continue to receive valid entries plus line-level database::DatabaseIssue values.

JSON is available for analysis, comparison, and deduplication reports when the reporting feature is enabled; it is not a persisted Manifest input format.

§Verify a folder

verify_folder recomputes every digest stored for every expected file. A file counts as a match only when all its stored digests match. The report keeps algorithm-specific mismatches, missing paths, new regular files, and recoverable issues separate.

use std::path::Path;
use quichash_core::{verify_folder, FailurePolicy, NoopObserver};
use quichash_core::database::DatabaseHandler;

let manifest = DatabaseHandler::read_manifest(Path::new("hashes.hashdeep"))?;
let report = verify_folder(
    &manifest,
    Path::new("assets"),
    FailurePolicy::FailFast,
    &NoopObserver,
)?;
let valid = report.mismatches.is_empty()
    && report.missing_files.is_empty()
    && report.new_files.is_empty()
    && report.issues.is_empty();
assert!(valid, "folder contents changed");

FailurePolicy::FailFast returns the first operational or parsing error. FailurePolicy::Continue retains item-level errors in the corresponding report, but cancellation still stops immediately.

§Analysis, comparison, and deduplication

Compatibility engine modules expose the remaining QuicHash functionality:

  • analyze::AnalyzeEngine summarizes a manifest and identifies duplicate digest groups.
  • compare::CompareEngine compares two databases and reports unchanged, changed, moved, removed, added, and duplicate files.
  • dedup::DedupEngine scans a directory and groups duplicate files. It reports candidates only and never deletes data.
  • benchmark::BenchmarkEngine measures algorithm throughput.
  • wildcard::expand_pattern expands CLI-style filesystem patterns.
  • path_utils contains cross-platform normalization and relative-path helpers.

The older hash::HashComputer, scan::ScanEngine, and verify::VerifyEngine accept string algorithm names and mirror historical CLI behavior. New embedded applications should prefer the typed functions at the crate root because they provide validated digests, multi-hash manifests, structured progress, and explicit error policy.

§Error handling

All primary operations return HashUtilityError, which preserves the path and operation where possible. Match the variants when recovery differs, or display the error for a contextual message.

use quichash_core::{Algorithm, HashUtilityError};

let result = "not-an-algorithm".parse::<Algorithm>();
assert!(matches!(
    result,
    Err(HashUtilityError::UnsupportedAlgorithm { .. })
));

The library never starts an async runtime, performs network requests, or parses process arguments. The typed APIs do not render progress; presentation belongs to the embedding CLI, GUI, service, or application.

Re-exports§

pub use error::HashUtilityError;
pub use hash::Algorithm;
pub use hash::AlgorithmInfo;
pub use hash::DigestValue;
pub use hash::HashComputer;
pub use hash::HashMode;
pub use hash::HashRegistry;
pub use hash::HashResult;
pub use hash::Hasher;
pub use hash::HasherSet;
pub use hash::hash_bytes;
pub use hash::hash_file;
pub use hash::hash_file_mode;
pub use hash::hash_reader;
pub use hash::hash_reader_observed;
pub use manifest::DigestMismatch;
pub use manifest::ManifestVerifyReport;
pub use manifest::OperationIssue;
pub use manifest::ScanOptions;
pub use manifest::ScanReport;
pub use manifest::scan_folder;
pub use manifest::verify_folder;
pub use manifest::FolderDigest;
pub use manifest::Manifest;
pub use manifest::ManifestEntry;
pub use operation::FailurePolicy;
pub use operation::NoopObserver;
pub use operation::OperationObserver;
pub use operation::ProgressEvent;
pub use operation::ProgressPhase;

Modules§

analyze
Manifest analysis.
benchmark
Hash algorithm benchmarks.
compare
Manifest comparison.
database
QuicHash and hashdeep manifest formats.
dedup
Duplicate-file analysis.
error
Error handling for the reusable QuicHash core.
hash
Hash computation primitives and optimized file helpers.
ignore_handler
.hashignore handling.
manifest
Typed multi-algorithm manifests and deterministic folder digests.
operation
Shared control and progress types for long-running operations.
path_utils
Cross-platform path utilities.
scan
Recursive directory scanning.
verify
Manifest verification.
wildcard
Wildcard expansion utilities.