gravityfile-scan

High-performance parallel file system scanner for gravityfile.

This crate provides the scanning engine that traverses directories and builds the file tree structure used by other gravityfile components.

Features

Parallel Traversal - Uses jwalk for multi-threaded directory scanning
Progress Updates - Subscribe to real-time scan progress via broadcast channels
Hardlink Detection - Tracks inodes to avoid double-counting hardlinked files
Cross-Filesystem Support - Optionally traverse mount points
Ignore Patterns - Skip directories matching configured patterns
Size Aggregation - Automatically calculates directory sizes from contents

Usage

Basic Scan

use gravityfile_scan::{JwalkScanner, ScanConfig};

let config = ScanConfig::new("/path/to/scan");
let scanner = JwalkScanner::new();
let tree = scanner.scan(&config)?;

println!("Total size: {} bytes", tree.total_size());
println!("Total files: {}", tree.total_files());
println!("Total directories: {}", tree.total_dirs());

With Progress Updates

use gravityfile_scan::{JwalkScanner, ScanConfig, ScanProgress};

let scanner = JwalkScanner::new();
let mut progress_rx = scanner.subscribe();

// Spawn task to handle progress
tokio::spawn(async move {
    while let Ok(progress) = progress_rx.recv().await {
        println!(
            "Scanned {} files, {} dirs ({} bytes)",
            progress.files_scanned,
            progress.dirs_scanned,
            progress.bytes_scanned
        );
    }
});

let config = ScanConfig::new("/path/to/scan");
let tree = scanner.scan(&config)?;

Custom Configuration

use gravityfile_scan::{JwalkScanner, ScanConfig};

let config = ScanConfig::builder()
    .root("/path/to/scan")
    .max_depth(Some(5))           // Limit depth
    .include_hidden(false)         // Skip hidden files
    .follow_symlinks(false)        // Don't follow symlinks
    .cross_filesystems(false)      // Stay on same filesystem
    .ignore_patterns(vec![
        ".git".into(),
        "node_modules".into(),
        "target".into(),
    ])
    .threads(0)                    // 0 = auto-detect CPU count
    .apparent_size(false)          // Use disk usage, not apparent size
    .build()?;

let scanner = JwalkScanner::new();
let tree = scanner.scan(&config)?;

Output Structure

The scanner produces a FileTree containing:

root - Root FileNode with nested children
root_path - Canonical path that was scanned
stats - Summary statistics (total size, file count, etc.)
warnings - Non-fatal issues encountered during scan
scan_duration - How long the scan took
config - Configuration used for the scan

Children are automatically sorted by size (largest first) for efficient display.

Performance

Uses rayon thread pool via jwalk for parallel directory traversal
Hardlink tracking uses DashMap for lock-free concurrent access
Progress updates are batched (every 1000 files) to reduce overhead
Memory-efficient: only stores aggregated tree, not individual paths

Re-exports

This crate re-exports common types from gravityfile-core for convenience:

use gravityfile_scan::{
    FileNode, FileTree, NodeId, NodeKind,
    ScanConfig, ScanError, ScanWarning,
    Timestamps, TreeStats, WarningKind,
};

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

gravityfile-scan 0.2.4