# eDirStat
[](https://crates.io/crates/edirstat)
[](https://docs.rs/edirstat)
[](https://spdx.org/licenses/MIT)
**eDirStat** is a modern, high-performance, cross-platform disk usage analyzer written in Rust. Inspired by legacy utilities like [WinDirStat](https://windirstat.net/), it leverages an immediate-mode graphical interface [`egui`](https://egui.rs/) to provide a real-time, interactive treemap visualization of your filesystem.
Unlike traditional analyzers that crawl sequentially, **eDirStat** is engineered from the ground up for modern multi-core systems. It couples a highly optimized, work-stealing multithreaded directory walker with a zero-copy arena data structure. This allows you to scan millions of files, locate space-wasting files using a treemap diagram (among other plots), identify duplicate files, and save or load system snapshots in milliseconds using compressed snapshots.
[**Up to 2.8x speedup** vs `WinDirStat`](#vs-windirstat-v262)
[**Up to 2.5x speedup** vs `WizTree`](#vs-wiztree-v431)
[**Up to 9.6x speedup** vs `QDirStat`](#vs-qdirstat-v2001)
## 📦 Distribution Model & Project Support
eDirStat is, and will always remain, entirely open-source and free. However, if you'd like to support continuous development, engineering, and maintenance costs, **official pre-compiled binaries for Windows, macOS, and Linux are distributed exclusively as a paid download via [Itch.io - xangelix.itch.io/edirstat](https://xangelix.itch.io/edirstat)**.
- **100% Open Source:** The complete codebase is free of charge and publicly available under the permissive **MIT License**. If you prefer to build the tool yourself, you are welcome to compile the source code on Windows, Linux, or macOS at no cost. Instructions are provided below.
- **Package Managers:** Package managers and their repository maintainers are welcome to bundle and distribute eDirStat through their respective channels at no cost, provided that it is distributed un-paid, "unofficial", and under the MIT License.
Purchasing precompiled packages directly funds the engineering efforts required to keep eDirStat fast, secure, and compatible with the latest operating systems.
[](https://xangelix.itch.io/edirstat)
[](https://aur.archlinux.org/packages/edirstat)
---
## 📽️ Demo Video
<https://github.com/user-attachments/assets/22075865-c258-4cee-bae4-247a48702f25>
---
## 📸 Screenshots
---
## 🚀 Key Features
- ⚡ **Work-Stealing Multi-threading:** Powered by a lock-free task injector queue that keeps all CPU cores saturated during scanning-- inspired by `ripgrep`.
- 🪟 **NTFS MFT Scanner (Windows):** Accesses raw NTFS physical handles to parse the Master File Table directly, bypassing OS filesystem bottlenecks for near-instantaneous drive indexing (requires administrative privileges).
- 👥 **7-Stage Deduplication Engine:** Safely identifies byte-for-byte identical files using cryptographically secure BLAKE3 hashing. It is hardlink-aware to protect shared filesystem links.
- 📦 **Fast Compressed Snapshots:** Writes structured tree snapshots to disk with Zstd compression. Once loaded and decompressed, the flat binary representation is cast directly via `bytemuck`, eliminating parsing overhead. Cross-compatible on all Little-Endian platforms.
- 📊 **Dynamic Treemap Visualization:** Features a responsive layout canvas with smooth HSL gradient scaling based on file extensions.
- 🗂️ **Layout Modes and Plots:** Choose between the different layout modes, both featuring data visualizations that can be cycled between.
- 📋 **Bulk Operations & Multi-Select:** Select multiple rows in the directory tree or deduplicator to execute batch trashing, deletion, or linking.
- 🛡️ **Safe & Native:** Built completely in safe, pure Rust with immediate-mode UI rendering and cross-platform support.
---
## 🚀 Installation & Build
### Prerequisites
Ensure you have the Rust toolchain installed. See `rust-toolchain.toml` for the version currently required.
### Build from Source
```bash
# Clone the repository
git clone https://github.com/xangelix/edirstat.git
cd edirstat
# Build the release executable
cargo build --release
```
The compiled binary will be located at `target/release/edirstat`.
> **Note:** When building on **Windows** you must use the nightly compiler, as `edirstat` requires the nightly feature `windows_by_handle`.
---
## 📖 Usage Guide
To run the GUI application from the command line:
```bash
./target/release/edirstat
```
You can also pass a directory path as a positional argument to automatically launch and begin scanning that folder on startup:
```bash
./target/release/edirstat /path/to/scan
```
### Navigating the User Interface
1. **Scan a Directory:**
Click the **📁 Scan Directory** button in the top menu bar to open a folder picker. Select the target drive or folder to initiate the scan.
2. **Explore the Tree:**
The left-hand panel displays a hierarchical directory explorer. You can expand/collapse folders using the `[+]`/`[-]` toggles. Use the **🔍 Filter** input bar to narrow down the view to matching folders or files.
3. **Interact with the Treemap:**
The central panel displays a visual representation of your disk space. Larger rectangles correspond to larger files or directories.
- **Hovering:** Move your cursor over a block to view its full path and size in a tooltip.
- **Clicking:** Click on a block to automatically select it in the directory tree on the left.
4. **Inspect File Extensions:**
The right panel displays a sorted list of file extensions detected during the scan, complete with color-coded markers.
5. **Deduplicate Your Drive:**
Switch to the **👥 Deduplicator** tab to search for duplicate files on your scanned filesystem. Custom selection helpers allow you to automatically select duplicates while preserving the oldest, newest, or shortest-path file.
6. **Context Actions:**
Right-click any item in the left-hand explorer to open a context menu.
- **Open in File Manager:** Launches your operating system's default file browser (Explorer, Finder, or Files) at the selected path.
- **Copy Name:** Copies the name of the selected folder or file to the system clipboard.
- **Copy Path:** Copies the absolute path of the selected folder or file to the system clipboard.
- **Delete (Permanent):** Opens a safety dialog to permanently delete the target path from your disk.
---
## 💾 Saving & Loading Snapshots
If you need to analyze a server or remote environment:
1. `edirstat /path/to/my/dir --to mysnapshot`
2. Transfer the `mysnapshot.edst` file to another machine.
3. Launch `edirstat` and click **📖 Load Snapshot** to open and navigate the tree with full interactivity, requiring no active filesystem connection.
---
## ⚙️ Architectural Design & Internals
### 1. Parallel Work-Stealing Walker (`src/traversal.rs`)
The traversal engine avoids the performance bottlenecks of standard recursive single-threaded walkers. It utilizes `crossbeam-deque` for task scheduling inspired by `ripgrep`:
- **Workers & Stealers:** Each parallel thread operates on a local thread-safe FIFO task queue. When a thread runs out of directories to scan, it attempts to steal tasks from a global injector or peer worker queues.
- **Cycle Detection:** Avoids infinite directory loops (caused by recursive symbolic links) by checking filesystem identity descriptors (`dev`/`ino` on Unix, and `volume_serial_number`/`file_index` on Windows) against an inherited stack of ancestors.
- **Developer-Friendly Ignore Matching:** Evaluates file pathways against globally defined structures and localized directory-level `.gitignore` files using compiled `globset` configurations.
- **Device Boundary Restrictions:** Restricts the scan to the primary mount point or device boundary to prevent unintended traversal of system directories (e.g., `/sys` or `/proc`).
### 2. Lock-Free Snapshot Commit Loop (`src/coordinator.rs`)
To prevent traversal worker threads from blocking the UI rendering cycle, `edirstat` decouples directory scanning from interface updates through an event-driven coordinator model:
- **The Coordinator:** Worker threads stream compressed structural events (`ScanEvent`) over a lock-free channel to a dedicated background Coordinator thread.
- **Dynamic ID Map:** The Coordinator translates worker-local task identifiers to global array indexes in $O(1)$ amortized time.
- **Atomic Snapshot Publishing:** Instead of locking a mutable tree, the GUI accesses an immutable `FileArenaSnapshot` read-only copy via `arc_swap`. The Coordinator issues updated snapshots to the GUI every 100 milliseconds during an active scan.
### 3. Cache-Friendly Arena Representation (`src/arena.rs`)
To conserve memory and avoid pointer-chasing latency, the directory tree is flattened into a single contiguous array (arena):
```text
[ Root Node ] ---> [ Child A ] ---> [ Child B ] ---> [ Child C ]
|
v
[ Sub-child 1 ]
```
- **Index-Based References:** Individual `FileNode` blocks reference parent, child, and sibling nodes using raw `u32` indices rather than heap-allocated pointers (`Box` or `Rc`).
- **Plain Old Data (POD):** The `FileNode` struct is annotated with `bytemuck::Pod` and `bytemuck::Zeroable`, strictly aligned to 8-byte boundaries.
- **Compact String Pool:** Directory and file names are deduplicated and stored in a contiguous byte sequence (`StringPool`). Nodes reference these names via a lightweight `StringId` wrapper.
### 4. Zstd-Compressed Snapshot Persistence (`src/persistence.rs`)
The `.edst` snapshot file layout features a Zstd-compressed payload matched to the in-memory arena:
```text
+-------------------------------------------------------------+
| - Version: u16 |
| - Uncompressed Size: u64 |
| - Node Count: u64 |
| - String Pool Offset & Length |
+-------------------------------------------------------------+
| Zstd Compressed Payload: |
| - Array of FileNode Structs (Flat Binary Segment) |
| - String Pool Data (Serialized Offsets + Packed UTF-8) |
+-------------------------------------------------------------+
```
- **Zstd Compression:** Minimizes the disk storage footprint of snapshot files while maintaining high read/write speeds.
- **Zero-Parsing Deserialization:** Once decompressed into heap memory, the data is cast directly into a slice of `&[FileNode]` via `bytemuck`, avoiding costly parsing loops or node-by-node deserialization overhead.
### 5. Multi-Stage Deduplication Engine (`src/stats/deduplicator.rs`)
<https://github.com/user-attachments/assets/a5743098-e88f-4fb7-bc0d-df073ed0615f>
The deduplication module detects byte-for-byte identical files with minimal disk I/O. Candidate duplicate groups are identified and isolated through a 7-stage pipeline:
1. **Size Partitioning:** Scanned files are grouped by identical byte counts. Singleton sizes are discarded immediately.
2. **Prefix Hashing:** Worker threads read and hash the first 4KB of files to filter out non-matching formats.
3. **Midpoint Hashing:** Computes a hash around the center of the remaining files to detect differences inside similar files.
4. **Suffix Hashing:** Hashes the last 4KB of file data, which often contains unique trailing metadata.
5. **Multi-Range Hashing:** Performs periodic block sampling (every 100MB) across large files to ensure long-distance uniformity without scanning entire gigabyte-scale structures.
6. **Full Cryptographic Hashing:** Executes a full BLAKE3 cryptographic hash only over candidates that successfully cleared the previous five stages.
7. **Real-time Validation:** Performs timestamp checking on disk immediately before grouping and action triggers to protect you against modifying files changed since snapshot generation.
The engine remains hardlink-aware, allowing it to accurately differentiate between physical duplicate copies and single-inode hardlinks, which consume no additional storage.
---
## Benchmarks
### Vs `WinDirStat` (v2.6.2)
A dense Windows primary drive.
`NVMe PCIe Gen3 [ntfs]`
**Up to 2.8x speedup**
video coming soon!
### Vs `WizTree` (v4.31)
A dense Windows primary drive.
`NVMe PCIe Gen3 [ntfs]`
**Up to 2.5x speedup**
<https://github.com/user-attachments/assets/2190e1d1-51ac-4b5e-89e6-80dfd1d9c8de>
### Vs `QDirStat` (v2.0.01)
To evaluate traversal performance, `edirstat` includes a custom comparison benchmark target comparing it against `qdirstat-cache-writer` (the official headless command-line crawler shipped with `QDirStat` for background scanning).
#### How It Works & Why It's Fair
1. **End-to-End Subprocess Spawning:** Both tools are launched as independent external subprocesses (running the optimized release binary for `edirstat` and the `perl` script execution for `QDirStat`). This captures full end-to-end CLI execution time, including binary loading, runtime initialization (Perl interpreter boot vs. Rust startup), option parsing, and traversal startup.
2. **Warm Cache Inode Priming:** The benchmark performs 2 warm-up runs for each target directory to prime the OS directory entry page caches. This eliminates disk I/O bottlenecks and isolates CPU/algorithm execution efficiency (multi-threaded, work-stealing Rust vs. single-threaded Perl).
3. **Statistical Averaging:** Measurements are collected across 5 consecutive sample runs to compute a robust median and average traversal duration.
#### Results
Across a diverse suite of storage devices and directory layouts, **eDirStat** consistently exceeds the performance of `QDirStat` in duplicate, delivering up to a **9.6x speedup** (ranging from 6.0x to 9.6x) in scan time!
Whether crawling highly nested code repositories on high-speed `NVMe` drives, game installations on SATA SSDs, or deep directory trees on enterprise HDDs, `edirstat`'s parallel, work-stealing multithreading model, allows it to remain the fastest disk usage analyzer available.
Key Highlights:
- Up to **9.60x faster** than the `QDirStat` backend writer, especially on SSDs.
- Achieves a **6.04x speedup** on mechanical HDDs even when processing massive, deeply nested directory paths.
- Smoothly scales directory traversal workload across all available CPU threads.
#### System Details
AMD Ryzen 9 9950X3D (32)
Linux 6.18.34-1-cachyos-lts
#### `/home/tux/Documents/git`
`Samsung 990 Pro NVMe SSD (Gen 4) [btrfs]`
Dense, an enormous amount of small files and directories.
```text
Running benches/compare.rs (target/release/deps/compare-95ade4249a41f004)
==================================================
eDirStat vs QDirStat Benchmark
==================================================
Target Directory : /home/tux/Documents/git
CPU Cores Available: 32
==================================================
Performing 2 warm-up runs...
Performing 5 sample runs...
Run 1/5... edirstat: 856.10ms, qdirstat: 6.70s
Run 2/5... edirstat: 869.77ms, qdirstat: 7.11s
Run 3/5... edirstat: 881.21ms, qdirstat: 6.89s
Run 4/5... edirstat: 865.67ms, qdirstat: 6.91s
Run 5/5... edirstat: 851.16ms, qdirstat: 7.31s
================ RESULTS SUMMARY ================
eDirStat (Rust, parallel):
Min : 851.16ms
Max : 881.21ms
Median: 865.67ms
Mean : 864.78ms
QDirStat (Perl writer):
Min : 6.70s
Max : 7.31s
Median: 6.91s
Mean : 6.98s
Speedup (QDirStat / eDirStat): 7.99x
==================================================
```
#### `/run/media/tux/F1/Games/PC/SteamLibrary/steamapps/common`
`Samsung SSD 870 QVO 8TB [btrfs]`
Game files, a mix of large and small files on a SATA SSD.
```text
Running benches/compare.rs (target/release/deps/compare-95ade4249a41f004)
==================================================
eDirStat vs QDirStat Benchmark
==================================================
Target Directory : /run/media/tux/F1/Games/PC/SteamLibrary/steamapps/common
CPU Cores Available: 32
==================================================
Performing 2 warm-up runs...
Performing 5 sample runs...
Run 1/5... edirstat: 471.00ms, qdirstat: 4.24s
Run 2/5... edirstat: 478.63ms, qdirstat: 5.12s
Run 3/5... edirstat: 467.80ms, qdirstat: 4.54s
Run 4/5... edirstat: 475.84ms, qdirstat: 4.27s
Run 5/5... edirstat: 451.38ms, qdirstat: 8.45s
================ RESULTS SUMMARY ================
eDirStat (Rust, parallel):
Min : 451.38ms
Max : 478.63ms
Median: 471.00ms
Mean : 468.93ms
QDirStat (Perl writer):
Min : 4.24s
Max : 8.45s
Median: 4.54s
Mean : 5.32s
Speedup (QDirStat / eDirStat): 9.64x
==================================================
```
#### `/run/media/tux/D1`
`Seagate Exos X18 18TB HDD [btrfs]`
Large files, but fewer, on an HDD. Less directory nesting.
```text
Running benches/compare.rs (target/release/deps/compare-95ade4249a41f004)
==================================================
eDirStat vs QDirStat Benchmark
==================================================
Target Directory : /run/media/tux/D1
CPU Cores Available: 32
==================================================
Performing 2 warm-up runs...
Performing 5 sample runs...
Run 1/5... edirstat: 8.01ms, qdirstat: 33.43ms
Run 2/5... edirstat: 5.34ms, qdirstat: 33.03ms
Run 3/5... edirstat: 5.48ms, qdirstat: 32.67ms
Run 4/5... edirstat: 4.59ms, qdirstat: 33.06ms
Run 5/5... edirstat: 5.81ms, qdirstat: 33.49ms
================ RESULTS SUMMARY ================
eDirStat (Rust, parallel):
Min : 4.59ms
Max : 8.01ms
Median: 5.48ms
Mean : 5.85ms
QDirStat (Perl writer):
Min : 32.67ms
Max : 33.49ms
Median: 33.06ms
Mean : 33.14ms
Speedup (QDirStat / eDirStat): 6.03x
==================================================
```
#### `/run/media/tux/B4`
`Toshiba MG09SACA16EA 16TB HDD [btrfs]`
An enormous amount of tiny files with deep directory nesting, on an HDD.
```text
Running benches/compare.rs (target/release/deps/compare-95ade4249a41f004)
==================================================
eDirStat vs QDirStat Benchmark
==================================================
Target Directory : /run/media/tux/B4
CPU Cores Available: 32
==================================================
Performing 2 warm-up runs...
Performing 5 sample runs...
Run 1/5... edirstat: 515.15ms, qdirstat: 3.54s
Run 2/5... edirstat: 520.41ms, qdirstat: 3.51s
Run 3/5... edirstat: 537.08ms, qdirstat: 3.50s
Run 4/5... edirstat: 580.76ms, qdirstat: 4.24s
Run 5/5... edirstat: 612.70ms, qdirstat: 4.05s
================ RESULTS SUMMARY ================
eDirStat (Rust, parallel):
Min : 515.15ms
Max : 612.70ms
Median: 537.08ms
Mean : 553.22ms
QDirStat (Perl writer):
Min : 3.50s
Max : 4.24s
Median: 3.54s
Mean : 3.77s
Speedup (QDirStat / eDirStat): 6.60x
==================================================
```
> **Benchmark Disclaimer & Configuration:**
>
> - Comparisons were conducted against **WinDirStat v2.6.2**, **WizTree v4.31**, and **QDirStat v2.0.01** under controlled testing conditions with primed system caches.
> - eDirStat is an independent open-source utility and is not associated with, sponsored by, or endorsed by the trademark holders of those projects.
> - Performance measurements depend heavily on hardware setup, filesystem fragmentation, operating system scheduling, and disk caching behavior; individual test results may vary.
---
## 📝 License
This project is licensed under the [MIT License](LICENSE).
---
_Disclaimer: `WinDirStat`, `WizTree`, and `QDirStat` are trademarks of their respective owners. eDirStat is an independent open-source project and is not affiliated with, sponsored by, or endorsed by the trademark holders._