A cross platform Rust library for efficiently walking a directory recursively. Comes with support for following symbolic links, controlling the number of open file descriptors and efficient mechanisms for pruning the entries in the directory tree.
Dual-licensed under MIT or the UNLICENSE.
To use this crate, add
walkdir as a dependency to your project's
[dependencies] walkdir = "0.1"
The following code recursively iterates over the directory given and prints the path for each entry:
use WalkDir; for entry in new
Or, if you'd like to iterate over all entries and ignore any errors that may
filter_map. (e.g., This code below will silently skip directories
that the owner of the running process does not have permission to access.)
use WalkDir; for entry in new.into_iter.filter_map
Example: follow symbolic links
The same code as above, except
follow_links is enabled:
use WalkDir; for entry in new.follow_links
Example: skip hidden files and directories efficiently on unix
This uses the
filter_entry iterator adapter to avoid yielding hidden files
and directories efficiently:
use ; let walker = new.into_iter; for entry in walker.filter_entry
std::fs has an unstable
walk_dir implementation that needed some design
work. I started off on that task, but it quickly became apparent that walking
a directory recursively is quite complex and may not be a good fit for
This should at least resolve most or all of the issues reported here (and then some):
The short story is that performance is comparable with
find and glibc's
nftw on both a warm and cold file cache. In fact, I cannot observe any
performance difference after running
walkdir / and
nftw / on my
local file system (SSD, ~3 million entries). More precisely, I am reasonably
confident that this crate makes as few system calls and close to as few
allocations as possible.
I haven't recorded any benchmarks, but here are some things you can try with a
local checkout of
# The directory you want to recursively walk: DIR=$HOME # If you want to observe perf on a cold file cache, run this before *each* # command: sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' # To warm the caches find $DIR # Test speed of `find` on warm cache: time find $DIR # Compile and test speed of `walkdir` crate: cargo build --release --example walkdir time ./target/release/examples/walkdir $DIR # Compile and test speed of glibc's `nftw`: gcc -O3 -o nftw ./compare/nftw.c time ./nftw $DIR # For shits and giggles, test speed of Python's (2 or 3) os.walk: time python ./compare/walk.py $DIR
On my system, the performance of
nftw is comparable.