dua-core 3.3.0

Fast parallel filesystem traversal iterators
Documentation

Parallel filesystem traversal backed by a work-stealing worker pool.

[walk] yields the root first, then workers read directories and distribute newly discovered subdirectories among themselves. [Order::ParentFirst] publishes each directory's entries before scheduling its children, while [Order::Completion] allows descendant batches to arrive first when their reads finish sooner. Sibling order is unspecified in both modes.

The descend predicate controls which directories are traversed; rejected directories are still yielded (but not traversed). Symbolic links are reported but never followed, and filesystem errors are returned as iterator items. Dropping the iterator stops and joins its workers.

Scheduling

The root directory starts in a shared injector queue. On platforms where directory-entry metadata may require another syscall, directory reads enqueue small metadata batches, and metadata batches enqueue accepted child directories. Windows and macOS workers instead consume native metadata returned by directory enumeration and enqueue child directories immediately. Every worker can run available jobs from its local LIFO queue or steal from a peer. Each successful thief wakes another idle worker, ramping up only while work remains stealable. A worker parks when no queue has work and is unparked when new work arrives or the walk stops. The last completed job emits the finished event; dropping the iterator stops and joins all workers.