pub struct Walker { /* private fields */ }Expand description
A configured scan of one tree.
Implementations§
Source§impl Walker
impl Walker
Sourcepub fn new(root: impl AsRef<Path>, ruleset: Arc<Ruleset>) -> Self
pub fn new(root: impl AsRef<Path>, ruleset: Arc<Ruleset>) -> Self
A walk of root under ruleset, with the defaults the safety model asks for:
symlinks are not followed and mount points are not crossed.
The tier-two gitignore fallback is on, at the default floor. It is safe on by default because it never claims a directory holding a tracked file, and it is inert outside a git work tree.
Sourcepub fn excludes(self, excludes: Arc<Gitignore>) -> Self
pub fn excludes(self, excludes: Arc<Gitignore>) -> Self
Paths not to descend into, matched in gitignore syntax against paths under the root.
Different from every other refusal in this program, and reported differently. An unreadable directory makes the totals a lower bound and says so, because the scan wanted to look and could not. An excluded one is the reader saying “not there” — the totals are still not the whole tree, but nothing went wrong, and a run that cried “scan incomplete” over a choice its user made would be teaching them to ignore that sentence.
Gitignore syntax rather than a list of prefixes, because it is the matcher everybody
reading this already knows, and it brings negation with it: an exclude of
Library/Application Support and a re-include of !Library/Application Support/Zed is
one line each and needs no new grammar.
Sourcepub fn threads(self, threads: usize) -> Self
pub fn threads(self, threads: usize) -> Self
How many threads to walk with. Defaults to the machine’s parallelism.
Sourcepub fn max_depth(self, max_depth: Option<usize>) -> Self
pub fn max_depth(self, max_depth: Option<usize>) -> Self
How deep to descend below the root, unbounded by default.
Sourcepub fn follow_links(self, follow_links: bool) -> Self
pub fn follow_links(self, follow_links: bool) -> Self
Whether to follow symlinks. Off by default: a followed link leaves the root, and the deleter will not remove anything it cannot prove is under it.
Sourcepub fn same_file_system(self, same_file_system: bool) -> Self
pub fn same_file_system(self, same_file_system: bool) -> Self
Whether to stay on one filesystem. On by default.
Sourcepub fn fallback(self, fallback: bool) -> Self
pub fn fallback(self, fallback: bool) -> Self
Whether to run the tier-two gitignore fallback. On by default.
Sourcepub fn ignored_files(self, ignored_files: bool) -> Self
pub fn ignored_files(self, ignored_files: bool) -> Self
Whether tier two also claims gitignored files. Off by default.
Off rather than on because it is a different job from the one the sweep does, and the
difference is not a matter of degree: clearing fifty env files reclaims kilobytes, so
the value is hygiene rather than space and a list sorted by size is the wrong place to
discover it. A real ~/repos holds tens of thousands of them, and an unasked-for sweep
would bury a 40 GB node_modules under .DS_Store rows.
It costs an ignore query and an index lookup per file the walk sees, which the walk was previously getting for free by refusing to judge files at all — so it is opt-in in the library as well as on the command line.
Sourcepub fn min_size(self, min_size: u64) -> Self
pub fn min_size(self, min_size: u64) -> Self
The size floor a tier-two directory must clear, DEFAULT_MIN_SIZE by default.
It applies to tier-two directories only. A rule that names a directory has already said
the directory is output, and an empty node_modules is still a node_modules — and a
gitignored file is not on the list for its size in the first place, so a floor stated in
bytes has nothing to say about one.
Sourcepub fn run<F>(&self, on_found: F) -> WalkOutcome
pub fn run<F>(&self, on_found: F) -> WalkOutcome
Runs the walk, calling on_found as each claim is found and again as each is priced.
on_found is called concurrently, from the walker threads and from the pricing pool,
and while the walk is still running — that is the point, since the TUI renders rows as
they arrive. It must not block for long, or it becomes the walk’s bottleneck.
§Why a claim and its price are two events
Pricing a claim means walking the subtree the scan just pruned at, and that is an order
of magnitude more work than finding it. Measured over one real ~/repos, 10,599
claims, under a full breakdown:
| last claim published | run complete | |
|---|---|---|
| priced on the walker thread | 60.1 s | 60.1 s |
| priced on the pool | 7.5 s | 63.0 s |
Those two left-hand numbers are the whole change. Measuring on the walker thread makes every claim’s publication wait behind its own measurement, so the listing completes only when the last byte has been counted and a front end has nothing whatever to render for a minute. That is npkill’s bargain, and not making it is what the pruning was for.
So a claim is published the moment it is judged, carrying Size::Unmeasured, and is
then handed to a pool of pricing threads. Its size arrives afterwards as
Found::Priced, naming the same path, and a consumer updates the row in place.
run still does not return until the pool has drained, so every number in the
returned WalkOutcome is final. A consumer that only wants totals — the command
line, today — need not care that any of this happened.
The pool is one thread per walker thread. Oversubscribing it is the obvious next idea
and it was measured, because the deleter oversubscribes for exactly this reason: at
four times the threads the same scan takes 85.8 s and does not publish its last
claim until 30.7 s. Pricing is readdir and lstat, which is 97% kernel time and
contends; unlink and rmdir wait on the disk and do not. The conclusion from the
deleter does not carry over here.
Sourcepub fn run_to_tree(&self) -> (Tree, WalkOutcome)
pub fn run_to_tree(&self) -> (Tree, WalkOutcome)
Runs the walk and files every hit into a rollup tree, pricing included.
The tree is correct at every moment — after each claim and after each late price — so a
caller that wants to render while scanning can build the same thing itself around
Walker::run and read the shared tree between updates.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for Walker
impl RefUnwindSafe for Walker
impl Send for Walker
impl Sync for Walker
impl Unpin for Walker
impl UnsafeUnpin for Walker
impl UnwindSafe for Walker
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more