Skip to main content

Walker

Struct Walker 

Source
pub struct Walker { /* private fields */ }
Expand description

A configured scan of one tree.

Implementations§

Source§

impl Walker

Source

pub fn new(root: impl AsRef<Path>, ruleset: Arc<Ruleset>) -> Self

A walk of root under ruleset, with the defaults the safety model asks for: symlinks are not followed and mount points are not crossed.

The tier-two gitignore fallback is on, at the default floor. It is safe on by default because it never claims a directory holding a tracked file, and it is inert outside a git work tree.

Source

pub fn excludes(self, excludes: Arc<Gitignore>) -> Self

Paths not to descend into, matched in gitignore syntax against paths under the root.

Different from every other refusal in this program, and reported differently. An unreadable directory makes the totals a lower bound and says so, because the scan wanted to look and could not. An excluded one is the reader saying “not there” — the totals are still not the whole tree, but nothing went wrong, and a run that cried “scan incomplete” over a choice its user made would be teaching them to ignore that sentence.

Gitignore syntax rather than a list of prefixes, because it is the matcher everybody reading this already knows, and it brings negation with it: an exclude of Library/Application Support and a re-include of !Library/Application Support/Zed is one line each and needs no new grammar.

Source

pub fn threads(self, threads: usize) -> Self

How many threads to walk with. Defaults to the machine’s parallelism.

Source

pub fn max_depth(self, max_depth: Option<usize>) -> Self

How deep to descend below the root, unbounded by default.

Whether to follow symlinks. Off by default: a followed link leaves the root, and the deleter will not remove anything it cannot prove is under it.

Source

pub fn same_file_system(self, same_file_system: bool) -> Self

Whether to stay on one filesystem. On by default.

Source

pub fn size_mode(self, size_mode: SizeMode) -> Self

How hard to work for each claim’s size.

Source

pub fn fallback(self, fallback: bool) -> Self

Whether to run the tier-two gitignore fallback. On by default.

Source

pub fn ignored_files(self, ignored_files: bool) -> Self

Whether tier two also claims gitignored files. Off by default.

Off rather than on because it is a different job from the one the sweep does, and the difference is not a matter of degree: clearing fifty env files reclaims kilobytes, so the value is hygiene rather than space and a list sorted by size is the wrong place to discover it. A real ~/repos holds tens of thousands of them, and an unasked-for sweep would bury a 40 GB node_modules under .DS_Store rows.

It costs an ignore query and an index lookup per file the walk sees, which the walk was previously getting for free by refusing to judge files at all — so it is opt-in in the library as well as on the command line.

Source

pub fn min_size(self, min_size: u64) -> Self

The size floor a tier-two directory must clear, DEFAULT_MIN_SIZE by default.

It applies to tier-two directories only. A rule that names a directory has already said the directory is output, and an empty node_modules is still a node_modules — and a gitignored file is not on the list for its size in the first place, so a floor stated in bytes has nothing to say about one.

Source

pub fn run<F>(&self, on_found: F) -> WalkOutcome
where F: Fn(Found) + Send + Sync,

Runs the walk, calling on_found as each claim is found and again as each is priced.

on_found is called concurrently, from the walker threads and from the pricing pool, and while the walk is still running — that is the point, since the TUI renders rows as they arrive. It must not block for long, or it becomes the walk’s bottleneck.

§Why a claim and its price are two events

Pricing a claim means walking the subtree the scan just pruned at, and that is an order of magnitude more work than finding it. Measured over one real ~/repos, 10,599 claims, under a full breakdown:

last claim publishedrun complete
priced on the walker thread60.1 s60.1 s
priced on the pool7.5 s63.0 s

Those two left-hand numbers are the whole change. Measuring on the walker thread makes every claim’s publication wait behind its own measurement, so the listing completes only when the last byte has been counted and a front end has nothing whatever to render for a minute. That is npkill’s bargain, and not making it is what the pruning was for.

So a claim is published the moment it is judged, carrying Size::Unmeasured, and is then handed to a pool of pricing threads. Its size arrives afterwards as Found::Priced, naming the same path, and a consumer updates the row in place.

run still does not return until the pool has drained, so every number in the returned WalkOutcome is final. A consumer that only wants totals — the command line, today — need not care that any of this happened.

The pool is one thread per walker thread. Oversubscribing it is the obvious next idea and it was measured, because the deleter oversubscribes for exactly this reason: at four times the threads the same scan takes 85.8 s and does not publish its last claim until 30.7 s. Pricing is readdir and lstat, which is 97% kernel time and contends; unlink and rmdir wait on the disk and do not. The conclusion from the deleter does not carry over here.

Source

pub fn run_to_tree(&self) -> (Tree, WalkOutcome)

Runs the walk and files every hit into a rollup tree, pricing included.

The tree is correct at every moment — after each claim and after each late price — so a caller that wants to render while scanning can build the same thing itself around Walker::run and read the shared tree between updates.

Trait Implementations§

Source§

impl Clone for Walker

Source§

fn clone(&self) -> Walker

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Walker

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.