Module home 

The home screen: pick a dataset without first knowing where it is.

It lists roots (places to look) and catalogs (named datasets and directories):

1. The working directory.
2. Catalogs: catalog.toml (Ctrl+D adds to it), the files catalogs lists, and the bundled public catalog. A directory in one is a row to step into.

RECENT lists every dataset opened, grouped under the directory or prefix it lives in; Enter on such a place row browses it. It is derived state, cheap to rebuild or discard.

Modules

catalog

Catalogs: named datasets, local or remote, one TOML file each.

codebook

A catalog dataset’s documentation: what its columns mean, in their publisher’s words.

discover

Dataset discovery for the home screen. Not a catalog: listings are computed from the filesystem when asked and forgotten at session end; between runs datui keeps only recent paths and measured shapes (remembered, valid while the files are unchanged). Every function scans one directory level, since data lives on slow mounts and huge partition trees.

fuzzy

Fuzzy matching scored as fzf scores it, so ranking matches the finder people already know (fzf, fzf-lua, Telescope’s fzf-native and snacks.picker, a port of fzf/src/algo/algo.go, all agree). The scoring constants are fzf’s:

home_preview

The home screen’s ROWS preview: the first rows of the selected file, read the way the open reads them and handed to the open as its first page (#547 M4).

locality

Where a dataset physically lives, and what that implies about opening it (2 GB on tmpfs and on hotel-wifi NFS are the same row and a thousandfold apart). Derived from /proc/self/mountinfo, a kernel-generated local read that cannot block on a share that stopped answering.

search

Recursive search for datasets below a directory: one bounded background walk of the working directory, its results filtered in memory like listed rows; nothing repeats per keystroke. Every limit exists because some real directory needs it (see Limits). The walk keeps every data file, scored off the UI thread (score); the listing cap counts matches, so a match is never lost behind non-matches.

Structs

CatalogPlaces

The catalogs’ datasets and bookmarks indexed by place, for per-row lookups.

CloudSource

One cloud source on home: a row under CLOUD and its bucket list. Kept apart from Section to survive rebuilds: re-listing buckets would be a billed round trip per keystroke.

Hit

How a row answers the filter: its score and the matched characters, in its name or a matched column’s name. Computed when rows are listed, not when drawn.

HomeState

Home screen state.

Listing

What a listing pass produced.

ListingRequest

Everything build_listing needs, gathered on the UI thread so the worker never reaches into the app.

Mark

Where the cursor was in a listing the user went inside from, by row identity: the listing is rebuilt on a worker and may change.

Measured

What measuring a dataset yielded; each part absent when unknowable without reading the data.

Narrowed

What a cut-short cloud directory holds under one name prefix, asked of the server for a typed filter.

PathListing

What the ~ prompt lists: the directory part of what is typed, and what is in it.

PathName

One name in the directory the ~ prompt is typing.

Probes

Every probe, by the place it lists.

Root

A place datui will look, and whether it can currently be read.

RowLabel

What a home row is called, decided once for the list and the pane beside it.

RowsCache

HomeState::visible’s rows, cached until their inputs change: building scores every row and sorts each section, and every pass reads them. HomeState’s own row-changing methods drop it; fields set elsewhere (filter, sort, folds) are compared on every read.

ScoreJob

What a worker needs to score the filter against a walk’s files.

SearchState

One recursive walk below the working directory. Kept apart from sections, which rebuild often: re-walking each time would cost per keystroke.

Section

A titled group of rows on the home screen.

ShownCatalog

A catalog as the home screen shows it: a section of named datasets, local and remote alike.

ShownDataset

One dataset of a ShownCatalog.

Enums

CloudLook

A directory in an object store that has not said what it holds yet.

CloudStatus

Where a cloud source’s listing stands.

DoorKind

What a directory’s door opens, from the kind and the tally already in hand.

Probe

Where the listing of one network root or remote directory stands; read off the UI thread (see HomeState::pending_probes).

RootOrigin

Where a root came from, shown subtly in the UI.

Row

One line of the home screen; headers are selectable to fold. Places and more rows are view rows, not entries: an Entry’s kind triggers probes, measurement and caching, none of which may happen to a place.

RowKey

A row’s identity apart from its index, to find it again after a rebuild or a cap change: the cursor’s index points at different rows whenever a listing lands or the terminal resizes, which would make Enter open the wrong row.

SortMode

How rows are ordered within each section.

Constants

BUNDLED_ORIGIN

The chip on the bundled catalog’s section.

CATALOG_ORIGINS

The chips a catalog’s section carries, and nothing else does.

CLOUD_PLACE

How a cloud source is addressed on home: cloud://<id>, naming the level above its buckets, which no real URL can.

Functions

build_listing

Build the home listing. A free function so it runs on a worker: it is home’s only filesystem access, and a wedged mount, FIFO or failing disk blocks here, so never call it from the drawing thread.

catalog_entry_for

The catalog entry path is or is inside, with its catalog’s label: the innermost, or the first listed of two at one place.

catalogs

The catalogs home shows, in order. The bundled catalog keeps only what this build can open (all its datasets are remote); a user’s catalog is shown whole. An empty catalog has no section.

cloud_account

The source ID and account of a cloud://<id>/<account> place: an Azure storage account, which has no URL of its own.

cloud_place

The place for one cloud source.

cloud_source_id

The source ID of a cloud://<id> place.

codebook_for

The column notes of the catalog dataset path is or is inside (the innermost), among datasets with notes.

complete_path

Complete a partly typed path against its directory: the longest unambiguous extension of typed and the candidate count. Reads a directory, so only on a worker.

describe

What entry is called. look is a bucket directory’s look-up state, drawn at spinner frame; known_sources are the sources a URL can name, None where the trail already names it.

desktop_recent_dirs

Directories holding data files the desktop recorded you opening (recently-used.xbel), so a fresh install has places to suggest. Only the directories are used, never the files: the list holds whatever was opened anywhere, which may be private.

directory_dataset_url

The URL that opens a cloud directory as one dataset: with a trailing slash, so it is scanned as a prefix rather than fetched as an object.

dirs_from_xbel

Extract directories of data files from XBEL content. Scans for href="file://…" rather than parsing XML, to avoid an XML dependency.

display_path

Abbreviate a path with ~ for display.

door_kind

Which DoorKind a door is. A directory the footers or headers turned down as one table is Directory with one format in its tally; the listing alone calls it MultiFile.

door_lands

Whether stepping into a directory puts the cursor on its door: only when the door opens the one dataset the directory’s own Enter opens; elsewhere it would start an unasked combined read.

door_name

The door’s name: the directory, and what Enter on it opens.

door_reads

What Enter on a door that is not one table reads and leaves out, for the details pane. Locally: the commonest format’s files directly inside, or a whole Parquet scan when there are subdirectories or no own files. In an object store: the prefix scanned whole in its commonest format.

expand_user_path

Expand ~ and $VAR in a path the user typed.

facts_for

The record to keep for a row that has just been measured.

fuzzy_positions

Character positions in haystack that needle matched, from the same alignment that scored it.

fuzzy_score

Whether and how well needle matches haystack (higher is better), through crate::home::fuzzy::best_match like every ranking and highlight.

holds_nothing_to_open

Whether a directory holds nothing a (all files) row could read. Shared by the door and the details pane so they agree.

index_key

The key a record about path is filed under in the dataset index. Opens record under the resolved URL (s3://bucket/x for s3://lab@bucket/x, one abfss:// spelling for Azure) while recents are stored as typed; both must meet here.

is_catalog_origin

Whether a section’s origin chip says it is a catalog.

is_cloud_place

Whether path is one of datui’s own cloud:// places rather than a real location.

is_network_path

Whether path is on a network filesystem by the mount table (longest matching mount point). False where the table is unavailable: a hint, never a gate. Uses crate::home::locality::Mounts::cached, since this is asked per row per frame.

is_object_store_url

Whether path is a place in an object store: s3://, gs://, or Azure.

is_remote_path

Whether reading path could block: an object-store or HTTP URL, or a network filesystem. Decides what home may touch on the UI thread; answered from the string and mount table alone.

list_typed_dir

A local directory typed at ~, for the prompt’s list; reads it, so runs on a worker. Nothing typed lists the working directory.

login_of

How a catalog URL is read, in words: what auth and connection say.

look_into_as

Find out what a row is, then what is in it, in one pass on the same filesystem (measure_row does nothing for a plain directory), reading files as the following open will: the command line passes the user’s reader settings; listing passes use the defaults, as a home open does.

look_into_batch

Look into a batch of rows on a worker, passing each answer to each and caching what was learned. Shared by the measure and classify passes. Every row is classified before any is measured: a kind is one directory read, a count up to sixty-four footers.

match_hit

match_score, with what the row is marked by when drawn.

match_score

How well an entry answers the filter, by name or by column (the footer’s column names: “which has a customer_id?”). A name match always outranks a column match. Higher is better, as in fzf; see crate::home::fuzzy.

matching_column

The first column of entry containing filter, case-insensitively. Substring, not subsequence: fuzzy matching dozens of names matches nearly everything.

measured_from

Fold a measured probe into the record kept for a row.

name_by_spec

Name files a format spec’s glob matches as data, under the spec’s name, from the name alone. Files a spec’s magic matches were named by the scan’s sniff.

names_a_file

Whether a remote path’s name says it is a file: a data extension or any dot in its last segment. A trailing slash is always a prefix.

names_under

The names one level below dir among urls: how s3://, gs:// and az:// complete, from what was listed, opened or cataloged. Nothing is asked of the store.

object_place_label

The service’s word for the top of an object-store place: a source, account, bucket or container. None otherwise, including directories inside a bucket, which are labeled by their contents like local ones.

parent_location

The location one level up from path, or None at the top. A URL’s top is its bucket or host (Path::parent would make gs://bucket into gs:).

place_is_browsable

Whether a place can be listed: a directory or an object-store prefix. An HTTP server has no listing, so a URL recent’s place is a heading, not a door.

place_of

The place a recent lives in: its directory or object-store prefix; a bare bucket or host is its own place.

substring_positions

Character positions of the first case-insensitive occurrence of needle, since column matching is a substring test.

typed_dir

The directory part of a typed path, through its last separator; for a URL at least its scheme (s3://), so buckets list under it.

typed_dir_is_url

Whether a typed directory is a URL, listed from what datui knows rather than read.