Module cloud_browse 

Finding the object stores this machine can already read, and listing their contents, so buckets can be browsed like local directories. Two rules:

Never list a provider datui cannot then read. Discovery looks only at the credentials object_store will use to open a file (not, say, gcloud’s own), so every listed bucket opens.

Nothing here runs on the drawing thread. Every network function is async and driven from a worker, as remote filesystem roots are.

Structs

Environment

Everything discovery may look at, in one place so a test can supply it; Environment::current is the real one.

InstanceIdentity

Which clouds’ VM or platform identity may be used. Finding one queries a metadata service that can hang, so only when configured or signaled by the platform (K_SERVICE on Cloud Run/Functions; IDENTITY_ENDPOINT or MSI_ENDPOINT on Azure App Service, Functions, Container Apps). ECS and EKS are found by their own variables.

Level

What one level of a place listed.

Listed

A source’s first level as home lists it: buckets for S3 and Google, storage accounts for Azure.

Provider

An object store datui believes it can read, and why it believes that.

Watch

How a listing is watched while it runs.

Constants

MAX_LEVEL_ROWS

The most rows one bucket level lists, as for a local directory; past it the listing stops and says so (141,000 partitions would be 141 requests held in memory).

Traits

Store

An object store that also lists a page at a time: every store datui builds is both.

Functions

adc_path

The application default credentials file where object_store reads it (%APPDATA%\gcloud\ on Windows, $HOME/.config/gcloud/ elsewhere), kept in step so discovery agrees with opening.

detect

Object stores this machine can read, in display order. Empty is normal without cloud credentials; nothing prompts, installs or logs in.

gcs_next_page_token

The pageToken for the next page of a bucket list, when the response has one.

instance_identity

is_marker

A key that is not data to open: job receipts and folder markers. Narrower than crate::home::discover::is_bookkeeping (which decides a directory’s kind): a leading _ is not enough here, since _manifest.parquet may be worth opening.

is_refusal

Whether an error is the service refusing the request, rather than failing to answer.

list_account

The containers of one account in an Azure source, as rows to step into.

list_buckets

Every bucket the provider’s credentials can see. Per provider, since object_store is bucket-scoped; both borrow its credential handling (no new crypto), so listing and opening use the same credentials.

list_first_level

Everything at the top of a source.

list_objects

One level of a bucket or prefix as home rows, up to MAX_LEVEL_ROWS. A delimited listing: a million objects under a hundred prefixes is one request, a hundred rows.

list_objects_watched

list_objects a page at a time: watch sees rows as they come and can stop between pages.

look_at_listing

The kind and holdings of a listing by crate::home::discover::classify, the local rule; a prefix’s row label comes from the holdings.

narrowing_prefix

The server-side prefix a home filter can ask a cut-short level for, if any. The filter is fuzzy and the prefix literal, so it asks for the names’ shared part up to their last separator (STATION=, year=) plus the filter, in the names’ case; a filter already spelling the shared part is taken as typed.

object_path

The object_store path for a key as stored. Path::from percent-encodes %, so 100%.csv.gz became 100%25.csv.gz and 404’d; existing keys are parsed as is, unless they cannot be a path.

parse_gcs_buckets

Bucket names from a GCS storage/v1/b response. Tolerant: no items means no buckets, and an entry without a usable name is skipped.

parse_s3_buckets

Bucket names from an S3 ListBuckets response. A custom endpoint may be hostile, so quick-xml parses it with a depth cap no legitimate response reaches.

peek_kind

What a cloud directory holds, from the first page of a delimited listing: Hive for key=value children, MultiFile for Parquet files, else Directory. One request; no object is read.

probe_unsigned

Whether resolved’s place reads unsigned: Some(true) if an unsigned request succeeds, Some(false) if refused, None otherwise (no network, missing object, custom endpoint). One request: a HEAD, or a one-key listing.

s3_bucket_region

Where S3 keeps bucket, from the x-amz-bucket-region header sent unauthenticated whatever the status; once per bucket per session, None if unsaid.

s3_builder

The one S3 builder, so listing and opening authenticate identically (separate builders once let the listing drop the configured endpoint and keys). settings are one source’s (cloud_sources::resolve). Addressed by bucket name: only s3://bucket/key URLs.

split_bucket_url

Split a gs:// or s3:// URL into bucket and prefix (no leading or trailing slash, empty for the root, as object_store wants). A source id (s3://<id>@bucket) is dropped.

store

The store for a resolved place, signed as the resolver decided, and the key inside it.

unreadable_google_login

The credential type of a Google login object_store cannot read (workload identity federation, an impersonated service account), when the environment or ADC file has one.

Type Aliases

Progress

What a listing hands each page of rows as it comes.