Module cloud 

Data that is not in a local file: object stores and their logins, HTTP, downloads and local copies.

Modules

aws_profiles

AWS profiles, read from the files the AWS CLI and SDKs share.

azure

Azure Blob Storage: logins, finding storage accounts, and listing what is in them.

cloud_browse

Finding the object stores this machine can already read, and listing their contents, so buckets can be browsed like local directories. Two rules:

cloud_command

Running the commands other tools provide for credentials: aws, a profile’s credential_process, gcloud, and az and PowerShell for Azure.

cloud_env

Cloud variables from [cloud] env_files, beside the real environment.

cloud_sources

Every object store datui can read, as a list of sources: a store plus the login reaching it (default S3 settings, a Google login, a [[cloud.connections]] entry). Each has an id that tells apart same-named buckets on different S3-compatible stores (s3://<id>@bucket/key); elsewhere URLs stay standard, usable in other tools. Discovery reads only environment variables and file existence; listing is cloud_browse’s, on a worker.

download

Remote files downloaded to local ones, and stdin spooled to one. Reading and writing are on separate threads (writing on the caller’s, never a runtime worker) with at most QUEUED_CHUNKS between them, so a slow disk backs the transfer up instead of memory. Store streams poll on the app’s runtime; blocking readers (HTTP, stdin) get their own thread.

gcloud

Google Cloud logins through gcloud, and the projects a login can see.

local_copy

A local copy of a remote dataset’s objects, for Data Quality’s full scan.

s3_tools

S3-compatible servers other tools already know about: the MinIO client’s aliases, MC_HOST_<alias>, and s3cmd’s config.

source

Input source detection for local paths vs remote URLs (S3, GCS, Azure, HTTP/HTTPS).

user_agent

The User-Agent header on every request datui makes: ureq’s, and the object stores’ (ours and the ones Polars builds for a scan).