Skip to main content

Crate quarb_objstore

Crate quarb_objstore 

Source
Expand description

Object-store adapter for the Quarb query engine: Google Cloud Storage, Amazon S3, and Azure Blob Storage as lazy directory trees.

Object keys with / separators span the tree the way a filesystem does — the adapter lists one “directory” per touch (delimiter listing, paginated), and an object’s content is its value, fetched on first read and cached. Under composition (qua wraps object stores by default), a bucket of JSON, CSV, or source files is directly queryable: the object is a leaf, its parsed content the subtree — grafting is the point of this adapter.

Targets:

  • gs://BUCKET[/PREFIX] — GCS, JSON API. Public buckets work anonymously; private ones authenticate like the other GCP drivers (QUARB_GCP_TOKEN, else gcloud auth print-access-token, ?account=EMAIL to pick the account — set ?auth=1 to force a token for non-public buckets).
  • s3://BUCKET[/PREFIX][?region=R][&endpoint=URL][&anon=1] — S3, ListObjectsV2. Requests are SigV4-signed whenever the standard credential chain resolves (env keys, then ~/.aws/credentials; see quarb-aws), so private buckets just work; without credentials — or with anon=1 — the request goes out unsigned, which public buckets accept. endpoint=URL points at any S3-compatible store (MinIO, Cloudflare R2, …) using path-style addressing.
  • az://ACCOUNT/CONTAINER[/PREFIX][?endpoint=URL][&sas=TOKEN] — Azure Blob Storage. Public containers read anonymously; a sas= token rides every request; otherwise, when AZURE_STORAGE_KEY holds the account key, requests carry a SharedKey signature. endpoint=URL points at Azurite or any compatible endpoint (path-style, account in the path).

Metadata: ;;;size, ;;;updated on objects; traits <object> / <prefix>. Read-only, as always.

Structs§

ObjstoreAdapter
A bucket (or prefix of one), exposed as an arbor.

Enums§

ObjstoreError
An error connecting to a bucket.