Expand description
object_store integration for Unity Catalog.
This crate adapts Unity Catalog’s credential-vending APIs to the
object_store trait, so any framework
that accepts an Arc<dyn ObjectStore> (DataFusion, delta_kernel,
parquet, …) can read and write data governed by Unity Catalog
volumes, tables, or external locations with no extra glue.
§Quickstart
use unitycatalog_object_store::{Operation, UnityObjectStoreFactory};
let factory = UnityObjectStoreFactory::builder()
.with_uri("https://my-workspace.cloud.databricks.com/api/2.1/unity-catalog/")
.with_token(std::env::var("DATABRICKS_TOKEN").unwrap())
.build()
.await?;
// Address a UC securable directly with a `uc://` URL …
let store = factory
.for_url("uc:///Volumes/main/default/landing/raw/", Operation::Read)
.await?;
// … and use it like any other `object_store`.
let listing = futures::StreamExt::collect::<Vec<_>>(store.as_dyn().list(None)).await;§URL scheme
See UCReference for the full grammar. In short:
uc:///Volumes/<catalog>/<schema>/<volume>[/<path>]uc:///Tables/<catalog>/<schema>/<table>s3://,gs://,abfss://,r2://, … — raw cloud URLs, vended viatemporary-path-credentials.
The kind segment is case-insensitive (so /Volumes/, /volumes/,
and /VOLUMES/ all work); the capitalised form is canonical because it
mirrors the Databricks workspace POSIX path convention. The Databricks
vol+dbfs:/Volumes/... alias is also accepted.
Structs§
- UCStore
- A configured Unity Catalog
ObjectStoreready for use. - Unity
Object Store Factory - Factory that mints
object_storeinstances backed by Unity Catalog credential vending. - Unity
Object Store Factory Builder - Builder for
UnityObjectStoreFactory.
Enums§
- Error
- Operation
- Unified read/write operation used by
UnityObjectStoreFactory::for_url. - Path
Operation - The kind of access requested for a storage path.
- Table
Operation - The kind of access requested for a table.
- Table
Reference - A reference to a table in unity catalog.
- UCReference
- A parsed reference to a Unity Catalog securable.
- Volume
Operation - The kind of access requested for a volume.
- Volume
Reference - A reference to a volume in unity catalog.
Functions§
- store_
from_ vended_ credential - Build a
UCStorefrom a credential that has already been vended out-of-band, using a static credential provider (no Unity Catalog client, no auto-refresh).