Skip to main content

Crate unitycatalog_object_store

Crate unitycatalog_object_store 

Source
Expand description

object_store integration for Unity Catalog.

This crate adapts Unity Catalog’s credential-vending APIs to the object_store trait, so any framework that accepts an Arc<dyn ObjectStore> (DataFusion, delta_kernel, parquet, …) can read and write data governed by Unity Catalog volumes, tables, or external locations with no extra glue.

§Quickstart

use unitycatalog_object_store::{Operation, UnityObjectStoreFactory};

let factory = UnityObjectStoreFactory::builder()
    .with_uri("https://my-workspace.cloud.databricks.com/api/2.1/unity-catalog/")
    .with_token(std::env::var("DATABRICKS_TOKEN").unwrap())
    .build()
    .await?;

// Address a UC securable directly with a `uc://` URL …
let store = factory
    .for_url("uc:///Volumes/main/default/landing/raw/", Operation::Read)
    .await?;

// … and use it like any other `object_store`.
let listing = futures::StreamExt::collect::<Vec<_>>(store.as_dyn().list(None)).await;

§URL scheme

See UCReference for the full grammar. In short:

  • uc:///Volumes/<catalog>/<schema>/<volume>[/<path>]
  • uc:///Tables/<catalog>/<schema>/<table>
  • s3://, gs://, abfss://, r2://, … — raw cloud URLs, vended via temporary-path-credentials.

The kind segment is case-insensitive (so /Volumes/, /volumes/, and /VOLUMES/ all work); the capitalised form is canonical because it mirrors the Databricks workspace POSIX path convention. The Databricks vol+dbfs:/Volumes/... alias is also accepted.

Structs§

UCStore
A configured Unity Catalog ObjectStore ready for use.
UnityObjectStoreFactory
Factory that mints object_store instances backed by Unity Catalog credential vending.
UnityObjectStoreFactoryBuilder
Builder for UnityObjectStoreFactory.

Enums§

Error
Operation
Unified read/write operation used by UnityObjectStoreFactory::for_url.
PathOperation
The kind of access requested for a storage path.
TableOperation
The kind of access requested for a table.
TableReference
A reference to a table in unity catalog.
UCReference
A parsed reference to a Unity Catalog securable.
VolumeOperation
The kind of access requested for a volume.
VolumeReference
A reference to a volume in unity catalog.

Functions§

store_from_vended_credential
Build a UCStore from a credential that has already been vended out-of-band, using a static credential provider (no Unity Catalog client, no auto-refresh).