DataFusion Delta Sharing

This create provides the integration between Delta Sharing and DataFusion.

Delta Sharing is an open protocol for securely sharing large datasets. It is a REST protocol that securely provides access to a part of a cloud dataset stored on popular cloud storage systems like S3, ADLS, and GCS.

Installation

The crate can be installed using cargo:

cargo add datafusion-delta-sharing

Example

An example of the integration is shown below:

use std::sync::Arc;
use datafusion::prelude::*;

use datafusion_delta_sharing::DeltaSharingTable;

let ctx = SessionContext::new();
let table = DeltaSharingTable::try_from_str("./path/to/profile.share#share.schema.table").await?;

ctx.register_table("demo", Arc::new(table))?;
ctx.sql("select * from demo").await?.show().await?;