Deltalake
Native Delta Lake implementation in Rust
Usage
API
let table = open_table.await.unwrap;
println!;
CLI
)
Examples
The examples folder shows how to use Rust API to manipulate Delta tables.
Examples can be run using the cargo run --example
command. For example:
Optional cargo package features
s3
- enable the S3 storage backend to work with Delta Tables in AWS S3.s3-rustls
- enable the S3 storage backend but rely on rustls rather than OpenSSL (native-tls
).glue
- enable the Glue data catalog to work with Delta Tables with AWS Glue.azure
- enable the Azure storage backend to work with Delta Tables in Azure Data Lake Storage Gen2 accounts.gcs
- enable the Google storage backend to work with Delta Tables in Google Cloud Storage.datafusion-ext
- enable thedatafusion::datasource::TableProvider
trait implementation for Delta Tables, allowing them to be queried using DataFusion.parquet2
- use parquet2 for checkpoint deserialization. Sincearrow
andparquet
features are enabled by default for backwards compatibility, this feature needs to be used with--no-default-features
.
Development
To run s3 integration tests from local machine, we use docker-compose to stand
up AWS local stack. To spin up the test environment run docker-compose up
in
the root of the delta-rs
repo.