Deltalake
Native Delta Lake implementation in Rust
Usage
API
let table = open_table.await.unwrap;
println!;
CLI
)
Examples
The examples folder shows how to use Rust API to manipulate Delta tables.
Examples can be run using the cargo run --example
command. For example:
Optional cargo package features
azure
- enable the Azure storage backend to work with Delta Tables in Azure Data Lake Storage Gen2 accounts.datafusion
- enable thedatafusion::datasource::TableProvider
trait implementation for Delta Tables, allowing them to be queried using DataFusion.datafusion-ext
- DEPRECATED: alias fordatafusion
featuregcs
- enable the Google storage backend to work with Delta Tables in Google Cloud Storage.glue
- enable the Glue data catalog to work with Delta Tables with AWS Glue.hdfs
- enable the HDFS storage backend to work with Delta Tables in HDFS.json
- enable the JSON feature of theparquet
crate for better JSON interoperability.parquet2
- use parquet2 for checkpoint deserialization. Sincearrow
andparquet
features are enabled by default for backwards compatibility, this feature needs to be used with--no-default-features
.s3
- enable the S3 storage backend to work with Delta Tables in AWS S3. Uses rustls.s3-native-tls
- enable the S3 storage backend but rely on OpenSSL.
Development
To run s3 integration tests from local machine, we use docker-compose to stand
up AWS local stack. To spin up the test environment run docker-compose up
in
the root of the delta-rs
repo.