Skip to main content

Crate aqueducts_delta

Crate aqueducts_delta 

Source
Expand description

§Aqueducts Delta Lake Integration

This crate provides Delta Lake support for Aqueducts pipelines, enabling both reading from and writing to Delta tables with full transaction support and ACID guarantees.

§Features

  • DeltaSource: Read from Delta tables with optional time travel and filtering
  • DeltaDestination: Write to Delta tables with various modes (append, overwrite, upsert)
  • Object Store Integration: Automatic registration of object stores for cloud storage
  • Schema Evolution: Automatic schema merging and evolution support

§Usage

This crate is typically used through the main aqueducts meta-crate with the delta feature:

[dependencies]
aqueducts = { version = "0.10", features = ["delta"] }

The Delta integration is automatically registered when the feature is enabled. Configure Delta sources and destinations in your pipeline YAML/JSON/TOML files:

Modules§

handlers
Delta Lake object store handler registration.

Functions§

prepare_delta_destination
Prepare a Delta Lake destination for writing.
write_to_delta_destination
Write data to a Delta Lake destination.