Crate deltalake[−][src]

Native Delta Lake implementation in Rust

Usage

Load a Delta Table by path:

async {
  let table = deltalake::open_table("./tests/data/simple_table").await.unwrap();
  let files = table.get_files();
};

Load a specific version of Delta Table by path then filter files by partitions:

async {
  let table = deltalake::open_table_with_version("./tests/data/simple_table", 0).await.unwrap();
  let files = table.get_files_by_partitions(&[deltalake::PartitionFilter {
      key: "month",
      value: deltalake::PartitionValue::Equal("12"),
  }]);
};

Load a specific version of Delta Table by path and datetime:

async {
  let table = deltalake::open_table_with_ds(
      "./tests/data/simple_table",
      "2020-05-02T23:47:31-07:00",
  ).await.unwrap();
  let files = table.get_files();
};

Optional cargo package features

s3 - enable the S3 storage backend to work with Delta Tables in AWS S3.
azure - enable the Azure storage backend to work with Delta Tables in Azure Data Lake Storage Gen2 accounts.
datafusion-ext - enable the datafusion::datasource::TableProvider trait implementation for Delta Tables, allowing them to be queried using DataFusion.

Re-exports

`pub use self::partitions::*;`
`pub use self::storage::get_backend_for_uri;`
`pub use self::storage::parse_uri;`
`pub use self::storage::StorageBackend;`
`pub use self::storage::StorageError;`
`pub use self::storage::Uri;`
`pub use self::storage::UriError;`

Modules

action	Actions included in Delta table transaction logs
delta_arrow	Conversion between Delta Table schema and Arrow schema
partitions	Delta Table partition handling logic.
storage	Object storage backend abstraction layer for Delta Table transaction logs and data

Structs

CheckPoint	Metadata for a checkpoint file
DeltaTable	In memory representation of a Delta Table
DeltaTableMetaData	Delta table metadata
DeltaTransaction	Object representing a delta transaction
DeltaTransactionOptions	Options for customizing behavior of a `DeltaTransaction`
Schema	Represents the schema of the delta table.
SchemaField	Describes a specific field of the Delta table schema.
SchemaTypeArray	Schema definition for array type fields.
SchemaTypeMap	Schema definition for map type fields.
SchemaTypeStruct	Represents a struct field defined in the Delta table schema.

Enums

ApplyLogError	Error related to Delta log application
DeltaTableError	Delta Table specific error
DeltaTransactionError	Error returned by the DeltaTransaction struct
LoadCheckpointError	Error related to checkpoint loading
SchemaDataType	Enum with variants for each top level schema data type.
TransactionCommitAttemptError	Error that occurs when a single transaction commit attempt fails

Functions

crate_version	Returns rust create version, can be use used in language bindings to expose Rust core version
open_table	Creates and loads a DeltaTable from the given path with current metadata. Infers the storage backend to use from the scheme in the given table path.
open_table_with_ds	Creates a DeltaTable from the given path. Loads metadata from the version appropriate based on the given ISO-8601/RFC-3339 timestamp. Infers the storage backend to use from the scheme in the given table path.
open_table_with_version	Creates a DeltaTable from the given path and loads it with the metadata from the given version. Infers the storage backend to use from the scheme in the given table path.

Type Definitions

DeltaDataTypeInt	Type alias for i32/Delta int
DeltaDataTypeLong	Type alias for i64/Delta long
DeltaDataTypeTimestamp	Type alias representing the expected type (i64/ms since Unix epoch) of a Delta timestamp.
DeltaDataTypeVersion	Type alias representing the expected type (i64) of a Delta table version.
Guid	Type alias for a string expected to match a GUID/UUID format

Crate deltalake⎘[−][src]

Crate deltalake[−][src]