lake-pulse 0.3.0

A Rust library for analyzing data lake table health across multiple formats and storage providers.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

# Examples

This is a folder where some examples are provided to understand the use of the library.

## Prerequisites

Some tables need to be created, some are easy to create, some are not. The `data/README.md` file
contains instructions on how to create the tables.

For Azure ADLS, AWS S3, and HDFS storage examples you will need to provide the needed credentials or access to the respective storage systems.

## Run the Examples

To run the examples, you need to have `cargo` installed. Then, you can run the examples as follows:

```bash
cargo run --example <example_name>
```

For example, to run the `local_store.rs` example, you can run:

```bash
cargo run --example local_store
```

## Feature-Gated Examples

Some examples require specific feature flags to be enabled:

| Example | Required Feature |
|---------|------------------|
| `local_store_hudi` | `hudi` |
| `local_store_lance` | `lance` |
| `local_store_paimon` | `paimon` |

To run these examples, use the `--features` flag:

```bash
cargo run --features hudi --example local_store_hudi
cargo run --features lance --example local_store_lance
cargo run --features paimon --example local_store_paimon

# Or use the all feature to enable all table formats:
cargo run --features all --example local_store_hudi
```