datafusion-loki 0.1.0

A DataFusion table provider for querying Loki data
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

# datafusion-loki

![License](https://img.shields.io/badge/license-MIT-blue.svg)
[![Crates.io](https://img.shields.io/crates/v/datafusion-loki.svg)](https://crates.io/crates/datafusion-loki)
[![Docs](https://docs.rs/datafusion-loki/badge.svg)](https://docs.rs/datafusion-loki/latest/datafusion_loki/)

A Datafusion table provider for querying Loki data.

## Features

1. Use SQL to query Loki logs
2. Insert logs into Loki
3. Support pushing down filters and limit to Loki
4. Execution plan can be serialized for distributed execution

## Usage

```rust
#[tokio::main]

pub async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let loki_table = LokiLogTable::try_new("http://localhost:33100")?;

    let ctx = SessionContext::new();
    ctx.register_table("loki", Arc::new(loki_table))?;
    ctx.register_udf(ScalarUDF::new_from_impl(MapGet::new()));

    ctx.sql(
        "insert into loki values (current_timestamp(), Map{'app': 'my-app'}, 'user login failed')",
    )
    .await?
    .show()
    .await?;

    ctx.sql(
        r#"
select * 
from loki 
where map_get(labels, 'app') = 'my-app' 
    and timestamp > '2025-11-12T00:00:00Z' 
    and line like '%login%' 
limit 2
"#,
    )
    .await?
    .show()
    .await?;

    Ok(())
}
```
Output
```
+-------+
| count |
+-------+
| 1     |
+-------+

+--------------------------------+--------------------------------------------------------------+-------------------+
| timestamp                      | labels                                                       | line              |
+--------------------------------+--------------------------------------------------------------+-------------------+
| 2025-11-17T09:34:36.418570400Z | {app: my-app, detected_level: unknown, service_name: my-app} | user login failed |
+--------------------------------+--------------------------------------------------------------+-------------------+
```