skardi 0.1.1

High performance query engine for both offline compute and online serving
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101

# skardi

High-performance query engine for AI and agents, powered by [Apache DataFusion](https://datafusion.apache.org/).

Query files, databases, data lakes, and vector stores with SQL — and serve results as parameterized pipelines.

[![Crates.io](https://img.shields.io/crates/v/skardi.svg)](https://crates.io/crates/skardi)
[![Docs.rs](https://docs.rs/skardi/badge.svg)](https://docs.rs/skardi)
[![License](https://img.shields.io/crates/l/skardi.svg)](https://github.com/SkardiLabs/skardi/blob/main/LICENSE)

## Modules

| Module | Description |
|--------|-------------|
| `engine` | SQL query execution engine backed by DataFusion |
| `pipeline` | Declarative SQL pipelines with parameter inference |
| `model` | ONNX model loading and inference (behind `onnx` feature) |
| `sources` | Data source connectors: CSV, Parquet, PostgreSQL, MySQL, SQLite, MongoDB, Redis, Iceberg, Lance |

## Installation

```toml
[dependencies]
skardi = "0.1"

# With ONNX model inference
skardi = { version = "0.1", features = ["onnx"] }
```

## Quick Start

### Execute a SQL query

```rust,no_run
use skardi::engine::datafusion::DataFusionEngine;
use skardi::engine::Engine;
use datafusion::prelude::*;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let ctx = SessionContext::new();
    ctx.register_csv("products", "data/products.csv", CsvReadOptions::new()).await?;

    let engine = DataFusionEngine::new(ctx);
    let result = engine.execute("SELECT * FROM products WHERE price < 100").await?;

    println!("{} rows returned", result.num_rows());
    Ok(())
}
```

### Load and run a pipeline

```rust,no_run
use skardi::pipeline::pipeline::{Pipeline, StandardPipeline};
use datafusion::prelude::*;
use std::sync::Arc;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let ctx = Arc::new(SessionContext::new());

    // Pipeline YAML defines a parameterized SQL query
    let pipeline = StandardPipeline::load_from_file("pipeline.yaml", ctx).await?;

    println!("Pipeline: {} v{}", pipeline.name(), pipeline.version());
    Ok(())
}
```

A pipeline YAML looks like this:

```yaml
metadata:
  name: product-search
  version: 1.0.0

query: |
  SELECT name, price FROM products
  WHERE ({brand} IS NULL OR brand = {brand})
    AND price < {max_price}
  LIMIT {limit}
```

Parameters are automatically inferred from `{placeholders}` in the SQL.

## Feature Flags

| Feature | Description |
|---------|-------------|
| `onnx` | Enables ONNX model inference via the `model` module |

## Further Reading

- [API Reference](https://docs.rs/skardi) — full rustdoc for all modules
- [GitHub Repository](https://github.com/SkardiLabs/skardi) — server, CLI, demos, and deployment guide
- [Crate Page](https://crates.io/crates/skardi)

## License

Apache-2.0