arrow-udf-wasm 0.2.1

WebAssembly runtime for Arrow UDFs.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

# WebAssembly UDF for Apache Arrow

[![Crate](https://img.shields.io/crates/v/arrow-udf-wasm.svg)](https://crates.io/crates/arrow-udf-wasm)
[![Docs](https://docs.rs/arrow-udf-wasm/badge.svg)](https://docs.rs/arrow-udf-wasm)

For untrusted user-defined functions, you can compile them into WebAssembly and run them in a sandboxed environment.

## Build UDF in WebAssembly

Create a project and add the following lines to your `Cargo.toml`:

```toml
[dependencies]
arrow-udf = "0.1"
```

Define your functions with the `#[function]` macro:

```rust,ignore
use arrow_udf::function;

#[function("gcd(int, int) -> int")]
fn gcd(mut a: i32, mut b: i32) -> i32 {
    while b != 0 {
        (a, b) = (b, a % b);
    }
    a
}
```

Then compile the project into WebAssembly:

```sh
cargo build --release --target wasm32-wasi
```

You can find the generated WebAssembly module in `target/wasm32-wasi/release/*.wasm`.

## Run UDF in WebAssembly

Add the following lines to your `Cargo.toml`:

```toml
[dependencies]
arrow-udf-wasm = "0.1"
```

You can then load the WebAssembly module and call the functions:

```rust,ignore
use arrow_udf_wasm::Runtime;

// load the WebAssembly module
let binary = std::fs::read("udf.wasm").unwrap();
// create a runtime from the module
let runtime = Runtime::new(&binary).unwrap();
// list available functions in the module:
for name in runtime.functions() {
    println!("{}", name);
}
// call the function with a RecordBatch
let input: RecordBatch = ...;
let output = runtime.call("gcd(int4,int4)->int4", &input).unwrap();
```

The WebAssembly runtime is powered by [wasmtime](https://wasmtime.dev/). 
Notice that each WebAssembly instance can only run single-threaded, we maintain an instance pool internally to support parallel calls from multiple threads.

See the [example](./examples/wasm.rs) for more details. To run the example:

```sh
cargo build --release -p arrow-udf-example --target wasm32-wasi
cargo run --example wasm -- target/wasm32-wasi/release/arrow_udf_example.wasm
```

## Build WASM UDF at Runtime

Enable the `build` feature to build the wasm binary from source:

```toml
[dependencies]
arrow-udf-wasm = { version = "0.1", features = ["build"] }
```

You can then build the WebAssembly module at runtime:

```rust,ignore
let manifest = r#"
[dependencies]
chrono = "0.4"
"#;

let script = r#"
use arrow_udf::function;

#[function("gcd(int, int) -> int")]
fn gcd(mut a: i32, mut b: i32) -> i32 {
    while b != 0 {
        (a, b) = (b, a % b);
    }
    a
}
"#;
let binary = arrow_udf_wasm::build::build(manifest, script).unwrap();
```

See the [`build`] module for more details.