valentinus 0.1.0

Proof-of-concept vector db built on top of LMDB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

[![.github/workflows/rust.yml](https://github.com/kn0sys/valentinus/actions/workflows/rust.yml/badge.svg)](https://github.com/kn0sys/valentinus/actions/workflows/rust.yml)

# valentinus
vector db built on top of lmdb

# dependencies

* lmdb-rs        - database bindings
* linfa          - machine learning
* ndarray        - numpy equivalent
* rust-bert      - embeddings encoding
* libtorch 2.1.1 - required by rust-bert
* see [rust-bert/tch-rs](https://github.com/guillaume-be/rust-bert)

# getting started

```bash
git clone --recursive https://github.com/kn0sys/valentinus
```

```bash
git submodule --init --update
```

```bash
export LIBTORCH="/home/user/libtorch"
export LD_LIBRARY_PATH="$LIBTORCH/lib"
export LIBTORCH_BYPASS_VERSION_CHECK=1
```

# examples

see [examples](./examples/embeddings.md)

# reference

[inspired by this chromadb python tutorial](https://realpython.com/chromadb-vector-database/#what-is-a-vector-database)