lnmp-embedding 0.5.16

Vector embedding support for LNMP (LLM Native Minimal Protocol) with similarity calculations and delta encoding
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

# lnmp-embedding

Native vector embedding support for the LNMP protocol.

> **FID Registry:** Embedding FIDs are defined in the Standard range (512-767) in [`registry/fids.yaml`](../../registry/fids.yaml).

This crate provides the core data structures and logic for handling vector embeddings within LNMP, enabling efficient storage, transport, and processing of high-dimensional vectors for AI and ML applications.

## Features

- **Multiple Data Types**: Supports `F32`, `F16`, `I8`, `U8`, and `Binary` embeddings.
- **Similarity Metrics**: Built-in calculation for Cosine Similarity, Euclidean Distance, and Dot Product.
- ** efficient Serialization**: Optimized binary format for minimal overhead.

## Usage

```rust
use lnmp_embedding::{Vector, EmbeddingType, SimilarityMetric};

// Create a vector from f32 data
let v1 = Vector::from_f32(vec![1.0, 0.0, 0.0]);
let v2 = Vector::from_f32(vec![0.0, 1.0, 0.0]);

// Calculate similarity
let similarity = v1.similarity(&v2, SimilarityMetric::Cosine).unwrap();
assert_eq!(similarity, 0.0);
```

## Examples

This crate includes several examples in the `examples/` directory:

- **[delta_compute](./examples/delta_compute.rs)**: Computing embedding deltas
- **[batch_ops](./examples/batch_ops.rs)**: Batch vector operations (similarity, etc.)

Run examples with:
```bash
cargo run --example delta_compute -p lnmp-embedding
cargo run --example batch_ops -p lnmp-embedding
```

## Integration

This crate is designed to be used with `lnmp-core` and the LNMP SDKs. It is the underlying implementation for the `Embedding` mode (0x06) and `LnmpValue::Embedding` variant.