rust_metrics 0.1.7

Incremental evaluation metrics for various machine learning pipelines.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

# rust_metrics

`rust_metrics` is an ML evaluation toolkit that brings [Torchmetrics](https://github.com/Lightning-AI/torchmetrics)-style metrics to Rust. Each metric implements the same incremental
`Metric` trait, so you can feed batched predictions over time and ask for the final score when ready.
Every metric shares the same Torch-inspired test cases and examples so that the
crate docs mirror the upstream behavior where the functionality matches.

Some benefits of `rust-metrics`:

- A standardized interface to increase reproducibility
- Reduces Boilerplate
- Rigorously tested
- Automatic accumulation over batches


## Getting started

Add the crate to your project:

```bash
cargo add rust_metrics
# or enable the BERT-based similarity metric
cargo add rust_metrics --features text-bert
```

## TorchMetrics-aligned examples

All snippets below (and method examples) reuse the exact inputs from the public TorchMetrics docs so you can cross-check the expected values.

### Classification

```rust
use rust_metrics::{BinaryAccuracy, BinaryAuroc, Metric};

let target = [0_usize, 1, 0, 1, 0, 1];
let preds = [0.11, 0.22, 0.84, 0.73, 0.33, 0.92];

let mut acc = BinaryAccuracy::default();
acc.update((&preds[..], &target[..])).unwrap();
assert!((acc.compute().unwrap() - 2.0 / 3.0).abs() < f64::EPSILON);

let mut auroc = BinaryAuroc::new(0); 
let auroc_scores = [0.0, 0.5, 0.7, 0.8];
let auroc_target = [0_usize, 1, 1, 0];
auroc.update((&auroc_scores, &auroc_target)).unwrap();
assert!((auroc.compute().unwrap() - 0.5).abs() < f64::EPSILON);
```

### Regression

```rust
use rust_metrics::{MeanSquaredError, MeanAbsoluteError, Metric};

let mut mse = MeanSquaredError::default();
mse.update((&[3.0, 5.0, 2.5, 7.0], &[2.5, 5.0, 4.0, 8.0])).unwrap();
assert!((mse.compute().unwrap() - 0.875).abs() < f64::EPSILON);

let mut mae = MeanAbsoluteError::default();
mae.update((&[2.5, 0.0, 2.0, 8.0], &[3.0, -0.5, 2.0, 7.0])).unwrap();
assert!((mae.compute().unwrap() - 0.5).abs() < f64::EPSILON);
```

### Clustering

```rust
use rust_metrics::{MutualInfoScore, Metric};

let preds = [2, 1, 0, 1, 0];
let target = [0, 2, 1, 1, 0];

let mut metric = MutualInfoScore::default();
metric.update((&preds, &target)).unwrap();
assert!((metric.compute().unwrap() - 0.500402423538188).abs() < f64::EPSILON);
```

### Text

```rust
use rust_metrics::{Bleu, EditDistance, Metric};

let preds = ["the cat is on the mat"];
let targets = ["a cat is on the mat"];

let mut bleu = Bleu::default();
bleu.update((&preds, &targets)).unwrap();
assert!(bleu.compute().unwrap() > 0.5); 

let mut edit = EditDistance::default();
edit.update((&["rain"], &["shine"])).unwrap();
assert_eq!(edit.compute(), Some(3.0));
```

For `SentenceEmbeddingSimilarity` enable the `text-bert` feature; it mirrors the `BERTScore` example sentences and
reports cosine similarities for each pair instead of precision/recall triples.

## Implemented metrics

### Classification

- `BinaryAccuracy`, `MulticlassAccuracy`
- `BinaryPrecision`, `BinaryRecall`, `MulticlassPrecision` 
- `BinaryF1Score`, `MulticlassF1Score`
- `BinaryHingeLoss`, `MulticlassHingeLoss`
- `BinaryJaccardIndex`, `MulticlassJaccardIndex`
- `BinaryConfusionMatrix`
- `BinaryAuroc`

### Regression

- `MeanSquaredError`
- `NormalizedRootMeanSquaredError`
- `MeanAbsoluteError`
- `MeanAbsolutePercentageError`
- `R2Score`

### Clustering

- `MutualInfoScore`

### Text

- `Bleu` with optional smoothing and arbitrary n-gram depth
- `EditDistance` with sum or mean reduction
- `RougeScore` 
- `SentenceEmbeddingSimilarity` (requires the `text-bert` feature) backed by [`fastembed`]. This
  metric embeds each sentence pair with lightweight BERT embeddings and reports cosine similarity
  scores.

[`fastembed`]: https://crates.io/crates/fastembed

## Feature flags

| Feature    | Default | Description                                                  |
| --------- | ------- | ------------------------------------------------------------ |
| `text-bert` | no    | Enables BERT sentence embedding similarity via `fastembed`. |