Skip to main content

Module evaluation

Module evaluation 

Source
Expand description

Evaluation metrics and infrastructure for ANN benchmarks.

Standard metrics used in ANN literature:

MetricFormulaInterpretation
Recall@K|approx ∩ true| / KFraction of true neighbors found
Precision@K|approx ∩ true| / |approx|Same as recall when |approx| = K
MRR1 / rank_of_first_trueReciprocal of first relevant result
QPSqueries / secondsThroughput

§Standard Benchmark Datasets

DatasetSizeDimDistanceSource
SIFT-1M1M128L2INRIA Texmex
GIST-1M1M960L2INRIA Texmex
GloVe-1001.2M100AngularStanford NLP
Fashion-MNIST60K784L2Zalando
Deep1B1B96AngularYandex

Reference: https://ann-benchmarks.com/

Re-exports§

pub use crate::distance::angular_distance;
pub use crate::distance::cosine_distance;
pub use crate::distance::inner_product_distance;
pub use crate::distance::l2_distance;
pub use crate::distance::normalize;
pub use crate::distance::DistanceMetric;

Structs§

EvalDataset
A dataset with ground truth for evaluation.
EvalResults
Evaluation results for a single run.

Functions§

compute_ground_truth
Compute ground truth for a dataset.
evaluate
Evaluate an algorithm on a dataset.
generate_clustered_dataset
Generate a clustered dataset (more realistic).
generate_normalized_clustered_dataset
Generate a normalized dataset (for cosine/angular metrics).
generate_uniform_dataset
Generate a synthetic dataset with uniform random vectors.
mrr
Compute mean reciprocal rank for a single query.
recall_at_k
Compute recall@k for a single query.