Module aggregation 

Turning many raters’ labels into one answer: majority, plurality and weighted votes (exact), the Dawid–Skene EM model of rater confusion, Bradley–Terry strengths from pairwise comparisons, and the worker quality helpers that go with them (accuracy against gold labels, per-category precision / recall / F₁, confidence intervals for a proportion, gold-question screening).

Labels are category indices 0..n_categories; a LabelTable holds them items × raters with None for a missing label. Votes and accuracies are exact (Q); the two iterative estimators (dawid_skene, bradley_terry) and the confidence intervals are f64.

use symplex::stats::aggregation::{majority_vote, worker_accuracy};
use symplex::linprog::q;

let v = majority_vote(&[Some(2), Some(0), Some(2), None, Some(1)]);
assert_eq!((v.winner, v.counts), (Some(2), vec![1, 1, 2]));
let acc = worker_accuracy(&[Some(2), Some(0), Some(2), None, Some(1)], &[2, 0, 1, 1, 1])?;
assert_eq!((acc.correct, acc.answered, acc.accuracy), (3, 4, Some(q(3, 4))));

Structs

Accuracy

A worker’s accuracy against gold labels.

BradleyTerry

Bradley–Terry strengths.

BradleyTerryOpts

Options of bradley_terry.

CategoryMetrics

Precision, recall and F₁ of one category, over the answered items.

DawidSkene

The Dawid–Skene estimates.

DawidSkeneOpts

Options of dawid_skene.

LabelTable

Labels of n_items items by n_raters raters (rows are items), each a category index below n_categories or None when missing.

PairwiseOutcome

One pairwise comparison: player winner beat player loser (both indices below the number of players).

Vote

The outcome of a vote over category indices.

WeightedVote

The outcome of a weighted vote.

Enums

DawidSkeneInit

How the Dawid–Skene EM iteration is started.

IntervalMethod

Confidence-interval methods for a binomial proportion (statsmodels.stats.proportion.proportion_confint(method=…)).

Functions

bradley_terry

Maximum-likelihood Bradley–Terry strengths from a wins matrix (wins[i][j] = times i beat j) by the MM algorithm of Hunter (2004, Ann. Statist. 32, 384–406):

category_metrics

Per-category precision, recall and F₁ of a worker against gold labels, over the items the worker answered.

dawid_skene

The Dawid–Skene EM estimate of true classes, rater confusion matrices and class prevalences (Dawid & Skene 1979, Applied Statistics 28, 20–28). Iterates, from the DawidSkeneInit posteriors T_ij:

dawid_skene_counts

The Dawid–Skene model from items × raters × categories counts n_ikl (how many times rater k gave item i the label l; the general form of Dawid & Skene 1979 where a rater may label an item several times). See dawid_skene.

gold_screening

Pass / fail of every rater on the gold items: Some(true) when the rater’s accuracy on the gold items answered is at least threshold, None for a rater who answered no gold item. gold[i] is the gold label of item i or None for a non-gold item.

majority_vote

The category chosen by most raters (missing labels are ignored; the count vector runs to the largest label seen).

majority_votes

The majority vote of every item of a LabelTable (counts have length n_categories).

plurality

A plurality vote with a quorum: the top category wins only if it has at least threshold of the votes cast (threshold in [0, 1]).

proportion_interval

A two-sided confidence interval for the success probability behind successes out of trials, at level confidence (e.g. 0.95). With p̂ = k/n, α = 1 − confidence and z = Φ⁻¹(1 − α/2):

weighted_vote

A vote where rater k’s label counts weights[k] (non-negative, exact).

wins_matrix

A wins matrix (wins[i][j] = times i beat j) from a list of PairwiseOutcomes among n players — the input of bradley_terry.

worker_accuracy

A worker’s accuracy on items with gold labels (missing labels are not counted as answered).