Expand description
Balanced few-shot selection and the ordered selected-ID manifest model.
Implemented by plan 02-05. Balanced few-shot selection and the ordered selected-ID manifest model.
Partial Fisher-Yates over sorted per-class buckets, with the swap index for step i drawn at ordinal i in that class’s domain. The output order IS the draw order, which is what makes the ordered manifest statable as a contract equation rather than as an implementation detail.
§One dataset value in, one selection out
[FewShotSelector::select] takes a single &PreparedDataset<Canonical>. That one
argument supplies the training pool, the validation witness, the exclusion record and
the fingerprint, so there is no signature into which a caller could feed a training
split from one dataset and a validation split from another. Witness mixing is not
rejected here; it is unrepresentable. A compatibility dataset is a different type and
cannot be passed at all (D-19).
§Every selected row carries its LABEL
[SelectedExample] holds (id, label, exact_hash, normalized_hash) and [SelectedId]
is an opaque ordinal into one [Selection]. Downstream pair construction therefore
derives its targets from real per-row labels rather than from class-size totals, which
would silently mislabel any layout the totals happen to be symmetric in.
§Determinism, and what it does and does not survive
The ordered selection is a pure function of (post-exclusion sorted pools, root_seed, shots_per_class). It survives thread count, iteration order and permuted ingest order,
because the buckets sort before any draw and draw i is a pure function of i.
The semantic_hash deliberately does NOT survive permuted ingest order, and that is
correct rather than a gap: the payload embeds dataset_fingerprint, which is partly a
digest of the split’s canonical JSONL bytes in ingest order. A permuted file is
different bytes and therefore a different dataset for provenance purposes. The
permutation test below asserts both halves — same selection, different fingerprint —
so neither can regress unnoticed.
Structs§
- FewShot
Selector - Balanced few-shot selection over one canonical prepared dataset.
- Selected
Example - One selected row, with its LABEL and both content hashes.
- Selected
Id - An opaque ordinal into ONE
Selection. - Selection
- A completed few-shot selection.
- Selection
Config - What a caller asks a selection for.
Constants§
- SELECTION_
ALGORITHM_ VERSION - The selection-algorithm version. A change here changes selected IDENTITIES, which is why it is versioned separately from the manifest schema.