Module eval 

eval and gate (spec 006, 3.9), and the dataset and bundle handling calibrate fit shares, for every host.

Structs

EvalArgs

What eval reads and writes.

JudgeArgs

What eval judge reads and writes (spec 041, 3.10.4).

Constants

MAX_SCORES

The most --scores documents one eval reads (spec 041, 3.10.2).

OUTPUTS

VIEWS

The score views eval adds to out when any scorer is declared.

Functions

check_case_ids

Case ids that map to bundle files (spec 006, 3.9.1): 1 to 128 bytes of [A-Za-z0-9._-] starting with an alphanumeric, distinct ignoring ASCII case.

eval

eval: a report, its detail, the configuration and the adapter, published in out.

eval_scored

eval with the scores documents of a version 3 configuration’s scorers (spec 041, 3.10.2), and its score views added to out when any scorer is declared. With no scores and another version this is eval, byte for byte.

gate

gate: one declared gate of the candidate’s configuration over the two sides eval wrote.

id_of

Identity of a document, as text, for output.

judge

eval judge (spec 041, 3.10.4): reproduce the judge’s bundles for each view entry and write the judge scorer’s rustev.scores/1 to out.

read_bundles

The bundles of cases from dir (spec 015, 3.3): absent files are left out for the library to judge (bundle-missing); a file that exists but cannot be read, is past the document limit or does not parse is supplied as that case’s load failure, and the command continues. Only an exhausted read budget stops it, because that depends on read order rather than on the bundle.

read_doc

Read and parse one document of type T under limit, as invalid_input when it is refused.

split_named

A split by its name (spec 006, 3.9).

views

eval views (spec 041, 3.10.3): the score views of the evaluation eval would run over the same inputs, written to the file out.

write_all

Write every (name, bytes) into dir, all or none. The bytes move into the host’s list; nothing is copied.